Big Ball of Mud

Big Ball of Mud

Brian Foote andJoseph Yoder

Department of Computer Science
University of Illinois at Urbana-Champaign
1304 W. Springfield
Urbana, IL 61801 USA

[e-mail][phone]
[e-mail] [phone]

Saturday, June 26, 1999

Fourth Conference on Patterns Languages of Programs (PLoP '97/EuroPLoP '97)
Monticello, Illinois, September 1997
Technical Report #WUCS-97-34 (PLoP '97/EuroPLoP '97),September 1997
Department of Computer Science, Washington University
Chapter 29
Pattern Languages of Program Design 4
edited by Neil Harrison, Brian Foote, and Hans Rohnert
Addison-Wesley, 2000
This volume is part of the Addison-Wesley Software Patterns Series.

This paper is also available in the following formats:
[PDF] [Word] [RTF] [PostScript]

Also by Brian Footeand Joseph Yoder
Architecture, Evolution, and Metamorphosis
The Selfish Class

This paper was twice featured in Slashdot

Contents

1. Abstract
2. Introduction
3. Forces
4. Big Ball Of Mud
5. Throwaway Code
6. Piecemeal Growth
7. Keep It Working
8. Shearing Layers
9. Sweeping It Under The Rug
10. Reconstruction
11. Conclusion
12. Acknowledgments
13. References

Abstract

While much attention has been focused on high-level software architectural patterns, what is, in effect, the de-facto standard software architecture is seldom discussed. This paper examines this most frequently deployed of software architectures: theBIG BALL OF MUD. ABIG BALL OF MUD is a casually, even haphazardly, structured system. Its organization, if one can call it that, is dictated more by expediency than design. Yet, its enduring popularity cannot merely be indicative of a general disregard for architecture.

These patterns explore the forces that encourage the emergence of a BIG BALL OF MUD, and the undeniable effectiveness of this approach to software architecture. What are the people who build them doing right? If more high-minded architectural approaches are to compete, we must understand what the forces that lead to aBIG BALL OF MUD are, and examine alternative ways to resolve them.

A number of additional patterns emerge out of the BIG BALL OF MUD. We discuss them in turn. Two principal questions underlie these patterns: Why are so many existing systems architecturally undistinguished, and what can we do to improve them?

Introduction

Over the last several years, a number of authors [Garlan & Shaw 1993] [Shaw 1996] [Buschmann et. al. 1996] [Meszaros 1997] have presented patterns that characterize high-level software architectures, such as PIPELINE and LAYERED ARCHITECTURE. In an ideal world, every system would be an exemplar of one or more such high-level patterns. Yet, this is not so. The architecture that actually predominates in practice has yet to be discussed: theBIG BALL OF MUD.

A BIG BALL OF MUD is haphazardly structured, sprawling, sloppy, duct-tape and bailing wire,spaghetti code jungle. We’ve all seen them. These systems show unmistakable signs of unregulated growth, and repeated,expedient repair. Information is shared promiscuously among distant elements of the system, often to the point where nearly all the important information becomes global or duplicated. The overall structure of the system may never have been well defined. If it was, it may haveeroded beyond recognition. Programmers with a shred of architectural sensibility shun these quagmires. Only those who are unconcerned about architecture, and, perhaps, are comfortable with the inertia of the day-to-day chore of patching the holes in these failing dikes, are content to work on such systems.

Still, this approach endures and thrives. Why is this architecture so popular? Is it as bad as it seems, or might it serve as a way-station on the road to more enduring, elegant artifacts? What forces drive good programmers to build ugly systems? Can we avoid this? Should we? How can we make such systems better?

We present the following seven patterns:

BIG BALL OF MUD

THROWAWAY CODE

PIECEMEAL GROWTH

KEEP IT WORKING

SHEARING LAYERS

SWEEPING IT UNDER THE RUG

RECONSTRUCTION

Why does a system become a BIG BALL OF MUD? Sometimes, big, ugly systems emerge from THROWAWAY CODE.THROWAWAY CODE is quick-and-dirty code that was intended to be used only once and then discarded. However, such code often takes on a life of its own, despite casual structure and poor or non-existent documentation. It works, so why fix it? When a related problem arises, the quickest way to address it might be to expediently modify this working code, rather than design a proper, general program from the ground up. Over time, a simple throwaway program begets a BIG BALL OF MUD.

Even systems with well-defined architectures are prone to structural erosion. The relentless onslaught of changing requirements that any successful system attracts can gradually undermine its structure. Systems that were once tidy become overgrown asPIECEMEAL GROWTH gradually allows elements of the system to sprawl in an uncontrolled fashion.

If such sprawl continues unabated, the structure of the system can become so badly compromised that it must be abandoned. As with a decaying neighborhood, a downward spiral ensues. Since the system becomes harder and harder to understand, maintenance becomes more expensive, and more difficult. Good programmers refuse to work there. Investors withdraw their capital. And yet, as with neighborhoods, there are ways to avoid, and even reverse, this sort of decline. As with anything else in the universe, counteracting entropic forces requires an investment of energy. Software gentrification is no exception. The way to arrest entropy in software is to refactor it. A sustained commitment to refactoring can keep a system from subsiding into a BIG BALL OF MUD.

A major flood, fire, or war may require that a city be evacuated and rebuilt from the ground up. More often, change takes place a building or block at a time, while the city as a whole continues to function. Once established, a strategy ofKEEPING IT WORKING preserves a municipality’s vitality as it grows.

Systems and their constituent elements evolve at different rates. As they do, things that change quickly tend to become distinct from things that change more slowly. TheSHEARING LAYERS that develop between them are like fault lines or facets that help foster the emergence of enduring abstractions.

A simple way to begin to control decline is to cordon off the blighted areas, and put an attractive façade around them. We call this strategySWEEPING IT UNDER THE RUG. In more advanced cases, there may be no alternative but to tear everything down and start over. When totalRECONSTRUCTION becomes necessary, all that is left to salvage is the patterns that underlie the experience.

Some of these patterns might appear at first to be antipatterns [Brown et al. 1998] or straw men, but they are not, at least in the customary sense. Instead, they seek to examine the gap between what we preach and what we practice.

Still, some of them may strike some readers as having a schizoid quality about them. So, for the record, let us put our cards on the table. We are in favor of good architecture.

Our ultimate agenda is to help drain these swamps. Where possible, architectural decline should be prevented, arrested, or reversed. We discuss ways of doing this. In severe cases, architectural abominations may even need to be demolished.

At the same time, we seek not to cast blame upon those who must wallow in these mires. In part, our attitude is to "hate the sin, but love the sinner". But, it goes beyond this. Not every backyard storage shack needs marble columns. There are significant forces that can conspire to compel architecture to take a back seat to functionality, particularly early in the evolution of a software artifact. Opportunities and insights that can allow for architectural progress often are present later rather than earlier in the lifecycle.

A certain amount of controlled chaos is natural during construction, and can be tolerated, as long as you clean up after yourself eventually. Even beyond this though, a complex system may be an accurate reflection of our immature understanding of a complex problem. The class of systems that we can build at all may be larger than the class of systems we can build elegantly, at least at first. A somewhat ramshackle rat's nest might be a state-of-the-art architecture for a poorly understood domain. This should not be the end of the story, though. As we gain more experience in such domains, we should increasingly direct our energies to gleaning more enduring architectural abstractions from them.

The patterns described herein are not intended to stand alone. They are instead set in a context that includes a number of other patterns that we and others have described. In particular, they are set in contrast to the lifecycle patterns,PROTOTYPEPHASE, EXPANSIONARY PHASE, and CONSOLIDATION PHASE, presented in [Foote & Opdyke 1995] and [Coplien 1995], theSOFTWARE TECTONICS pattern in [Foote & Yoder 1996], and the framework development patterns in [Roberts & Johnson 1998].

Indeed, to a substantial extent, much of this chapter describes the disease, while the patterns above describe what we believe can be the cure: a flexible, adaptive, feedback-driven development process in which design and refactoring pervade the lifecycle of each artifact, component, and framework, within and beyond the applications that incubate them.

Forces

A number of forces can conspire to drive even the most architecturally conscientious organizations to produceBIG BALLS OF MUD. These pervasive,"global" forces are at work in all the patterns presented. Among these forces:

Time: There may not be enough time to consider the long-term architectural implications of one’s design and implementation decisions. Even when systems have been well designed, architectural concerns often must yield to more pragmatic ones as a deadline starts to loom.

One reason that software architectures are so often mediocre is that architecture frequently takes a back seat to more mundane concerns such as cost, time-to-market, and programmer skill. Architecture is often seen as a luxury or a frill, or the indulgent pursuit of lily-gilding compulsives who have no concern for the bottom line. Architecture is often treated with neglect, and even disdain. While such attitudes are unfortunate, they are not hard to understand. Architecture is a long-term concern. The concerns above have to be addressed if a product is not to be stillborn in the marketplace, while the benefits of good architecture are realized later in the lifecycle, as frameworks mature, and reusable black-box components emerge [Foote & Opdyke 1995].

Architecture can be looked upon as a Risk, that will consume resources better directed at meeting a fleeting market window, or as anOpportunity to lay the groundwork for a commanding advantage down the road.

Indeed, an immature architecture can be an advantage in a growing system because data and functionality can migrate to their natural places in the system unencumbered by artificial architectural constraints. Premature architecture can be more dangerous than none at all, as unproved architectural hypotheses turn into straightjackets that discourage evolution and experimentation.

Cost: Architecture is expensive, especially when a new domain is being explored. Getting the system right seems like a pointless luxury once the system is limping well enough to ship. An investment in architecture usually does not pay off immediately. Indeed, if architectural concerns delay a product’s market entry for too long, then long-term concerns may be moot. Who benefits from an investment in architecture, and when is a return on this investment seen? Money spent on a quick-and-dirty project that allows an immediate entry into the market may be better spent than money spent on elaborate, speculative architectural fishing expedition. It’s hard to recover the value of your architectural assets if you’ve long since gone bankrupt.

Programmers with the ability to discern and design quality architectures are reputed to command a premium. These expenses must be weighed against those of allowing an expensive system to slip into premature decline and obsolescence. If you think good architecture is expensive, try bad architecture.

Experience: Even when one has the time and inclination to take architectural concerns into account, one’s experience, or lack thereof, with the domain can limit the degree of architectural sophistication that can be brought to a system, particularly early in its evolution. Some programmers flourish in environments where they can discover and develop new abstractions, while others are more comfortable in more constrained environments (for instance, Smalltalk vs.Visual Basic programmers.) Often, initial versions of a system are vehicles whereby programmers learn what pieces must be brought into play to solve a particular problem. Only after these are identified do the architectural boundaries among parts of the system start to emerge.

Inexperience can take a number of guises. There is absolute, fresh out of school inexperience. A good architect may lack domain experience, or a domain expert who knows the code cold may not have architectural experience.

Employee turnover can wreak havoc on an organization’s institutional memory, with the perhaps dubious consolation of bringing fresh blood aboard.

Skill: Programmers differ in their levels of skill, as well as in expertise, predisposition and temperament. Some programmers have a passion for finding good abstractions, while some are skilled at navigating the swamps of complex code left to them by others. Programmers differ tremendously in their degrees of experience with particular domains, and their capacities for adapting to new ones. Programmers differ in their language and tool preferences and experience as well.

Visibility: Buildings are tangible, physical structures. You can look at a building. You can watch it being built. You can walk inside it, and admire and critique its design.

A program’s user interface presents the public face of a program, much as a building’s exterior manifests its architecture. However, unlike buildings, only the people who build a program see how it looks inside.

Programs are made of bits. The manner in which we present these bits greatly affects our sense of how they are put together. Some designers prefer to see systems depicted using modeling languages or PowerPoint pictures. Others prefer prose descriptions. Still others prefer to see code. The fashion in which we present our architectures affects our perceptions of whether they are good or bad, clear or muddled, and elegant or muddy.

Indeed, one of the reasons that architecture is neglected is that much of it is "under the hood", where nobody can see it. If the system works, and it can be shipped, who cares what it looks like on the inside?

Complexity: One reason for a muddled architecture is that software often reflects the inherent complexity of the application domain. This is whatBrooks called "essential complexity" [Brooks 1995]. In other words, the software is ugly because the problem is ugly, or at least not well understood. Frequently, the organization of the system reflects the sprawl and history of the organization that built it (as perCONWAY’S LAW [Coplien 1995]) and the compromises that were made along the way. Renegotiating these relationships is often difficult once the basic boundaries among system elements are drawn. These relationships can take on the immutable character of "site" boundaries thatBrand [Brand 1994] observed in real cities. Big problems can arises when the needs of the applications force unrestrained communication across these boundaries. The system becomes a tangled mess, and what little structure is there can erode further.

Change: Architecture is a hypothesis about the future that holds that subsequent change will be confined to that part of the design space encompassed by that architecture. Of course, the world has a way of mocking our attempts to make such predictions by tossing us the totally unexpected. A problem we might have been told was definitely ruled out of consideration for all time may turn out to be dear to the heart of a new client we never thought we’d have. Such changes may cut directly across the grain of fundamental architectural decisions made in the light of the certainty that these new contingencies could never arise. The "right" thing to do might be to redesign the system. The more likely result is that the architecture of the system will be expediently perturbed to address the new requirements, with only passing regard for the effect of these radical changes on the structure of the system.

Scale: Managing a large project is a qualitatively different problem from managing a small one, just as leading a division of infantry into battle is different from commanding a small special forces team. Obviously, "divide and conquer" is, in general, an insufficient answer to the problems posed by scale. Alan Kay, during an invited talk at OOPSLA '86 observed that "good ideas don't always scale." That observation promptedHenry Lieberman to inquire "so what do we do, just scale the bad ones?"

BIG BALL OF MUD

aliasSHANTYTOWNSPAGHETTI CODEBrazil (Brasilia) and Nigeria (Abuja) started as paper cities as well. Other cities, such as Houston, have grown without any overarching plan to guide them. Each approach has its problems. For instance, the radial street plans in L’Enftant’s master plan become awkward past a certain distance from the center. The lack of any plan at all, on the other hand, leads to a patchwork of residential, commercial, and industrial areas that is dictated by the capricious interaction of local forces such as land ownership, capital, and zoning. Since concerns such as recreation, shopping close to homes, and noise and pollution away from homes are not brought directly into the mix, they are not adequately addressed.

Most cities are more like Houston than Abuja. They may begin as settlements, subdivisions, docks, or railway stops. Maybe people were drawn by gold, or lumber, access to transportation, or empty land. As time goes on, certain settlements achieve a critical mass, and a positive feedback cycle ensues. The city’s success draws tradesmen, merchants, doctors, and clergymen. The growing population is able to support infrastructure, governmental institutions, and police protection. These, in turn, draw more people. Different sections of town develop distinct identities. With few exceptions, (Salt Lake City comes to mind) the founders of these settlements never stopped to think that they were founding major cities. Their ambitions were usually more modest, and immediate.

vv v

It has become fashionable over the last several years to take pot shots at the "traditional" waterfall process model. It may seem to the reader that attacking it is tantamount to flogging a dead horse. However, if it be a dead horse, it is a tenacious one. While the approach itself is seen by many as having been long since discredited, it has spawned a legacy of rigid, top-down, front-loaded processes and methodologies that endure, in various guises, to this day. We can do worse that examine the forces that led to its original development.

In the days before waterfall development, programming pioneers employed a simple, casual, relatively undisciplined "code-and-fix" approach to software development. Given the primitive nature of the problems of the day, this approach was frequently effective. However, the result of this lack of discipline was, all too often, a BIG BALL OF MUD.

The waterfall approach arose in response to this muddy morass. While the code-and-fix approach might have been suitable for small jobs, it did not scale well. As software became more complex, it would not do to simply gather a room full of programmers together and tell them to go forth and code. Larger projects demanded better planning and coordination. Why, it was asked, can't software be engineered like cars and bridges, with a careful analysis of the problem, and a detailed up-front design prior to implementation? Indeed, an examination of software development costs showed that problems were many times more expensive to fix during maintenance than during design. Surely it was best to mobilize resources and talent up-front, so as to avoid maintenance expenses down the road. It's surely wiser to route the plumbing correctly now, before the walls are up, than to tear holes in them later. Measure twice, cut once.

One of the reasons that the waterfall approach was able to flourish a generation ago was that computers and business requirements changed at a more leisurely pace. Hardware was very expensive, often dwarfing the salaries of the programmers hired to tend it. User interfaces were primitive by today's standards. You could have any user interface you wanted, as long as it was an alphanumeric "green screen". Another reason for the popularity of the waterfall approach was that it exhibited a comfortable similarity to practices in more mature engineering and manufacturing disciplines.

Today's designers are confronted with a broad onslaught of changing requirements. It arises in part from the rapid growth of technology itself, and partially from rapid changes in the business climate (some of which is driven by technology). Customers are used to more sophisticated software these days, and demand more choice and flexibility. Products that were once built from the ground up by in-house programmers must now be integrated with third-party code and applications. User interfaces are complex, both externally and internally. Indeed, we often dedicate an entire tier of our system to their care and feeding. Change threatens to outpace our ability to cope with it.

Master plans are often rigid, misguided and out of date. Users’ needs change with time.

Change: The fundamental problem with top-down design is that real world requirement are inevitably moving targets. You can't simply aspire to solve the problem at hand once and for all, because, by the time you're done, the problem will have changed out from underneath you. You can't simply do what the customer wants, for quite often, they don't know what they want. You can't simply plan, you have to plan to be able to adapt. If you can't fully anticipate what is going to happen, you must be prepared to be nimble.

Aesthetics: The goal of up-front design is to be able to discern and specify the significant architectural elements of a system before ground is broken for it. A superior design, given this mindset, is one that elegantly and completely specifies the system's structure before a single line of code has been written. Mismatches between these blueprints and reality are considered aberrations, and are treated as mistakes on the part of the designer. A better design would have anticipated these oversights. In the presence of volatile requirements, aspirations towards such design perfection are as vain as the desire for a hole-in-one on every hole.

To avoid such embarrassment, the designer may attempt to cover him or herself by specifying a more complicated, and more general solution to certain problems, secure in the knowledge that others will bear the burden of constructing these artifacts. When such predictions about where complexity is needed are correct, they can indeed be a source of power and satisfaction. This is part of their allure of Venustas. However, sometime the anticipated contingencies never arise, and the designer and implementers wind up having wasted effort solving a problem that no one has ever actually had. Other times, not only is the anticipated problem never encountered, its solution introduces complexity in a part of the system that turns out to need to evolve in another direction. In such cases, speculative complexity can be an unnecessary obstacle to subsequent adaptation. It is ironic that the impulse towards elegance can be an unintended source of complexity and clutter instead.

In its most virulent form, the desire to anticipate and head off change can lead to "analysis paralysis", as the thickening web of imagined contingencies grows to the point where the design space seems irreconcilably constrained.

Therefore, incrementally address forces that encourage change and growth. Allow opportunities for growth to be exploitedlocally, as they occur. Refactor unrelentingly.

Successful software attracts a wider audience, which can, in turn, place a broader range of requirements on it. These new requirements can run against the grain of the original design. Nonetheless, they can frequently be addressed, but at the cost of cutting across the grain of existing architectural assumptions. [Foote 1988] called this architectural erosionmidlife generality loss.

When designers are faced with a choice between building something elegant from the ground up, or undermining the architecture of the existing system to quickly address a problem, architecture usually loses. Indeed, this is a natural phase in a system’s evolution [Foote & Opdyke 1995]. This might be thought of asmessy kitchen phase, during which pieces of the system are scattered across the counter, awaiting an eventual cleanup. The danger is that the clean up is never done. With real kitchens, the board of health will eventually intervene. With software, alas, there is seldom any corresponding agency to police such squalor. Uncontrolled growth can ultimately be a malignant force. The result of neglecting to contain it can be aBIG BALL OF MUD.

In How Buildings Learn, Brand [Brand 1994] observed that what he calledHigh Road architecture often resulted in buildings that were expensive and difficult to change, while vernacular,Low Road buildings like bungalows and warehouses were, paradoxically, much more adaptable. Brand noted thatFunction melts form, and low road buildings are more amenable to such change. Similarly, with software, you may be reluctant to desecrate another programmer’s cathedral. Expedient changes to a low road system that exhibits no discernable architectural pretensions to begin with are easier to rationalize.

In the Oregon Experiment [Brand 1994][Alexander 1988] Alexander noted:

Large-lump development is based on the idea of replacement. Piecemeal Growth is based on the idea ofrepair. … Large-lump development is based on the fallacy that it is possible to build perfect buildings. Piecemeal growth is based on the healthier and more realistic view that mistakes are inevitable. … Unless money is available for repairing these mistakes, every building, once built, is condemned to be, to some extent unworkable. … Piecemeal growth is based on the assumption that adaptation between buildings and their users is necessarily a slow and continuous business which cannot, under any circumstances, be achieve in a single leap.

Alexander has noted that our mortgage and capital expenditure policies make large sums of money available up front, but do nothing to provide resources for maintenance, improvement, and evolution [Brand 1994][Alexander 1988]. In the software world, we deploy our most skilled, experienced people early in the lifecycle. Later on, maintenance is relegated to junior staff, when resources can be scarce. The so-called maintenance phase is the part of the lifecycle in which the price of the fiction of master planning is really paid. It is maintenance programmers who are called upon to bear the burden of coping with the ever widening divergence between fixed designs and a continuously changing world. If the hypothesis that architectural insight emerges late in the lifecycle is correct, then this practice should be reconsidered.

Brand went on to observe Maintenance is learning. He distinguishes three levels of learning in the context of systems. This first is habit, where a system dutifully serves its function within the parameters for which it was designed. The second level comes into play when the system must adapt to change. Here, it usually must be modified, and its capacity to sustain such modification determines it’s degree of adaptability. The third level is the most interesting:learning to learn. With buildings, adding a raised floor is an example. Having had to sustain a major upheaval, the system adapts so that subsequent adaptations will be much less painful.

PIECEMEAL GROWTH can be undertaken in an opportunistic fashion, starting with the existing, living, breathing system, and working outward, a step at a time, in such a way as to not undermine the system’s viability. You enhance the program as you use it. Broad advances on all fronts are avoided. Instead, change is broken down into small, manageable chunks.

One of the most striking things about PIECEMEAL GROWTH is the role played by Feedback. Herbert Simon [Simon 1969] has observed that few of the adaptive systems that have been forged by evolution or shaped by man depend on prediction as their main means of coping with the future. He notes that two complementary mechanisms, homeostasis, and retrospective feedback, are often far more effective. Homeostasis insulates the system from short-range fluctuations in its environment, while feedback mechanisms respond to long-term discrepancies between a system's actual and desired behavior, and adjust it accordingly. Alexander [Alexander 1964] has written extensively of the roles that homeostasis and feedback play in adaptation as well.

If you can adapt quickly to change, predicting it becomes far less crucial. Hindsight, as Brand observes [Brand 1994] is better than foresight. Such rapid adaptation is the basis of one of the mantras ofExtreme Programming [Beck 2000]: You're not going to need it.

Proponents of XP (as it is called) say to pretend you are not a smart as you think you are, and wait until this clever idea of yours is actually required before you take the time to bring it into being. In the cases where you were right, hey, you saw it coming, and you know what to do. In the cases where you were wrong, you won't have wasted any effort solving a problem you've never had when the design heads in an unanticipated direction instead.

Extreme Programming relies heavily on feedback to keep requirements in sync with code, by emphasizing short (three week) iterations, and extensive, continuous consultation with users regarding design and development priorities throughout the development process. Extreme Programmers do not engage in extensive up-front planning. Instead, they produce working code as quickly as possible, and steer these prototypes towards what the users are looking for based on feedback.

Feedback also plays a role in determining coding assignments. Coders who miss a deadline are assigned a different task during the next iteration, regardless of how close they may have been to completing the task. This form of feedback resembles the stern justice meted out by the jungle to the fruit of uncompetitive pairings.

Extreme Programming also emphasizes testing as an integral part of the development process. Tests are developed, ideally, before the code itself. Code is continuously tested as it is developed.

There is a "back-to-the-future" quality to Extreme Programming. In many respects, it resembles the blindCode and Fix approach. The thing that distinguishes it is the central role played by feedback in driving the system's evolution. This evolution is abetted, in turn, by modern object-oriented languages and powerful refactoring tools.

Proponents of extreme programming portray it as placing minimal emphasis on planning and up-front design. They rely instead on feedback and continuous integration. We believe that a certain amount of up-front planning and design is not only important, but inevitable. No one really goes into any project blindly. The groundwork must be laid, the infrastructure must be decided upon, tools must be selected, and a general direction must be set. A focus on a shared architectural vision and strategy should be established early.

Unbridled, change can undermine structure. Orderly change can enhance it. Change can engender malignant sprawl, or healthy, orderly growth.

vv v

A broad consensus that objects emerge from an iterative incremental evolutionary process has formed in the object-oriented community over the last decade. See for instance [Booch 1994]. TheSOFTWARE TECTONICS pattern [Foote & Yoder 1996] examines how systems can incrementally cope with change.

The biggest risk associated with PIECEMEAL GROWTH is that it will gradually erode the overall structure of the system, and inexorably turn it into aBIG BALL OF MUD. A strategy ofKEEPING IT WORKING goes hand in hand withPIECEMEAL GROWTH. Both patterns emphasize acute, local concerns at the expense of chronic, architectural ones.

To counteract these forces, a permanent commitment to CONSOLIDATION and refactoring must be made. It is through such a process that local and global forces are reconciled over time. This lifecyle perspective has been dubbed thefractal model [Foote & Opdyke 1995]. To quote Alexander [Brand 1994][Alexander 1988]:

An organic process of growth and repair must create a gradual sequence of changes, and these changes must be distributed evenly across all levels of scale. [In developing a college campus] there must be as much attention to the repair of details—rooms, wings of buildings, windows, paths—as to the creation of brand new buildings. Only then can the environment be balanced both as a whole, and in its parts, at every moment in its history.

 

KEEP IT WORKING

aliasVITALITYBABY STEPSDAILY BUILDFIRST, DO NO HARM

Probably the greatest factor that keeps us moving forward is that we use the system all the time, and we keep trying to do new things with it. It is this "living-with" which drives us to root out failures, to clean up inconsistencies, and which inspires our occasional innovation.

Daniel H. H. Ingalls[Ingalls 1983]

Once a city establishes its infrastructure, it is imperative that it be kept working. For example, if the sewers break, and aren’t quickly repaired, the consequences can escalate from merely unpleasant to genuinely life threatening. People come to expect that they can rely on their public utilities being available 24 hours per day. They (rightfully) expect to be able to demand that an outage be treated as an emergency.

vv v

Software can be like this. Often a business becomes dependent upon the data driving it. Businesses have become critically dependent on their software and computing infrastructures. There are numerous mission critical systems that must be on-the-air twenty-four hours a day/seven days per week. If these systems go down, inventories can not be checked, employees can not be paid, aircraft cannot be routed, and so on.

There may be times where taking a system down for a major overhaul can be justified, but usually, doing so is fraught with peril. However, once the system is brought back up, it is difficult to tell which from among a large collection of modifications might have caused a new problem. Every change is suspect. This is why deferring such integration is a recipe for misery. Capers Jones [Jones 1999] reported that the chance that a significant change might contain a new error--a phenomenon he ominously referred to as a Bad Fix Injection-- was about 7% in the United States. This may strike some readers as a low figure. Still, it's easy to see that compounding this possibility can lead to a situation where multiple upgrades are increasing likely to break a system.

Maintenance needs have accumulated, but an overhaul is unwise, since you might break the system.

Workmanship: Architects who live in the house they are building have an obvious incentive to insure that things are done properly, since they will directly reap the consequences when they do not. The idea of the architect-builder is a central theme of Alexander's work. Who better to resolve the forces impinging upon each design issue as it arises as the person who is going to have to live with these decisions? The architect-builder will be the direct beneficiary of his or her own workmanship and care. Mistakes and shortcuts will merely foul his or her own nest.

Dependability: These days, people rely on our software artifacts for their very livelihoods, and even, at time, for their very safety. It is imperative that ill-advise changes to elements of a system do not drag the entire system down. Modern software systems are intricate, elaborate webs of interdependent elements. When an essential element is broken, everyone who depends on it will be affected. Deadlines can be missed, and tempers can flare. This problem is particularly acute inBIG BALLS OF MUD, since a single failure can bring the entire system down like a house of cards.

Therefore, do what it takes to maintain the software and keep it going. Keep it working.

When you are living in the system you’re building, you have an acute incentive not to break anything. A plumbing outage will be a direct inconvenience, and hence you have a powerful reason to keep it brief. You are, at times, working with live wires, and must exhibit particular care. A major benefit of working with a live system is that feedback is direct, and nearly immediate.

One of the strengths of this strategy is that modifications that break the system are rejected immediately. There are always a large number of paths forward from any point in a system’s evolution, and most of them lead nowhere. By immediately selecting only those that do not undermine the system’s viability, obvious dead-ends are avoided.

Of course, this sort of reactive approach, that of kicking the nearest, meanest woolf from your door, is not necessarily globally optimal. Yet, by eliminating obvious wrong turns,only more insidiously incorrect paths remain. While these are always harder to identify and correct, they are, fortunately less numerous than those cases where the best immediate choice is also the best overall choice as well.

It may seem that this approach only accommodates minor modifications. This is not necessarily so. Large new subsystems might be constructed off to the side, perhaps by separate teams, and integrated with the running system in such a way as to minimize distruption.

Design space might be thought of as a vast, dark, largely unexplored forest. Useful potential paths through it might be thought of as encompassing working programs. The space off to the sides of these paths is much larger realm of non-working programs. From any given point, a few small steps in most directions take you from a working to a non-working program. From time to time, there are forks in the path, indicating a choice among working alternatives. In unexplored territory, the prudent strategy is never to stray too far from the path. Now, if one has a map, a shortcut through the trekless thicket that might save miles may be evident. Of course, pioneers, by definition, don’t have maps. By taking small steps in any direction, they know that it is never more than a few steps back to a working system.

Some years ago, Harlan Mills proposed that any software system should be grown by incremental development. That is, the system first be made to run, even though it does nothing useful except call the proper set of dummy subprograms. Then, bit by bit, it is fleshed out, with the subprograms in turn being developed into actions or calls to empty stubs in the level below.

…

Nothing in the past decade has so radically changed my own practice, and its effectiveness.

…

One always has, at every stage, in the process, a working system. I find that teams cangrow much more complex entities in four months than they can build.

-- From "No Silver Bullet"[Brooks 1995]

Microsoft mandates that a DAILY BUILD of each product be performed at the end of each working day. Nortel adheres to the slightly less demanding requirement that a working build be generated at the end of each week [Brooks 1995][Cusumano & Shelby 1995]. Indeed, this approach, and keeping the last working version around, are nearly universal practices among successful maintenance programmers.

Another vital factor in ensuring a system's continued vitality is a commitment to rigorous testing [Marick 1995][Bach 1994]. It's hard to keep a system working if you don't have a way of making sure it works. Testing is one of pillars of Extreme Programming. XP practices call for the development of unit tests before a single line of code is written.

vv v

Always beginning with a working system helps to encourage PIECEMEAL GROWTH. Refactoring is the primary means by which programmers maintain order from inside the systems in which they are working. The goal of refactoring is to leave a system working as well after a refactoring as it was before the refactoring. Aggressive unit and integration testing can help to guarantee that this goal is met.

 

SHEARING LAYERS

Conclusion

In the end, software architecture is about how we distill experience into wisdom, and disseminate it. We think the patterns herein stand alongside other work regarding software architecture and evolution that we cited as we went along. Still, we do not consider these patterns to be anti-patterns. There are good reasons that good programmers buildBIG BALLS OF MUD. It may well be that the economics of the software world are such that the market moves so fast that long term architectural ambitions are foolhardy, and that expedient, slash-and-burn, disposable programming is, in fact, a state-of-the-art strategy. The success of these approaches, in any case, is undeniable, and seals their pattern-hood. People build

BIG BALLS OF MUD because theywork. In many domains, they are the only things that have been shown to work. Indeed, they work where loftier approaches have yet to demonstrate that they can compete.

It is not our purpose to condemn BIG BALLS OF MUD. Casual architecture is natural during the early stages of a system’s evolution. The reader must surely suspect, however, that our hope is that we can aspire to do better. By recognizing the forces and pressures that lead to architectural malaise, and how and when they might be confronted, we hope to set the stage for the emergence of truly durable artifacts that can put architects in dominant positions for years to come. The key is to ensure that the system, its programmers, and, indeed the entire organization, learn about the domain, and the architectural opportunities looming within it, as the system grows and matures.

Periods of moderate disorder are a part of the ebb and flow of software evolution. As a master chef tolerates a messy kitchen, developers must not be afraid to get a little mud on their shoes as they explore new territory for the first time. Architectural insight is not the product of master plans, but of hard won experience. The software architects of yesteryear had little choice other than to apply the lessons they learned in successive drafts of their systems, sinceRECONSTRUCTION was often the only practical means they had of supplanting a mediocre system with a better one. Objects, frameworks, components, and refactoring tools provide us with another alternative. Objects present a medium for expressing our architectural ideas at a level between coarse-grained applications and components and low level code. Refactoring tools and techniques finally give us the means to cultivate these artifacts as they evolve, and capture these insights.

The onion-domed Church of the Intercession of the Virgin on the Moat in Moscow is one of Russia's most famous landmarks. It was built by Tsar Ivan IV just outside of the Kremlin walls in 1552 to commemorate Russia's victory over the Tatars at Kazan. The church is better known by it's nickname, St. Basil's. Ivan too is better known by his nickname "Ivan the Terrible". Legend has it that once the cathedral was completed, Ivan, ever true to his reputation, had the architects blinded, so that they could never build anything more beautiful. Alas, the state of software architecture today is such that few of us need fear for our eyesight.

Acknowledgments

A lot of people have striven to help us avoid turning this paper into an unintentional example of its central theme. We are grateful first of all to the members of theUniversity of Illinois Software Architecture Group,John Brant, Ian Chai, Ralph Johnson, Lewis Muir,Dragos Manolescu, Brian Marick, Eiji Nabika, John (Zhijiang) Han, Kevin Scheufele, Tim Ryan, Girish Maiya, Weerasak Wittawaskul, Alejandra Garrido, Peter Hatch, andDon Roberts, who commented on several drafts of this work over the last three years.

We’d like to also thank our tireless shepherd, Bobby Woolf, who trudged through the muck of several earlier versions of this paper.

Naturally, we’d like to acknowledge the members of our PLoP ’97 Conference Writer’s Workshop, Norm Kerth, Hans Rohnert, Clark Evans, Shai Ben-Yehuda, Lorraine Boyd, Alejandra Garrido,Dragos Manolescu, Gerard Meszaros, Kyle Brown,Ralph Johnson, and Klaus Renzel.

Lorrie Boyd provided some particularly poignant observations on scale, and the human cost of projects that fail.

UIUC Architecture professor Bill Rose provided some keen insights on the durability of housing stock, and history of the estrangement of architects from builders.

Thanks to Brad Appleton, Michael Beedle, Russ Hurlbut, and the rest of the people in the Chicago Patterns Group for their time, suggestions, and ruminations on reuse and reincarnation.

Thanks to Steve Berczuk and the members of theBoston Area Patterns Group for their review.

Thanks too to Joshua Kerievsky and theDesign Patterns Study Group of New York City for their comments.

We'd like to express our gratitude as well to Paolo Cantoni, Chris Olufson, Sid Wright, John Liu, Martin Cohen, John Potter, Richard Helm, andJames Noble of the Sydney Patterns Group, who workshopped this paper during the late winter, er, summer of early 1998.

John Vlissides, Neil Harrison, Hans Rohnert, James Coplien, and Ralph Johnson provided some particularly candid, incisive and useful criticism of some of the later drafts of the paper.

A number of readers have observed, over the years, that BIG BALL OF MUD has a certain dystopian, Dilbert-esque quality to it. We are grateful to United Features Syndicate, Inc. for not having, as of yet, asked us to remove the following cartoon from the web-based version ofBIG BALL OF MUD.

References

[Alexander 1964]Christopher AlexanderNotes on the Synthesis of FormHarvard University Press, Cambridge, MA, 1964[Alexander 1979]Christopher AlexanderThe Timeless Way of BuildingOxford University Press, Oxford, UK, 1979[Alexander et. al 1977]C. Alexander, S. Ishikawa, and M. SilversteinA Pattern LanguageOxford University Press, Oxford, UK, 1977[Alexander 1988]Christopher AlexanderThe Oregon ExperimentOxford University Press, Oxford, UK, 1988[Bach 1997]James Bach, Softwae Testing LabsGood Enough Software: Beyond the BuzzwordIEEE Computer, August 1997[Beck 1997]Kent BeckSmalltalk Best Practice PatternsPrentice Hall, Upper Saddle River, NJ, 1997[Beck & Cunningham 1989]Kent Beck and Ward CunninghamA Laboratory for Teaching Object-Oriented ThinkingOOPSLA '89 ProceedingsNew Orleans, LAOctober 1-6 1989, pages 1-6[Beck 2000]Kent BeckEmbracing Change: Extreme Programming ExplainedCambridge University Press, 2000[Booch 1994]Grady BoochObject-Oriented Analysis and Design with ApplicationsBenjamin/Cummings, Redwood City, CA, 1994[Brand 1994]Stewart BrandHow Buildings Learn: What Happens After They're BuiltViking Press, 1994[Brooks 1995]Frederick P. Brooks, Jr.The Mythical Man-Month (Anniversary Edition)Addison-Wesley, Boston, MA, 1995[Brown et al. 1998]William J. Brown, Raphael C. Malveau,Hays W. "Skip" McCormick III, and Thomas J. MobrayAntipatterns: Refactoring, Software Architectures, and Projects in CrisisWiley Computer Publishing, John Wiley & Sons, Inc., 1998[Buschmann et al. 1996]Frank Buschmann, Regine Meunier, Hans Rohnert, Peter Sommerlad, and Michael StahlPattern-Oriented Software Architecture: A System of PatternsJohn Wiley and Sons, 1996[Coplien 1995]James O. CoplienA Generative Development-Process Pattern LanguageFirst Conference on Pattern Languages of Programs (PLoP '94)Monticello, Illinois, August 1994Pattern Languages of Program Designedited by James O. Coplien and Douglas C. SchmidtAddison-Wesley, 1995[Cunningham 1999a]Ward CunninghamPeter Principle of ProgrammingPortland Pattern Repository13 August 1999http://www.c2.com/cgi/wiki?PeterPrincipleProgramming[Cunningham 1999b]Ward CunninghamThe Most Complicated Thing that Could Possible WorkPortland Pattern Repository13 August 1999http://www.c2.com/cgi/wiki?TheMostComplexWhichCanBeMadeToWork[Cusumano & Shelby 1995]Michael A. Cusumano and Richard W. ShelbyMicrosoft SecretsThe Free Press, New York, NY, 1995[Foote 1988]Brian Foote (Advisor: Ralph Johnson)Designing to Facilitate Change with Object-Oriented FrameworksMasters Thesis, 1988Dept. of Computer Science,University of Illinois at Urbana-Champaign[Foote & Opdyke 1995]Brian Foote and William F. OpdykeLifecycle and Refactoring Patterns that Support Evolution and ReuseFirst Conference on Patterns Languages of Programs (PLoP '94)Monticello, Illinois, August 1994Pattern Languages of Program Designedited by James O. Coplien and Douglas C. SchmidtAddison-Wesley, 1995This volume is part of the Addison-Wesley Software Patterns Series.[Foote & Yoder 1996]Brian Foote and Joseph W. YoderEvolution, Architecture, and MetamorphosisSecond Conference on Patterns Languages of Programs (PLoP '95)Monticello, Illinois, September 1995Pattern Languages of Program Design 2edited by John M. Vlissides, James O. Coplien, and Norman L. KerthAddison-Wesley, 1996This volume is part of the Addison-Wesley Software Patterns Series.[Foote & Roberts 1998]Brian Foote and Don RobertsLingua FrancaFifth Conference on Patterns Languages of Programs (PLoP '98)Monticello, Illinois, August 1998Technical Report #WUCS-98-25 (PLoP '98/EuroPLoP '98), September 1998Department of Computer Science, Washington University[Foote & Yoder 1996]Brian Foote and Joseph W. YoderEvolution, Architecture, and MetamorphosisSecond Conference on Patterns Languages of Programs (PLoP '95)Monticello, Illinois, September 1995Pattern Languages of Program Design 2edited by John M. Vlissides, James O. Coplien, and Norman L. KerthAddison-Wesley, 1996This volume is part of the Addison-Wesley Software Patterns Series.[Foote & Yoder 1998a]Brian Foote and Joseph W. YoderThe Selfish ClassThird Conference on Patterns Languages of Programs (PLoP '96)Monticello, Illinois, September 1996Technical Report #WUCS-97-07, September 1996Department of Computer Science, Washington UniversityPattern Languages of Program Design 3edited by Robert Martin, Dirk Riehle, and Frank Buschmann Addison-Wesley, 1998http://www.laputan.orgThis volume is part of the Addison-Wesley Software Patterns Series.Brian also wrote an introduction for this volume.[Foote & Yoder 1998b]Brian Foote and Joseph W. YoderMetadataFifth Conference on Patterns Languages of Programs (PLoP '98)Monticello, Illinois, August 1998Technical Report #WUCS-98-25 (PLoP '98/EuroPLoP '98), September 1998Department of Computer Science, Washington University[Fowler 1999]Martin FowlerRefactoring: Improving the Design of Existing CodeAddison Wesley Longman, 1999[Gabriel 1991]Richard P. Gabriel Lisp: Good News Bad News and How to Win Bighttp://www.laputan.org/gabriel/worse-is-better.html[Gabriel 1996]Richard P. GabrielPatterns of Software: Tales from the Software CommunityOxford University Press, Oxford, UK, 1996http://www.oup-usa.org/[Gamma et al. 1995]Eric Gamma, Richard Helm, Ralph Johnson, and John VlissidesDesign Patterns:  Elements of Reusable Object-Oriented SoftwareAddison-Wesley Longman, Reading, MA, 1995[Garlan & Shaw 1993]David Garlan and Mary ShawAn Introduction to Software ArchitectureV. Ambriola and G. Totora, editorsAdvances in Software Engineering and Knowledge Engineering, Vol 2.Singapore: World Scientific Publishing, 1993, pp. 1-39[Ingalls 1983]Daniel H. H. IngallsThe Evolution of the Smalltalk Virtual MachineSmalltalk-80: Bits of History, Words of Adviceedited by Glenn KrasnerAddison-Wesley, 1983[Johnson & Foote 1988]Ralph Johnson and Brian FooteDesigning Reusable ClassesJournal of Object-Oriented ProgrammingVolume 1, Number 2, June/July 1988[Marick 1995]Brian MarickThe Craft of Software TestingPrentice-Hall, Upper Saddle River, NJ, 1995[Meszaros 1997]Gerard MeszarosArchi-Patterns: A Process Pattern Language for Defining ArchitecturesFourth Conference on Pattern Languages of Programs (PLoP '97)Monticello, Illinois, September 1997[Roberts & Johnson 1998]Don Roberts and Ralph E. JohnsonEvolve Frameworks into Domain-Specific LanguagesThird Conference on Patterns Languages of Programs (PLoP '96)Monticello, Illinois, September 1996Technical Report #WUCS-97-07, September 1996Department of Computer Science, Washington UniversityPattern Languages of Program Design 3edited by Robert Martin, Dirk Riehle, and Frank Buschmann Addison-Wesley, 1998[Shaw 1996]Mary ShawSome Patterns for Software ArchitecturesSecond Conference on Patterns Languages of Programs (PLoP '95)Monticello, Illinois, September 1995Pattern Languages of Program Design 2edited by John M. Vlissides, James O. Coplien, and Norman L. KerthAddison-Wesley, 1996[Simon 1969]Herbert A. SimonThe Sciences of the ArtificialMIT Press, Cambridge, MA, 1969[Swift 1726]Johnathan SwiftTravels Into Several Remote Nations Of The World. In four parts. By Lemuel Gulliver, First a Surgeon, and then a Captain of several Ships.B. Motte, London, 1726.[Vitruvius 20 B.C.]Marcus Vitruvius Pollio (60 B.C-20 B.C.)De Architecturatranslated by Joseph GwiltPriestley and Weale, London, 1826

---