Real Software Engineering

Talks

Glenn Vanderburg

@glv

#software-engineering

#agile

#quality-assurance-qa

Real Software Engineering

by Glenn Vanderburg

The video features Glenn Vanderburg discussing the shortcomings of traditional software engineering practices as taught in universities and by companies. He argues that these practices often do not control costs, ensure quality, or even produce functioning software reliably. Vanderburg highlights a critical misunderstanding among software practitioners regarding both software itself and the true nature of engineering.

Key points discussed include:

- The Historical Context of Software Engineering: The term 'software engineering' gained prominence after the 1968 NATO conference aimed at addressing the 'software crisis' characterized by unreliable and error-prone software projects.

- Misinterpretation of Engineering Principles: Vanderburg outlines that the practices of software engineering became distorted, leading to defined processes that did not yield effective results, contrasting with engineering fields that utilize proven methods.

- The Waterfall Model: He uses the introduction and misinterpretation of the waterfall model as a historical example of how reasonable suggestions were misconstrued, illustrating the pitfalls of rigid process adherence.

- Empirical vs. Defined Processes: Emphasizing the value of empirical processes, Vanderburg discusses the importance of adaptive approaches that can respond to the complexities of software development rather than strictly defined methodologies.

- The Role of Testing and Iteration: He stresses that effective engineering, including software engineering, involves continuous testing and iteration, allowing for adjustments based on real-world demands.

- Creative Nature of Engineering: Vanderburg emphasizes that engineering is fundamentally a creative activity that requires flexibility and responsiveness to new discoveries and challenges throughout the development process.

In conclusion, Vanderburg proposes that software engineering should be viewed as both a science and an art, demanding elegance and adaptability in system design, which aligns more with true engineering principles. He advocates for a shift towards empirical models that enhance creativity and responsiveness in the software engineering field, urging practitioners to focus on iterative learning and adapting rather than rigidly adhering to predefined processes. Vanderburg's session calls for ongoing dialogue in the engineering community to continually innovate and refine software engineering practices.

00:00:10.320 Good morning. I'm Glenn Vanderburg. I work for InfoEther with Rich Kilmer, Chad Fowler, and Bruce Williams, who I suspect many of you know here.

00:00:16.199 I'm here to talk about real software engineering. This is my fifth LoneStarRuby Conference, and I really enjoy them.

00:00:23.599 To start off, software engineering doesn't work—at least as it's currently taught in universities and in training programs at companies. If you happen to work for one of the few companies that cares about teaching such things, you'll notice that the techniques that are called software engineering simply don't work.

00:00:31.119 They don't reliably control costs, and they don't reliably produce quality software. Sometimes, even when they're practiced rigorously by people who've been trained to do so, they don't produce working software at all. This shouldn't be surprising, as it seems to be common knowledge among working programmers all over this country. Yet, while it's not surprising, it is odd, because in every other field that aspires to the title of an engineering discipline, the term engineering is reserved for practices that work. In fact, that's a good definition of engineering, independent of any particular discipline: a set of practices and techniques that have been determined to work reliably through experience.

00:01:06.960 Yet, in software, we have a set of practices that don't work, and we call that engineering. This has sparked a lot of discussion, especially over the past year or so, about whether software is genuinely an engineering discipline or whether software development fits the metaphor at all. Some argue it's not engineering; it's more of a craft, art, or something else entirely, like gardening or movie-making.

00:01:19.680 I don’t believe that to be true. I think software is an art, a craft, and in some respects, a science. That doesn't mean it can't also be engineering. The problem lies in the fact that the people who have defined the set of disciplines we call software engineering have misunderstood two very important concepts: software and engineering.

00:01:30.959 This misunderstanding has resulted in software engineering being a caricature of a true engineering discipline. I will explain what I mean by this and, to do so, I'll first explain how software became a caricature of engineering and how it developed that way. Next, I will describe what real engineering looks like.

00:01:58.680 So, how many people in the room were trained as engineers rather than software developers? Interesting. I'm especially interested in feedback from individuals with a classical engineering background because I’ve had to learn this all on my own. However, I think I've gained a pretty good understanding of where software engineering went wrong.

00:02:09.439 We'll explore what real software development looks like, and in the process, I think we'll highlight what real software engineering is. The first time the term 'software engineering' really got popularized was in 1968 at a NATO-organized conference in Garmisch, Germany.

00:02:25.199 At that time, people were dealing with what was called the software crisis. Software projects were notoriously unreliable, flaky, error-prone, and subject to massive cost overruns—and they didn't know much about how to manage them effectively. Thus, they concluded that they needed to mature as a discipline and become an engineering discipline.

00:02:41.680 I've known about and heard of this conference for many years but never really knew the details, so out of curiosity, I went and read the proceedings—which are available online. I expected to uncover where the madness began. However, I found that the participants were largely very smart people working as software developers. There were a few academics as well, but they also had valuable insights.

00:03:06.240 Overall, the findings of this conference were entirely reasonable and reflected a great deal of wisdom about the state of software at the time. One notable aspect of the conference was their willingness to admit their ignorance—in fact, it was a common theme. The conference was filled with statements about their lack of understanding, uncertainties, and speculations about what software engineering could be.

00:03:39.239 At the conference's conclusion, Alan Perlis delivered a talk in which he summarized three points to which the group could broadly agree about what software engineering should entail. The first point was that a software system is best designed when testing is interwoven with the design process rather than treated as a separate, post-design activity. At the time, this was quite reasonable by today’s standards.

00:04:09.439 The second point related to the idea that a simulation matching the requirements contains the controls that organize the design of the system. The exact terminology has changed since then, but when you combine that with the third point, it becomes clearer. The third point posited that through successive iterations of interlacing testing and design, the model ultimately becomes the software system itself.

00:04:51.519 This in effect means that testing and the replacement of simulations with more detailed modules will continue as the model directs the sequence and timing in which these tasks are executed. He was effectively discussing concepts similar to unit testing, mocking, and iterative design and development, which we now recognize as core practices.

00:05:27.440 A year later, there was a second NATO software engineering conference, but the tone of the proceedings from that conference was entirely different. During the year following the first conference, everything went wrong, and software engineering devolved into a field dominated by academics who designed processes for practitioners.

00:05:41.239 What went wrong during that year? I don’t know. I was alive at the time, but I wasn't a computer programmer and wasn't involved in these processes. However, I have a theory about what happened, and I can illustrate it through another example in history where reasonable statements were misconstrued, resulting in unreasonable conclusions.

00:06:00.040 An example of this is the introduction of the waterfall process. The waterfall model was introduced by Winston Royce in a 1970 paper titled 'Managing the Development of Large Software Systems.' Royce spent the rest of his career constantly insisting that he had been misquoted and that he never advocated for the waterfall model.

00:06:21.399 When I read this paper, I was astonished to find that he was indeed telling the truth; he didn’t recommend the waterfall model at all. However, I realized that the problem is rooted in the paper’s poor information design. It presents the opposite impression of what Royce was trying to convey, and if you were an engineering manager struggling with software project management in 1970, you might misinterpret his points.

00:06:47.520 If a manager received this paper on their desk from a colleague, they might skim it and look primarily at the diagrams. Initially, they see a simplified waterfall diagram that shows a linear process starting with analysis and moving through coding, testing, and operations. This is misleading, as the caption states that it describes implementation steps for developing a large software program, creating a sense of understanding that it is a clear-cut process.

00:07:09.120 But upon reading the next line, which states that the model invites failure, the reader encounters confusion. The very next page requires the reader to flip back and forth to comprehend Royce’s ideas regarding what a proper implementation plan should contain.

00:07:34.759 It becomes evident that if you take a defined process model as gospel, you can easily fall into a belief that this exact, simplistic model ties into success. Instead, the feedback loops and the realities that must influence earlier steps often make things appear messy, confusing, and unmanageable.

00:08:02.479 The end result was the waterfall model quickly became a standard in the industry, and it's easy to critique waterfall now since it’s recognized as ineffective by many. Still, I encountered groups that adhere religiously to the waterfall methodology and consider any alternative anathema even as recently as three or four years ago.

00:08:33.440 The Department of Defense mandated its use for all projects they funded well into the late 90s. It took a long time for the recognition of this idea, presented as a solution, to work its way out of the industry’s consciousness. This phenomenon occurs because people tend to simplify complex issues; they hear buzzwords, associate those terms with their understanding, and proceed without critically analyzing their context.

00:09:03.920 This pattern also happens with the term software engineering. H.L. Mencken famously said, 'For every complex problem, there is a solution that is simple, neat, and wrong.' This certainly applies to our field. Even as people began to understand the weaknesses of the waterfall model and sought alternatives, software engineering was still plagued by a bias toward the defined process model.

00:09:48.640 This model, championed by some chemical engineering researchers, advocates for a total understanding of every piece of work so processes can be completed consistently and yield predictable results. However, this defined process model disproportionately affects software engineering literature and practice. An example can be found in the work of David Parnas, a prominent figure in the software engineering field, who detailed a rational design process in a 1986 paper.

00:10:27.640 He describes an ideal process where one establishes and documents requirements before designing, documenting, coding, and maintaining software. Yet he later admits in the same paper that the idea of a software designer deriving a rational design from requirements is unrealistic—no system has ever been developed this way and likely never will.

00:10:55.080 This viewpoint, however, failed to prevent him from clinging to the defined process model and labeling the actual, non-ideal systems as irrational. He persisted in this notion, which he himself called the rational design process, and that's a telling choice of terminology.

00:11:26.080 Furthermore, this particular bias towards defined processes led to significant misunderstandings in software engineering practices. This misunderstanding is illustrated in the famous cost of change curve in Barry Boehm's 'Software Engineering Economics.' This was well understood long before, but Boehm gathered real data on actual software projects in 1981 to emphasize the notorious truth that the cost of finding and fixing errors climbs steadily as you proceed through the project life cycle.

00:11:52.679 Initially, it's relatively inexpensive, becoming increasingly horrendously expensive at the later stages. Boehm's solution to this challenge was to push all changes forward in the process. The intention was to reduce costs and identify errors as early in the process as possible, thereby eliminating change at later stages.

00:12:19.800 However, such practices often come with secondary effects. Processes designed to identify and eliminate errors early led to ballooned costs across the project, causing budgetary issues that organizations often resisted. It's a bit like trying to step on a tube of toothpaste—pressing down on one end pushes material out the other side.

00:12:43.639 While we now recognize there was an unseen bias in Boehm’s original data—namely that all the projects he surveyed were waterfall projects—we failed to realize at the time that he was essentially measuring the cost of feedback loop length. Before we understood that, it was normal to regard his findings as accurate for any software project type.

00:13:13.760 The belief that all projects were conducted in a waterfall manner skewed our understanding of effective practices. It turns out that when measuring projects that utilize iterative processes—where teams validate and begin continuously delivering working software every few weeks—the cost dynamics appear less harsh due to reduced time between making and resolving mistakes.

00:13:39.560 Another major way software engineering strayed was through excessive focus on modeling, or what some call 'math envy.' Many assert that software development needs to grow up and adopt a framework similar to traditional engineering disciplines. Yet even in some traditional engineering fields, notable figures like David Parnas have stated that the main job of an engineer is to write documentation.

00:14:07.239 Now, let’s consider some of the documentation types that have been promoted in the software engineering field over the years. Some notable examples include the Zed specification language and tabular mathematical expressions proposed by Parnas; while they may sound precise, they end up being challenging to understand.

00:14:36.079 A key question arises: is it any easier to derive correct code from such mathematical documentation than it would be to develop from standard programming code? In practice, many of these ambitious proposal documents have failed to remain relevant and are now largely obsolete.

00:14:55.119 Bruce Eckel, a prominent writer in programming circles, noted in a blog post last year that software engineering is sometimes not as clear as we'd like to believe. He implied engineering isn’t purely a matter of throwing things into a system and waiting for the design to manifest, which is a common misconception in software development.

00:15:31.040 The first thing I want to address is the core nature of engineering: it is inherently a creative activity. Engineering entails designing, building, and creating new things. Engineers often encounter blind alleys or missteps, making adjustments as discoveries unfold throughout the process.

00:15:54.279 A key point to recognize is that different engineering disciplines operate with unique materials, forces, and scales. They also represent varying degrees of reliance on mathematical modeling versus experimentation.

00:16:14.920 In addition, different engineering disciplines vary significantly in their reliance on empirical processes versus defined ones. Structural and chemical engineering tend to lean more toward empirical designs, while software engineering has historically favored predefined models.

00:16:37.600 Interestingly, the definition of the empirical process model offers control through frequent inspections and an adaptive approach to undefined processes yielding unpredictable outputs. Software engineers' emphasis on defined models has misled many in the field—this bias has led to misunderstandings, confusion, and ultimately affected outcomes.

00:17:01.960 Take, for example, Boehm’s cost of change graph, which made clear that all projects he studied utilized the waterfall model. It was paramount to recognize that this also colored perceptions of effective practices while failing to capture the true nature of iterative designs where feedback loops are shorter.

00:17:38.760 Additionally, the emphasis on mathematical models can lead to the faulty conception that they are the only path to success in engineering. In reality, mathematical modeling was introduced as a cost-cutting mechanism in traditional engineering fields, and their use has only been able to increase savings in practice.

00:18:09.760 Calvin and Hobbes humorously encapsulated the oddity of how load limits on structures are understood with a cartoon showing an exaggerated premise. The idea that engineers only build bridges until they collapse to gather data is, of course, farcical. This would carry enormous costs if done literally.

00:18:47.360 With proper models and mathematical understanding, engineers are able to assess capabilities without incurring significant expenses. For instance, a famous example is Boeing's testing of a 777 wing, where full-scale prototypes suffered extreme stress at defined limits and validated the work of engineers against real-world expectations.

00:19:24.120 In the end, all engineering takes intricate testing, modeling, and prototyping—all of which contribute to achieving successful outcomes. Models remain useful approximations, but they are no substitute for critical testing against real-world demands.

00:19:48.320 Even the best engineering teams still perform rigorous testing to validate their models, whether testing prototypes or experimenting in wind tunnels. This illustrates that models are approximations rather than definitive declarations about reality.

00:20:14.640 The best engineering definition encompasses both science and art. Structural engineers emphasize the importance of designing structures that resist forces with creativity and elegance to meet safety standards.

00:20:37.239 Based on this understanding, I propose a definition of software engineering: software engineering is the science and art of designing and making systems which are economical and elegant, enabling easy adaptations as needed.

00:20:56.120 Software engineering is inherently different from other areas of engineering; it's influenced by the complexity of systems and the simplicity required in physical interactions. While we seek to model structures and systems, what we encounter in software must adapt quickly to new demands.

00:21:33.760 Our understanding of effective software engineering ties back to the initial challenges around software engineering that resulted in deviations from traditional methods. Each engineering project will invariably navigate complexities and variations that other disciplines might not encounter.

00:22:02.480 From early software engineering dialogues to the ongoing challenges we face today, we must always seek the empirical process model that resembles true engineering practices. In doing so, we must prioritize adapting to feedback and understanding principles rooted in creativity and discipline.

00:22:25.320 Software engineering, ultimately, must be seen through a lens that is less about process and more about practitioners—those who are harnessing ideas, iterating through solutions, and learning as we progress.

00:22:50.640 I appreciate all of you for your attention and hope this session ignites thoughts and motivates further conversations about what we can do to understand how to innovate within software engineering. Thank you!

LoneStarRuby Conf 2010