00:00:05.440
You look just like they said. My boys will be immensely pleased by that introduction. Thanks.
00:00:10.639
Am I on the screen? In one of the many pearls of wisdom in Gerald Weinberg's great book, 'Secrets of Consulting,' is one that he calls Ronda's First Revelation.
00:00:19.400
Ronda's First Revelation says, 'It may look like a crisis, but it's only the ending of an illusion.'
00:00:26.240
The field of software engineering originated in the middle of what was called the software crisis in the 1960s. As software projects got bigger, more complicated, and more ambitious by the standards of the time, they also became more difficult to control, manage, and predict.
00:00:38.680
They started failing in various ways, and this became common. So, NATO of all things called people from industry and academia, practitioners and theorists, a whole big group of people to Germany for the first software engineering conference in 1968.
00:00:53.120
It actually got off to a kind of promising start, but then for various reasons that I won't go into in detail, it went wrong. The result was that we labored under traditional software engineering ideas for about 30 years.
00:01:10.600
That's a little unfair because the software engineering folks did realize some of the limitations of the waterfall model, and they tried other methods. However, in my mind, all of the traditional software engineering processes suffer from the same fundamental limitations that waterfall did.
00:01:35.840
Things were pretty unpleasant in large parts of the software industry for about 30 years until the late '90s when the Agile movement came along. Practitioners and programmers began taking ownership of the processes and methods they used to deliver software.
00:02:00.200
Things got a lot better, but the failure of the traditional software engineering ideas has given rise to another crisis. A couple of years ago, Tom DeMarco, who in some ways was one of the fathers of software engineering, published an article where he suggested that the idea of programming as an engineering discipline was a failed idea that should be abandoned.
00:02:17.519
Closer to home, my colleague Bruce Williams tweeted, 'I am not an engineer, and neither are you. I doubt you're an architect either.' Both crises—the original software crisis and the crisis of identity we are witnessing now—are driven by the same illusion: a thorough misunderstanding of engineering and what it is.
00:02:37.400
This was my response to Bruce. It was a little more curt than maybe he deserved, but you know, I was busy. Let's talk about engineering for a bit and what it is. When you discuss engineering with people who lack an engineering training and background, they tend to think of engineering in terms of modeling, mathematical modeling, formal analysis, and a heavy emphasis on documentation.
00:03:07.360
The goal of this rigorous approach is to ensure that the end product is safe, reliable, and predictable in terms of budget, time, and quality. Additionally, people often think about engineering as related to civil engineering, particularly large-scale structural engineering like bridges and highways. This perspective is natural because those structures are tangible and easy to understand.
00:03:44.599
If we take that engineering checklist and apply it to civil engineering, we find that modeling is indeed a significant part of the process. Mathematical modeling, formal analysis, and extensive documentation are vital elements. We trust the safety of bridges and civil engineering projects; however, failures do occur every few years despite these precautions.
00:04:29.360
While these projects are largely safe, it's not guaranteed, and predictability can be just as illusory. Anyone involved in local or state government sees this firsthand, often observing significant cost and time overruns. Sometimes the predictable parts of civil engineering come down to construction processes rather than the engineering itself, which adds to the illusion that the engineering processes are truly predictable.
00:05:37.120
It’s important to note that there are many kinds of engineering—civil, structural, aerospace, mechanical, among others. These fields differ in terms of scale, complexity, and levels of abstraction, but as you go down the list from large-scale concrete works to smaller, more abstract kinds of engineering, the rules change substantially. In fields like chemical, industrial, and electrical engineering, costs are less often dominated by materials and labor and more by design processes.
00:06:13.320
For instance, when we look at industrial engineering, which focuses on designing and optimizing processes (often factory processes), the approach differs. Industrial designers collaborate with mechanical engineers to create solutions for factory machinery. A prime example would be the automatic donut machine in a Krispy Kreme shop. Its design was not predicated on mathematical modeling and formal analysis; it's likely the result of iterative tinkering—trying different configurations until finding one that worked effectively.
00:07:55.919
This iterative approach is common in engineering, and it's often about balancing against documentation and mathematical rigor. Furthermore, as you work toward smaller and more abstract types of engineering, the propensity for prototyping increases, indicating a shift away from classical engineering models which are often criticized for being overly rigorous. We have an idealized view of engineering as a very sophisticated and planned endeavor; however, it can also simply consist of common sense and iterative processes.
00:08:57.920
The aerospace engineer Eugene Ferguson tells the story of his first engineering invention as a boy when he observed a Douglas DC-3 landing in 1937. He noted how tires that weren't rotating yet made contact with the runway, creating smoke and possibly harming the integrity of the tires under stress. This inspired him to sketch a design for a device to pre-rotate tires before landing, which he later submitted to the Douglas Aircraft Corporation.
00:09:41.120
The response he received illustrated how often technical problems have multiple solutions. The corporation informed him that this issue, known as pre-rotation, had already been recognized, and while there were multiple proposals, they ultimately rejected those solutions that added excessive machinery and instead used the weight saved to improve tire tread thickness. Ferguson realized that every technical problem often has alternative solutions; the key is knowing the constraints and the impact of those solutions.
00:10:41.680
Fast forward to today, many in the engineering community still debate whether software engineering is a true discipline. My response to Bruce, who expressed a change of heart after our discussions, led him to ask about architecture. My reluctance shows a prevailing attitude towards the debate of whether software architecture counts as real architecture. Most misunderstandings stem from a lack of clarity on what engineers do and the true nature of engineering itself.
00:12:22.560
We tend to idealize engineering, viewing it through the lens of documentation and mathematical modeling. However, many important aspects of engineering go unrecognized in traditional discussions. Books like 'What Engineers Know and How They Know It' by Walter Vincenti and 'Engineering in the Mind's Eye' by Eugene Ferguson are essential for those wanting to understand the broader scope of engineering qualities and practices.
00:13:07.720
Programming fits into engineering on that continuum toward abstraction. However, it differs from other engineering disciplines in two critical ways. First, the conceptual model for software engineering, as derived from traditional engineering, assumes that engineers produce designs handed off to laborers who build the finished product, a notion that doesn't hold true in the realm of software.
00:14:16.440
Software engineers often find themselves creating ongoing modules and systems without a clear handoff process or distinct division of labor. This was critiqued by Jack Reeves in 1992 when he argued that software engineering processes needed to depart from traditional design documents. He suggested that the source code itself acts as the primary design, pointing out that while source code is crucial, it does not represent the finished artifact or what customers truly pay for.
00:15:40.800
He encouraged viewing source code not only as a working artifact but as an evolving design, with the key point being that running software expresses what customers want, fundamentally changing the economic model of software engineering. Unlike traditional engineering, where material and labor are significant costs, software programming often requires iteration, allowing engineers to avoid over-engineering.
00:16:38.360
The implication of this is important: the proper approach to software engineering isn't to front-load analysis and documentation but rather to be iterative, developing multiple prototypes that can be tested rapidly. This shifts the responsibility for engineering's accuracy away from rigid structures toward flexible, guided experimentation.
00:17:55.360
When considering whether there is a model in programming, source code serves as both the model and verification document. Mathematical principles exist in programming languages, although not always as formal specifications. The nature of the programming disciplines means we exploit both the documentation and execution of our code, yielding an advantage not readily available in traditional engineering realms.
00:19:11.799
There’s a distinct upside-down economics in software, where traditionally expensive approaches become cheaper, redefining our understanding of documentation. Engineering is more than structured documentation; it allows for the execution of specifications, providing a dual layer of verification compared to classical methods.
00:20:01.240
Additionally, many successful engineering disciplines embrace processes without questioning their fundamental validity. Electrical engineers never ponder if their field is real engineering; they simply embrace their practices. I propose we adapt the same attitude for software programming—it is time we stop questioning our place and embrace our practices.
00:20:35.600
In the current landscape, we face a dilemma: would we prefer to think of ourselves primarily as engineers, or would we rather recognize the craft within our work? Both engineering and craftsmanship play distinct roles in the software development process, prompting the need to consider how craft influences engineering outcomes.
00:21:07.520
The distinction between craftsmanship and engineering often appears when discussing documentation. An artisan may produce documents to help conceptualize their work but ultimately builds based on intuition. Engineers, however, must generate documents to convey their designs to builders because they are typically separate from the actual construction.
00:22:12.179
To understand this better, consider civil and structural engineering where documentation mainly serves as formal communication with builders. These engineering models may seem arbitrary but are crucial for conveying complex structures, while software's abstraction allows flexibility and allows it to feel more intimate versus separated.
00:23:08.360
As we explore engineering documentation from structural through to software engineering, we note that while early documentation resembles the tangible product being constructed, as we arrive at software engineering, the models appear completely abstract. However, programming languages are meticulously designed to maintain isomorphism with the constructs that they execute on the computer, rendering our code the ultimate blueprint of our final product.
00:24:49.760
This relationship implies that as software engineers, we experience our craft in unique ways that intertwine the roles of designer and builder; the act of coding applies direct pressure on the abstractions and constructs we create. Software engineering is the most abstract of engineering disciplines, yet our models have concrete relationships with our actions, enabling us to explore our work directly without a separation that might exist in traditional methods.
00:26:28.040
As engineering disciplines evolve, especially with the integration of technology, traditional models become less mathematical and more simulation-based, highlighting the need for engineers to adapt. In electrical engineering, circuit simulations offer efficiencies that parallel the advantages we’ve adopted in coding, where rapid prototyping becomes not only viable but often necessary to lead innovation.
00:27:58.720
In conclusion, the models we work with in software engineering provide a direct sense of working with the artifacts we create. These constructs possess an isomorphic relationship with what gets executed, becoming evident in our interactions with coding. This dual relationship, where we design and build simultaneously, fosters deeper engagement with our creations and emphasizes that as software professionals, we must embrace both the aspects of engineering and craftsmanship.
00:29:49.799
We are designers and makers, deeply engaged in our creations. Understanding this balance brings luxury and responsibility to our role, ensuring we never lose sight of the importance of structure, quality, and the long-term impact of our digital creations. Thank you very much.
