Talks
How Programs Learn, and What Happens After They're Built

Summarized using AI

How Programs Learn, and What Happens After They're Built

Phil Crissman • November 13, 2023 • San Diego, CA

In this RubyConf 2023 talk titled "How Programs Learn, and What Happens After They're Built," Phil Crissman draws parallels between building architecture and software development, drawing inspiration from Stewart Brand's book "How Buildings Learn" and the influential paper "The Big Ball of Mud" by Brian Foote and Joseph Yoder.

Key Points Discussed:

- Introduction to the Speaker: Phil Crissman, a Microsoft software engineer with a passion for Ruby and functional programming, shares his journey in software development and his long-standing interest in agile methodologies.

- Concept of MTTA (Mean Time to Apologize): Phil introduces his personal metric for assessing codebases—how quickly he would need to apologize for the quality of the code when explaining it to others.

- Architectural Metaphors in Software Development: The talk highlights how software architecture can be compared to building architecture, including concepts like time layering and piecemeal growth.

- Stewart Brand's Influence: Brand’s observations on building adaptability, shearing layers, and low-road versus high-road architecture offer insights into how software systems can be designed to evolve effectively over time.

- The Big Ball of Mud: Phil discusses common pitfalls in software development, including the tendency to create poorly designed systems due to various pressures (time, cost, experience). The concept emphasizes how software often becomes an unwieldy mass over time.

- Design Philosophy: The importance of designing software with change in mind is underscored, advocating for modularity and adaptability in architecture.

- Communication and Management: Phil emphasizes the need for effective communication and process management to mitigate the creation of big balls of mud in software, suggesting that poor communication can severely impact effectiveness beyond technical challenges.

- Lessons from Architecture: The talk recalls insights from architectural practices, such as scenario planning and building with change in mind, which can lead to more sustainable software solutions.

- Change and Prediction in Development: Phil concludes that, like buildings, software is often built on predictions that may not hold true, and emphasizes the continuous need to be adaptable and open to change.

Conclusion and Takeaways: The presence of "big balls of mud" in software systems is seen as an inevitable outcome of various interacting forces, yet proactive communication and mindful design practices can help mitigate these issues. Crissman advocates for a mindset that embraces change and highlights the importance of learning from past experiences to improve future development cycles.

How Programs Learn, and What Happens After They're Built
Phil Crissman • November 13, 2023 • San Diego, CA

In 1994, Stewart Brand published a book called "How Buildings Learn, and What Happens After They're Built". As well as a fascinating account of architecture and the history of various buildings and building styles, some ideas from this book were inspirational to the famous "Big Ball of Mud" paper, by Brian Foote and Joseph Yoder.

Can we learn anything about building software systems from the observations in Brand's book and the Big Ball of Mud paper? Spoiler alert: yes, I think so! Let's talk about them.

RubyConf 2023

00:00:17.920 Okay, welcome everyone. My name is Shaina, and I'm part of the RubyConf programming planning committee. Today, we're delighted to have a senior Microsoft software engineer from Seattle joining us.
00:00:24.000 Our speaker, a Ruby and functional programming enthusiast, has been in the programming world since 2008. He is passionate about learning new languages and exploring software engineering processes and organization. Let's warmly welcome Phil Crissman to the stage!
00:01:05.280 I made a lot of new Rails apps from the ground up; it was a great experience. In about 2013, I moved to Seattle and took a short contract at Amazon, working on a couple of Seattle-based startups. Then in 2015, I started at Microsoft. For anyone who’s too young to know, that was a real Microsoft logo at some point in the '80s—the most metal of all Microsoft logos.
00:01:22.000 Yes, and the part of Microsoft that I joined used to be a startup called Yammer. Yammer was built on top of Ruby on Rails and had a bunch of Java microservices, and they decided to keep that tech stack because it would have been crazy to rewrite it.
00:01:38.520 This talk is called 'How Programs Learn, and What Happens After They’re Built.' I got a little clicker, maybe I’ll try that. It’s not about machine learning, oh that goes backwards. Okay, I know how to do this.
00:01:58.920 Sorry, so let’s see. I’ll start with a little preliminary stuff. There’s a book called 'How Buildings Learn' that inspired the title of this talk. A lot of the content comes from a paper from the late '90s called 'The Big Ball of Mud'.
00:02:15.680 So that’s where some other material for the talk comes from. Then I’ll try and think about: given all this, now what? Oh yeah, there’s our source material.
00:02:29.000 Clicking is good. Who’s ever seen this acronym? What does that mean? Meantime to failure? Yes, we get that from manufacturing, and I think kind of DevOps and computing have adopted it a little bit. A little while ago, when I saw these, I thought about a metric I like to use for my own codebases; that is MTTA.
00:02:58.640 It started as a thought experiment. I imagined, okay, if we have a new person at work or maybe there’s someone visiting someone I really respect, and it’s my job to walk them through our codebase, how long will that walkthrough take before I find myself apologizing for something? An apology in my head is very broad—it could be anything from 'we’ve really been meaning to refactor this' all the way to 'I’m sorry you have to see this' and 'I’m sorry it exists.'
00:03:25.160 I realized that in a lot of codebases I’ve worked on, my MTTA would be effectively zero because I’d preemptively apologize for what I was about to show them. But I’m also Canadian, so that might have something to do with it.
00:03:41.200 When I started, I also had the good fortune to learn about agile software development processes, because even the Rails book at the time was about agile web development with Rails. I loved the idea; it made a lot of sense to me immediately. This is sort of how I imagined our software would grow.
00:03:59.680 It would grow incrementally and iteratively; parts of it would be different, but it would all fit together wonderfully. It was all going to be beautiful. However, how I really felt about the software usually felt a little more like this. I got gradually to work on larger systems, and I felt like this was the world that Kbac wants me to live in.
00:04:30.160 But again, the reality often felt a lot more like this. You think of a new feature or a new thing you need to do, and you can draw it; it’s pretty and small and simple. Chat GPT drew this, so some of the arrows don’t actually connect, but let's imagine it makes sense.
00:05:08.120 But again, the reality always felt a little bit short. Within a couple of weeks of doing the work, I would take a look at it and wonder how it even works. Do the tests pass? One thing I really liked about that first eight or so years I got to spend contracting is that I was the guy typing 'rails new' a lot.
00:05:44.720 I got to witness that exact moment when the code goes from something that’s small and beautiful and fits in your head to when it doesn’t anymore and someone will come up to you and ask you how this works and what it does. Well, I don’t think anyone really knows.
00:06:23.160 What is knowledge anyway? How do we know anything? Software architecture, as a metaphor for building software, has a long history in the software development and software engineering community. It’s by no means the only metaphor for building software, and it’s possibly not even the best one. But because of the source material I’m using, we often use the word architecture and architect for roles and for features of what we build.
00:06:54.640 Other metaphors exist; some people think of software as more organic, growing like a garden. There are other ways to think of it, but for better or worse, today we're going to think about it as it relates to buildings.
00:07:34.000 This is the book, 'How Buildings Learn' by Stewart Brand. You can see right on the cover kind of where he’s going with it. That’s a photograph of two buildings in New Orleans, about 30-40 years apart. You can see they’ve both grown, and they’ve both changed, but they haven’t changed in the same way.
00:08:05.680 That’s kind of an ongoing theme of this book. Stewart Brand is mostly known for the Whole Earth Catalog. He’s written a lot of books over the years. He always kind of had one foot in the Berkeley '60s counterculture and the other in technology and the development of new things.
00:08:30.000 The themes in his book deal with adaptations—how buildings do or don’t adapt easily. There’s the concept of time layering; different layers of architecture change at different rates. Ideally, your structure remains static for a long time, but what is inside that structure can change at different rates and with different levels of difficulty.
00:09:03.440 Almost no buildings adapt well; they’re designed not to adapt. This is a building known as the 'Salt Tower' in England. It was originally a water tower, but at some point, when the water tower was no longer needed, someone decided to convert it into a house.
00:09:20.000 It seems to have worked but at the cost of creating an unconventional house. Brand uses the term 'shearing layers,' which will come up again. He makes this statement because of the different rates of change among a building’s components: a building is always tearing itself apart.
00:09:47.760 The diagram is great because it uses the thickness of the line to show how hard it is to change something, while the arrows and the number of arrows indicate how frequently you might need or want to change that thing. Some parts of a building change very slowly, while other things change frequently.
00:10:06.640 Fortunately, most of those things that change frequently are also easier to change, like the stuff inside your building. Brand got this idea of layers from architect Frank Duffy, who said, 'the unit of analysis isn't the building; it's the use of the building through time.' Time is the essence of the real design problem.
00:10:30.000 There’s what we’re building today and what we’ll need in the future. Brand identifies a couple of different categories of architecture, not in the sense of Georgian, Victorian, modern, but in his concept of low-road architecture.
00:11:00.160 Low-road architecture refers to what he thinks of as office parks and industrial warehouse spaces; it’s cheaper to build and easier to change. This contrasts with high-road architecture, which is meticulously designed, very beautiful, but difficult and expensive to change.
00:11:25.000 Building 20 at MIT was considered a temporary structure, built in the '40s for radar research in World War II. It was briefly designed in one afternoon and built in six months. Yet it lasted for 55 years. The exposed pipes and wires made it easier for students and faculty to modify the building.
00:11:58.000 They considered it the best experimental building ever built because they could cut holes in the floor or walls without concern. The Tech Model Railroad Club at MIT, a precursor to early MIT computer hacker culture, was housed there because it was a place where they could build their things.
00:12:30.160 This library in Boston, known as the Boston Athenaeum, has been around since at least 1915. In the book, Brand includes photos showing one of the reading rooms, where tables and chairs remain unchanged over a span of 40 to 50 years.
00:12:59.040 It turns out that if you know exactly what you want and are never going to change your mind, big design upfront is fine. But the issue is, we rarely have the luxury of knowing that.
00:13:30.000 You can imagine if you had to repurpose this space, how would you even approach it? It is built to be a library and would be difficult to repurpose.
00:13:58.120 Here’s another building at MIT, designed by renowned architect I.M. Pei. Brand mentions that there were problems with planning between the faculty and the architect. Initially, the researchers were too vague, and eventually, they became too specific, wanting rooms designed exactly around their current research without thinking of future use.
00:14:40.000 As a result, there are many unusable spaces in the building that are not conducive to being occupied by people. The concept of piecemeal growth is another thing examined by Brand, which is also applicable to software.
00:15:24.000 Structures often grow without a plan, much like how a final vision of a software project might not align with its original conception. The burden of past choices often weighs us down. Brand notes that buildings are a prediction and that predictions are always wrong.
00:16:00.320 Interestingly, authors like Mark Richards and Neil Ford have made similar statements about software architecture: that it is also a prediction, and we are often wrong.
00:16:24.000 As Brand explores solutions to the issues he discusses, he asserts that buildings can be designed so that it doesn’t matter when they’re wrong, which piques my interest as a software engineer.
00:16:55.120 One idea is scenario planning, wherein various potential future needs are considered in the design process. This helps designers create buildings that can accommodate different needs.
00:17:23.360 The architectural profession often employs procedures to understand what the client needs, what they want, and what they can afford. This brings to mind the poignant rule: whatever you think will happen, won’t; whatever you're ready for won't happen, and whatever you're unprepared for, will.
00:18:00.120 There are also useful illustrations of scenario planning used by a film company, which looked at growth potential versus market stability. They aimed to avoid the worst-case scenario of a large structure in a volatile market.
00:18:37.680 Another concept he discusses is building with change in mind. You don't have to hide your services inside walls; instead, if you want to change services or pipes, it should be straightforward.
00:19:16.840 This reminds me of software: the term 'connaissance'—meaning if one element is connected to another, a change in one necessitates a change in the other. We want to minimize this.
00:19:59.000 Another influential figure in architecture is Christopher Alexander, who suggested buildings should be designed to last 300 years. This idea makes me think about how I design software.
00:20:26.440 In the late '90s, connected to these patterns, there were conferences called 'Pattern Languages of Programs.' In 1997, Brian Foote and Joseph Yoder submitted a paper there called 'The Big Ball of Mud,' which many of you may have heard of.
00:21:06.680 Initially, I thought the term was just a humorous reference to poorly designed software. However, they were serious about their point, highlighting that we have many smart engineers who care about design, yet we often see a 'big ball of mud.'
00:21:43.920 Why does this happen? They identify several forces at play in creating big balls of mud: time, cost, experience, skill, visibility, complexity, change, and scale.
00:22:16.560 Time and cost are probably the most familiar and significant forces impacting software development. Deadlines, whether self-imposed or set externally, often lead to sacrifices in design.
00:22:45.680 Cost may relate to time, but it can also simply mean that the time to design and build an ideal version exceeds the budget, leading to compromises.
00:22:57.920 They also discuss experience and skill, which encompass not only inexperience affecting code quality but also varying levels of skill among team members. Even a seasoned developer may face challenges in a new domain without prior experience.
00:23:26.600 This raises questions about whether we learn from mistakes or if it’s necessary to make mistakes to learn important lessons. If all of us have to learn the same lessons repeatedly, it would hamper the overall development of our industry.
00:23:58.720 In response, what we hope for is that the mistakes we make tomorrow are different from those we've made yesterday. Visibility refers to the ugly code that often goes unnoticed by senior leadership, product managers, and users; they only see that the code works.
00:24:27.440 Complexity has been widely discussed in software development, as we prefer code that fits well within our understanding. Complexity is necessary, but confusion is undesirable.
00:24:50.480 There’s a real comment in a codebase I worked on, and while we all believe in the necessity of good comments, by the time I read this one, it was 11 years old, and the person who wrote it was no longer available.
00:25:29.960 Change in scale arises when we find ourselves needing to scale and are unprepared or trying to scale prematurely. The agile process encourages us to embrace change and adapt accordingly.
00:25:57.600 Brian Foote, one of the authors of the paper, tells a story about passing a house under renovation. He realized that building work on solid foundations is ultimately simpler than many of the complex changes we make in software.
00:26:32.520 In the paper, they identify some patterns or anti-patterns associated with the big ball of mud. Throwaway code, familiar to many, is a quick fix that often becomes critical to infrastructure.
00:27:01.920 Likewise, piecemeal growth refers to the incremental, ad hoc manner in which software evolves, while hotfixes, which address urgent concerns, rarely lead to necessary refactorings.
00:27:35.520 Another concept they mention is 'shearing layers,' where layers of software change at different rates, suggesting that components changing at similar rates should be kept together.
00:28:08.760 Sweeping it under the rug is another problematic approach—when software is too fragile or complex, it can lead us to avoid necessary changes entirely, resulting in an even scarier situation.
00:28:48.040 The last pattern they mention is 'giving up and ripping it up to start again.' Paul Buchheit, known for leading the Gmail project, shared his realizations about what 'good enough' truly means, conveying how sometimes this mentality can lead us down a problematic path.
00:29:29.760 This reconnects to Richard Gabriel, who contrasts two software development ideologies: the 'right thing,' emphasizing correctness, and 'worse is better,' emphasizing simplicity over other attributes.
00:30:05.040 Overall, there's a tension between aiming for the best quality and the need to innovate quickly. Kent Beck's advice to make software work, make it right, and then make it fast creates a scenario where we often prioritize short-term needs over necessary improvements.
00:30:41.000 The question becomes whether big balls of mud are inevitable. While it may be necessary to create some disposable products to maintain progress, we should not let that philosophy become accepted.
00:31:22.520 To reduce the impact of these forces, we must hold conversations, perhaps by establishing book clubs that discuss these topics. If we neglect these discussions, we risk losing sight of challenges and progress.
00:31:45.720 There's an effectiveness formula noted in a book called 'Improving Software Development Productivity,' which suggests that effective communication and strong management are critical. Even with high skill levels, poor communication or management can drag down overall effectiveness.
00:32:27.760 Communication can be improved more readily than technical or management skills. As a speaker at this conference, I've had multiple opportunities to converse about these topics.
00:33:09.040 Rich Hickey once mentioned that all he wanted from his talk was for attendees to go back to their teams and discuss the ideas.
00:33:48.920 Pushing back can take the form of mentoring, ensuring more experienced members support the inexperienced, and making the ugliness in our code visible to others—whether through metrics or other means.
00:34:56.199 Scenario buffered strategies prompt discussions about the various possibilities, though they can become complex without clear, collective understanding.
00:35:28.999 Understanding our tools helps us design code that remains flexible under changes, much like ships have maintenance logs for ongoing conditions.
00:36:12.000 Chesterton’s fence illustrates the necessity of understanding the rationale behind existing systems before deciding to change or remove them.
00:36:52.000 When we write software, we often oscillate between viewing it as either temporary—ready to be disposed of—or as permanent. This perception often leads us to mismanage our progression with software.
00:37:43.200 Brand’s observations about buildings and their adaptability invoke relevant parallels to software systems. While his solutions focus on buildings, they resonate with many aspects of software design.
00:38:26.000 Ultimately, there’s no one-size-fits-all approach. The key takeaway is that accommodating change and understanding growth become pivotal as we navigate the landscape of software engineering.
00:39:09.600 Thank you for listening, everyone!
Explore all talks recorded at RubyConf 2023
+38