00:00:10.670
Good morning! Today, we are going to be talking about the metaphors we use for software architecture, how all of them are broken, and why set design is my favorite of these broken metaphors.
00:00:15.870
Before I became a developer, I worked in theater. I primarily designed and painted scenery, but I had a little bit of everything as long as it was backstage.
00:00:29.130
Now, usually when I talk about cross-applying my theater skills to tech, I tell about this one thing that kept happening. Let's say, completely hypothetically, that we're doing a play where we need to fly an actress in — maybe "Heaven" by George Walker.
00:00:41.250
Now, this is the character in Heaven, Judy, who is literally a black belt commando avenging angel. In one scene, she descends from the sky, by which I mean the catwalks, where she just enacts some heavenly wrath.
00:01:00.210
Let's say, again hypothetically, that the harness keeping her in the air in that first photo kind of looks like a cross between a diaper and a swingset, and that the entire production staff figures this out when it arrives in the mail a week before opening.
00:01:20.340
Now we have opposing needs. On one side, our director, hypothetically named Kenyatta, champions his artistic need to make Judy look awesome. On the other side, our lead rigger, Mike, champions his safety need to not kill Judy's actress, Michelle.
00:01:35.100
Perhaps this dilemma reminds you of something closer to home. If we allow ourselves to be reductive, we could frame the dilemma this way: we could call the director's objection to the swingset-diaper thing him risking Michelle's life unnecessarily. Then, as technicians, we would obviously be right, and we could win the argument, and then reviewers would pan the show for looking ugly.
00:01:51.570
Now, don't get me wrong — this is a much better solution than an actress dying, which would, in fact, be catastrophic. But we can also aim a little bit higher than just not catastrophic, can't we? Let's look at the situation a different way.
00:02:16.020
Maybe we can have our costume designer work some trickery so the swingset diaper gets to disguise it as Judy's camo outfit. Maybe we can keep a bright spotlight on her torso so that the audience's eyes are drawn away from the harness.
00:02:35.100
By approaching the issue in the spirit of collaborative problem-solving and negotiating to meet everyone's needs, we can have our cake and eat it too. We can have something that looks okay — a bit less cool than the director had hoped, but still pretty cool — and we could do it in a way that doesn't risk safety at all.
00:02:58.200
I use this experience a lot as an engineer. Then there's another big theater skill that I apply as a programmer, although honestly, this one took me a bit longer to relearn. It's a skill that theater people call 'telling the story,' and programmers call 'focus on business value.'
00:03:11.790
In 2006, I saw the Forum Theatre production of Caryl Churchill's "The Striker," a somewhat dark play about ecological decay, postpartum depression, and the creepier parts of British folklore. One character is literally called 'Raw Head and Bloody Bones.' In the script, there are no spaces between those words; I just added them for clarity.
00:03:35.250
Now, the theater where this show was held has been torn down; gentrification happens. I couldn't find photos of it on the internet, and it wouldn't have mattered much if I had. The set was not that much to look at — gray floor, gray columns, some debris scattered about.
00:04:01.829
It didn't matter; there were things chittering in the rafters, and there were creepily dressed actors crawling towards you. Then blackout — so crawling towards you in the dark. I have seen a lot of plays by that director since. Kathleen Ackerley is the best director in DC.
00:04:10.650
But I will never sit in the front row of one of her shows again; nope, cannot make me. And that is how you know it worked. For "The Striker" to work as a play, the audience needs to believe that its two human characters, Josie and Lily, are both very haunted and suffering from postpartum depression.
00:04:37.710
If they aren't convinced that these are human experiences, then the second bit — the depression — gets lost, and the climax of the finale just does not make sense. So it's up to the rest of the production to convince us that fairies are real, so that Josie and Lily's actresses can focus on being humans.
00:05:05.430
The set for "The Striker" wasn't impressive objectively, but it helped us tell the story. It gave the fairy actors lots of opportunities to appear creepily from high places, and it kept the action way too close to the stage for my personal comfort.
00:05:37.800
In short, it made it easy to believe the things that Josie and Lily were seeing were real. This is a parable about the beauty of our code: what matters about our code is what it does.
00:05:49.099
We talk a lot about business value, and sometimes, honestly, it just seems like a buzzword to me. It also doesn't really capture what a lot of nonprofits or open source projects are doing.
00:06:03.750
So whenever I hear the phrase 'business value,' I mentally substitute 'usefulness.' Beautiful, bulletproof, well-tested code is often more useful than other kinds, but it is rarely the most important thing about a project.
00:06:15.120
Thus, the set in 'The Striker' wasn't impressive on its face, but the play still gave me some nightmares. Similarly, I have written some pretty ugly code in the service of helping diabetics track their blood sugar better.
00:06:39.240
I know which part of that sentence I care about most. Now, these are good general lessons, but I want to talk about software architectures specifically, not only about small line-by-line design decisions.
00:06:44.999
Software architecture is controversial, and I think that's because of the metaphor we use for it. We visualize it as a building and think: 'Okay, the architect draws up the plans, then the builders execute.' The architect has the creativity and a white-collar job, while the builders have no creativity, and you can tell because they work with their hands.
00:07:04.559
The entire thing is this very hierarchical, classist piece of Great Man theory nonsense. It caught on in the software industry because software has historically been quite susceptible to hierarchical, classist Great Man theory nonsense.
00:07:27.000
Take the 10x Developer meme. As many of you know, it comes from a book written in the 70s called 'The Mythical Man-Month.' It was by a guy named Frederick Brooks. In this book, Brooks misreads a narrowly targeted scientific paper to make a broad claim that some developers are innately 10 times as good as the rest of us.
00:07:54.409
It gets worse. This is lesser-known today, but Brooks goes on to recommend that engineering managers organize a surgical team around each of their 10x developers.
00:08:07.079
Brooks does not mean 'surgical' here in the sense of having a strictly defined purpose or executing very precisely; he means surgical in the sense of calling calipers to a nurse.
00:08:12.809
That idea is what has transformed into how we think of software architects today. That's where we get this misguided notion that an architect should specify everything out and that a good architect will design something perfect for the entire life of a project.
00:08:19.199
That they will know, based on a similarly perfect list of business requirements, all of the classes that a project will need, and the correct database, and every microservice, and every single detail of the API with which these microservices will communicate.
00:08:38.819
That nonsense doesn't even reflect how buildings work in the real world; it does not reflect what actual building architects do. I come from DC, a city that is slowly turning each and every one of its warehouses and some of its theaters into condo complexes with hipster bars on the first floor.
00:08:53.059
Building architects are mostly not building beautiful, timeless art; they're taking a past architect's creative work and the ways it was embellished by its builders and the ways that were changed by the people who lived in it, and they are trying to turn it into something new — something new that meets the needs people have now as opposed to the needs they had back then.
00:09:21.839
This sounds a lot like working in legacy code to me. Not glamorous, but useful.
00:09:33.509
We've been talking a lot — a depressingly amount, really — about the impermanence of software this week. Building architects and urban planners often have jobs where they are doing very little but repurposing the past.
00:09:50.579
That's not quite what we're doing in software; often, we at least have the illusion of starting fresh. What if we want to aspire to more than just turning today's legacy code into tomorrow's legacy code?
00:10:04.060
When I got bumped into software architecture, I had no idea what I was doing or what I was even supposed to be doing. Mostly, I knew what I didn't want: I didn't want to specify wildly unrealistic waterfall nonsense that my implementing teams would then have to just ignore.
00:10:20.310
I didn't want my codebase to stratify into archaeological layers, each representing a different Rails fad and each completely unreadable once that had passed.
00:10:31.529
I didn’t want to be any of the architects the building metaphor told me to be; I needed new metaphors. I ended up thinking a lot about set design, particularly this one show.
00:10:44.040
My set design mentor, a woman named Elizabeth Jenkins McFadden, once designed a production of "The Merry Widow" for the Washington Savvy Arts. Watching what she did with it taught me a crucial lesson about set design: it's about the ways that actors move through space.
00:11:07.560
The Merry Widow is a comic opera set in the high society of a tiny made-up European country at the beginning of the 20th century. It's a culture of formal etiquette that still dictates that people be announced when entering a room.
00:11:35.160
So, the back of the stage was occupied by a long, high promenade for folks to enter on, with exactly one staircase for them to go down. You could see them walking for a long while before they would make it to the staircase and join the rest of the actors downstage.
00:11:54.840
Except in a few scenes, in which people would kind of sneak in from the wings in front of the promenade. Funnily enough, they were always the characters who were cheating on their spouses or plotting something.
00:12:12.420
The set design created a visual language that indicated this character is being a sneaky, scheming jerk right now. Of course, that visual language would not have meant anything without the director and actors using it.
00:12:43.160
Okay, you might be thinking, 'That's interesting, but what does it have to do with code?' Let's look at a good old model-view-controller architecture. This is a set of boxes to put code in.
00:13:10.440
We know going into a new MVC codebase that most, if not all, of the code in the models will have to do with the domain concepts and data that our application deals with, but most of the code in the views will have to do with displaying data to users.
00:13:37.830
And most of the code in the controllers will translate incoming user requests into things that the models and views need to do. This contextualizes our code.
00:13:58.500
If I put these three lines of code into a person model, I am saying that a person's full name is an important part of how our codebase understands a person. If I put these other three lines of code, which are practically identical to those first three lines, into a user's index, I am saying that a full name is not intrinsic to a person.
00:14:17.180
I am saying it is only a formatting detail that matters when we've got a whole table of users to display. That's a low-level design choice, but it's one that only exists because of the architecture.
00:14:33.520
In 'The Merry Widow,' there wouldn't have been a visual distinction between people who announce themselves like regular individuals and those who snuck in like jerks if the grand promenade didn't exist.
00:14:45.059
Just as importantly, without actors to walk on it, the grand promenade would not be a way of separating sneaky people from upstanding ones; it would just be a very large and attractively painted balcony with a staircase hanging off it.
00:15:04.620
A model-view-controller architecture doesn't mean anything until engineers start putting code into each of these boxes. The point of architecture is to give other engineers meaningful choices so that those choices can silently annotate all the implementation code they write.
00:15:28.770
So maybe architecture is about giving people a lot of boxes, since putting something in a model means something different than putting the same thing in a view. Maybe it'll also mean something different if we put it in a form object or a service object.
00:15:52.920
You can add a lot of letters to MVC that way. But I did not like that definition of architecture; I had a really strong gut reaction to it.
00:16:02.900
In part, I have this reflexive assumption that if an architecture has twelve patterns in it, there are going to be two that are very alike. Maybe the architecture has a pattern called presenters for display, and the code that applies to all instances where a domain concept might be displayed.
00:16:31.740
Another pattern called view objects could govern all the code that underlies a given view template. Then I need to think about splitting that hair all the time when I'm trying to code!
00:16:56.860
I need to think about that very fine distinction a lot, and I potentially need to defend that decision in code reviews. And frankly, if there's anyone on my team who has low-level issues with female engineers, that person is not going to be able to resist being super passive-aggressive about whatever decision I made there.
00:17:19.610
All of a sudden, a lot of energy from everyone is being poured into this one decision, and I think: 'Is this decision generating business value? Is this decision making my code more useful by making it easier to read? Is it making my code enough more useful to be worth the energy me and my teammates are spending on it?'
00:17:41.470
Sara Mae has this awesome metaphor called 'livable code.' Like a lot of architecture metaphors, she asks us to visualize a building, but this metaphor works a lot better than the others because she asks us to visualize a specific building — namely, our own house.
00:17:53.240
If we let clutter build up everywhere, then it's hard to use our house as a house! If the kitchen table is covered with stuff or if you can't get to the kitchen table because you've just moved, and the coffee table is strewn with books and unsorted mail all the time, we have to eat on the couch — stuff like that.
00:18:19.810
It goes further. In Olivier Lakhan's talk on errors on Wednesday, he talked about the intimidation factor of scary errors. There is always something in any given codebase that is as scary as the closet of doom in your house that you do not open.
00:18:40.500
People similarly avoid this area; they change the code they write. They alter the paths they take through your house to never have to deal with the unclear metaprogramming stack trace that vomits out at them if they open that closet.
00:19:06.420
But the other half of livable code is that you can get it clean if you have a really complicated organizational system or a super minimalist interior. If you have something that can't abide a stack of books you happen to be reading somewhere, you end up spending more time catering to your house than living in it.
00:19:20.710
I think there's even more to it than that. I think part of the problem with many box architectures is that they're ultimately just like architectures where the architect names every single class in the application up front.
00:19:39.830
To talk about that problem, I'm going to share another story from my theatre days — the story of one of my more embarrassing mistakes: "Stuart Little," a kids' play based on the book of the same name by E.B. White about a human family in New York that adopts a mouse.
00:19:57.380
It's also the first show I ever did a full design for instead of just designing bits and pieces. I really wanted to impress my director, so I tried to incorporate all those ideas about distorted perspective and scale, playfulness, and historical context that we had discussed in the first design meeting.
00:20:12.930
It wasn’t just this; I mean, these were pretty drawings, but as a theatrical design, it had too many elements. There was this really cool idea I had about putting doorframes in these moving skyscrapers, but there just wasn't time to build it.
00:20:41.520
And that one really cool idea was buried under all of the other things that were going on in the design. My director, Michael Bobbitt, was really nice about it, but it was clear that I'd need to change a lot of stuff.
00:21:05.490
We started talking about what I'd been trying to do with my design, so we could figure out how to cut things and stay true to the core of it. Michael said something that has stuck with me for a really long time: 'Betsy, you’re trying to tell the entire story with your set.'
00:21:39.120
'You do not need to do that; you have to leave something for the actors to do.' It wasn’t just that I didn’t need to do everything with my set; it was that I was trying to do everything, and as I said, I ended up succeeding at nothing.
00:21:57.740
A good set didn't try to do everything; it would only try to do enough to support the actors as they told the story. It would trust the actors to do a good job with that and focus on its job instead.
00:22:16.940
So I took that feedback and pulled back a lot. We wound up building a set that had the things I cared most about from my original design, like skyscrapers so much taller than the kids in our audience.
00:22:36.680
But they would think that sure, maybe my scene kids could be about the same size. It turned out pretty good, especially given our budget.
00:23:03.700
If they do, I am not leaving anything for the implementing developers to do, and if that happens, I am insulting them. I am saying I don't trust them; I am saying I don't respect their intuitions, their analytical abilities, their creativity, their domain knowledge, or their subject matter expertise.
00:23:28.040
There are three million and five really hard and crucial skills that go into making an effective software engineer. When I specify everything, I am NOT enabling meaningful choices because I am NOT enabling choices at all.
00:23:44.900
When I give people too many boxes, I am NOT leaving them room for judgment calls. I am not leaving them meaningful choices because none of the choices mean anything.
00:24:09.360
I am doing all of the horrible 10x developer surgical team nonsense I was decrying earlier. So maybe we should just have no architecture and no architect and let the team figure everything out organically.
00:24:33.060
One of my favorite shows to work on didn't even have a set — just a chair. It was so minimalist; the characters did not even have names; they were just called 'this one' and 'that one.' The actors were both named Jason!
00:24:48.120
It was called 'In On It,' and I ran the soundboard. I swear to God, I cried every single night! There was no set to constrain where the actors went, and it felt like the plain rack of theater.
00:25:04.370
But they kept on returning to the same three positions anyway. Sure, there were points where they were elsewhere, but that made me cry.
00:25:20.060
If they were sitting against the back wall of the theater, they kept on returning to these same three positions: one where their relationship was falling apart and they were yelling at each other about petty things like taste in music or dishes in a sink.
00:25:39.700
The second where they were facing the audience together, playing baseball or dancing or otherwise happy, and the third where they were having very emotional discussions about the spoiler. It turns out that structure, or design, doesn't really meet the purpose.
00:26:09.300
If it's that design or an architect that isn't providing that structure, then something else needs to. The reason architects are useful is not because we have special skills or knowledge.
00:26:22.990
The reason architects are useful is that our primary responsibility is to the macro structure, and this frees engineers up to think about feature details.
00:26:49.650
Designating an architect is not the only way to protect that macro structure and keep everyone pointed in the same direction. If you’re careful and intentional and take your time, team consensus can get there too.
00:27:03.060
It’s just easier to fool yourself then because you’re focused on both the macro and the micro at once. It’s hard to tell the difference between them.
00:27:34.550
Whether your team designates an architect or relies on consensus, maintaining the architecture is a collective responsibility. Each of our low-level meaningful design choices is contextualized by the architecture.
00:27:56.440
But each of them can change it, too. If you put enough of your business logic in your controllers, then guess what? Service objects are not your business logic box; your controllers are.
00:28:14.330
I don't care what your architect or your diagram says — people's understanding of what goes where is the real architecture, and it is much harder to change than the documentation or the code.
00:28:32.340
And again, this means that all of this is our responsibility, whether we call ourselves architects or not. It’s heavy stuff because we've gone over all the ways to get it wrong.
00:28:56.600
If too much architecture is stifling and too little is confusing, how do we Goldilocks our way out? Maybe we're still looking at this the wrong way.
00:29:12.420
Maybe the idea that there is a correct set of three or six or twelve patterns to pick for any given codebase, given the hierarchy of classes to define a given anything, is the thing that doesn't work.
00:29:26.800
Maybe we need to look at the process rather than the product. We're looking at the structures in our heads after all, not the structures in our code.
00:30:06.560
It has to be pretty waterfall that way since the set design helps the director define the visual language they use to position the actors. It kind of has to be ready when the actors are.
00:30:25.520
But it's not all waterfall. There are countervailing things throughout the rehearsal process. The stage manager sends out rehearsal reports every night.
00:30:51.260
These summarize any information that the director and actors learned about the show’s technical requirements during that rehearsal.
00:31:03.260
If they realize that the payphone needed to be upstage left in order to make this transition work, then they can let the set designer know that, yep, that's where it needs to be.
00:31:27.700
And the set designer and director have an opportunity to have a conversation about that. The final set of rehearsals goes through the technical run, with all of the designers there along with the actors and the orchestra.
00:32:04.100
If you've got one, it's an opportunity for the designers to observe how elements work in the actual context of the actual play. Sometimes, there are lots of opportunities for the set designer to change things.
00:32:19.670
There have been times when I have gone up onstage during a pause, moved some scenery around, and called out to the director, 'Hey, I think this looks better — how do you feel?'
00:32:30.870
But the set is often already built, so sometimes there are as many — not always already built, let’s be real — there are always some little tweaks to be made.
00:32:43.000
Though sometimes all you can do is adjust a paint color to look better under light — but you can always at least do that.
00:32:54.800
The reason the traditional building metaphor for architecture persists is that it's about waterfall and hierarchy.
00:33:00.400
It's about a 10x person directing a team of individual cogs to follow a plan exactly. The way to escape it is, therefore, to be agile — to constantly talk to one another.
00:33:18.270
To have implementing developers talk back to their architect or talk to each other, saying, 'Hey, this isn't working; how can we make it work better?'
00:33:39.410
To hypothetically tell your architect for two weeks that the service would really be more stable — and luckily, this totally hypothetical architect did wise up and listen.
00:33:58.300
There seems to be a lot of anxiety about the future of Ruby lately. It seems like we're saying 'this isn't working' a lot and wondering if features from other languages, like static typing and immutability, would lock down Ruby's weirder edges and make Ruby work better.
00:34:18.800
The reason that this is not a tangent is the same as why I think that this is totally the wrong direction. Let's take monkey-patching.
00:34:38.000
I have definitely looked at monkey patches a lot and said, 'This isn't working,' but more and more, I still find they symbolize the things I love most about Ruby.
00:34:56.670
So this framework class doesn't do what you want it to. Well, fine — just open it back up, override that method, and carry on. Do you want to rewrite random number generation to enable really cool mystical effects like Sonic Pi does? Go ahead.
00:35:21.970
Do you want to have your own special file for a special version of file details? Well, let’s see what happens after Jettagodme fides standard libraries hit this Christmas!
00:35:38.800
Bundle maintainer doesn't really think this is fine, but at least you can write it. Monkey-patching can create more problems than it solves, but it’s also a promise that no matter what the problem is, there will be some way to fix it.
00:35:55.400
Matt made some architectural decisions when he built Ruby, but he had the humility to recognize that his decisions wouldn't work for everyone, and so he also gave us the power to change them.
00:36:12.500
Sometimes our decisions are going to be stupid, but you can’t empower people to do cool, creative, mold-breaking stuff without also empowering them to make mistakes.
00:36:26.600
And because Ruby lets us make our own mistakes and build our own things, it empowers us all to participate in architecture in these macro-level decisions, not just line-by-line ones.
00:36:48.300
It lets everyone participate; it let everyone share the responsibility. That’s the positive way of looking at monkey-patching.
00:37:07.070
I think that we get to make meaningful architecture choices all the time, whether we designate an architect or not. We're all collectively responsible for the architecture of our systems.
00:37:27.420
An architecture, whether you're a capital-A architect or not, needs to be about empowering people. It needs to be about giving them the tools to build their thing, to focus on what they find useful, to tell their story.
00:37:46.710
So, yeah, I’m Betsy. I just founded a software consultancy called Cohere. If you like my perspective on architecture and would like me or my co-founders to apply it to your codebase, this can happen.
00:38:05.680
You can also subscribe to our newsletter at TinyLetter.com/cohere. I forgot to put this on the slide, but if you want more of my feelings on architecture, then I am guest-starring in a course that Obvi Grimm is putting out starting in early December.
00:38:21.410
You can find me personally on the internet at @Betsymuffin on Twitter.
00:38:36.360
Since I assume you are all going to follow me right now, I’ll give you a fun fact.
00:38:45.420
The bubble-wrapped couch in my cover photo is from my favorite of my set designs for Priscilla. In this photo, Priscilla explains to a boy she is dating just why her couch is covered in bubble wrap.
00:39:06.180
Although I’m afraid that won’t illuminate much for you, you see the aliens who believe she will save the world want to make her apartment as low-risk a place as possible.
00:39:32.200
One last note: as an experienced software developer, I make more in an hour than I did most days as a theater professional.
00:39:52.300
I have made more in a day than I made in an entire ten-week theater internship. Sure, we're not all rich, but on average, we do okay.
00:40:13.480
The recession of 2008 never really left America, and internet-based entertainment is cheap nowadays, so people aren’t going out and spending twenty-five bucks on a theater ticket as often.
00:40:41.490
It is killing the industry. A lot of theater professionals aren't doing so hot right now, including crucially some of the people I mentioned as mentors in this talk.
00:41:07.710
Including — to go back to legacy — some of the people whose legacy I am so thankful for. If you have that twenty-five bucks and you want to spend it on theater per ticket, know that you will be doing at least three things.
00:41:27.490
You will be paying someone's grocery bill for a week. You'll be providing jobs near downtown that aren’t hipster bars, and you will be turning off Twitter for at least ninety straight minutes.
00:41:59.050
I can promise you these three things, and you may even enjoy the show.
00:42:16.140
Alright, now I’m actually done.
