Keynote: Growing Old

by Chad Fowler

In the keynote titled 'Growing Old' by Chad Fowler at RubyConf 2017, the speaker reflects on the aging of the Ruby programming language and software systems in general, using the notion of legacy as a central theme. Fowler starts by reminiscing about his contributions to the Ruby community and his experiences with legacy systems in software development. He underscores a prevalent sentiment regarding Ruby's libraries and features feeling outdated, emphasizing that while the language has aged, it remains relevant, influenced by demands for modern programming features akin to those found in TypeScript.

Fowler illustrates the concept of legacy through various anecdotes, contrasting software with arts and music, showcasing figures like Beethoven and John Coltrane who left behind celebrated legacies. He posits that the negative perception of 'legacy' in the software world overlooks its potential to signify something of enduring value.

He uses the Standish Group's data to underscore the software industry's struggles with project success rates, where many systems fail or become obsolete shortly after birth. To counteract these trends, Fowler discusses methods to create sustaining software systems, advocating for a biological metaphor of regeneration, where small and simple components replace larger, monolithic systems, akin to cells in a living organism.

Key points include:
- The necessity of creating systems with longevity and stability, as seen in successful legacy systems from the UNIX world.
- The encouragement of small, manageable components within software development that can be easily updated or replaced to ensure longevity.
- The concept of homeostasis from biology, where various subsystems balance each other out to maintain stability.
- The importance of reducing code complexity and focusing on the value of the overall system rather than the code itself.

Fowler concludes with reflections on the adaptability of systems, resting on the importance of having clean interfaces and the virtue of allowing systems to fail and recover, rather than striving for a perfect state of operation. His call to action is for developers to learn to create thoughtful code that contributes to a legacy of stability and innovation in the software domain.

00:00:10.590 Come on, so Rich Kilmer, who is one of the co-founders of Ruby Central along with David Allen Black and Dave Thomas, isn't here. I just wanted to include that for completeness. I wanted to show you this site real quick; this is LifeIsTooShortForMeetings.com.

00:00:28.630 Just one second, I made that site as a reminder that we're getting old. I haven't been here in like six years. I think by 'here' I mean RubyConf, because RubyConf is just a location that floats around through space.

00:00:37.030 It's true; I’ve never keynoted at RubyConf before. In fact, the only talk I ever gave at RubyConf was in 2005, in San Diego. Does anybody remember that? Matt's, Rich, Jeremy, Eric? It was a POSH E, a continuations tutorial. Jim Wyrick has since left us because I haven’t been here for a while. This is the first time I've come here and Jim isn't here.

00:01:00.670 So, I have this lazy way of doing talks. Matt even retweeted this for me. Coming back to the Ruby community after a while, I just wanted to see what people think, even outside the Ruby community, that we need to be aware of.

00:01:12.070 I'm not going to put the answers up because it’s not really worth it, but there were a lot of answers that were useful. I'm not saying the answers weren't good, but what I observed was a lot of stuff that sort of verified that Ruby is getting old. It's things like some of the libraries are getting old and decrepit, they don't work very well, or the documentation is out of sync with the code. Things are slow; it's not all about Rails.

00:02:00.569 That was a pretty constant theme, followed by a whole bunch of feature requests for the language that we've been hearing since 1999. I mean, a lot of them. It used to be the case that when we started with Ruby, English speakers on the Ruby Langley list would see pretty much every new person that joined request the same set of features to make it more like Java. Now, I think they all want it to be like TypeScript.

00:02:39.959 I wrote this blog series back in 2006 after what I thought was a pretty full career already of being what I like to call a systems euthanizer. You know, euthanasia for those who speak English as a second language refers to killing something for mercy because the thing needs to die.

00:03:40.470 From system to system, I have been part of projects to painfully kill those systems. They're sick, they're slow, they're legacy systems, they're dying, they're unhealthy, and so we put them out of their misery. In the process of doing it, we usually create new systems. Of course, you don’t just kill it and leave a dead carcass behind. I realized that because I work so hard, I put so much of myself into this work as I'm killing these systems.

00:04:09.500 I realized I’m also crushing the last remnants of the hopes and dreams of the developers that created the systems that I've killed. So, it's a pretty miserable job, really, both the job of a euthanizer and the job of someone who creates systems because they tend to die so frequently.

00:04:40.070 If you don’t know this, it's sort of a non sequitur, but you'll see it kind of matters: I came into the software world originally as a musician. This is me doing a recent jazz show in Little Rock, and it changed the way I thought about how software development should be done and how I assumed it would be done.

00:05:06.030 There was one major thing that stood out, and it is this word. I said it a minute ago when I said legacy systems. How many of you thought, "Oh yes, legacy systems… I love legacy systems?" Thank you, one person. We've created this negative feeling around this word.

00:05:40.890 But I am a musician. I still think of myself as a musician who just happens to be programming because it pays better. As a musician, I came in with a different assumption of what legacy meant. So, when people would say legacy, I would think of it as something that someone leaves behind, like an inheritance, a bequest, a gift from a past generation. A legacy is a really nice thing when you think about it from that perspective.

00:06:06.360 But in our world, we think of it as a negative thing, a nasty old messy pile of junk left behind by someone else whose name we usually curse in our everyday work. It's kind of strange. You know, as a musician, right now we’re listening to the second movement of Beethoven's seventh symphony. This is a picture of Beethoven. Beethoven left behind such a legacy of music that even we Americans can’t pronounce his name properly.

00:07:12.680 I swear, this piece, the seventh symphony—everyone knows the ninth, everyone knows the fifth, but this is the best one. Listen to it! I'm a saxophone player, and this is John Coltrane. How many of you have heard of John Coltrane? Most people have! Every saxophone player obsessively tries to imitate John Coltrane, even today. Even fifteen-year-old kids in New York City are playing, and you can hear Coltrane in their sound. Jazz is irrelevant at this point, and still, they're doing it because he left behind this legacy.

00:07:57.050 The same is true in the world of literature. One of my heroes, Kurt Vonnegut, for example, left behind words that I have read, even after he died, that have shaped the way I think. Probably the way that I’m speaking to you now is heavily influenced by having read Kurt Vonnegut’s work. Architecture is the same thing. This is Gaudi—walk through Barcelona, and you see his legacy and his imprint on the city.

00:08:50.000 Even the world of fashion—this is a piece by the Japanese group Comme des Garçons, which is my favorite fashion house. I’m not sure what your favorite is, but mine is this one, not just because I want to impress my friends, but because they create these pieces that will theoretically live forever if they could be preserved but also change the course of fashion.

00:09:31.220 The same thing is true with art. This is Joan Miró. I like Joan Miró so much that I have basically matching tattoos on my arm. Joan Miró left behind a massive body of work that we can all still enjoy, still be influenced by, and he also pioneered the idea of visual poetry. He created a visual language of symbols; his paintings in certain periods of his career were poems.

00:10:16.960 Now, we have a problem in the software world. Like I said, we use this word "legacy" negatively. We also have a really hard time shipping software. You’re not going to be able to read this, but you'll get the gist of it. This is some questionable data from a group called the Standish Group.

00:10:31.960 I say it's questionable meaning you might challenge the scientific merits of it. However, you will probably agree with the directionality of the data. They survey projects across the world and categorize them into successes and failures.

00:10:55.660 What we have here is the green row at the bottom or the bottommost part, which represents successful projects; the top, the red, are failed projects that were simply scrapped, and in the middle are challenged projects. For challenged projects, they consider them significantly over time or over budget, which to me is also a failure.

00:11:23.979 So, look at how bad we are at making software! And this is just for creating software in the first place. Once we create it, I made up a statistic but I think it’s right. I travel around the world and kill these things, and it seems like they only live about five years in business systems, especially. Unfortunately, software, when it is initially birthed, like most things that are creative, is not immediately perfect.

00:12:07.180 So, if you create something, it may live for five years on average, but it may take longer than that to even have it be good.

00:12:13.490 This is a bad situation—software might only live five years, and it takes ten years for it to even get good. It often takes that long to iterate it and actually meet the user's needs and perfect the User Experience (UX) and make it perform. So, we have a problem.

00:12:31.130 So, how do you create a legacy software? How do we turn this around? Well, this has become my quest. Some of you, very few I would guess, have seen me do a talk like this before. This one has different slides and different content, but I've been talking about this for the last seven years or so.

00:12:40.310 Much of my work has been really influential, like with Mike Feathers. He wrote this book; if you work with legacy code, which you all deal with in the bad definition of it, you all work in code that already exists that needs to be modified—you should read this book.

00:12:59.660 You might have to hold your nose and read through the C++ and Java stuff, but it's worth it. So, I asked Mike—I actually got on a video chat with him a few years ago and said, "How do we create legacy code?" I was incredibly disappointed by his answer. He said that people need to be afraid of it. It needs to be static, basically, and very difficult to change, so I don’t think he understood the question.

00:13:58.700 So, I turned to Twitter, as I like to do, and did this again recently. I got very similar answers: what are the characteristics that make the software we still use, which is old, still in use? I also got the same negative vibe because people think of old software as being bad—fear of awful, sunk cost fallacy.

00:14:46.040 Old systems still in use means managers, and that was one of my favorites, but I got some good ones too: it works, it's stable, it’s valuable, and it provides value to the users, etc. So, I was sort of encouraged by that, and I got a lot of things that got me thinking.

00:15:37.090 Back to around the time that I wrote that big rewrite article series, which was sort of relevant timing-wise. There was this post by DHH on the Basecamp blog, the Signal vs. Noise blog, for some 37signals. It was joking that 'Enterprise is going to be the new legacy.' He really ran a propaganda campaign against the word 'Enterprise.' If any of you were doing Rails or watching back then, we went from 'enterprise' being a good word that was often used for negative purposes to something that you would chuckle about.

00:16:44.420 But the sad thing about this is David's way of panning the word 'Enterprise' and making us all hate it was comparing it to legacy and saying it was going to be like legacy, which is obviously bad.

00:17:05.920 I found this comment to be really well-written, and I'll read it to you: "Careful, legacy isn't a bad word. Legacy usually means tried and true, and of enough value that it lasted long enough to be old and outdated. True legacy is to look at the successes of the past and declare that they aren't to be revered or respected. Most of what runs our economies is legacy, and in the future, I hope the software I'm creating now is highly regarded enough that it's still around and being referred to as legacy." I thought that was really inspiring.

00:18:21.100 Unfortunately, the software I was creating at that time was not highly regarded enough. In fact, it was one of those 'bread boxes' on the Standish Chaos report. There was a year of my life, and of a colleague's life, that I swear we still have PTSD over. And I mean that literally—it’s one of the only periods that I regret in my career.

00:18:43.160 It was dismal. I still look at it as like I died a year early in 2005. I can't get about how miserable it was before that. On a more positive note, toward the beginning of my career, I worked for General Electric. At GE Appliances, we had this system that was running on a Honeywell Bull mainframe. Who has a Honeywell Bull mainframe?

00:19:11.630 The funny thing about this is that I think it was 25 years old at the time. It had a custom TCP/IP stack; it had a custom RDBMS; we had created this RPC mechanism that we could use to talk to it. It was terrifying. The hardware was so old that you could not get replacement parts from the manufacturer because it wasn't manufactured anymore. So, we had a team of people who knew how to manufacture hardware replacement parts. Replacement parts were not available, but we could not get rid of it because what it did was both so valuable and performed so well that every attempt to replace it failed because the users would reject it.

00:19:50.120 There was also just too much information in the system. However, unexpectedly, I think when I came in, I thought, well this is going to be terrible. But I left that job feeling a great deal of respect for what the Honeywell team had created at GE. There was even a team called the Bull exit team that had existed longer than my software career at that point.

00:20:29.270 When I left, that team still existed, and eventually it was disbanded, but I heard recently that this thing is still running. It’s amazing and terrifying but amazing. So, I had that influence at that time. I think the system was my age, actually. I was 25 when I started there, and speaking of which, I'm 43 now, and I don’t deserve to be alive because I take really bad care of this system.

00:20:49.090 You know, refactoring and maintenance, and you may have seen me before—I’ve looked a lot better in the past. I also looked worse, but how do we create systems that survive? Well, the first step like the Standish thing: the system has to be born well. When Twitter first got launched, did you know that it was a database-backed web app written in Rails?

00:21:29.850 Everyone knew it at the time in our community, but a lot of people don’t realize it now. If you tell them that, they think, well, that’s ridiculous. A database-backed web app for Twitter? Because Twitter is a messaging system that’s this massive asynchronous thing with all these crazy optimizations.

00:22:03.620 Because you've got people like Justin Bieber with millions of fans, you've got Trump, and you've got all these different things requiring high volume to do this in a database. I mean, they really typed like Rails new Twitter or whatever it was, probably script/generate Rails Twitter.

00:23:01.010 But it’s not actually stupid. We used to see these failures all the time, and for a while, we would make fun of it. There was the 'fail whale' you would see. But you know, if they hadn't created it like this, we wouldn't have Twitter today because it never would have amounted to anything. They would have run out of money while trying to build the distributed messaging queuing system that Twitter needed to be.

00:23:40.610 Thank God they didn't realize that's what it needed to be! They just made this microblogging system in Rails. So, they got it to be born. Had they not, we wouldn’t care about the legacy of the software behind Twitter. It’s harder to create software birthing than we might believe.

00:24:32.050 It’s like when you get older; you can’t feel the keys like you used to, and you press the wrong ones. So, the conversation with Mike Feather wasn’t all doom and gloom. He gave me a link to a tiny URL link, I don’t know why, but it’s tiny URL code alive. It was an article he wrote back in 2006 where he said that your software is alive.

00:25:11.080 He starts thinking in terms of this biological metaphor for software, which is really interesting for someone who’s thinking heavily about how to create systems that can live a long time and live healthier lives. What he says here in the article is it all comes down to one thing: code survives by providing value and being difficult to replace.

00:25:58.480 So, the value has to be greater than the difficulty. This is an interesting point—the biological metaphor is what got me interested when I talked to Mike about this. Your code is alive.

00:26:29.470 He sent me looking at this paper by Dick Gabriel. If you’re not aware of Dick’s work, he is, I believe, the first person to have written about software in terms of patterns. He took Christopher Alexander's patterns ideas and applied them to software development. He wrote the Common Lisp Object System, he did a bunch of really influential, important stuff that not enough people know about.

00:27:00.340 He wrote this paper that you can find— I have the URL there; it says in the title: 'You Were Only Designed Beyond Human Abilities,' which is a pretty enticing title, anyway. Basically, he looks at make-believe systems to explore what you would do if some crazy parameters were present—like trillions of lines of code.

00:27:45.799 How would you maintain a system that lived like that, which is not even likely to ever happen unless it were general? But what really got me interested is that he ends up turning to this biological metaphor for systems and he explains that biological systems are much larger than anything coherent people have built—not just in software, but in general.

00:28:42.780 There are a bunch of interesting things that come out of this, but as we talk about complexity and systems, if you look at biology—the human body, for example—you can get some interesting work done. So, we’re biological, and like I said, I have no business even being alive at 43, given how I’ve treated my body.

00:29:29.890 Yet, your software systems can’t compete with me generally. That’s pretty sad; we should both be embarrassed. So, how do we create systems that outlast us, or at least me? This sent me down this path. First, I started thinking, how did the system stay alive like me when I'm not giving a lot of input to make them be better?

00:30:13.950 There's a concept in biology called homeostasis, which I honestly don't understand that well because I'm not a scientist. It's insane, but homeostasis is a balancing process in biological systems where different subsystems in the organism play different roles that might be at odds with one another. They balance each other out using an effect called negative feedback.

00:30:55.520 So something happens that’s not bad, and something else happens that's not good, and in a certain amount of something else happens to counteract it. You have like the brain, which manages the whole thing; the liver, which metabolizes toxic substances; and the kidneys, which deal with blood water levels, etc. It’s a system that just balances each other out, and all the right components are there to make it work.

00:31:57.380 In the same way that we talk about really good agile development systems as systems where all the components need to be present to balance each other out, or management systems with executives playing different functions that balance each other out—that’s the simplified kind of dumbed-down version. That’s what homeostasis is.

00:32:50.270 When you are a biological organism and you’re unable to maintain homeostasis, you reach a state called homeostatic imbalance, where these checks and balances aren’t happening, and that leads to…good news, though: in that case, you’re already dying. We’re all dying at an alarming rate—50 trillion cells in your body, 3 million die per second. So just look around at the death that’s happening around you—it’s truly disgusting.

00:33:49.810 It's like a zombie movie in here! But that’s interesting because here I am, I’m still me. You know, we all grow up hearing about this, and like skin cells falling off and regenerating, we say we’re not even the same organism we were, but somehow I’m still me.

00:34:28.010 You can recognize me fairly when I come through, even six years later. You still know it’s probably me, or it might be this guy named Chaz that goes to the RailsConf and looks exactly like me. It’s weird, have you seen him? But it turns out it’s not really true that all the cells in your body regenerate, but a lot of them do.

00:35:33.680 Maybe that’s a key. When Vanderburg responded to one of my many tweets about this subject, he said we learned that software should start small and grow—not attempting to replace an existing system that way. What he's saying is we already know that dealing with small pieces and growing them from there is a good idea, but it's very hard to take a whole system and replace it by starting small.

00:36:25.161 And that's why we run into problems. I also asked the question of a bunch of people many times: what are the oldest surviving systems that you use regularly? I’m trying to get a sense of like old systems people are still using, probably not because they’re forced to. You can see most of these systems are from the UNIX world, probably partially because that’s the best software.

00:37:11.310 You can categorize these things into two categories. They are either really small components, you know the UNIX philosophy of creating tools that do one thing and do them well, or they’re systems made up of a bunch of small components. So stuff like the X Window System or Apache or probably a better version of Apache would be something that's Internet-based.

00:38:06.500 You get the idea; we’ve been using systems like this for a long time. Here’s a row of them that are still driving down the street and are much older than software, and even older than me. They are systems; each one is a system that is a vehicle made of all these little components that can be replaced.

00:39:05.740 The slides are so irrelevant; they’re only distracting me. Then I started asking myself, what if we built software this way? What if we thought in terms of the biological metaphor, the idea of regeneration? What is a cell in this context? What is a cell in the context of the software system, and what is the system? What are the lines between those, and how do you know what to build and when?

00:39:44.140 Well, the cell concept to me is easy; a cell is just a small thing. It's a tiny component. If you’ve ever had to work on something and you go to a piece of code, and you have a task to complete and you look at it, and it’s tiny, you think, oh good! This is going to be easy. Right?

00:40:30.950 If you go look at something and it’s big, you think, now I have to read it and try to understand it. Who knows? You know it might not go well. You could watch this talk from RailsConf 2014 by Sandy. It’s right at the beginning; she says something so insightful, you could almost not have done the rest of the talk.

00:41:13.180 All of the problems we cause have the same simple solution: make smaller classes, make smaller methods, and let them know as little about each other as possible. Turns out this is a summary of my talk too, but you know probably we could all just get up on the mic, say that, and be done with almost every talk at the conference.

00:41:58.470 When I was working on Wunderlist, which was the last job I had before we were acquired by Microsoft, I made a rule—I was CTO and I would say you can write code in any language you want, any technology you want, and you don’t have to ask me. It doesn’t matter who you are; you could be an intern as long as the code was this big or smaller.

00:42:34.330 Then I would just hold up my fingers like this. There were a few other rules: like it had to work with our deployment system, and you had to get someone else you thought it would be a good idea to use that technology. You know, simple obvious ones, but you didn’t even have to ask me about it, though I would prefer to know about it later.

00:43:26.380 Even if you wrote a whole critical service in a programming language that probably only 70 or 80 percent of us are experts in in this room, if someone came along and was on call and there was a problem and the code was this big, they could probably read it well enough to understand what it did and rewrite it and deploy it—that was the idea.

00:44:09.960 And if they did that, I would like that. As it happens, I wrote with one of my coworkers who was one of the most junior people on the team; we wrote our one and only set of Haskell services. Some of the people on the team were angry that we wrote Haskell services because they thought that was like academic elitist crazy stuff they couldn’t understand.

00:44:56.390 Then one of the angry people came and rewrote it. Another person rewrote it in JavaScript, and another person rewrote it in Ruby. As CTO, I did not think that was a waste of time because it was this much code. They didn't want to deal with the Haskell, and I was fine with that. It turned out the Haskell performed so much better than all the other things, and we stayed with the Haskell.

00:45:49.450 What we didn't do, though, is we didn’t destroy the code regularly enough. So, you know, having small things is good, but if you want to really use this biological metaphor, you should be destroying those things. And like I said, I'm okay with it if you destroy something that’s big and you recreate it or replace it. They did that, but then we ended up sticking with the Haskell.

00:46:28.710 If you replace the cells regularly, it shows that the interfaces to whatever it is are clean enough that the rest of the system doesn’t have to be bothered with the fact that you could replace that component. It also proves to you that you can replace it, which I think is an important meta system around software development.

00:46:59.366 If you don’t have a meta system for knowing that you can replace code, then you probably cannot replace code. At Wunderlist, we built up this crazily seemingly over-engineered system. In fact, when they did due diligence on us for the acquisition, I got a glowing review—a 40-page report that used the words 'over-engineered' in a positive light to describe what we had done.

00:47:58.560 We had this crazy over-engineered system of cells that were regenerating. However, because the system ran so well, especially the Haskell one, we didn’t have to change it. You know, when you write Haskell code, if it compiles, it works—this is how Haskell works; there were never any bugs in the Haskell code because it compiled!

00:48:45.340 But a problem occurred where the dependencies changed, and the Cabal system changed out from under it, which is the package management system for Haskell. By the time we did the acquisition, we could no longer actually compile and deploy the code. So we had a critical problem with the system that never had issues; because the ecosystem around it changed, we learned we should be destroying and replacing things even when they're not broken.

00:49:35.200 When they don't need to be replaced, it's a healthy meta system for development. And along with this whole thing, as I started thinking about throwing away code in the big organization with a lot of old code, for example, Excel, I realized that developers have a fetish for code. They’re obsessed with code; they think code matters. But code doesn’t matter at all— the system is the thing that matters; that is the asset you create.

00:50:37.510 The code is a liability, and that sounds weird, but the more code you have, the worse your system is, in a sense. If you could take a system that does exactly the same thing and two different systems and one has 200% more code than the other, the one with more code is worse. You wouldn't want that.

00:51:05.520 Think about it this way: you need to be creating a system where the code is just part of it. It just plays a part. Back to Dick Gabriel's idea of designing beyond human abilities, he reached the conclusion that cells also need to be destroyed sometimes.

00:51:55.300 He talked about how an external cell can command the cell to destroy itself without releasing toxins and without destroying anything around it. This influenced how we deployed as well. I’ve been talking about code, but runtime is the same way. In the ‘90s at GE, we had a server that we set up.

00:52:31.450 I remember looking at the uptime, and it was like 600 days up time on this UNIX server. We kind of high-fived and said, great job! It didn’t reboot for a crash. But then we realized this was terrifying because we didn’t know what was on this, and there’s no way we could reproduce this.

00:53:07.520 So we coined this term, 'immutable infrastructure,' to describe it where we would do the same sort of thing at runtime in our backend systems. Whenever you want to change a piece of software on a server, you would have to throw away the server and replace it with a new one. In the ‘90s, that would have been hundreds of thousands of dollars every time you did it, so it wouldn’t work.

00:53:58.440 You know, six years ago, this started to make more sense with AWS and EC2. This is how everyone does it. And by everyone, I mean, you know, bleeding-edge nerds that are using Docker. Which doesn’t seem like leading edge to a lot of you, but it actually still is. So, never upgrade software on an existing node; always just throw it away, create a new one, and replace it.

00:54:43.250 Then always deploy. So, we had a rule, we were still using EC2 containers at the time, or what are they called—instances. If I had a graph of uptime on all the instances, like I used to be proud of my 600 days of uptime, now if I saw something that was more than a couple of hours, I would get worried: Why is that server not being recycled and destroyed?

00:55:05.820 Because the system is not healthy in that case. We don’t have a meta system for proving that we can replace things. My goal—and we got acquired, so everything got ruined, you know, in terms of my nerdy plans, but everything wasn’t ruined otherwise, but in terms of my nerdy desires—my goal is to have uptime be less than an hour.

00:56:01.650 I want to have everything just cycling through servers in an automated fashion. Speaking of this immutability, it brings me to the idea of immutability in code, which I think is one of the most important things that we can learn from biological systems. Immutability and impermanence in code—if we can think in terms of pure functions?

00:56:41.870 Think about it—an immutable pure function actually gives you an immutable disposable scope. You can’t create huge long pure functions. It ends up being harder than just creating small ones because you don’t have scope—you don’t have mutable state. So, pure functions, meaning functions that don’t have side effects, force you into a mindset.

00:57:29.150 You can do this in Ruby, but you have to enforce your own rules. But thinking in terms of your functions is a valuable way to force yourself to think in small pieces and create beautiful stuff. There’s this whole new wave of functions as a service.

00:58:08.740 Some of you have probably played with AWS Lambda or Azure Functions. Standard Live is a company that my VC firm has invested in, and we did it because they were trying to reinvent how you build things based on pure functions with no side effects and no state.

00:58:47.380 Anyway, what if we had a rule: never edit a method, always rewrite it instead? What if you had code folding on, and there was a need to enhance your code, and the only way to do it on your team is that you can’t unfold the code—you just have to make a new method with the same name and replace the functionality every time. This is a crazy idea!

00:59:39.410 But I like the mindset. Imagine how it would change what you would write when you created a method; it would be like Sandy was sitting there looking intently at your code. This would be a summary of these ideas: the mutability of a system is enhanced by the immutability of its components. I just sat there silently so you could think about that for a second.

01:00:31.320 Once you have all these little components, they need to communicate with each other. The best way to do this shows the interface—a stupid, inefficient, simple interface. Don’t worry about, you know, binary protocols unless that’s fun for you, of course.

01:01:06.890 You’ve got to do it once and do the stupidest thing you can. An example of that is the UNIX standard in and out of idea. UNIX utilities just read from standard input, they print to standard output. Most apps, and/or most tools in UNIX—that’s what they do. In our Bull mainframe, we had built this Bull RPC, which was a really dumb input-output thing using TCP.

01:01:39.750 This allowed us to create this extensible ecosystem around the Bull and our custom TCP/IP stack. This was my first Ruby experience at work, by the way. I was generating Java classes to talk to the Bull mainframe by getting the COBOL copybook off the Bull mainframe—that’s basically the schema, the data structures in COBOL.

01:02:27.210 I would parse them with Ruby that would generate Java code and I would commit that to CVS or whatever we were using at the time so that no one would know I was using Ruby. I didn’t have to write in Java in a sense.

01:03:06.370 These interfaces, if you zoom out, are the system. This is just one stupid simple concept, but if you think about the fact that code doesn’t matter that much because it’s so small, so simple, then the interfaces and the way the code components communicate becomes the system. That’s how much attention you should pay to this.

01:03:49.500 Even though I’m telling you to make it dumb, I mean that in a good way. Make it as simple as possible. Here’s a rule to keep in mind, in cursive. It’s probably too hard to read, but it says to be conservative in what you produce and what you do and liberal in what you accept. That’s a good way to deal with something that evolves.

01:04:27.940 Here’s something I did that’s maybe okay for a Ruby conference. We said, okay, we’re going to be heterogeneous by default. When I started there, I started talking about Scala a little bit because I’d been watching the stuff Twitter and LinkedIn had been doing.

01:05:13.300 People would really snicker in the same way that in the 90s people snickered at me when I talked about Ruby to PE. What I wanted to do was get everyone into a mindset where they weren’t dealing with these monolithic processes, monolithic memory spaces, or monolithic acceptance of languages and libraries.

01:06:03.300 All of these boundaries of creating small things that have to communicate through clean interfaces. It’s a lot harder to produce tight coupling between your Haskell code and your Ruby code than it is to not produce tight coupling, like you have to go out of your way to be Aaron Patterson with his PHP interpreter at Ruby, for example.

01:06:43.720 So we didn’t use all these languages or platforms or whatever they were, but by the time we were acquired, we had something like 12 or 13 back-end languages—Haskell, as I said, Rust, Go, Ruby, Closure, Scala, etc.—and I think that contributed significantly to creating the system we wanted to create.

01:07:36.890 This is Joe Armstrong. If you were at RubyConf 2006, you probably learned about this video. Assume that things are going to fail—failure in a system like this, where you want things destroyed, is a virtue both at runtime but also in your code, which leads you to the concept of 'mean time between failure' optimization versus 'mean time to resolution' optimization.

01:08:34.130 A lot of people spend a lot of time thinking about how can I never have a failure, which is probably impossible in most cases. Rather, if we spent all our time thinking about how to recover from failures and forcing them, we would be in a better state. One classic example of MTBF is testing.

01:09:16.580 This slide probably speaks for itself. Testing—hi, I’m surprised no one booed! I guess we’ve gotten past that point now; we’re too old to worry about it anymore. But there was a time at RubyConf when people would have been upset about this thing: 'tests are coupling!'