00:00:13.400
Let's keep it going for John Scheffler. I got it nailed in one! Before we start, I want to say I'm so happy to see all of you wonderful people here. How many of you are new? Wow, this is your first time! That is more than half, which is amazing. Can we have a round of applause for the new people?
00:00:21.570
And I also want to say, weren't all of the talks great this year? So good! I'll try not to screw that up now. And in the spirit of gratuitous pandering, I want to show this thematic picture of John in his construction earring onesie.
00:00:39.180
Now, to the matter at hand. In the course of my career, as it is, I have made a number of observations about software. Based on these observations, it is my conclusion that software fails a lot. Therefore, I have written this talk, which I am now giving to you. The subject is, 'But Why?' Like all good talks do, I will begin by defining some terms.
00:00:56.520
We're going to be talking about how our software is defined by the processes we use, and how the processes we utilize to build our software are governed by our culture. So, I want to start by defining culture because it's a word that has been used and misused a lot. I want to be clear that I don't mean bro fests and beers after work, nor do I mean preferring to hire people who look like you.
00:01:11.880
What I mean by culture is the set of behaviors, beliefs, and values that we share as a group of people. Since this talk is about the quality of software, I also want to address quality itself. Quality is, generally speaking, not an ethical problem. Software is neither good nor bad; it is merely valuable to someone.
00:01:27.890
And since we're talking about failure, I want to define failure as well. I define failure as the difference between what we expect and what we observe. So, failure is a particular kind of change, specifically the kind we don't like.
00:01:45.110
Before I proceed, I'm going to show you a picture that Amy lovingly drew for me. I want you to take a look at it for 30 seconds, trying to remember as many significant details as you can about this picture. You will literally be quizzed on it later, so pay attention to significant details, like how adorable it is!
00:02:02.090
What I want to start talking to you about today is a concept that I call process patterns, which Gerald Weinberg referred to as sub-cultural patterns. These are ideas we share within our culture about how software should be built, what we value when it comes to building software, and how we define our processes.
00:02:20.650
There is no single process followed by all organizations, and that's fine. Different organizations face different trade-offs and challenges. However, we can identify some common patterns and start to discuss them. If you need to increase your capability as an organization, you might have to reach for new patterns.
00:02:35.360
Gerald Weinberg wrote a book titled 'How Software is Built,' where he talks about these software sub-cultural patterns, and I recommend you read it. The first pattern, or rather, the zeroth pattern, we're going to discuss is the oblivious pattern. This is where we don't actually know we're performing a process.
00:02:51.769
How many of you have written some small program, perhaps a shell script on your own machine? You didn't check it into version control, you didn't create a ticket system for it—you just wrote it, used it, and were done. You were following an oblivious process—the null process.
00:03:10.060
It’s not that following an oblivious process is bad; it's good for certain things but not so good for others. If you need a more capable system, you will naturally evolve out of this into a new pattern. To know when this oblivious pattern can be successful, you need a few things.
00:03:25.900
You need to be capable of solving your own problem because that problem isn't too big for you to handle. You also need to know exactly what you want to build—you're the source of the requirements, you're the one building, and you're the one using it.
00:03:43.020
The next pattern, which I can't remember whether it's the first or second, is called the variable pattern. Phil Crosby, a management consultant, wrote about his own version of these sub-cultural patterns, which started with the pattern I'm about to show you. Gerald Weinberg later added the zeroth pattern.
00:04:03.416
The variable pattern is characterized by poor adherence to plans and processes and is dependent on the performance of individuals. It’s the first pattern where people are aware that they're performing a process. This pattern separates the developer from the user, and it’s also the first time blaming appears as a significant software development activity.
00:04:20.340
This pattern is especially common among small, young companies that build software products for microprocessors. To be successful in this pattern, you need to have a good enough relationship with your customer so that the requirements process isn't poor.
00:04:35.840
You also need to be a group of highly competent professionals—because we're dependent on individual performance—to ensure that there can be some expectation of individual performance. Lastly, the problem you're solving shouldn't require any additional effort.
00:04:51.310
There’s a myth among variable organizations when they try to figure out how to be successful, and it’s the Rockstar Programmer myth. They post ads for rock star programmers because they believe hiring one is the only hope they have of shipping a successful product.
00:05:12.910
They think that because software performance depends on their rock stars, software quality is as variable as human performance—spoiler alert: it's quite variable. These organizations try to find rock stars, yet managers often aren't capable of selecting the right candidates from their pool of applicants.
00:05:31.250
The other issue is that if they have processes, they only follow them when it's convenient. During a crisis, these processes are often disregarded entirely. As a result, the best indicator of success for a software project is which programmer happens to be working on it.
00:05:47.150
In this context, programmers receive all the credit but also bear all the blame. The next pattern is what Jerry calls the routine pattern. There’s a lot of emphasis on creating a plan—because if we can just come up with a robust plan, it will solve all our problems.
00:06:03.590
The first step in achieving statistical control is to attain predictability of schedules and costs. For organizations where the variable pattern isn't achieving this anymore, teams that need to coordinate with others start moving into the routine pattern.
00:06:21.500
It’s no longer sufficient to just lock programmers away and wait to see what they do. Routine organizations depend on plans and managers to execute those plans, but they struggle to deal with the actual changing requirements of software.
00:06:36.220
If a routine organization is to succeed, the problem cannot be bigger than what a small team can handle, and they must utilize their routine process to solve the problem. They also need to ensure that developers follow the process and avoid any extraordinary circumstances that could disrupt it.
00:06:52.370
These organizations have their own myth—the Rockstar Manager myth. This myth suggests that if the right manager can be hired, they will ensure developers follow the plan, and everything will work out perfectly.
00:07:06.410
Routine organizations can achieve a repeatable level of control and quality through rigorous management of commitments, costs, schedules, and changes. However, the catch is that they don't always know what they can do.
00:07:15.900
Usually, this is due to a lack of understanding of why they follow certain processes. Following processes blindly, especially in exceptional circumstances, is generally not successful. Routine managers tend to worsen situations during crises rather than improving them.
00:07:31.610
Routine organizations also believe in what anthropologists call named magic. Named magic is the belief that simply saying the name of a certain methodology—be it Agile, behavior-driven development, or lean—endows us with power.
00:07:46.370
We all know that there aren't silver bullets that can magically improve the quality of our software, so why do we keep investing in these named magic methodologies?
00:08:05.340
The next pattern is what Jerry calls the steering pattern. This is where you start to gain control over the quality of the software you're building. You choose the processes you want to use based on the results you’ve obtained from those processes.
00:08:22.080
Steering managers don’t depend on magic; they depend on understanding. While managers from routine organizations often have successful programming backgrounds, they lack the training, talent, or interest in management. Steering managers, conversely, usually possess some management training.
00:08:39.370
I personally moved from an individual contributor programmer position to a management role, which highlights how we cannot simply assume knowing how to write a program translates into managing people.
00:08:50.520
In steering organizations, processes aren't always well defined, but there is a shared understanding of the goals and why those processes were selected. Even if a process doesn't work, they are capable of recovering from it due to this understanding.
00:09:08.580
This is also why steering managers and teams can tackle harder, more challenging, and riskier projects. If you find yourself in a routine organization and struggle to solve harder problems or ship software to demanding customers, leveling up to a steering organization may be what you need.
00:09:27.290
Steering organizations are suited for problems that are too difficult for a simple routine to handle. Steering organization managers must be able to negotiate with their bosses and vendors and maintain control over external factors while rejecting arbitrary schedules and constraints.
00:09:42.720
There are two more patterns Jerry discusses in his book, but I will not cover them. Here's why: there was a Department of Defense study conducted in the 70s that surveyed various organizations to rank them in what was then called a maturity model.
00:09:57.759
I'm not in favor of the term maturity model, as I don't think I was being immature when I wrote that shell script; I thought it was simply the solution to my problem at that time. This study found that eighty-five percent of projects were at the lowest maturity level, which we can call variable, while only fourteen percent were at level two (routine) and only one percent at level three.
00:10:11.640
No projects or organizations even existed at levels four or five, so I won't waste your time discussing unicorns, and we'll move on.
00:10:27.320
How do you conduct a steering organization? What separates it from a routine organization? The answer lies in the concept of cybernetic control, which involves taking in feedback and using it to make informed decisions.
00:10:43.410
As we receive feedback from the system, we continuously make choices and refine our processes. How many of you remember Triple-A? Before there were Google Maps, you would go to Triple-A for travel directions. They would compile a spiral-bound book listing your route with landmarks along the way.
00:10:59.450
However, these directions lacked context; if you took a wrong turn, you wouldn't have the necessary context to recalibrate your route at all. Essentially, you had to backtrack to find your way.
00:11:15.720
Now, let's look at a more modern example of routing: GPS. A GPS provides a complete map that offers both context and understanding of the route, along with automated recalibration if you get off course.
00:11:31.030
This exemplifies a cybernetic system where your GPS recognizes you've veered off track and helps you make necessary course corrections.
00:11:46.680
To visualize this concept, consider a simple model without a controller, focusing on the inputs and outputs of a system. For a system producing software, outputs include not just software but also other effects like increased staff confidence and numerous failure reports.
00:12:01.230
Inputs need to include requirements, resources, and randomness.
00:12:20.530
When I refer to randomness, I'm speaking of the unpredictable elements that can derail your plans—essentially, the entropy in the system.
00:12:37.090
Variable organizations struggle with this model, as they don’t have a concept of a controller integrated into their systems, leaving them unable to exert control. In this case, requirements go in, and software comes out, but there's little understanding of that process.
00:12:54.790
For routine organizations, if we want to accept control over our system, we need to connect it to a controller so that individuals involved can assist in keeping software on track.
00:13:10.360
However, this level of control is still limited, as the controller can only enforce the plan without a comprehensive view of how far off the plan the system actually is.
00:13:25.440
To evolve into a steering organization, we need to take in feedback from the system's state and its outputs, which allows us to adjust requirements and exert the necessary control over our systems.
00:13:43.700
An effective method to mitigate productivity losses due to sickness is to send staff home when they begin showing symptoms. However, this isn't possible unless you’re aware that an individual is sick.
00:14:03.630
Controllers that only follow the routine pattern lack the ability to observe these signs. A more adaptable and effective control model is a feedback model, which enables us to measure and analyze our system's performance.
00:14:21.260
With effective feedback, we can predict how we want the system to behave. Assuming we can actually observe the real state of the system, we can compare desired outcomes with actual results.
00:14:37.290
If we have the means to act on the system and bring it into alignment with our intentions, we can initiate cybernetic control.
00:14:55.520
Cybernetics is a fascinating field encompassing the science of control and communication in animals and machines. These systems can be observed throughout life, from our human bodies, which are also cybernetic systems, to organizations with interconnected processes.
00:15:10.560
One common challenge is that many cybernetic systems exhibit non-linear behavior. A non-linear system is where inputs do not always correlate straightforwardly with outcomes.
00:15:25.370
When inputs vary, the outputs can also differ, which complicates prediction and management of the system. At the beginning of the talk, I presented Amy's picture. Can anyone recall how many cats were in it? Don't shout it out—just silently remember.
00:15:40.430
Now, think about whether you were correct in your answer. Many of you probably noticed that this is not the same picture I initially showed, showcasing the concept of change blindness.
00:15:55.410
What we have, in essence, is akin to a spot-the-difference game—failure detection, in a nutshell. We have expectations on one side, and what we observe on the other. We're trying to identify mismatches.
00:16:11.290
If I asked you to spot the differences now and we graph how many differences you noted over time, the graph would not depict a straight line. You'd start noticing a lot of differences but that rate would taper off.
00:16:27.740
This illustrates a non-linear system. So why is it not linear? Two simple reasons: first, as you find some differences, fewer remain; second, you begin by spotting the easiest differences before tackling the harder ones.
00:16:42.590
When you mark your progress and say, 'I've found 10 differences in two minutes,' and base your estimates solely on that, predicting how long it will take to finish becomes inaccurate.
00:16:59.180
Humans excel in linear systems but struggle in non-linear ones. Compounding that problem is that we often fail to understand the factors causing this nonlinearity.
00:17:15.530
Thinking about it: when you scan for differences, you lack a clear mechanism for prioritization. If you fix bugs, a natural bias might be to address the easy bugs first—potentially leading to an accumulation of risk over time.
00:17:32.470
Should you instead prioritize the riskiest bugs first, the impact is that you could work towards a less risky system over time, ultimately improving project outcomes.
00:17:49.640
Everywhere you look—requirements gathering, feature development, and bug fixing—non-linear systems appear, yet managers persist with ineffective linear models, yielding disappointing results.
00:18:05.840
The situation worsens as these non-linear interactions add even more complexity to the overall system. We term this the composition fallacy: the idea that the composite system is not more complex than the sum of its parts.
00:18:20.260
Reality shows that the interactions between components lead to increased complexity. I should also highlight the decomposition fallacy, which suggests that if all subsystems work, the overall system must also function.
00:18:35.160
Ignoring the interactions between these systems poses significant risks. Therefore, how can we navigate non-linear systems while avoiding pitfalls from disturbances or insufficiently understood elements? A few suggestions:
00:18:51.460
First, introduce governing actions in your feedback models. Prioritize addressing risky work first, for example. The system may remain non-linear, but you can regulate its behavior to prevent it from spiraling out of control.
00:19:07.010
Second, act promptly—small and often. The sooner you implement corrections, the smaller the needed adjustments become. Less risky changes allow for more accurate predictions.
00:19:24.840
The more frequently you implement modifications, the more opportunities you have to catch problems before they escalate. These small corrections fuel the engine of cybernetic control.
00:19:39.340
However, to address issues, those problems must be visible. If you’re unaware of a problem, you won't be able to recognize or rectify it. Controllers require feedback to function effectively.
00:19:56.590
How many of you utilize the GitHub pull request model? Once you have something to share, you put it up, create a pull request, and receive feedback. It's a solid system and facilitates valuable conversations.
00:20:14.680
More crucially, GitHub renders work products visible that might not be otherwise displayed. Managers cannot solve issues with work they cannot observe, which diminishes their effectiveness in managing software production.
00:20:30.300
This leads us to the next point: if you cannot comprehend a problem or system, you can’t fix it. If your model lacks supporting detail for accurate predictions, your decision-making process will be blind and arbitrary.
00:20:49.100
Regarding models: all cultural patterns employ models guiding their thoughts. These models—implicit or explicit—support the need for understanding and effective communication. Explicit models complement verbal communication with pictorial representations that can illustrate complex interactions.
00:21:07.880
Frederick Brooks, who authored the classic book 'The Mythical Man-Month,' noted that more software projects fail due to lack of calendar time than for all other causes combined. However, he doesn’t mean that running out of time is the root cause.
00:21:26.730
What he does is offer a model that people frequently use to explain project failure, which often represents merely a symptom stemming from deeper issues in their process.
00:21:43.930
If we neglect to expose our models of a system, when someone inquires about why a project failed, we won’t comprehend its underlying issues adequately. Consequently, responses may lean towards 'we just didn’t have enough time.'
00:22:01.200
Managers rely on models to guide their decision-making, whether those models are acknowledged or not. By making models explicit and sharing them, we foster collective understanding and invite constructive feedback.
00:22:17.200
Let’s talk about an implicit model managers often hold. Managers tend to believe that if they input 10 units of pressure, 10 units of increased productivity will result. But allow me to present a more nuanced perspective.
00:22:33.530
It's not just about the linear relationship; there's burnout to consider. When individuals burn out, productivity doesn’t plateau; it plummets. Not just productivity but overall happiness as well.
00:22:51.110
Here's another example of an implicit model, particularly prominent in variable organizations. They often operate under the misconception that they can easily identify the best programmers. This belief is foundational to the premise of their success.
00:23:08.490
Yet, there's scant evidence supporting their ability to identify top talent. In conclusion, I want to highlight the importance of these feedback control systems—these cybernetic systems—at all organizational levels.
00:23:26.500
They should inform our approaches to testing, strategic product decisions, and our understanding of systems. With these models, we can begin to think about and observe our systems through the lens of non-linear interactions.
00:23:43.850
Ultimately, if software is governed by process, and process by culture, then culture is governed by people. If you truly want to enhance software quality, we must dedicate more time to understanding people.
00:24:00.000
I have one last thing to share with you: I also have this GIF of Vader and a little girl. No, no, still no.
00:24:12.790
Okay, good talk, everyone!
00:24:16.480
You.
