I know I can, but should I? Evaluating Alternatives

Decision Making

Kevin Murphy

@kevin-j-m

#decision-making

#ruby-on-rails

#software-development

#project-management

#code-quality

#maintenance

I know I can, but should I? Evaluating Alternatives

by Kevin Murphy

The talk "I know I can, but should I? Evaluating Alternatives" by Kevin Murphy at RailsConf 2019 explores the complexities involved in making technical choices in software development. Kevin uses the metaphor of selecting between different tools (hammer, screwdriver, etc.) to illustrate how various factors influence decision-making. He proposes a framework for evaluating alternatives based on four key criteria: impact, cost, maintenance, and consistency.

Key points discussed include:

Making Choices: The importance of making informed technical decisions rather than relying on instinct or popular opinion.
Weighted Scoring Model: Introducing a systematic approach for evaluating options through a weighted scoring model to facilitate better decision-making. This includes determining what criteria matter most to the team.
Criteria for Evaluation:
- Impact: Assessing how decisions affect the team and codebase, along with potential non-functional requirements like performance and accessibility.
- Cost: Understanding the risks associated with solutions, including dependencies on third-party libraries versus developing in-house solutions.
- Maintenance: Considering the long-term implications of choices, including how easily future team members can work with the code.
- Consistency: Finding a balance between adhering to established practices and integrating new approaches, ensuring the codebase remains comprehensible to new team members.
Example Scenarios: Kevin illustrates his points through practical examples, such as a tool for food scientists at Hormel, where the audience evaluates alternatives for implementing specific functionalities.
Final Thoughts: The conclusion emphasizes the necessity of evaluating choices based on all four criteria while recognizing the context of a problem and being adaptable in the decision-making process. The talk encourages developers to engage in discussions with their teams to find solutions that align with both technical and business needs.

The core takeaway from Kevin's talk is that while it’s important to identify the right tool for the task, understanding the implications of each choice and collaborating with stakeholders is crucial for sustainable software development.

00:00:20.750 Hey everybody, thanks for coming today. We're going to be talking about making choices. There's a lot of information out there about making choices on what programming language to use or what framework to pick for your next project. We're here at RailsConf, so I'm going to assume that's a solved problem for everyone in the room. Today, we're going to talk about some choices you might make on a day-to-day basis in your job, such as trying to push tickets through the system.

00:00:38.519 Let's start with an example. Suppose I ask you to join two pieces of wood together. There are a couple of screws already started in there, and you have a hammer and a screwdriver available. I’d like a little audience participation. By a show of hands, who knows which option they would choose? It doesn’t matter which, but I know which one I would use. Okay, great! Wonderful! Now, let’s change it slightly. I’ll take that screwdriver away and give you a brick. Again, who knows which option they would choose? Awesome! One last scenario: Let’s remove the screws and instead give you your choice of a hammer and nails, tape, screws, and a screwdriver, or wood glue. Who knows which of those options they would use to join the two pieces of wood together?

00:01:02.000 Great! For anyone that raised their hand for any of those situations, I'm interested in why. There are probably a lot of reasons. You could have a lot of experience; maybe you’ve used the hammer a lot, and you think, “This is just another tool; it’s a situation where I can use the hammer.” Cool! Wonderful! You might just want to get it over with, looking at the tape and thinking, “I can just wrap the pieces of wood together and be done with it.” You might also be interested in the degree of difficulty. You see the brick and think, “I could use a challenge.” Maybe you’re interested in trying something new you’ve never used, like wood glue. It seems like a reasonable application, especially if everyone is talking about wood glue; it’s the new hot thing! Maybe you don't really know why, but you definitely have an opinion.

00:01:36.630 Today, we’re going to talk about how we can develop a system for thinking about all of these kinds of choices that we’re making. My name is Kevin Murphy, and I work at a software consultancy called The Gnar Company. We're located in Boston, Massachusetts, and we work really closely with our clients and our partners to make sure that we’re making the right technical choices for them, given the context of the problems they’re trying to solve. We try to think of some theoretical backing for how we’re going to make these decisions.

00:02:19.750 We can consult the sacred project management texts, and you’ll see a lot of references to alternatives analysis. This might sound like big words, but they usually propose something called a weighted scoring model. So let’s build a weighted scoring model today and talk about how we do it. Like any good project management tool, it’s just a spreadsheet. That’s fine. Start by building up some criteria that are important to making this decision. For this example, let’s say we care about price, ease of use, compliance with some regulatory body, and how fast we can get it.

00:02:49.540 I put those on the slide really quickly, but the fact is this can take a lot of time just to figure out what’s important to you, and that’s just step one. After that, you then need to pit the items against each other and say, “Yeah, these things are important to me, but really how important are they?” Out of a fixed pool of 100 points, how would you determine which one you really care about? Then you need to identify which choices you’re going to evaluate: for this example, I’m going to call them A and B, because I’m really good at naming things.

00:03:34.970 So be thankful you’re listening to a talk and not reading my code. Once you know which options you’ll be evaluating, you need to score them on the different criteria. For example, I would evaluate option A on price and all the other criteria, and I would do the same for B. To wrap up, I can compute a score for each of the alternatives by multiplying the weight by the score for each individual criterion and summing them all up. At the end of the day, I choose the option with the highest score, and we’re done. Great!

00:04:11.390 So what do we learn here? Why is this beneficial? Well, it takes some thought! We gave due consideration to each of the alternatives; we didn’t just make it up. We went through a thoughtful process that got us to an endpoint. It also provides justification for our decisions. If our boss comes to us and asks, “Hey, why did we choose A?” we actually have an artifact that we can point to, and say, “Well, the team went through this exercise, and at the end of the day, A won for these reasons.” Most importantly to me, it’s a huge consensus-building opportunity. You’re not going to be building this out by yourself; you’re going to involve your project stakeholders.

00:05:13.200 At the end of the day, everyone’s going to be super aligned on what’s important to you and how you’re going to get there. Everyone’s going to be on board with A because we all came together, and that’s what it ended up being. So great! However, it’s not all roses, right? These processes take a lot of time to do. Even knowing what’s important to you, like I said, is difficult. There are a lot of unstated assumptions you really need to bring to light. You need to learn a lot about the different options you’re choosing between before making a decision. To do this right, you need to have perfect knowledge, and that’s not reasonable.

00:05:56.750 You’re going to be making these choices before you’ve done anything with A or B; you just figure out which one should I do. Because of that, these things tend to be a bit rigid and inflexible. It takes a lot of time; it’s really hard to get it right, so you’re not likely to iterate on it. You might do it at the beginning of a project in a big kickoff meeting, but you probably won’t revisit it. Even with these limitations, that doesn’t mean it’s not valuable, but it may be a little difficult for us to use on a daily basis.

00:06:50.590 Let me try to give you some help here and let’s talk about how we can actually do that. I’m going to save everyone some time and propose some criteria we can use and punch into our weighted scoring model. To identify those criteria, we’re going to represent them with the four tools that we talked about at the beginning of the presentation.

00:07:01.500 We’re talking about the impact of the choices we make, what cost they introduce, what sort of maintenance activities we need to undertake, and lastly, how each of these options affects consistency. Let’s get started. We’ll start at the top with impact, which we, of course, will represent with a hammer, because what tool could be more emblematic of making an impact than a big old hammer? For all of these criteria, we’re going to grab a ticket out of our ticketing system and implement it all together.

00:07:34.839 For the purposes of this presentation, we all need to be working on a shared application. We’re going to be building a tool that food scientists can use to conduct their studies. The Hormel Corporation is based out of Minneapolis, and for all you meat aficionados, the Spam Museum is south of here in Austin, Minnesota. If you have some extra time and the means to get down there, check it out. We’re going to build a tool to help those food scientists that are working on Spam make it taste even better. They need our help in building a system to keep track of the food tests they’re running. So, our first ticket is that these food scientists need a way to welcome participants after they’ve enrolled in this study.

00:08:37.610 Alright, I can do that. We’ll start with a test. We’ll build a system test or a feature test. We'll go to the page where you can enroll as a participant. We’ll fill in that form with the necessary information, and when you click enroll, we’ll make sure that we send out an email. Okay, now we need to make this actually work. We already have a study participant model. This is all part of the existing system, and Rails knows how to save stuff. It also knows when it saves and creates things, so let’s just hook into an ActiveRecord lifecycle callback and say, after we create one of these things, we’ll call this deliver_welcome_mail method. As the name implies, this will call ActionMailer and send that welcome email to the participant. We’re going to run our test, and it passes, so great, we’re done!

00:09:38.580 Right? We can push it and go grab coffee, head home for the day, play ping pong, or whatever it is we do, right? Maybe! But this is a talk about alternatives, and it would be kind of disingenuous to say there’s only one way to do this. So let’s talk about a different way we could solve the same problem.

00:09:55.450 Instead, we could build a whole separate class called study enrollment. When you create a new study enrollment, it takes various parameters about a study participant. You’ll say you can save a study enrollment, and when you do that, it builds a new study participant. If you successfully save a study participant, it will send the welcome email as well. We need to change our existing controller a little bit before we create a study participant. We now need to create an enrollment, save that enrollment, and if it doesn't save, we need to tell the instance variable about what the participant is so we can address any validation errors it has. We run our test, and it passes.

00:11:11.710 So now we’ve got two solutions that both work equally well, and we need to figure out which one we’re going to do. Let’s talk about how impact can help us make that decision. The first thing that's important in impact is to talk about how your team is going to react to these choices. Presumably, you’re working on code with other people who probably have great experiences and you love working with them. They likely have opinions, and you might agree with them or you might not, but those opinions are still valid.

00:11:32.360 You may know that someone on your team has deep-seated issues with ActiveRecord callbacks, or perhaps there’s someone else who doesn’t like layers of code and figuring out all the indirection and where things are coming from. Understanding this information about your team can help sway your decision on how your implementation is going to look. You also need to consider what sort of standard you’re setting. Any code you write that ends up being merged into the main branch of your application is going to be used as an example in the future.

00:12:18.060 For instance, when building the next feature that sends an email as a result of creating some object, people may reference back to your implementation. So it’s a lofty concept, but it’s important to consider what sort of legacy you’re leaving by introducing this code. You should also be wary of what non-functional requirements may be associated with this work. One example of a non-functional requirement could be performance.

00:12:46.880 If performance is particularly important at this moment, then being more performant than the alternative could help sway your decision. Accessibility is another incredibly important non-functional requirement we want to emphasize. We need to ensure that the systems we write are usable by the majority. For the remainder of this talk, I’ll be giving you justifications for advocating a particular option, but I’ll draw a hard line here: If you have two alternatives and one is more accessible, just pick that one. If there are trade-offs you’re making, find other ways to mitigate them.

00:13:19.320 With accessibility being the deciding factor, it’s an easier choice. Now let’s revisit our second criterion, which is cost. For this, we will represent it with tape. Tape is relatively abundant, you can find it almost anywhere, and it will get the job done quickly. Plus, if you screw up, you can just remove it and try again. So, we’ll grab our second ticket now. We’re going to ensure that only principal investigators can create study protocols. These folks have PhDs; they went to college for a million years, and this is their big payoff. Congratulations, you’re the only people who can do this!

00:14:22.740 We’ll start with a test, just like we did in our last exercise. We’ll sign in as a non-principal investigator and go to the page to try to create a new study protocol. We’ll make sure to see a message that says, “Sorry, you can’t really do this,” and you’re redirected to some other page. Now let’s actually make this work. We already have a way to create a new study protocol, so we can write an if statement. If you are a principal investigator, cool, you can continue. If not, you’ll see a message that says you can’t do that and you’ll be redirected to some other place.

00:15:23.230 We run our test, and it passes. Once again, we’re done! However, since this is a talk about alternatives, let’s revisit that controller from before. Instead of pulling in a simple if test, let’s include some third-party authorization library and ask it to handle whether this person can perform this action on this resource. For this example, I choose Pundit. We’ll do something special here: we’ll show a flash message to recover from that authorization error.

00:15:55.170 For any authorization library authors in the room, I didn’t use your tool. Sorry; I’m sure it’s great, but I had to pick one. If we do that, we run the test, and it passes again! So, we have two solutions: we can either write an if statement to authorize or pull in this dependency. How do we know which one to choose? Let's consider cost. As Aaron might say, we can do these things, but at what cost?

00:16:37.270 When I think about cost, I also talk about what your risk appetite is like. What kind of risk are you willing to accept? Each of these solutions is going to introduce some level of risk. Maybe they both introduce the same risk, but they manifest differently. When you pull in a third-party dependency, you’re accepting any performance problems or security issues they might have in their code. You’re relying on them to maintain it and hoping that they continue to support it.

00:17:14.460 If you write the code yourself, you have the same problems, but you are handling them yourself. Neither option is wrong or bad; they’re just different. You need to evaluate what’s important to you and how you're going to mitigate the risks you’re introducing. You also need to be aware of the context within which you’re being asked to solve the problem. Let’s say we picked this ticket up in a sprint planning meeting Monday morning.

00:18:01.960 We sit down with our coffee and bagel while our product manager says, “Great news! We have new work for this week. Only certain people should be able to perform certain actions in the system. Let’s make it happen!” We all high-five and head off to tackle that! This is how your Monday mornings go, right? But let’s say someone runs up to my desk at 4:30 PM on Friday and says, “Co-investigators can create study protocols! If we don’t fix this right now, we’re all fired!” Now, that’s a radically different situation!

00:18:50.230 My dinner reservations are looming, and I’m leaving for the weekend. I just want to get this done! The choices I make in this situation are going to be impacted by the context in which I’m being asked to solve this. There’s a cost associated with urgency, and that’s perfectly fine! We just need to recognize that.

00:19:38.790 Let’s consider the bigger picture; we’re working on that one ticket and only making sure that principal investigators can create study protocols, but if we regard this as one ticket, we might miss the fact that we want to do the same thing in the create action as well. So if someone finds a way to post to our form, we don’t want them to create a new study protocol unless they’re an investigator. We can add the if clause to the create action too, and we can extract it into a private method in the controller to keep it DRY.

00:20:39.450 Then, a couple of weeks later, we have to guard some other resource against some other action, and we’ll do something similar. We’ll think back and say, “Hey, we did this in another controller. Let’s extract it into its own module or class.” That’s awesome and super cool; you’re now maintaining an authorization library! Maybe that’s what you want to do—and that’s great—but maybe you don’t want to maintain that. That’s perfectly fine too. If you knew how your system would change and what risks you’re willing to take on at the onset, that might impact which approach you choose.

00:21:36.550 This idea of the big picture will return with our next criterion: maintenance. I don’t know about you, but I spend every Monday morning walking into my Rails app like I’m putting up my screws, making sure everything’s good to go, and going about my business. What that actually ends up being for me is updating gems, but we’re going to use a screwdriver to represent any maintenance activities we’re going to take on.

00:22:35.740 Let’s grab another ticket. Our food scientist friends have the ability to look at the data they’re collecting about how much people are loving Spam, but for some reason, those food scientists can’t see when we actually got this data—when the actual taste test happened. So, we need to make sure that information is available. We’ll write a test that sets the time the data was collected and creates a data collection event, saving the timestamp for that time. We’ll check our show action to ensure that it outputs that date in the expected format. When we go to our show view, we’ll see a list that says, “Hey, here’s when this data was collected,” and it will output the Rails created_at timestamp.

00:23:25.890 Once that’s done, we’ll assume that format is correct and run our tests. They pass! But this is still a talk about alternatives! Let’s explore a different way to solve that. For inspiration, we’ll change the test a little. We had a variable called collected_at, which we used as an attribute named created_at. What if instead we had a separate attribute on the model called collected_at? We can make that work! We can add a migration to create the new attribute.

00:24:45.250 Next, we’ll make a subtle change to our view. Instead of using the created_at timestamp, we’ll use the new collected_at attribute. We run our specs, and they pass! We can use Rails’ timestamp default of created_at time or introduce a whole new attribute. How can maintenance help us solve this problem? There are really two sides to maintenance. You need to be able to predict the future.

00:25:38.960 I’m going to ask you to do that because I’m not great at it. Yet, I can tell you that it helps to consider the big picture. I might know a little more about my application than just the current problem I’m trying to solve. For example, I could be importing a lot of data from another system that has food science experiments over the last fifteen years. If I know this, it informs my choice: using the Rails timestamp for created_at might represent the time it was entered into the database or the time that the data collection event actually took place, but not both. That could be a problem. Therefore, this knowledge could lead to a better choice.

00:26:47.780 If I’m unaware that this data will ever come from another place—or if I just don’t know it’s not going to happen—as far as I know, we can get away with just using the Rails timestamp. If it seems acceptable to you, let’s make that choice! I’d like to propose a radical idea: Talk to other people about it. But don’t do most of the talking; do a lot of listening. Particularly, talk to folks who aren’t doing the same work as you. Talk to developers on other teams, your users, your support teams, project management, and your ops team. I’m sure they’d love to hear from you before you make something their problem for the foreseeable future.

00:27:57.190 Explore what they’re thinking when you say, “I’m trying to solve this problem.” They might have some insights or information that can radically influence your current work. Lastly, for maintenance, think about how you can make things easier on yourself. We’ve all encountered situations where we’re trying to implement a new feature in an application and something doesn’t sit well with us. We might be looking at a code segment, and we’re unsure why it’s not working. Whether due to procrastination or just a yielding to temptation, we blame that line of code from a couple of months ago. We realize, that was code we wrote, and we had no idea.

00:29:13.950 We take a deep breath and think, “Sure, I had my reasons.” And you did! Presumably, we’re all doing the best we can at the time. That’s wonderful! But can you look back at code you’ve written in the past and think, “There’s no way I’d solve that problem this way now?” That’s something to be celebrated as you learn and grow. Maybe you’ve discovered a design pattern that could radically simplify the problem you need to solve. But that’s not what I mean when I say think about your future self. I’m talking about those moments when you’re in the present, trying to solve a future problem. You know you could take another hour to fix it properly, but instead you think, “I just need this to be done.” That has its place!

00:30:19.920 It’s 4:30 PM on a Friday, and you don’t want to get fired. But if you have the opportunity to step back and think, consider how you can put in just a little bit more effort to ensure that your future self won’t be mad at you for making choices in the past that could lead to future frustration. We’ve got one last criterion to consider: consistency. It’s the glue that holds everything together. Metaphors are great!

00:31:20.170 Let’s work on one last ticket together. We have a page where folks can edit their personal information, and we want to continue allowing them to do that. But if they do so, we need to ensure they’re aware they may no longer be eligible to participate in the study. Perhaps Hormel is only interested in people within certain age demographics or with particular food preferences. We run a test like all the others; we’ll navigate to the page where you can edit your personal information and make sure there’s a message stating, “You can totally do this, but there are consequences for your actions.”

00:32:07.220 We already have an existing controller that enables users to edit personal information. So we’ll add a flash message that says, “Hey, just show this immediately.” As long as that localization reflects what’s in the test, we’re good to go. Once again, let’s say it together: this is a talk about alternatives! Great job, everybody! A subtle twist I want to propose is that I’m not going to suggest a new implementation. That’s fine; it’s just a flash message, a single line of code. However, I want you to question whether you even need this test. What value does it provide?

00:32:58.590 How do I know how to test something? How do I know the right level for a test? How do I know whether I should test something at all? Consistency can help you here; it’s vital to understand the existing institutional knowledge. What are the current best practices in the field regarding testing flash messages? What do industry thought leaders say about it? What does your organization value concerning testing flash messages?

00:33:43.570 If your company has large posters stating, “We do not test flash messages,” that will probably influence your decision. If you work at such a company, come talk to me afterward—I have many questions! If you work somewhere where there’s a standard, it will likely guide your choice. You should also consider what existing standards exist. This flips the impact side we discussed. Remember when I said any work that ends up being merged into the main branch of your application is going to be used as an example? This is when you’ll want to leverage that.

00:34:35.950 If I glance through the codebase and don’t see any tests for flash messages, that gives me some insight. But if I spot tests that assess various aspects of flash messages, that gives me a different signal. Lastly, with consistency, think about how easy or difficult it is for someone new to understand or modify this code.

00:35:22.820 Since we’re at RailsConf, most of us have some Rails knowledge. The framework emphasizes convention over configuration, which adds value; it means most of us could swap Rails apps and quickly grasp what’s happening. This shared understanding is valuable, even if there are reasons to deviate from the standard approach.

00:36:09.290 Allowing ourselves to introduce new developers to the code base in an efficient manner is essential. So, we’ve discussed our four criteria separately, but annoyingly, we can actually think about all of them together. We must consider how each applies to the current situation we’re trying to solve.

00:37:10.470 Even though I would never actually sit down with my team or by myself and create a weighted scoring model spreadsheet, I probably have some intuition about how important these elements are based on working with the team for a while. These numbers are malleable and will change based on the particular problem we are trying to solve.

00:37:59.540 Let’s return to our ticket because I initially omitted the beginning. Remember, the reason we’re doing this is to ensure compliance with some FDA regulation. For anyone in the audience dealing with an application subject to FDA regulations, please hang on! Don’t run out of the room screaming, “We’ve got to fix this!” I made up this scenario, so I’m sure your application is compliant, and you’re good to go.

00:38:53.920 However, even knowing that the reason for these actions is to meet government regulations might adjust our baseline. We realize we might need to put together an attestation report for auditors to clarify that our application remains compliant. Therefore, we need to ensure this never breaks. So, consistency might take a backseat while impact and maintenance costs could increase.

00:39:56.540 Even if I work at the company with the poster that says we do not test flash messages, now we are testing this one because we don’t want to face penalties. If we can avoid it, we want to keep our fellow developers from complaining about our CI times being faster if we didn’t test the flash message.

00:40:52.850 We’ve probably already established happy path test coverage for the editing of personal information page. Rather than creating a new test, I can simply integrate with that existing spec and say, “Hey, while you’re doing this, just make sure I don’t go to jail and continue on.” We altered our approach slightly but stayed on course.

00:41:54.560 So even though we shifted our perspective, we still prioritize what’s most essential. Each time you encounter a problem to solve—whether at work, in personal projects, or anywhere—consider the impact it has not only on the codebase and the product but also on your team. Evaluate the costs of introducing new solutions, how you will be aware of those costs, and how you can mitigate them to match the maintenance activities you’ll undertake when implementing these options.

00:43:31.230 Also, think about how it aligns with the approach you've previously taken to solve similar problems. If you’d like copies of these slides or wish to see a complete Rails app that’s fully developed, you can visit The Gnar Co/RailsConf or my GitHub at Kevin-J-Test-Em/EvaluatingAlternatives. I have stickers that look like this; if you’d like one, come chat with me afterward.

RailsConf 2019