00:00:14.570
Hello! Hi everybody! Does everyone still have energy at the end of the day? We're still here attending. I'm sorry, what? So, alright, I want to start this with what is going to seem like possibly an unrelated question: How far do you live from an airport? Think about what you would say about that. You could say it out loud or just think about it.
00:00:27.980
When this question comes up, I have no problem saying that I live 20 minutes from an airport. That's basically factual as far as I'm concerned; I will drop that into conversation. But, of course, that's not really true. It's dependent on a series of assumptions. It assumes, for example, that I'm in a car driving to the airport. In my case, O'Hare would take a lot longer. It also assumes that the weather is good, which in Chicago is dicey for half the year. It assumes that it's not rush hour, which means another ten percent of the day. And finally, nothing untoward happens—like blowing out a tire on my way here. Yesterday I learned another assumption, which is that the cab driver takes the optimal route to the airport. If the cab driver messes around on the back roads of my suburb, it takes a couple minutes longer.
00:01:01.310
Despite the fact that all of these factors conspire to make my trip longer, I still say that my trip to the airport takes 20 minutes, even though there's no way it could possibly take less than 20 minutes unless I teleported. There are all kinds of ways for it to take longer. In other words, I still find this 20-minute estimate, for lack of a better term, to be so valuable that I use it. I don’t even know the actual distance that I live from the airport in miles; I only know it in minutes, even though it's problematic. So, my name is Noel Rappin and this talk is estimated to take 20 to 30 minutes.
00:02:20.540
I realized I had more time in the slot, so it's 30 minutes, give or take 10. It's going to be very embarrassing when I go over at the end, so let's get that out of the way right now. I work for Table XI, which is a consulting firm in Chicago. If you need consultants in Chicago or you want to be a consultant in Chicago, come find me over the next couple of days. If you just want a nice green circle sticker with an 'X' on it, I have a bunch of those as well.
00:02:40.100
You can also follow me on Twitter at @noelrap, if you wanted to say something about this talk or just say hi in general. My qualifications for talking about estimates: once upon a time, I gave an entire full-day workshop that ended exactly at four o'clock on the dot, as scheduled. This was evidenced by one tweet from a guy. Since I actually got an estimate right once, I think I am very qualified to talk about estimates.
00:03:05.340
This is also slightly idiosyncratic; I am slightly idiosyncratic about this. You will probably disagree with some of what I say—many people do. Some people are somehow successful even if they do estimates a different way than I do. I don't know how that works, but apparently they are, and that's fine. This is a way that works for me and a method that works for me, and you may do something different or have different opinions, and that's fine. One thing that you hear about estimates is that developers are bad at them. I think that's something that almost everybody in this room has probably heard: 'Developers suck. Developers are bad at estimates!' We can't do it right. And sometimes you hear the question: 'Are software developers uniquely bad at estimates?'
00:03:51.780
Sometimes you hear a phrase like that we are so bad that we have problems that no other industry has because of our estimates. The only thing I can say to something like that is that people who say that have obviously never been involved in any kind of home remodeling, construction project, or work involving any kind of home contractor. That is an entire industry dedicated to time overruns and cost overruns, with a vocabulary dedicated basically to scope creep. It has many similarities to software in that many times you don't really understand the problem until you try to solve it. You don't really know what's behind the wall until you tear the wall down. There are similar pressures in construction in terms of the pressure to deliver an initial estimate that's low for obvious reasons, such as to make a sale.
00:05:25.290
When I can point you to the Big Dig in Boston—those of you who live near Boston—I lived in Boston for a good chunk of this. This was the most ambitious public works project in America in the 2000s. It was an attempt, a successful attempt, to replace the north-south overpass highway through Boston with a series of north-south tunnels. It occurred to me as I was putting this together that in relation to the airport analogy, it also involved an exciting game of how will we get to the airport this week for about two years, as the access roads got messed up. The Big Dig was originally estimated to cost $5.8 billion, which is a lot of money. It came in a tiny bit over that at a mere $21.93 billion and counting, because they are still fixing a couple of things that weren't installed correctly the first time.
00:06:29.840
So, estimation is not just our problem. There are other industries and other kinds of engineering that have similar issues when trying to come up with a time or money estimate for the work that they do. But with software, we have a slightly more unique solution, because at least in theory, it's easier to change software estimates and move software around than it is to move walls or gigantic earth tunnels underneath Boston or things like that. Potentially, there’s something that we can do about it.
00:06:58.560
Sometimes, you will hear the suggestion from people who think about Agile projects that we shouldn't estimate. Some believe that software estimates are so bad that they are actually a net negative to project success or project happiness. You can find these people sometimes through hashtags like #NoEstimates or through blog posts. They suggest that we don't estimate and that we just do things and get paid. The ideal world here is that you are continually in a very tight feedback cycle where you tackle the highest priority task remaining until it's done or until the client runs out of money or becomes dissatisfied.
00:07:28.240
I can imagine this working in maybe a startup environment—a product environment where you don't really have deadlines, customers, or clients, and there is enough money to cover all of this. But I'm a consultant working with clients with tight budgets, and it feels somewhat similar to me saying, 'We're just going to tear up your kitchen, and you just keep writing me $15,000 checks and eventually you'll probably get your kitchen back.' It is not effective. It's not useful to our customers. Customers reasonably expect to have some kind of idea of what they’re getting into before they start a project, and it's our responsibility to provide them with a good faith plan.
00:08:02.540
You sometimes hear that projects don’t fail for technical reasons; they fail in part for communication reasons. I think estimates are a very important piece of communication between the team doing the work and the team that is paying for the work, managing it, and overseeing it. However, you need to be careful with it, because it’s communication that has dollar signs attached. Anything that has a number attached suddenly gains a certain weight and gravity in discussion. It’s hard to move it. Especially once you put a dollar sign on something, no matter how crazy or faulty your process was to arrive at that number, once you put a dollar sign by it in the mind of the person you're talking to, that is now a real thing that you need to take into account in future discussions.
00:08:46.110
So, if estimates are communication, what are they communicating? Well, at the most basic level, you’re communicating the investment involved. In this case, not just a monetary investment; you're communicating all the things that need to be provided to the development team in order to get the project done. So that is money, time, potentially certain amounts of requirements analysis, some kind of data format, some sort of business plan; all of these can be part of the estimation process.
00:09:00.790
Additionally, the estimate enables you to communicate progress. It's hard to say you're fifty percent done if you don't know what 'done' is. Estimates help you define what is in scope and what is out of scope for the purposes of being done. These are all necessary aspects of a good project, and discussing the estimate in planning is a really good framework for achieving that shared understanding upfront.
00:09:34.860
However, a bad estimate does have consequences. Many of us have probably been on projects where there was an unrealistic estimate upfront. This creates unrealistic expectations and results in pressure on the development team which leads to poor work and further slipping of guidelines. Bad estimates also allow for a certain kind of micromanaging, making it critically important to track progress: 'You said this feature would be done on Tuesday, yet here we are on Thursday—where is it?'
00:09:52.800
That kind of very fine-grained management, whether it comes from a client or your actual manager, can be extremely counterproductive during a project. So if bad estimates lead to these issues, I think we should simply aim to do good estimates. The easiest way I know of to produce accurate estimates is to wait until the end of the project and then create your estimate. This technique, although underrated, has been utilized at large companies. They produce the estimate at the end to cover and retroactively justify their budgets.
00:10:14.090
However, most of the time you actually need to produce an estimate at the beginning of the project, and estimating at the project’s start is basically the worst. You don’t know anything! Gary was talking about knowns, unknowns, and unknown unknowns. At the start of a project, you have the most unknowns of all kinds, known and unknown. This situation only gets incrementally better over time. Unfortunately, clients insist on receiving an estimate at the beginning.
00:10:38.920
This is somewhat like trying to figure out how long it’s going to take to get to the airport if you don’t even know where the airport is or if you haven’t even decided if you’re going to need to fly yet. What can make an estimate at the beginning of a project effective, even when we know very little about the details? I’m going to talk about a couple of processes by which you can create an estimate: first, one that works throughout the project’s life cycle, then how to modify that to create an estimate that might work at the beginning of a project and have a reasonable chance of success.
00:11:04.720
This method has worked for me in the past, but first I need to ensure that we’re all clear on a couple of Agile project software terms because I’ll be using them, and I want to make sure we all have a common understanding. We’re going to talk about a feature or a story, which is the unit of work in a project. Typically, these are relatively small, ideally on the order of a couple of days or a week or so. The main thing about them is that they are, at least in theory, independent of each other and independently verifiable by the customer or manager regarding when they’re done.
00:11:42.200
An iteration is the unit of planning in an Agile project. It’s typically one or two weeks. The way that this works is that at the beginning of each iteration, the team gets together to discuss which stories they’re going to complete over this iteration. This is how planning is traditionally done in iterations. A project point is the size of a story; we will talk more about exactly what points are in a second.
00:12:10.160
Velocity is the most important number. It represents the number of points that your team can complete in an iteration. You can say, for instance, 'We've done ten points in this iteration; we can do another ten points in the next iteration; we have about a hundred points left in the project, so we have about ten iterations to go.' This is how we use velocity effectively.
00:12:35.900
So, here's a theory about how to do this, or a couple of things that have worked for me. When you start thinking about how to create an estimate, remember that there are two different scales. We can estimate the entire project—e.g., 'This will be live by December 15th,' versus estimating individual features like, 'I can get this reporting feature done by Tuesday.' The first thing to realize in creating a successful estimate over the life of a project is that nobody really cares about fine-grained feature estimates.
00:13:16.700
A lot of effort goes into those estimates; however, at the same time, it's not important over the life cycle of the project whether an individual feature is done on Tuesday or Thursday, or if it takes a little less or a little longer. Over time, the important part is knowing when you can actually go live and when people will start using the software. Individual estimates are only useful so long as they serve as the building blocks for arriving at the aggregate estimate.
00:14:08.920
Estimates are, by nature, imprecise. If they were exact, we'd call them measurements. Instead, they are probabilistic. For example, if I see a weather report that says there is a 50% chance of rain, and it rains, was that forecast accurate? You actually can't tell from one instance of data. You need a larger sample to determine accuracy. If I've said there was a 50% chance of rain 20 times and it rained 11 of those times, you would deem that process reasonable. Conversely, if it rained only four times, there’s likely a miscalculation in my estimation process.
00:14:36.010
Weather forecasters are known to overestimate the chance of rain because if they predict rain and it doesn't, nobody gets mad at them. But if they predict that it won’t rain and then it does, people get upset. This scenario is somewhat similar to the pressure we face surrounding software estimates. Similarly, if I define a story as a one-point story and it takes six hours to finish, that doesn’t say much about how that estimate was derived or how it relates to future one-point stories. What I need is to have a collection of stories defined as one-point stories and observe their duration in the aggregate.
00:15:36.240
This process only works in the aggregate. A key mistake is the tendency to believe that individual point estimates hold more significance than they do, when in fact, they primarily hold validity in aggregate. Larger issues such as bad assumptions can cause this whole estimation process to veer off course. For example, I might assume a reporting feature only needs to be delivered on the web, when in actuality it also requires CSV and PDF formats.
00:16:18.460
You also have scope creep, where, once the user sees the initial product, they might say, 'Oh, you know, the columns are in the wrong order. I want to be able to alphabetize them.' Now they want features that were not included in the original scope, requiring additional estimates on time. There may also be unknown complexities that arise, like needing to integrate with PayPal unexpectedly. Lastly, there's rework; this involves more back-and-forth as developers and clients work towards defining what needs to be done together. Developers often overlook rework when they create their time estimates because of an optimistic outlook about both their own capabilities and the support time necessary to bring something to production.
00:17:04.770
As developers, we also often overlook project management and testing time. When we’re creating individual estimates, we can feel pressured to provide a lower estimate. This can happen in many ways—like a manager stating, 'This must be done by the 15th: I don't care! It just has to be done, or else you're all fired!' Or it can happen if a developer believes that it is easier to say a feature will take a day rather than discussing its true three-day requirement with a client who may not understand.
00:18:01.040
Or perhaps, if you know this project is going to be around $400,000, but you won't be able to sell it at that price, you might offer it as a $250,000 project instead. None of these scenarios represent the ideal decisions in our profession, but they happen. The fundamental issue here is that delivering a lower estimate doesn’t simplify the complexity; it won’t make it more likely for the project to be completed by the 15th, nor does it change the budget from $400,000 to $250,000 or make a feature implementation magically take one day. In reality, you're just setting yourself up for significantly more difficult conversations down the line. I don't think this approach is a great way to run projects.
00:18:53.520
To improve our estimations, we should embrace uncertainty and not chase after false precision. Adding decimal points to estimates does not yield accuracy; it merely indicates your proficiency in operating a calculator. It's vital to target the areas of a project that team members are naturally better at estimating. For instance, if I take an individual feature—say a reporting feature, I need to create a bunch of administrative reports—and think about how much calendar time that might take, there are three primary factors to consider: the actual complexity of the task, the proficiency of the developer or team producing the code, and the time they can actually dedicate to the task.
00:19:26.810
Let’s examine those three factors a bit more. We are typically poor at estimating how much time we will spend on any single task. Developers often display an overly optimistic outlook regarding that time. If you measured how much time you spend with your code editor open versus doing something else, you might have discovered that the numbers are shockingly low—at least that’s what happened to me. This number is rather consistent from week to week. Similarly, the skill of each developer is fairly stable. The development team's proficiency tends to remain constant because they’re working together on projects. Ideally, they get a bit better over time with practice.
00:20:25.320
As for complexity, developers are generally adept at estimating that. If I showed you five stories from a project I was working on, even without context, I'm confident we would be able to discern which features are more or less complicated. We could assess relative complexity, agreeing that one feature is one and a half times as complex as another. This type of reasoning is something that developers excel at.
00:20:55.350
If you think of it graphically, while almost none of us can estimate how many blue diamonds would fill an entire slide, each of us could conclude that we'd need more red triangles to fill that slide compared to blue diamonds. Most of us could also estimate that it takes about four red triangles to fill the space of one blue diamond. If we focus our estimates on the complexity of each story and not too much on time, and let that time work itself out, we align with the definition of a point in Agile projects—being a unit of complexity and not a measure of time.
00:22:06.580
If someone starts suggesting points should equate to about four hours or six hours, in a restricted context, I’ll note there will be people suggesting such things, who are either wrong or oversimplifying the matter. A point is a measure of complexity—effectively dimensionless. Teams generally figure out their own scales for points. When stories are assigned points, a team can average their productivity and figure that, for instance, they do ten points of work a week.
00:22:37.520
If everything else remains stable, there is no need to estimate the individual tasks, and it can be observed that the average will generally stabilize over time. This average is akin to figuring out how far I typically drive to the airport and how fast I usually drive. Although it won't be precise, it's fine over the long term. Over extended periods, this precision may not matter since, on average, with a consistent approach, you can estimate the time it will take to reach your destination.
00:23:38.370
However, it’s essential to maintain consistency in a couple of different dimensions. For example, the team size should remain consistent. Adding or subtracting people from the team can significantly disrupt the process. It’s also crucial to be consistent in how stories are graded. Having a relatively similar mix of small and big stories at all times promotes consistency. The hidden skill here is to take a complex problem and break it down into manageable, more or less uniformly-sized chunks. If you can achieve this, most aspects of creating reliable estimates become manageable.
00:24:21.540
However, you require some data for this process to succeed. You need at least a few weeks’ worth of data to accurately estimate the team’s velocity. This creates challenges at the beginning of a project as this initial estimate is still fraught with issues, and there remains uncertainty—especially when the client comes to you asking how long a request will take, which is often problematic to answer.
00:25:11.240
The technique I typically use at the beginning of a project is what I like to call the worst way to estimate—except for every other way I've seen. This involves 'faking' your knowledge of the project by estimating your team's speed to accomplish a point's worth of work responsibly. Don’t waste time trying to achieve precision in this step; it'll be a fruitless pursuit.
00:25:43.240
What this process yields is a range of probabilities—essentially a low estimate to a high estimate. The presentation style for these findings will depend on context, but eventually, this should result in both low and high values. This begins by taking your best understanding of the project’s scope, ideally while including the client in this process, and breaking down tasks along with assigning points to them, as we previously discussed.
00:26:15.180
Each task is designated a point value; for example, a one-point task is straightforward, while a three-point task carries more complexity or risk. A five-point task indicates a significant complexity or risk inherent. Traditionally, many teams use a Fibonacci sequence for supporting numbers, where 8 and 13 indicate rising complexity and risk. Avoid spending too much time arguing the distinction between one-point and three-point tasks—any attempt to assign an exact number can lead to protracted debates that are unproductive.
00:27:16.710
Simply round up among these classifications; this approach will help establish your consistency and counter balance your tendency to be overly optimistic. When initiating an estimate for the beginning of a project, I would label the stories about a reporting feature so that if everything goes well the development will yield a three-point story.
00:27:54.430
However, if those conditions change or the gem we are using fails, it may transform into an eight-point story. As the project progresses, I hope to refine these estimates; but at the beginning, to compensate for not having complete information, I will assign a range of values. Hopefully, without too much discomfort, you compile a range of point estimates; simply total the low numbers and the high numbers to present a project scope indicating, say, between 100 and 120 points of work.
00:28:41.490
The estimation can then be refined based on priorities, as needed. The method I use to derive a time estimate goes like this: I’ll assign a point range of five to seven hours, which combines with the point estimate to yield lower and higher hour estimates. It's crucial to note, however, that anyone, including me, who states points correlate strictly to a specific hour count is oversimplifying or misrepresenting the reality. To arrive at an overall number, an approximation based on range is most effective.
00:29:34.420
For example, a project estimated at 100 to 120 points at five to seven dollars per point, would yield a general estimate of $500,000 to $840,000. This estimating process inherently produces broader ranges; therefore, it's common to focus only on the mid-range during presentations, deemphasizing the extremes that are less likely. Additionally, these findings can be multiplied by a load factor to determine total calendar time or by a dollar amount to estimate budget. Presentation is vital, as the approach can vary considerably based on context.
00:30:16.410
At Table XI, we often utilize elaborate scripts or spreadsheets with graphs illustrating the possible estimate ranges, depending on the priority of stories. A key challenge arises when expressing that a project will cost between $75,000 and $126,000; what many perceive is simply that it will cost $75,000. One way to mitigate this misunderstanding is to emphasize the conditions necessary to reach the minimum estimate. Emphasizing everything that must unfold ideally in order to reach that target is essential: A minimum value needs to be understood as potentially unlikely.
00:31:27.710
An active, ongoing discussion about how likely the project will maintain that budget throughout development is critical. Additionally, it’s crucial to adapt as changes occur. One of the advantages of Agile methodologies is that you can examine whether a project is on track to complete on schedule, allowing for ongoing conversations around scope, rework, and shifting priorities. Estimations are not static; they're evolving documents across the project's lifespan, ultimately ensuring that everyone is headed towards the common goal.
00:32:36.540
Thank you! That was approximately 29 minutes, if I'm counting right. I have one image credit and two items I would like to mention. You can find me on Twitter at @noelrap. Feel free to connect for questions. I work at Table XI in Chicago. If you finish this and think, 'Wow, I want to hear everything Noel has to say on running software projects and Agile methodologies,' I wrote a self-published book called 'Trust Driven Development,' available at t.nlrapin.com/trdd. Normal price is $15, but for the next couple of days, using the coupon code 'Ruby2015,' you can get it for ten dollars, likely the lowest you'll see it!
00:33:06.600
I also wrote a book on testing for Pragmatic, found at prag.com/tests. Thank you for your time and attention; I hope you enjoy the conference over the next few days. I still have time for questions, I guess, but thank you!
00:34:11.210
A couple questions? Well, okay, so the question is, 'Does Agile imply that you don't have a master plan at the beginning?' I think that there is sometimes some confusion here. There’s a distinction between the ideal Agile project and how things work in reality. I find it challenging to work with clients on projects without any master story plan at the outset, even if it’s just an outline of what should occur in the first release.
00:34:36.570
The expectation of an initial endpoint, such as defining features that must be present for the first release, is beneficial; without it, it becomes difficult to gauge whether a project is on course. The Agile manifesto itself doesn’t deeply address these levels of detail, and though some people might see it as a deviation from the Agile approach, it may be a necessary condition. Regarding the next question: 'Do points get assigned to items that aren’t features, like bugs or chores?' My answer is no. While I see the argument for estimating them, I find it incredibly hard to assign accurate points to bugs and chores, as they are not user-facing stories in the Agile sense.
00:35:56.900
The math works out to ensure they hold a consistent percentage of the work, meaning if every team member spends about 25% of their time on bug fixes and chores, then that can maintain a regular rhythm. However, as the project progresses, typically more chores are handled at the start and more bugs towards the end, leading to a challenging dynamic that can complicate point estimations.
00:36:28.610
Ultimately, the estimate point should represent complexity; bugs providing user value can be considered features, but chores usually are not seen the same way. Finally, the question arose: does my process imply a direct, proportional relationship between complexity and time? My inclination would be not to overthink. In large projects, I must point out that a single point task may take a lot of time because it also has multiple verifiable elements.
00:37:15.180
In a project, once you get into later complexities—if a one-point task is surprisingly involved—it may necessitate splitting it into several individual, one-point stories that can be verified and deployed independently. While this situation is not a straightforward answer, it is often the least bad option. To wrap up, I can say that estimations may have outliers; sometimes a one-point story might take quite longer than expected, and sometimes it may only take minutes.
00:38:09.030
That's just how it works, similar to a 50% chance of rain; it’s a probabilistic affair. It's not a linearly satisfying answer, but it serves as a fitting response in many cases. Are there any other questions?
00:38:14.530
Okay, I think we’re good! I’ll be around for the next couple of days. I’m glad to talk about this topic, so feel free to approach me with questions or even to tell me where I’m incorrect, as that's also helpful. Thanks!
