Software Estimation: You are not as good as you think

Note: This blog post was originally written when I was a consultant for OpenSource Connections.

I know it’s a bit harsh to insult you in the title of this post, but the odds are, you really stink at estimation. Even if you know you’re bad at it, you’re probably even worse than you think.

Don’t take it so personally – I’m certainly not saying I’m perfect at it either! But you’re among friends here, so let’s have an honest discussion.

While you will never be perfect at estimation (it is after little more than prediction of the future), you can get better at building and communicating estimates if you remember a few key reasons why you’re so bad at it. You’re still human after all, and so I guarantee you will fall into some of these traps.

Reason #1: You’re doing it alone

Too often, estimates are done alone. Typical scenarios include one person (such as the team lead) estimating an entire project, or a project manager going to a specific developer and asking them for an estimate. One person giving an estimate by themselves is one of the most unreliable ways to give an estimate, because there is no one to check you against common biases. And since all initial estimates tend to be very optimistic, you are almost guaranteed to set unrealistic expectations.

Reason #2: You’re doing it for others

When the team lead gives an estimate on behalf of their team, several problems ensue. First, you are relying solely on the judgment of one person (the team lead), and no matter how smart that person thinks they are, they are susceptible to biases when they go it alone. Second, when the rest of the team is told what the estimate for the project is, they had no voice in it, and therefore, they have no sense of ownership in that estimate. A lack of ownership means they will have no sense of accountability, and they will not go the extra mile to stick to the estimate.

Reason #3: You’re being way too optimistic

Very few of us give pessimistic estimates. For one reason, we want to please the customer. When they say they want something in two weeks, we make mental leaps in how we will build the software to try and get them what they want. Or we discount the likelihood of obstacles, and hope for the best case scenario.

Reason #4: You think you’re an expert

There’s a great chapter in the book «The Black Swan» about the problems with predictions, and one problem described is «the expert problem.» Studies have shown that experts tend to underestimate their own margins of error on predictions, because they are overconfident in their expert status. As software developers, we fall into this trap regularly when we say things like «oh yeah, that will be easy, I’ve done something similar before.» When we do remember that task last time was not as easy as we thought it would be, we often discount those obstacles we encountered as outliers. We assume that «now that we know how to do it», we won’t encounter those obstacles again, and no other obstacles will arise. We are usually wrong.

Reason #5: You’re using single point estimates

Using single point estimates makes all of these previous reasons worse. Single point estimates encourage you to go with your gut and the first number you think of, or the number that the customer wants to hear. Think how ridiculous it would be someone asked you «How long does it take to get to your house?», and you answered «Oh, about 25.6735 minutes». By including all those extra decimal points, you are creating a false impression of accuracy. Single point estimates are the same idea. If I say «it takes 25 minutes to get home», then you will expect to be there in exactly 25 minutes. But what about traffic, or other variables? A more accurate answer is to say that it takes «25-30 minutes, depending on traffic.»

A solution to estimation biases: Range estimation in Scrum Poker

Using a range instead offers a lot of benefits. By saying «that will take me 2-5 days» instead of «I can get that to you in two days», you are communicating the uncertainty that is inherent in all software development. The size of the range you provide communicates risk. And it prepares the customer for a more realistic schedule, instead of setting you up for failure when they expect completion on the most optimistic schedule possible.

Combining range estimation with the Agile practices of team estimation is doubly effective. In Agile teams we often play «Scrum Poker.» I’ll avoid the long description of it here, but in short, each team member gets a deck of cards. When you are estimating a story or a task, each team member privately chooses a card (or range of cards) from their deck. When everyone has chosen, they all show their cards at the same time.

Showing the cards at the same time combats a number of the reasons I list above, because you get a true sense of the variety of estimates across the team without introducing the biases of the team lead or architect. Those who picked the lowest or highest estimates are not discounted as outliers, but encouraged to describe to the team why they chose that number. This encourages the team to discuss the idea until they have reached a common vision for the complexity of that story or task.

Agile team estimation is great, but one big problem I still have with it is the reliance on single point estimates. That’s why we use range estimation in our Scrum Poker at OpenSource Connections, and we find it to be more effective. By holding two cards instead of one, each team member is indicating a range estimate and communicating lot more about the risk and complexity that they see in that project.

Next Tuesday (August 10th, from 11am-noon in room Asia5) I’ll be presenting at the Agile 2010 conference on «Building a More Accurate Burndown: Using Range Estimation in Scrum». If you’re going to be in Orlando, I hope you’ll stop by to hear more! You can also see my slides on slideshare.