Stop rolling the dice: how to estimate effort in software development
The usual disclaimer: this is a small contribution to a huuuuuge topic. It does not claim to be exhaustive. Our hope is to provide you with some useful and good-to-have-at-hand guideline. You may need it the next time you estimate some software dev. Check out the references at the bottom of the article for additional insights.
When it comes to software development estimates there are two major methods one can adopt: man/days or function points.
Function points, often combined with COCOMO or similar methods, move from the idea that estimates should be based on weighing what is in the software (inputs, outputs, functions, files, processes, etc.). This article does not focus on function points analysis, but some references are provided below.
Man/days move from the idea that estimates should be based on the time and resources needed to implement what is in the software. In this approach you would normally divide the software into sub-tasks or activities, and associate a man/days estimate to each of them. The baseline estimation normally accounts for one Full-Time Employee (or FTE), that is, how long one employee working full-time would take to conclude a specific activity (please check the note at the end of the article for some details on this).
A typical estimate would look something like this (numbers are random):
Requirements analysis = 20 Man/Days
Project scaffolding = 10 Man/Days
Functionality 1 = 50 Man/Days
Functionality 2 = 50 Man/Days
Functionality 3 = 50 Man/Days
Project / Demand Management = 30 Man/Days
Testing / deployment = 20 Man/Day
------
Total = 230 Man / Days
Moving from this baseline estimate, some additional operations are made, namely:
Even if your organization adopted Lean / Agile methods, you will still go through these steps, as they are necessary to provide the Business Dept. or the final client with clear figures. Sometimes, it is just a matter of producing a commercial offer or some sort of feasibility report.
There is nothing wrong in this method, provided that you pay a lot of attention to 3 golden rules. And in my personal experience these rules are constantly ignored or violated, even by experienced project managers.
Rule 1: you should account for planning and retrospective days
For one day a developer spends programming, there is an associated effort dedicated to planning and doing retrospective on the job done. In a typical Agile / Scrum method, there is one day at the end of each sprint cycle dedicated to retrospective and planning. The method may also include pair programming sessions and code reviews. Of course, you can skip some of these activities (and you will increase your technical debt and all its bugs), but you cannot skip all of them. Meaning, some man/days dedicated to planning and reviews must be taken into account.
When estimating a single activity in man/days this is often ignored. The estimate, at its best, will include testing and not just pure development. But that is usually it. However, your programmers will need time to understand what they have to do, organize their work, and discuss the planning and how they are doing.
In our company, we add 1 day every 10 days, because we have two-week sprints. And this will be multiplied by the FTEs involved in the retrospective and planning activity. It does not matter how you weight these activities but you have to adjust your man/days calculations accordingly.
Rule 2: velocity and number of FTEs are not linearly correlated
Let’s start from a classic example.
For a woman, it takes nine months to give birth to a baby. But if you take nine women, you will not have your baby delivered in one month. And this is the result of a simple fact: a pregnancy is not an activity that can be divided in sub-tasks and performed in parallel. It does not matter how many women you put on the task: a baby can be born from one at a time. This means that, while you can certainly have nine babies in nine months, if you take nine pregnant women there is no chance of getting one baby in one month with the same “team”.
Velocity is indeed an asymptotic function, as described by Amdahl’s Law.
If F is the fraction of an activity that can be performed in parallel and ( 1 — F) is the fraction that cannot be performed in parallel, then the maximum velocity improvement one can observe using N resources / FTEs is equal to:
Below you can see what happens to velocity when you apply this rule. We assumed that F = 0.7 (or 70% of the job can be done in parallel, which is quite an optimistic assumption).
The X-axis is the number of FTEs (N in the formula), and the Y-axis is velocity. With 1 FTE velocity equals 1, which is our baseline. By adding FTEs we observe a steep increase at the beginning. With 4 FTEs we double our velocity (note: it is x2, not x4!). However, it takes 50 (!!) FTEs to triple it. The curve becomes asymptotic. You can add as many FTEs you want, but you will not tangibly improve your speed (but you will increase your costs).
The conclusion is that you should estimate the percentage of the work that can be done in parallel, and consequently adjust your man/days calculations.
A note on FTEs, their availability, skills, and parallelism
The estimated percentage of parallel work is not dependent only on the nature of the task (i.e. pregnancy cannot be made in parallel, requirements’ analysis may), but also on the resources’ availability during the span of the project and their individual skills.
Even if, on average, a working year has 220/230 working days, your employees will not all go on vacation at the same time, and will not be sick all together. When we talk about Full-Time Employees equivalents we refer to the fact that a single developer may work 50% and another one 50% and we will count them as one developer working full-time. But we are not implying that all FTEs will always be at their desks in the very same calendar days.
Also, the term “equivalent” refers to the employed time, not to professional skills or subsequent velocity. Some developer will be faster at backend and slower at frontend. Some the other way around. Even if you can make some task in parallel, the efficiency will be affected by individual competence.
All these factors will likely increase the percentage of work that will not be done in parallel.
Rule 3: velocity has a cost, and you should consider it
Let’s get back to our pregnancy example, and let’s pretend for a second it can accomodate for a lot of parallel work, with 9 women delivering a baby in 1.3 months ( not exactly in one, because of Amdahl’s Law we just saw).
Now, if you take the total individual Man/Days that a single woman takes to deliver a baby, the result would be 9 * 30 = 270 Man/Days (or, we should say, Woman/Days).
But if you hire a team of nine women for 1.3 months the total Man/Days would equal to 9 30 1.3 = 351 Man/Days.
If one Man/Day costs you — on average — 500$, the results are clear: if you can wait for a single woman to do all the job in nine months, you will pay 500 270 = 135'000$. But if just cannot wait, you will pay a total of 500 351 = 175'500$ !
The difference of 40'500$ is the price you are paying to get your newborn in a shorter time-frame considering that velocity cannot be linearly cut.
Recommended by LinkedIn
Why does this matter? It does because the initial Man/Days estimation is always made on an single-task basis. For example, in the case of pregnancy you would have something like this:
Fertilization = 30 Man/Days
Head formation = 80 Man/Days
Legs formation = 50 Man/Days
Arms formation = 50 Man/Days
Belly formation = 80 Man/Days
Birth preparation = 9 Man/Days
Birth delivery = 1 Man/Day
------
Total = 270 Man / Days
These numbers normally refer to 1 FTE. The total Man/Days are indeed 270, and they would perfectly match the expected elapsed time to deliver.
For this reason you would be tempted to conclude that this project requires a budget of 270 * 500$ = 135'000$, missing that you are assuming two things:
These assumptions are normally wrong. You Business Dept. wants the job done much quicker, and tells you to add extra FTEs. But, as we just saw, extra FTEs will not linearly cut the effort. Thus, with extra FTEs you will end up with an estimate less than 270 days, but greater than [ 270 / FTEs ] (it depends on how much work can be done in parallel). Let’s say your final estimate is 165 days with 3 FTEs. Your final budget will then be 165 3 500$ = 247'500$. You are paying more to get things done quicker.
A warning on adding extra FTEs
When you add extra FTEs you will pay more and your velocity may benefit from it. But this is true when you are estimating the effort at the early stages of the project, or before starting a new milestone. If your project is already late, you should not add extra people in the attempt to increase velocity. In fact, you will likely obtain the opposite result. This is known as the Brooks’ Law (more on this in the references):
Adding manpower to a late software project makes it later.
Let’s be practical: an estimation tool and template
If you have followed me so far, you might have realized that you ignored some of the rules in your past estimations. I want to comfort you: I did the same. At my company, we elaborated a simple template / tool / estimator to help us avoid these common mistakes.
You can find it here. Beware that this is the result of a lot of discussions and adjustments, and it is still “work in progress” rather than “final”. However, it is a good starting point. To use it, just list your activities in the bottom (or detail) table and add individual man/days estimate to each of them. You can then input your parallel work percentage estimate (e.g. 0.7) and your dev cycle duration (to account for planning and retrospective days). The system will calculate for you the elapsed time needed to deliver and the total man/days, adjusting for the number of FTEs and applying an adjusted Amdahl’s Law (you can see more about it here, a great keynote by Kevlin Henney). As an extra, you can assign every activity to a Release / Milestone and the system will divide the calculations accordingly.
I hope you will find the rules above, together with the tool, useful in your everyday job. There is an urgent need in the software industry to improve the standards for estimating deliveries, setting the right expectations, and go beyond dice-rolled figures and numbers.
A note on activity split and individual man/days per activity
One may have two sets of questions:
The answer to these questions would require a dedicated article (or an entire library of articles…), but in our professional experience there are some common factors that influence the estimation, namely:
Clear requirements. We all know this is something hard to get, especially at the early stages of a project. It is often the case that the activity to detail specifications is one of the items to be estimated, and therefore you cannot get it beforehand. However, there is a lot of space in between knowing almost nothing and getting a decent understanding of the problem our software is meant to solve. Spend time to get as much detail as possible. If an activity is still a little fuzzy, increment the estimate.
Development lifecycle. This is a handy criteria to break activities at a good level of granularity. Take it as a rule-of-thumb: if you have a two-weeks lifecycle (as in most Agile projects), no activity estimate should exceed its number of working days (10 in this case) plus some margin of error. There are exceptions, but try to stick to this rule, and never put any activity estimate larger than 30. It means you should break it. Remember: the larger the estimate, the greater the margin of error.
Team competence. Or, in simple terms, how skilled is your team. Not the single person, but the team as a whole. You should ask yourself something like “ on average, how skilled are we in doing frontend work in ReactJS?”. Having some sort of team-skill-matrix is a fundamental step not only when it comes to estimates, but also in the broader practice of software decision making (see Francesco Strazzullo's contribution for more).
Team experience. Repetition normally improves efficiency, all other things being equal. Although Lean methods stress the power or good interactions over repetition and processes, there is no doubt that experience with a task will improve velocity and performance.
Margin of error. No matter what numbers comes to your mind: adjust it for a margin of error. This should fall something in between 5 to 10 percent of your estimate. As mentioned before: the smallest the estimate, the smallest the margin of err.
Final advice: estimation is not a solitary experience. Never, ever, estimate alone. You can choose to be very democratic and go with some planning poker game estimation, or be more conservative and just talk with one or two colleagues. But you need someone to confront with, no matter how skilled or experienced you are.
Additional references
On making better software estimates
On estimation and velocity
General project management