Recently Kirk Gray wrote a piece — more of a plea really — titled Software Estimation is Hard. The problem at hand is that there doesn’t seem to be a silver bullet that delivers accurate software project cost estimation. Software cost estimation (and here, I mean “cost” in the sense of effort, time and money) is hard, as Kirk points out:
“The problem is that, especially in a small shop, we:
a) Don’t have comparable projects that can be used to provide historical data, especially for partner projects that are often more one-off than core product development. I can say with some confidence how long it takes to add a new Struts 2 action and mapping – that part is easy, but what about the elements on the page? What about the logic?
b) Don’t have the infrastructure in place to really record time spent on any one task, and often find that if we did, it would be inaccurate because we wear so many hats, each hat being taken on and off randomly throughout the day/week.
c) Can’t afford just yet to spend a lot of time making exhaustive estimates, because that’s time that won’t be spent in construction and testing.”
This is probably the most common problem facing software teams today and, quite simply, the root cause of most project failures. Even projects that are regarded as ultimately successful suffer from this issue, as evidenced by frequent project overruns, software delays and adjusted delivery goals. And yet, we have the tools at hand to solve the problem: Function Point Analysis, Proxy-Based Estimating, Evidence-Based Scheduling, and similar tools offer the basic software estimation processes needed to accurately predict the overall cost of a project.
The problem is not with lack of process: It’s with lack of proper adoption. As Kirk points out, in his own situation a host of pressures prevent dedicating the necessary effort, time and resources to do the job right. Put bluntly, relatively unseasoned teams often lack the experience and wherewithal to set up the right processes and make the right investments. These “investments” typically translate into very real costs to the business, longer development schedules being among the most obvious. The pressure applied by a business to set a delivery date now, before enough data has been gathered, can be daunting. It often takes a combination of two things to successfully pull off software cost estimation: First, a team that has the ability to make accurate statements regarding a software project, and second, a business that is willing to listen to that team.
An experienced team
During my years performing project auditing I’ve noticed a trend throughout Silicon Valley, particularly among startup companies. This trend involves creating teams from remarkably bright individuals that have very little formal training or experience in software development. Other sectors, including large scale enterprise and the military, take a different approach: Assembling teams that have the senior leadership necessary to create an effective team. This means a formal understanding of development process as well as experience — because, quite simply, technical knowledge without experience generally results in missteps. Those experienced, senior dinosaurs are actually quite valuable in providing guidance and mentoring to the development team as a whole. But, again being quite blunt, these senior experts (SME’s or Subject Matter Experts) support their teams in a very effective way by providing the voice of experience and authority to the business. This is the bit about “standing up to the business,” and is critical when the business tries to step in and force decisions that just shouldn’t be made yet.
A listening business
Perhaps equally important as building a team with experienced, authoritative leadership is having a business that recognizes the value — and expertise — of its team. That has to extend to a willingness to listen to the team, and take the team’s recommendations at face value. I’ve personally walked away from more jobs than I’ve taken where it was apparent that the business believed in a purely “top down” management style, not giving the team the resources or time it needed to make the right decisions to deliver a quality software product.
Let’s make this concrete: What does “listening to the team” mean? More than anything it means listening to the needs of the team and addressing them. Kirk points out that they have a lack of expertise in software cost estimation: Hire a consultant to work with the team for three to six months to address this need. He points out a lack of infrastructure: Buy the right tools and give the team time to put them in place (they aren’t that expensive). Most appalling to me, he points out a lack of time to invest in estimation because that means the team isn’t coding: Give them the time to do their job right. The long term benefits will be realized ten-fold through accurate software estimation, greater reproducibility and improved software quality (you know how the software is always late or ships with too many bugs in it? This is the root cause).
Strategies for success
For projects that don’t have a body of work product to use for historical data, look outside the project. Unless the team consists of entirely junior programmers with no experience under their belt this won’t be hard to do (the formal nomenclature for this is wideband delphi, an easy-to-use and surprisingly accurate method for discovering historical data where none seemed to exist). The basic process is simple: A moderator and an estimation team with three to seven members is selected for two meetings run by the moderator. In the first meeting the estimation team creates a work breakdown structure (WBS) and discusses assumptions. After the meeting, each team member creates an effort estimate for each task. The second meeting is the estimation session, in which the team revises the estimates as a group and achieves consensus. I’ve had excellent success using this process as a basis for arriving at reasonably reliable estimates, but it’s important to add one more step: Make sure that the programmer that takes up the given task agrees with the estimate, and understands the task in full. Sometimes the group estimate will be more aggressive than an inexperienced programmer is willing to take on; likewise, sometimes an experienced programmer will know it can be done more quickly. Be conservative, and defer to the individual programmer when appropriate.
In regard to making time for the underlying effort in software cost estimation, one of the most common excuses I hear is “we don’t have the tools to record time, our team wouldn’t agree to it, and if we did, it would be inaccurate because we wear so many hats.” Frankly, this is just procrastination. First of all, read Joel Spolsky’s article on Evidence Based Scheduling. It’s an excellent introduction that clearly explains why wearing a lot of different hats, being distracted by unplanned tasks, and having a chaotic environment just doesn’t matter. A good system for measuring velocity is going to understand and accommodate those realities — in fact, the entire purpose for estimating velocity is because of all these unexpected, unplanned interruptions. If we had none, our estimates would be perfect, wouldn’t they? So, where does this excuse lead you but down the same, inevitable path: Lots of misunderstandings with the business, failed deliveries, slipped ship dates, poor software quality and an awful lot of late nights trying to get incomplete software out the door. Stop the addiction: Do the right thing and spend $200 on FogBugz or, if you want to go industrial choose my favorite, Atlassian JIRA (it’ll cost you a couple thousand dollars, but it’s probably the best investment your department will make, period). If you’re looking into JIRA, check out the Greenhopper plugin.
And finally, if you feel you just can’t take the time to estimate and manage your project, keep in mind this is the same as saying “we just can’t take the time to do it right — so we’ll hope for the best.” This is a classical problem for many Agile organizations that focus entirely on the short-term while sacrificing long-term strategy. It’s a situation that leads to a perpetual, self-feeding problem, and it gets worse not better. It’s time to break the pattern and make the decision to fix it.
Probably the most important piece of advice I offer team managers is simply this: “I don’t know” is OK. The worst thing you can do is commit too early. When the business challenges you for a timeline and presses for a hard and fast ship date, the best thing you can say is “I don’t know.” The natural follow on question is going to be “fine, what do you need to find the answer?” Stand firm and insist that you need the time, tools and resources to accurately estimate the job. Cite past performance — or past lack of performance, failed deliveries and customer service problems, if need be. Ask the business “do you want more of the same, or do you want to improve?” Ultimately, it’s up to you as the team lead, Scrum Master or project manager to make this happen.