Click here to monitor SSC

Tony Davis is an Editor with Red Gate Software, based in Cambridge (UK), specializing in databases, and especially SQL Server. He edits articles and writes editorials for both the Simple-talk.com and SQLServerCentral.com websites and newsletters, with a combined audience of over 1.5 million subscribers. You can sample his short-form writing at either his Simple-Talk.com blog or his SQLServerCentral.com author page. As the editor behind most of the SQL Server books published by Red Gate, he spends much of his time helping others express what they know about SQL Server. He is also the lead author of the book, SQL Server Transaction Log Management. In his spare time, he enjoys running, football, contemporary fiction and real ale.

Why Most Developers are Rubbish at Estimating

Published 28 April 2009 12:04 pm

Recently, a local builder handed me an estimate for some construction work on my home. Once my eyes had stopped watering, he calmly explained to me that I wasn’t paying him for what he could do, so much as what he knew. He’d been in the trade for 30 years. “That” he declared, pointing at the estimate “is exactly how much it will cost, and exactly how long it will take”. Three weeks later the work was complete, exactly as he said it would be, and I gratefully handed over a vast sum of money.

Why do so few IT projects proceed in this way? Why are most developers so poor at estimating projects? The first trick, when estimating, is to break down the project into all of its component parts. Accurately identifying all of these steps is difficult, and is a skill that comes with experience. The second task is to assign time estimates to each step. Again, the more times a developer has been through the mill, the higher the likelihood that a given task is one has been seen before, and the more accurate the estimations become.

Even then, given that software development always tends to be on the cutting edge of technology, there will always be a few steps that even the experienced developer will not have encountered before. It is at this point that a developer is likely, if pressed, to give his “best guess” as to how long the “unknown” step will take. It has been wisely observed that the inaccuracy of these “guesstimates” increases exponentially with time. If the estimate is a day, you can expect it to be finished in around a day. If the estimate is a week, it will probably be finished in 1-2 weeks. If the estimate is a month, then the developer really has no idea how long it will take.

Any step that is estimated at over 1 week is a danger signal. Any step that none of the team has encountered before is an “unknown” and therefore a risk. Instead of “guesstimating” such steps, the developer should apply solid three-valued logic. If a step is “unknown” then a NULL should be inserted into the time column for that step. Of course, thanks to NULL propagation, that means the entire project timeline is NULL. The developer must then insist on having time to prototype these NULL steps, so that estimates can be made based on hard evidence, rather than hunches and guesswork.

Inevitably, there are many “agile” techniques, such as planning poker and velocity measurements, which you can use to produce more accurate project estimates. Most of them are well-documented. However, the biggest problem still remains that the IT industry, unlike the building trade, is notoriously bad at retaining experience. In the main, companies hire and pay developers based on what they can do (write code) rather than on the knowledge that comes with experience, such as effective project estimation and management. If these skills were properly valued and rewarded, the industry would do a better job at retaining experience in the role, and the development team’s estimation skills, like fine ales, would refine and mature with age.

Cheers,

Tony.

14 Responses to “Why Most Developers are Rubbish at Estimating”

  1. Robert Seber says:

    That’s pretty much spot on. My first reaction to “how long will this take” now is “how long can I have to work out how long it will take”? This can be a problem with some customers who expect a quick answer though.

  2. Arles says:

    I concur, and please let me add three more contributing factors why developers are time-challenged when it comes to estimating a project’s completion:

    1. Rubbish requirements beget rubbish estimates.
    2. Moving targets constitute missed deadlines.
    3. A predilection to overvalue one’s skill and undervalue a project’s complexity leads to delayed deliverables.

    The first two are outside forces and the third one is self-inflicted.

  3. aj_raghuram says:

    Excellent article & choice of topic! I believe there are two more factors that may make a fairly new developer give over-optimistic estimates:

    Peer pressure – One may always be tempted to give estimates that are “better” than a fellow team member hoping for extra brownie points.

    Managerial pressure – A team leader/ project manager may want to prove to the upper management that the project that s/he leads gets done quicker and so would ask the developers to revise their estimates. After a couple of times, the developers will by default give conservative estimates in order to avoid the revision exercise and the project eventually goes out of schedule.

    I believe a team leader / project manager should have the necessary skills & experience to come up with a reasonable project estimate and encourage the developers to estimate their individual tasks realistically. This in the long run will help the developers in giving correct estimates instead of favorable-but-unachievable ones leading to project failure.

  4. kenzo says:

    < >

    Sorry, I cannot agree. The first trick is to understand basic principles of project management (that are almost always ignored in software projects):

    1: A business level goal/target – or more often demand, is not an estimate. An estimate is an attempt to predict the future. The future doesn’t care what anyone *wants*, even the CEO of Chrysler :) .
    2: The only people who are highly accurate at predicting the future are people who have tons of experience doing EXACTLY the work that is being contemplated, with nearly identical work environment, tools, and organizational structure. ANY deviation from the above mapping between historical experience and new project requirements represents an UNKNOWN.
    3:UNKNOWNS are the thermo-nuclear bomb that will torpedo almost every prediction of the future. This should be obvious, but because it is anethema to business demands, organizations pretend they aren’t significant (think OSTRICH).
    4. UNKNOWNS can be categorized into two types, those for which a range of possible outcomes is predictable (type 1-for example rolling dice-finite and limited possible outcomes), and those for which a range of possible outcomes is not unknowable (think of a programmer being asked to accomplish something they have never done before-infinite possibility for unexpected problems). Many predictive methods for unknowns in software estimating are based on type1 calculations (Monte Carlo simulations are an excellent example) when almost all unknowns in software development are of type 2.
    5.NO estimate is worth the name if it has not mitigated and accounted for the signficant unknowns. And no project can do that if it doesn’t detect, document, and analyze them in a structured and rigorous way PRIOR to attempting to predict the future (the estimate).
    6. Finally, we get to Tony’s first step…

    Just my opinion based on 25+ years of manufacturing experience (where failure to predict the future cannot be swept under the rug as it so often is in software dev.) and 10+ years of software programming/development experience.

    Ken

  5. kenzo says:

    sorry, typo in point #4. Should read:

    . UNKNOWNS can be categorized into two types, those for which a range of possible outcomes is predictable (type 1-for example rolling dice-finite and limited possible outcomes), and those for which a range of possible outcomes is not KNOWABLE…

  6. kenzo says:

    aj_raghuram points are MAGNIFICENT.

  7. puzsol says:

    Joel on Software (Fog Creek) has written two excellent articles on the subject:

    http://www.joelonsoftware.com/articles/fog0000000245.html
    http://www.joelonsoftware.com/items/2007/10/26.html

    For me one of the main problems with projects delivering, is that quite often it is not the people doing the work that do the estimates:

    Situation A: A salesman who has never programmed in his life sells the company’s abilities, agrees to an amount, takes his commission and then promptly forgets about the project.

    Situation B: A programmer who “graduated” to management bases the estimates on their experience and abiltities and agrees to a timeline… but it’s not him doing the work, not his experience that determines how long it will actually take – it’s the newer programmers (who haven’t “graduated”) who don’t have the experience (so there are really more nulls in the list than the manager thinks)

    Situation C: You actually take the time to ask the people doing the work, and yes their estimates are not accurate (at least to begin with), or they hate doing it, and you can’t get your quote into the customer on time.

    Joel’s articles should give you some idea how best to work with Situation C, which if you ask me is the only real way to do it… I mean, if you took the builder’s quote, then hired someone else, would it have been done the same? Same quality, same cost same time? I doubt it.

    Happy scheduling everyone!

  8. Pallavi says:

    I loved the wite up and the topic is really good!

    I think there are many dimentions to this topic. What I would like to add as per my experience is normally developer do not count time required for error detection and correction. I would say that there is always a temptation to commit to finish work even before actual time required. We should “Under commit and Over Deliver” but practically opposite is done.
    Also, much time is needed to know the actual requirements, most of the time requirements are not undersood/ given correctly and then planning based on that always failes. So “Requirements are not given, they should be taken”.

    I believe that cost is always related not to the work done but the way it has done, thats quality of wirk :)

  9. r_honey says:

    It also has to do with our developer’s mentality, and how we approach the projects. Many people will agree with this at the back of their mind, that we developers tend to present overly ambitious timelines.

    A client approaches us with an assignment. We have a quick look over it, we estimate cost & time (where the time is already arrived at by estimating stretched working hours in majority of the cases), and present it to the client.

    The client says, can’t it be done quicker, in less cost? No one likes to compromise on cost, and so we say, there is not much room on the cost side, but we would try to cough up the solution in lesser time. The client asks, how much less? We quickly go over our current assignments in our mind, decide to work overtime to get them finished, and so predict a sooner beginning & ending time for the new project (forgetting that our initial estimate was already arrived by stretched working hours, and squeezing every bit of our team productivity).

    The client agrees. And now, there are all sorts of problems. We have requirements for the projects we were working on currently (that we decided to finish before schedule). Someone else asks us whether we can quickly have a look over the code we had delivered to him/her sometime earlier, and so on…..

    This has has to do in part from clients expecting miracles from IT & IT professionals. Like the construction work, we humans have been doing it for centuries. But it hardly has been 5 decades as regards IT. Our own over-squizeed estimates coupled with client expectations, the ultimate recipe for disaster!!!

  10. randyvol says:

    Your analogy is flawed with respect to the world I live in. Were you living in the world I live in, the you would have told the contractor something along the line of “I need a house”. The contractor would have responded with something like “Do you have plans of the house you’d like me to build?” Your response would have been something along the lines of “That’s what I hired you to figure out. All I know is I need a house”. The contractor would have persisted in trying to ascertain exactly what it is you want, and during these cycles of discovery you would have continued to become more and more irritated until finally, the contractor would have realized that you refuse to take the time to sit down and really think through what it is you want built for yourself. The contractor, tired of being beaten on for not delivering and finally armed with enough information to take a swipe at delivering something that approximates what you have told him you want would have built a house. At that point you would have told him that “This is useless! It is nothing like what I wanted! Just another example of why home builders have a bad reputation – they simply cannot do the job they were hired to do!” Yep, that must be it, I’m just the only contractor on the planet that: a) doesn’t have the gene that equips him to read the minds of his customers; b) is too stupid to realize that he lacks the mind reading ability that everyone else on the planet enjoys…. Then again it may be that the person wanting a house isn’t really sure what they want; or is too busy with their lives to be bothered with the mundane process of transferring knowledge to the ‘hired help’; or is unwilling to admit to themselves they have no bloody idea what it is they want and are just hoping someone will work a miracle and solve their need without them having to do the ‘heavy lifting’ of detailing it out. Then again, if you don’t ever issue the builder a detailed specification of exactly what it is you desire, you can always stiff him on the bill for not delivering what it is you claim you wanted.

    Once we get that mess cleared up, then we can live in your world and I’ll agree that we need to do a better job. But, having detailed out the world I live in, I have to say that just like the builder in your example, I’ve been doing this for long enough that if I get good specs, I’m pretty spot on at delivering on time, on budget.

    Randy V

  11. Tony Davis says:

    Thank you for all the great feedback.

    Inevitably, I “oversimplified” the issues when I wrote this but, judging by the comments, there is at least some agreement that NULLS are the killer of project estimation. You have to prototype the unknowns or you are just guessing – and these guesses are likely to be influenced more by what you think the customer/manager wants to hear than anything else. As r_honey suggests, this road leads to 18 hour days and developer burnout.

    Some companies seem to thrive, for a short time at least, on pushing through a stream of “death march” projects, where the stock answer to “how long” question is “2 days”, regardless of what the request entails, and developers simply work relentlessly “until it is done”. In these circumstances, estimation skills are basically wasted.

    And, of course, randyvol is right: accurately breaking down and estimating a project relies on there being a shared understanding of exactly where it is you are trying to get to in the first place.

  12. Robert Seber says:

    I quite like using the “building” analogy myself. If you ask someone to build a brick wall they will be able to quote you pretty accurately. Most software projects, on the other hand, are more like building unique buildings such as the ones for the Olympics. They are unique and therefore guaranteed to run over-budget.

  13. BuggyFunBunny says:

    I was going to skip this one, since Tony said everything I’ve ever said on the subject, but this has inspired me to ruminate:

    Most software projects, on the other hand, are more like building unique buildings such as the ones for the Olympics.

    While I agree in principle, I don’t believe that this, alone, makes estimating impossible. Difficult, certainly. After all, most architects (the ones who design and build buildings) don’t keep designing the same building in perpetua, but, at least attempt, to make each project unique. On a percentage basis, they succeed at a much higher rate than software architects. In general, building architects don’t create new material or methods in the course of design (Pei and Gehry notable exceptions); we do so more often.

    (Here is a paragraph from the WikiPedia of Gehry: Gehry has gained a reputation for taking the budgets of his clients seriously. Complex and innovative designs like Gehry’s typically go over budget. Sydney Opera House, which has been compared with the Guggenheim Museum Bilbao in terms of architectural innovation, had a cost overrun of 1,400 percent. It was therefore duly noted when the Guggenheim Bilbao was constructed on time and budget. In an interview in Harvard Design Magazine Gehry explained how he did it. First, he ensured that what he calls the “organization of the artist” prevailed during construction, in order to prevent political and business interests from interfering with the design. Second, he made sure he had a detailed and realistic cost estimate before proceeding. Third, he used CATIA (Computer-Aided Three-dimensional Interactive Application) and close collaboration with the individual building trades to control costs during construction.)

    By way of specific example, from a field that keeps my attention. Boeing set out to make a completely new aircraft, the 787. They broke the mold in two important ways: choosing to build the aircraft mostly, and particularly the fuselage, from carbon fibre composite, and they chose to outsource virtually the entirety of manufacture. Not surprisingly, the aircraft is years behind schedule. Some believe that it will end up like the Comet.

    In the continuing critiques of Boeing’s (lack of) performance with the 787, some points continually arise: they didn’t have a handle on manufacturing process for the fuselage sections, and they didn’t have a handle on the project management requirements. Previously, Boeing had built aircraft in house with their employees. With the 787, they chose to outsource core competency (design and manufacture of aircraft subsystems) which they themselves hadn’t yet acquired. Sound vaguely familiar?

    Critics of Boeing come to some obvious conclusions: in order to hire out for cheap labour, Boeing sacrificed quality; Boeing did not initially have, and arguably still hasn’t, a grasp of the engineering requirements of carbon fibre layup for fuselage tubes; they didn’t establish stringent management controls over outside contractors. By assuming that testing and computer simulation of carbon fibre layup would scale reliably to fuselage sections, Boeing simply assumed that the 787 could be built to the desired schedule.

    It is possible to build a unique building, such as an Olympic venue, iff there are records of scalable activity of sufficient granularity. That’s where Boeing failed: they extrapolated optimistically. We in software do that as a matter of course. We also are lax in defining assembly line activities (we don’t do assembly line work, now do we?) and the record keeping associated. We’re artists, don’t you know. It’s never the same, so why bother keeping records? There was a time, years ago, when Function Points were the activity to track and use for subsequent estimation. Failed and faded. We need the new Function Point, whatever it might be.

  14. r_honey says:

    There are many things I believe people are forgetting (or ignoring) here. First of all, we can have analogies. But you should remember, that we have been constructing buildings for centuries, or even much beyond that. (I still cannot imagine how people without the modern mechanical support were able to arrange such large blocks to create a Pyramid, that might be a challenge to create even today with such state-of-the-art computer-aided mechanical support available).

    The second important point I think is important is that, remember an architect or builder is doing a job he/she excels in. This is his/her field.

    Yes, IT is our field as a developer, but majority of the projects we build are meant for use in non-IT fields, for which we do not have any intrinsic knowledge. We rely on a domain-expert to explain us the intricacies of his/her field. Our first important task as an analyst is to get into his/her shoes, and build a good understanding of his field, its environment, its rules etc. etc.

    No matter how much we stress on S/W Engg. models to extract maximum information from the domain-expert, but we as humans tend to think of such things being implicit in the field to which we work in daily (to the extent that my english sentences often end in semi-colons… huh)

    The builder has no such phase in his project. The client tells him information that he faces daily. He builds buildings with material that dont change so often (atleast not so often & fast as the .NET framework, the Sql Server, Silverlight, Javascript, ExtJs, and what not). After all, how much has brick & mortar changed over the years?
    And remember there are no breaking changes either. Atleast, I have not heard of a breaking change in the way concrete is prepared, or plaster is applied, that makes it unsuitable to apply on the existing app (a building is a builder’s app, that’s what he creates :) )

    And finally, its also about competition. Almost everyone would agree about the cut-throat bids in IT projects. So, if Company 1 has presented a timeline for Project P1, many other companies would approach the client. Some of them are really not interested, and hence provide schedules impossible to achieve, seeing which Company 1 has to present a slashed schedule. And the other case is a big fish eating a small one, where a larger company presents a lesser schedule relying on “Hire for Fire” contract modes to complete the project.

Leave a Reply