A simple multiple is not the answer. This is an incredibly well studied problem. At the basis of all of the reasoning is the inherent uncertainty of the task. Estimation is only accurate when all the details of the required task are known. For example, you can estimate how long it will take to assemble 5000 widgets if you know the average time it takes to assemble 10 widgets, and that time is consistent between executions.
With programming, there is no atomic unit of work that is consistent between projects, thus highly accurate estimation is next to impossible. What seems similar from a conceptual standpoint is often completely different depending upon the input factors: lanuage used, experience of the team, specific project requirements.
This problem has given rise to many new ways of working on programming projects. Solving this issue inside a company means building understanding of the inherent uncertainties with management and stakeholders. Solving this issue as a freelancer means inflating estimates and hoping and praying that you can keep the customer in check, because "we can't estimate that" just doesn't work for consultants.
Watts Humphreys made the point that any seriously constructed estimate is still more useful that no estimate at all.
The SEI have claimed that organisations with well-defined processes working on problems where they can control many factors such as language and tool choice, with stable staff and so on can in fact get to within 5% variance (sorry, can't find the source right now).
If your software shop is working on same-y tasks (and many do), using historical data as a guide to future performance is perfectly reasonable.
If your software shop is working on such same-y tasks that they can be precisely estimated, your software shop is leaving immense amounts of money on the table in the form of something repetitive that ought to be refactored and automated away.
Beyond question, there are shops that operate this way. However, trying to translate the lesson on how to schedule out of this context into the context of a higher-end shop that can and will so automate is not possible.
Sure, automation is a smart move. But by same-y I mean:
* We need a website. It will have these 5 pages. How long will that take?
* We need an iphone application. Here are the dozen or so screens and a few pages of what it needs to do. How long will that take?
Automation is actually a diminishing multiplier on the estimate -- it doesn't make estimation itself as a problem go away. If anything it reduces variance, making estimates more useful.
Take the website example. It might have been that originally you did everything by hand but now you have a collection of templates, generation tools, snippets and so forth. This has brought your development time down substantially -- but you still get asked for estimates. What used to take weeks might now take days; fine.
But the end product still has some degree of size and therefore takes some amount of time to develop, even if that size is a hundred thousand lines of code and the time is however long it takes for your code generator to spit out Generico Inc's new website from the you-beaut inhouse system.
If you are making a 5-page website and you have effectively no code to write, the reason you can make an estimate cleanly is that it is no longer programming. At that point you are estimating how long the content will take to produce, get signoffs, graphics, etc, and those are quite estimable. The worst case you're looking at there is CSS incompatibilities. An iPhone app that is just layout out some UI elements and doing something very simple (or just pulling up existing webpages) will similarly mostly not be programming.
Of course if you remove most or all of the programming elements, the programming no longer screws your estimates up.
There's a whiff of No True Scotsman in my post here, so let me nail this down a bit more and say that in this case I see programming as requiring some sort of logic in it. Laying out a form for your iPhone app may occur in a programming tool and with another definition may even be part of the "programming", but for the purposes of this post I am not including that. And while having, say, a conditional panel that only appears when certain things is true is indeed programming, if your project is dominated by content and forms and just has a trace of logic, your exposure to the chaos of programming is limited and negligible. It's a continuum, of course, not a binary thing.
I think it was good of you to cite No True Scotsman.
I still think any estimate, even hilariously broad ones, are useful. Watts Humphreys put it this way: when is an estimate going to be most accurate? When you've just finished. When is it most useful? When you're just starting. In between is a tradeoff between usefulness and accuracy. You accept that estimates are inaccurate -- that's why they're called 'estimates'. But even if you narrow estimation variance from +/- 400% to +/- 200% over time, that is still a valuable improvement.
When I'm working with clients I explain the cone of uncertainty and so far they've all been understanding of the fact that software work contains profound uncertainties that other lines of work won't.
In my anecdotal experience, at large corporations with well-defined processes, the 5% variance is the result of huge time estimates. Then you make sure you don't implement any faster than the estimate. (Even if it turns out you could have done it half the time.)
I was thinking something similar. In my anecdotal experience [1], large corporations can control variance by normalizing everyone's velocity. For example, in order for me to finish a feature, I had to write a document describing how I would solve the problem and convince my manager and tech lead that it would work. After spending about two months getting my plan approved [2], they let me implement it, which took me about four hours. At that ratio, the actual development time is practically negligible, so it's easy to have low variance. The side effect is that you also have very low productivity.
Imagine the opposite scenario, where they let me implement it first and then approve it if it works. Let's say it takes me a couple days to finish it (this time has to include some of the time that it took me to figure out how to solve it originally). Now the feature is getting done in a matter of days instead of months. The flip-side is that you can have higher variance, because if there's a problem with my initial implementation, that adds more time to fix the problem. In this scenario, the time it takes to finish features is more dependent on the engineer's ability than the time it takes to pass through the bureaucracy, and there's going to be more variance in the abilities of individual engineer.
[1] I worked at Microsoft as an intern in the summer of 2009.
[2] There were many days that I literally couldn't do anything except wait for clearance to go forward.
You're thinking of Parkinson's Law -- another well known finding in all industries.
A lot of comes down to how estimates are being used. A lot of places use estimates as the plan rather than ... an estimate. And that's where stuff goes awry, because as you say the incentives are to stick to the plan even if it's stupid.
If I asked you how long it would take you to write a program to sum a list of 1,000 numbers that varied from 0 to 20000 after reading them from a file, I'm sure you could give me an answer that would be very accurate on the time frame of days.
Most programming is applying known solutions to known problem types and can be estimated quite easily if people take the time to account for everything The problem I see is that most of the time, they don't. They ignore how long it will take to test it, or to update the documentation, or that they don't quite understand the problem, so they'll have to find Jean in Accounting to explain Requirement 2B to them, etc.
In a healthy development environment, you can separate the "problems we know how to solve" from "the problems we don't know how to solve yet" and deal with them separately.
> If I asked you how long it would take you to write a program to sum a list of 1,000 numbers that varied from 0 to 20000 after reading them from a file, I'm sure you could give me an answer that would be very accurate on the time frame of days.
Is a console utility sufficient, or do you need a GUI?
Is the file we're reading in a specific location, or do we need to be able to specify the location at execution?
If using a GUI, can the user type the path, or do we need to provide a specific file picker UI?
What platforms is the utility expected to run on?
Can we assume that the file is in the format we expect, or do we need to perform any validations?
If we find that the file is invalid, what action should we take?
Does the output of the tool need to be machine readable?
Yes, I'm being a bit silly, but that's kind of the point. Even those of us who know how hard the problem is often oversimplify matters. That's the root of the challenge, and the reason that even seemingly simple projects are difficult to estimate accurately.
To compound this problem, you'll often find that the customer will answer the set of questions you propose differently on the first day than they do two weeks in to the project. "The situation is fluid", so to speak. All of this adds up to a lot of uncertainty in the estimating process.
You are, of course, correct. And in the absence of any context, I'd say those questions must be answered. Note that the time required to get them answered (requirements gathering) should also be estimated.
However, in most cases, there is a surrounding context to that set of "requirements" I gave, so you'd probably be able to answer accurately.
The overall point is that you still have some idea of (a) how long it will take and (b) what level of confidence you have in the estimate.
I hate to go back and forth, but I spent 6 years doing freelance work, and I hate to see people repeating my mistakes. They can be costly. I would "probably" be able to answer accurately, but we're definitely talking probabilities here.
I learned that estimating was really a negotiation of risk. When I put an estimate down on paper, I'm saying to a customer, "I accept responsibility to deliver the product for this price." I'm taking on the risk if I don't, so I pumped the prices up. Also keep in mind that "the product" is what's in the customer's mind, not yours.
At some point I decided that I needed to have the "risk negotiation" conversation with all new clients in very candid language. Once a client is aware that they're paying for you to take the risk, they're usually willing to buy some of it back. Customers who were willing to buy back some risk always turned out to be better to work with than customers who wanted to offload all the risk to us. It turned out to be a marker by which I would steer the course of client development.
If people were actually to "account for _everything_" they would have, by definition, solved the problem already.
So the time to make the estimate would be 100% of the project time, and the estimate would be 100% accurate, since all the work would be done.
That is reductio ad absurdum, but the underlying problem is, as Donald Rumsfeld would say, there are "unknown unknowns" which cannot be accounted for ahead of time.
No. Accounting for everything is not the same as doing everything. You can tell me it will take a week to implement a feature and it may take you half an hour to come up with that estimate. It doesn't mean it took you half an hour to implement the feature!
Then I wouldn't be accounting for everything, and my estimate is most likely wrong.
I might not account for the cat tripping over my network cable.
I might not account for a server unexpectedly crashing.
I might not account for the office being too noisy to think in on Tuesday.
I might not account for Amazon having EC2 issues
I might not account for a DDoS attack on my DNS provider.
All of those things impact the time it would take, and the only way to fully account for everything that could happen would be to actually complete the project. Of course, if the project was identical to one I've done before, I could get a lot closer to an accurate estimate, with fewer real unknowns, but in 20 years of software, I've never had to create an identical solution. Because if it was an identical problem, I'd just use the previously developed software.
Exactly, the very reason why it is near to impossible to estimate time in computer programming is because programming has always, at least a little, to do with a research of a solution to a logic problem. And finding a problem solution means finding something unknown from the point of view of the programmer. Ultimately, the programmer don't even know if she will succeed in a finite time.
So we can't say either: "programmers are bad to estimate time" because nobody is expected to do the impossible.
It would be the same to ask: "can you estimate the time mankind will need to find the cancer vaccine? You can't? You're bad..."
Yes, it's hard to estimate time for problem solving. But the author has a very good point that you shouldn't neglect an hour for basic testing of a fix after solving it. And that you should take a moment to consider what classes depend on the code to both hinder changes and make you spend more time thinking about side effects.
With programming, there is no atomic unit of work that is consistent between projects, thus highly accurate estimation is next to impossible. What seems similar from a conceptual standpoint is often completely different depending upon the input factors: lanuage used, experience of the team, specific project requirements.
This problem has given rise to many new ways of working on programming projects. Solving this issue inside a company means building understanding of the inherent uncertainties with management and stakeholders. Solving this issue as a freelancer means inflating estimates and hoping and praying that you can keep the customer in check, because "we can't estimate that" just doesn't work for consultants.