Unit Testing Myths and Practices

We all understand the value of Unit Testing, but how come so few organisations maintain unit tests for their in-house applications? We can no longer pretend that unit testing is a universal panacea for ensuring less-buggy applications. Instead, we should be prepared to actively justify the use of unit tests, and be more savvy about where in the development cycle the unit test resources should be most effectively used.

Despite the pervasive dictum within software engineering that unit tests must exist for all code, such tests are little-used for the development of enterprise applications. In over fifteen years of consulting, I can count on the fingers of my hand the number of organizations maintaining unit tests for their applications. Why is it that so many organizations ignore such a popular software practice? In order to answer this question we first need to explore the myths and real world practices of unit testing, and then go on to describe the points within development where their value is well-established.

Myths

Two of the more generally-held myths that prop up the dogma of unit testing concern the professed benefits. The first myth, and most loudly claimed, states that unit testing inevitably saves money. The second myth, and zealously professed by many developers and engineers, promises a reduction in the bug count.

The Saving Money Myth

The idea that unit testing inevitably lowers the cost of application development rests on the reasonable assumption that fixing a bug as soon as possible saves money. Graphs such as the one below typically compare two application’s costs to support this claim. Each solid black line represents an application’s total cost over time.

1417-image001.jpg

The application labeled “No Unit Tests” starts out with lower expenses than that labeled “Unit Tests” as shown by the red line. Not a surprising fact since someone either coded or didn’t code unit tests. Over time though the “No Unit Tests” application costs more money because of increased support, bug fixes, deployment, lost customers, etc. According to this assumption about cost-savings, organizations can even calculate the “Breakeven” point at which their unit testing pays off. And at any time past this point they can find the total savings earned from writing unit tests as shown by the green line on the far right.

There’s a problem with the underlying assumptions supporting the above cost profiles. They do not factor in the financial repercussions of a delay in the delivery of today’s enterprise applications.

The first problem stems from the increasing importance of enterprise applications to an organization’s profitability. For example, it is not unusual for a company to build some widget that’ll save identifiable staff several hours a day. Delaying the widget’s release in order to write unit tests causes not only angst but measurable, lost productivity. I have rarely seen a developer convince an end user that adding unit tests justifies delaying a much-anticipated feature just to prevent of a few potential bugs.

The second problem is the cost associated with the consequences of writing tests. Time expended coding unit tests keeps other features un-built and idling on backlog. It disrupts the development process. The process of implementing unit tests cannot be readily scaled up to prevent backlog because, for most organizations, there are only one or two folks possessing the domain knowledge to add new or enhanced features to a specific application. When these developers code unit tests they are not coding new stuff.

When we update our cost assumptions for Unit Tests with only these two omissions, our graph alters as shown below. The “Unit Tests” application’s total cost over time line shifts up with dramatic implications.

1417-image003.jpg

Suddenly the savings advantage of the “Unit Tests” application shrinks. To make matters worse, it also takes longer for an organization to realize these diminished savings as shown by the rightward movement of the “Breakeven” point. No wonder that any IT management that takes a financial vantage will wish to minimise unit tests. Adding them to a project may not save an organization as much money as it had previously calculated.

The Reduce Bugs Myth

The myth of the ‘reduce bug count’ originated in the early days of software engineering. It sprung from the reality of working with the tools and technologies that were available at the time.

When enterprise applications were built 20 years ago with C and C++, unit tests helped to minimize the number of bugs escaping into the wild. Even the crudest test caught defects that were attributable to undeclared variables, mixed case method names, missing header file, calling functions without parenthesis, etc.

Enter today’s improved integrated development environment (IDE) tools and managed memory runtimes, and many C/C++ lifesaving checks handled via unit testing became superfluous. The technologies of choice for enterprise applications, such as, .NET and Java, placed many golden oldie bugs on the extinction list. Unit testing was no longer required to catch all those pesky technical code flaws.

Improved technology further eroded unit testing as the premier bug management tool by promoting software construction practices which leveraged multiple components to create an enterprise application. Dynamically cobbling together different data stores, libraries, web services via dependency injection, configuration files, and plug-ins spawned an entire new breed of defects for which unit testing proved marginally effective.

The below graphic depicts a simplified scenario of how “SomeMethod” might pull from several component’s to complete its task.

1417-image005.jpg

In such a scenario, it is only some form of integrated, or user acceptance, testing that will guarantee “SomeMethod” behaves appropriately. Properly-constructed unit tests only ensure that a component works as coded. Service X and Service Y may pass their respective unit tests but both may not produce the desired results when end users execute “SomeMethod” within a specific context.

Practices

If the preceding discussion leaves one believing that unit tests only provide mythical benefits, real-world experience suggests otherwise. In several circumstances they play a critical role in helping to deliver enterprise applications. This section discusses a few them.

Building Libraries

Even the most powerful frameworks and runtimes omit features that are required by an organization. Because of this, organizations wisely create their own custom libraries for all their developers. As these libraries change of over time, whether via refactoring or feature updates, unit testing is essential to quickly validate the change and can save an organization a great deal of time and money.

Enhancing Technology

Sometimes the required technology cannot work its magic alone. Unaided it allows developers to write buggy code all too easily. In such situations enhancing the technology with unit testing comes to the rescue.

Developing browser-based User-interfaces is an example of this situation. Despite the existence of such powerful plug-ins for building Web-Based User-Interfaces as Flex and Silverlight, it is HTML and JavaScript that remain the tools of choice for responsive web applications. Although JavaScript is powerful, it looks more like C++ than C#. This means developers will need unit tests to avoid the classic pitfalls of type checking, improperly cased names, overwritten functions, etc.

Helping Startup Projects

When a new project with several engineers starts from scratch it will initially generate much code that will ultimately require synchronization. But until reconciling all of the project’s components each engineer usually needs a way to execute their code without the other components. Unit testing tools and products can provide such services.

As an aside, I find test driven development (TDD) demonstrates this value proposition. It too does not require developers to have all of their components’ dependencies lined up and running. TDD allows engineers and developers to productively and safely write a lot code courtesy of unit testing. Developers only require the scantiest knowledge of the project’s other components.

Implementing Smoke Tests

Regularly deploying code to any environment can be a risky proposition for even the smallest fix or feature. We repeatedly relearn this fact. (Remember that little tweak that broke the entire application and left the testing staff without work?) In order to combat this problem many organizations write a few key unit tests which are automatically executed immediately after a build and before deployment. These few “smoke tests” provide enough coverage to ensure the basic functionality of the application.

If smoke testing sounds like continuous integration (CI), there’s a reason. They share similar objectives, albeit with vastly different scope. CI typically demands organizational commitment and resources; incorporating smoke tests typically requires a few commands by the build manager.

Writing Clearer Code

By having to Craft unit tests, developers are forced to think as consumers of their code. This gives them an extra motivation to build an application programming interface (API) that is clear, and easy to implement.

The drive to write better unit tests generally motivates better design, too. By incorporating unit tests into their components, Developers are encouraged to follow many good software design tenets, such as, avoiding hidden dependencies, and coding functions with a well-defined mission.

Note: Bug Management Practices

If unit testing provides less than ideal bug management support, what options exist for software developers and engineers? After all the integrated and UAT tests are completed prevailing practices suggest two broad strategies for better managing bugs, one passive and one active.

The passive policy amounts to organizations providing a full featured help desk. In such an environment end users report bugs which get triaged until resolved. While an effective practice from a managerial perspective, it tends to frustrate end users and place developers in a reactive mode.

Actively managing bugs is an alternative strategy of growing popularity. It requires applications to self-report exceptions and such. This can happen via home grown or 3rd party tools, such as, Red Gate’s SmartAssembly. This strategy acknowledges the difficulty of preventing bugs with the belief that knowing about them as soon as possible without explicit user complaints mitigates pain.

Conclusion

Forgive me if I have left any readers thinking ill of unit testing. Nothing could be further from my intent! The goal was to challenge the premise that writing unit tests is always a wise practice in all contexts. Because the IT culture within the enterprise application space is no longer so uncritically accepting of the value of unit testing it is up to software engineers and developers to actively justify the cost of building unit tests in terms of benefits to the enterprise. I’ve noted in this article several circumstances where unit testing is vital for the timely delivery of robust applications, and I strongly suspect that many enterprise application engineers and developers who read this will think of more.

We can no longer rely on a general acceptance of the myth that unit testing is a universal panacea, but need to focus unit testing on aspects of development where it is most effective, and be prepared to actively justify its use.

For more articles like this, sign up to the fortnightly Simple-Talk newsletter.

Tags: , , , , ,

  • 62940 views

  • Rate
    [Total: 190    Average: 3.3/5]
  • Rowland

    Well done sir!
    Nice to see someone evaluate this from the cost-analysis perspective regardless of popular belief!

    Thanks Mr T 🙂

  • Anonymous

    Summation
    I can sum up the entire article thus:

    Use common sense.

    Unit testing is simply another tool, use it when it’s appropriate to use it. Makes no sense to write a test suite for a widget but a whole lot of sense when you’re building a multimillion dollar platform.

  • Anonymous

    Uncommon Sense
    Really? common programmers with common sense?

  • Urban Björkman

    Smoke Tests
    I often hear that unit testing need to have a very high code coverage to be useful at all.
    Unless its an application for a nuclera plant or a space probe, i think that to much code coverage will make the tests too cumbersome to maintain.
    Having a few test, testing key classes and mehods is the way to go in most cases. Focus should be on writing code, not tests.

  • Rob Levine

    I suspect many will feel this article misses the point of unit tests.
    I have to say that reading this article and the comments make me realise that some people approach unit tests very differently to me.

    There is a lot in the article that talks about unit tests as if they are things to be added afterwards – with an additional cost over and above that of writing the code itself.

    For me, my unit tests are part of the code itself. When I first write code, I will write the unit tests *to help me write the code*. I may not cover every line, or every flow, but these tests are part and parcel of me writing and validating my code before I check it in. They help me write clear and decoupled code, and have high confidence as I do it. They are, basically, inseperable from the code I write, not a seperate task. After check-in, these same tests become a way of ensuring the code can be both maintained and changed with relatively high degrees of confidence.

    There is no “additional cost” writing the unit tests, unless it is the accusation that I code slower than other people, because the unit tests are not a seperate task from the process of me writing my code. In any case, I don’t believe I am substantially slower the most coders, althoughI know I am not the fastest hacker out there.

    In fact, the treatment of unit tests as a seperate task is pervasive throughout the article. It misses what many people believe to be the whole point of unit testing – as part of the process of writing the code itself.

  • David Musgrove

    Rob’s post
    Rob has worked for me for over two years now – he certainly does NOT code slower than other people…

  • Todd

    Bad assumptions
    This article is has a couple of glaring poor assumptions.

    The first, and most critical is that the cost of of an application is linear over time and that same linearity can be applied to both unit tested and un-tested code bases.

    In my real-world experience, as applications age they require more time and effort for changes and maintenance. This maintenance is rarely done by the original developer and requires a lot of addition time to understand the original code and intent. IF documentation is available it rarely matches what is actually implemented. Modifications often introduce subtle and unforeseen bugs even if the developer takes extra time to investigate the change.

    Unit tested code bases SIGNIFICANTLY reduce both the time it takes to make the modification AND the number of costly bugs produced by those changes. Unit tests make the intent of the code clear and also give a developer a usage pattern that may not be clear in other parts of the code base. In addition unit tests help catch bugs by validating rules and assumptions that are built in the application and after a change will help expose those bugs quickly and saving tons of time.

    The second bad assumption is that writing unit tests take so much more time. This may be true of developers that are new or resistant to writing test or who do them afterwards (and I even disagree with this since I’ve seen developers waste countless hours “debugging” simple problems that would have been caught by a few simple and quick unit test). But as Rob states above, developers who embrace testing write the tests as they write the code and they come an integral process of development. When used in this way they don’t slow development down at all and the quality of the code that is written is MUCH better.

    A quality unit tested code base promotes a better product that has fewer bugs and less technical debt and allows that product to be released and enhanced FASTER than an un-tested one.

  • Eric King

    What about confidence and flexibility?
    In my opinion, the biggest benefit of a robust suite of unit tests isn’t even mentioned in the article: the confidence that comes with quickly understanding the impact of changes on the code base.

    I’ve worked on several mature projects where the lack of unit tests means that even relatively small changes to the code base must kick-off a very heavy and labor intensive testing effort. Sure, *writing the code* is marginally quicker, but the effort required to be confident in understanding its impact is huge.

    In contrast, a mature project with good unit tests means that you run the tests, make the change (including its unit tests), and run the tests again to validate the impact. The time spent is *significantly* less, and the confidence level is *significantly* higher.

    It’s certainly not the end of the story when it comes to quality assurance, but it leads to *much* more time and resource savings than the article implies.

  • Al Taylor

    Good article, Todd makes some good points
    Interesting article and discussion…thanks!

    Todd makes some good points

  • alex_razv

    Source of data for graphs
    It is interesting to know a way of collecting data for graphs.

  • Chris Marisic

    Irony
    Found this blog post and read this directly after reading this: http://elegantcode.com/2012/01/06/you-are-responsible-for-making-that-feature-work-write-a-test-just-do-it/

    Which directly ties in the comments from Eric King. The unit tests assert confidence in your code changes.

  • Anonymous

    Right on the money
    Tom’s article is very timely, and I appreciate his insight. I know many developers swear by writing unit tests for everything, including as part of the initial code development.

    But, Tom’s goal to “challenge the premise that writing unit tests is always a wise practice in all contexts” is right on the money.

    Very few things in development-land fall under *always* or *never*. It is usually “it depends”, right? All of our environments are very different, with different requirements, SLA’s, and satisfaction tolerance.

    I manage a cloud-based SaaS system with 9 million lines of code – we make 150 deployments *per day* to our production system. Our code changes so much, and so fast, that writing unit tests for everything becomes untenable, as the tests themselves have to change constantly, effectively doubling or tripling our develoment time. This will kill us.

    However, we have identified key areas where unit testing is critical. It is my job to make sure we maintain the balance between the core value proposition to our customers and the pragmatic use of unit testing.

  • Florin

    Quality
    Tom,

    I disagree with you on this…

    One of the most benefits of writing good unit testing is in resulting a much higher quality code. Higher maintainability, extensibility, reusability. This is even more visible when you’re working with developers which are not very experienced with OOP patterns or design principles (SOLID). By doing unit tests or TDD they will use and apply these principle by just removing duplication. Plus it also enables refactories without fear of braking things.

    I think that your graphics do not take into account the costs benefits of a higher quality code base. We all know that requirements changes are a fact, so having a low cost of change and a reduced accidental complexity may balance the graphics in the other way. Without backing these graphics with some real data, they can be suspected of bias.

    I agree with what Rob says. The cost of writing unit tests as part of code is the cost of learning to do unit testing or TDD. Once you’ve learned how to write good tests and how to benefit from them, this cost is no longer there.

    I think we all test the code we’ve written immediately. Just to get a confirmation that our code works. Most of the times we test it through UI, logs, or debug. Unit tests only enables us to make these tests repeatable and runnable by anyone.

  • Bart McLeod

    What about our silly mistakes?
    We all make silly mistakes.

    Without unit tests you may be looking for them for hours. Literally, if you have a bad day.

    With unit tests, you will likely spot them within a few minutes, sometimes even before you finish the test.

    So, once you have become proficient at writing Unit Tests, they save time. Lots of time. But I agree with Tom that you do not need them for each and every widget and that you should carefuly choose when to write them and when not.

  • Mike Gale

    Shows how diverse programming is
    The article focuses on mindsets which the author (Tom) has found in organisations.

    Some of the comments are about mindsets that are different.

    The advice to follow depends on how you go about things. (There’s more than two mindsets…)

    The dimensions include things like:

    1) Bolt on tests after coding approach vs develop tests and code together.
    2) One programmer writing both tests and code (like Rob).
    3) Pair programming teams.
    4) Tests that are not unit tests at all, but test across components.

    It’s a good idea to challenge the vehemence with which this idea is sometimes pushed. A worthwhile article. Gets the thinking going.

  • AlexK

    mixed feelings
    I somewhat agree with most ideas presented by Tom – we do not have to always cover 100% of our code with unit tests, especially when we are lacking good integration tests.

    Some modules needs unit tests more than others.

    Suppose, for example, that for the next version of SQL Server we are implementing MERGE command and Intellisence. Getting MERGE right is crucial – if we release it with bugs, that can scare off potential customers. Intellisense, on the other hand, is not that important – if it has bugs, that might not be a show stopper, because we can live without it. If we do not have enough time, we can skip some unit tests on Intellisense and get away with it, but we should not ever skip any tests on MERGE.

    I agree with Todd: speed of development does slow down more and more because of maintenance.

    More specifically, proper test coverage of MERGE will save a lot on maintenance. Same effort spent on unit tests for Intellisense would probably yield much less return.

  • BuggyFunBunny

    Division of Labour
    Three points I didn’t see in the article or comments:

    – unit testing came to the fore with dynamic languages, where neither compilers nor analysis could come close to guaranteeing correctness (see: Knuth)
    – in Large Corp development, Business Analysts specify and test, coders code (I’ll include DB development), so there’s not much to be gained by shifting testing to coders
    – the TDD and “do the simplest thing that could possibly work” and so on (Q and D cowboying, but iteratively!) leads to code which has a distinct Italian flavor; some are convinced that unit testing makes maintenance of such codebases tractable (sure it does)

  • BrainiacV

    Unit testing for bug free code is older than 20 years
    Back when structured code was being introduced in the 70’s, the argument was that if each module was bug free, then the whole program would be bug free.

    Totally ignoring the probability of logic errors or to use more modern terms, lasagna logic.

  • Anonymous

    Opinions are not facts
    Just because you have graphs doesn’t mean that they are based on facts. If you could back up your arguments with some factual information, I might buy some of your arguments. As it is, this article doesn’t do anything to convince me.

    The paragraph:

    Even the crudest test caught defects that were attributable to undeclared variables, mixed case method names, missing header file, calling functions without parenthesis, etc.

    really makes me question your experience. You don’t need unit tests to find these issues in C/C++, the compiler would find them all. It appears that you are thinking of VB, not C/C++…

    After that, I didn’t even bother reading the rest of the article.

  • Miles B

    Test Driven Development
    I get the impression you don’t fully understand test driven development and judging by the wording “delay to add unit tests” implies the code isn’t being driven by faithful TDD – if that is the case then testing is indeed a waste of time and has no value.

    When you consider a unit test (TDD) a part of the specification, how a piece of code will be used then you flip the entire purpose away from “testing” code to validating business functionality, maintenance (huge monetary savings!) and knowledge sharing.

    You also imply testing doesn’t drive how the software is built. TDD promotes (through lazyness no less!) data hiding which is reinforced through the single responsibility principle. For example, each publicly accessible behavior needs to be tested. It is in a developers interest to minimise what behavior is exposed and the coupling (think mocking).

    If you imagine that an object or piece of code has an “bug attack surface” (the publicly accessible methods) then TDD promotes the reduction of these methods where possible.

    The secondary benefit is the overall production code written will likely be less than a non-TDD system. When you combine the test and production code LoC then contrast TDD vs. Code Driven Tests I’d be surprised if the difference is that major.

    In short, TDD assures code brevity, “lean” test coverage and maintainability. Each of these has a monetary short and long term value

  • Anonymous

    Interesting perspective
    I see a certain amount of idealism expressed in the comments. Fair enough – I think 99% of programmers support TDD and unit testing.

    I think there is a large amount of idealism going on. The article isn’t talking about the perfect case, it’s talking about the typical case – large companies with existing code bases that didn’t start with TDD. He’s correct in his ‘graphs’ in that the cost-benefit is significantly reduced.

    Now a company must maintain profitability to survive, so while there is some long term planning, if they don’t see short-medium term cost recovery it’s not worth it.

  • Lupestro

    I test therefore I think 🙂
    The quality of the code you write comes from the quality of the questions you ask yourself when writing it. Unit tests pose questions of the author. When I write a test in the course of writing a class, I end up writing the class differently. Each of these changes focuses responsibilities more crisply. I have a few more classes and a lot less code. This in turn provides continual opportunities to streamline and improve the details of the program architecture as I go.

    Unit tests are necessary but not sufficient. We’re no better at imagining clearly how pieces go together into an application than we are at imagining clearly what the pieces should be and what they should look like. What developers nurture to succeed, QA professionals tempt to fail. It is an incompatible mindset but necessary.

    Unit tests do address “the stupid stuff” of implementation – null pointers, senseless ifs, hanging whiles, etc. This in turn results in fewer cycles through QA and lets QA focus their testing on “the interesting stuff”. Fragile code is boring to test.

    Unit tests are useful for regression after later changes and for putting you on notice that a change had a subtle ripple effect. With it, you can change code more boldly late in the game. All of this is true, but the greatest advantage is in how it makes the code stronger before a single line gets executed.

  • Joe T

    Give QA a Bigger Role
    I agree with the concerns you point out early in the article, however I think you are proceeding under a mistaken assumption that only developers can write unit tests.

    One wrinkle we use that mitigates diverting programmers to writing unit tests is to have QA write or augment unit tests. They are familiar with the processes and some even code. We have our QA staff code up initial unit tests in parallel to the developers prototyping. That way the testing specialists write the tests and developers develop unhindered. Best of all, lengthy & high pressure QA reviews right before deadline are avoided.

  • Bill E

    Sometimes but mostly not
    I use unit testing when its necessary, but mostly I do not use them. I use unit testing on complex functions that have to handle many variations and the unit test makes sure I covered them all. I use unit testing in our pricing formulas to make sure things are rounded correctly and consitently and to ensure business rules are still left intact. But mostly I do not use unit testing.

  • Kurt Guntheroth

    Not buying this argument
    Unit tests on new code help the developer check the *design* of their interfaces by forcing the dev to use them. The result is cleaner, more maintainable code produced sooner, at reduced cost.

    Unit tests improve confidence that changes made to existing code don’t break it, enabling more staff to work in a code base, producing complete code faster, at lower cost.

    The discipline of writing unit tests helps less-experienced developers find bugs that would otherwise escape to production. This allows lower-cost workers to produce code of a given quality level, at lower cost.

    Unit tests are a best practice. If you don’t want to follow this best practice, you have something to prove or at least explain; how you can produce quality code without them, at lower cost than a team that embraces unit testing and takes it seriously.

  • Lupestro

    Green fields and brown
    > The article isn’t talking about the perfect case,
    > it’s talking about the typical case –
    > large companies with existing code bases
    > that didn’t start with TDD.

    I see no evidence of that in the article. All of the effects he’s talking about are valid in projects of any considerable size, even those that use unit tests. However, the arguments in the “Reduce Bug Count” section like:

    > In such a scenario, it is only some form of
    > integrated, or user acceptance, testing that
    > will guarantee “SomeMethod” behaves
    > appropriately.

    do seem to be ignorant of the benefits of dependency inversion. If the inverted dependencies accurately capture the behavior of the other units, then a unit test will test the concrete behavior of “SomeMethod” adequately. In fact, dependency inversion will allow testing of its handling of many error cases that would be devilishly hard to induce deliberately in an integration test.

    However, a single class dependent on that many things directly does suggest a “Ptolemaic manager” class, which is itself often a design smell, i.e. it needs to learn to delegate and trust its delegates to make decisions. This is one of those design issues that unit testing causes you to look at squarely.

    As for the perfect case vs. imperfect cases, you can introduce units into legacy projects piecemeal, usually starting with the buggiest or the most volatile parts, i.e. the ones you really want to tame. When introducing agile practices useful to unit testing, like dependency inversion, though, it is wise to schedule some extra focus on system testing, since you didn’t have the units there to fall back on when you started.

  • BuggyFunBunny

    What do you mean We, White Man?
    — We’re no better at imagining clearly how pieces go together into an application than we are at imagining clearly what the pieces should be and what they should look like.

    This, and other, musings in the comments is the heart of the arguments. In a Large Corp venue, which is what I infer from the article, it’s not the coder/database developer who gets to make those decisions. In a one person (delta, to a dozen) venue, it’s whole nuther story. With multi-hundreds of Cube Coders (and dozens of Business Analysts telling them what to do) wailing away, the Agile Experience (which is the other mantra Which Must Not Be Voiced) is not gonna happen.

    A Cube Coder paid (directly or indirectly) to make lines of code, will do so, even if it means writing (correct) code that’s an order of magnitude more bloated than an alternative. In such a situation, any code which is not “in-line” with the application’s execution is code which doesn’t count toward the paycheck. It’s just that simple. Change how Cube Coders get paid (and the autonomy to make meaningful decisions), and you’ll see them be more careful and more smarter.

  • John Hunter

    You need to get out more.
    “In over fifteen years of consulting, I can count on the fingers of my hand the number of organizations maintaining unit tests for their applications”. In just one year, in one organisation, I’ve worked on 4 projects with extensive unit tests driving the design of the code, backed up by integration and acceptance tests. Your very first premise sets the tone for the whole article – “We all understand the value of Unit Testing, but how come so few organisations maintain unit tests for their in-house applications?” – this must be based on your experience – happily, it’s not mine.

  • AlexK

    survivor bias
    @John Hunter: teams that do use unit tests do not need consultants as much as the teams which neglect unit testing. So naturally consultants’ perspective is skewed ;).

  • Malcolm Rasool

    Pleasure and Pain principle
    My experience:

    Writing code TDD way tends to cater for an alert but relaxed coder. thy knoweth thy code!!

    Writing code in code-now-think-later kind of encourages sel-immolation for the coder in near future or extreme distress for unfortunate support fella later.

    Happy people generally dont create nightmares for themselves or the others and organisation with happy people generally does well too.

    Scared, stressed-outs want to take sick leave when push come to shove.

    I will stick with my happy path. Too selfish to change.

    cheers

  • Dan Sutton

    Hmmm…
    I tend to find that if you put enough error handlers which can report on themselves in a verbose enough way, and enough catches for unexpected data situations, say, then unit testing becomes more cumbersome than it’s worth: the code itself tends to find out what’s wrong with it and inform you. Minor bugs usually get weeded out a little later: in reality, it doesn’t matter how many unit tests you do – the insidious bugs which sleep for months before showing up are going to do so no matter how rigorous you are. Like anything else in programming, this sort of thing needs to be taken with a grain of salt: as pointed out above by “Uncommon Sense”, the application of basic common sense in these matters will balance the testing against the lack of progress in overall development.

  • Anonymous

    No Proof
    The article doesn’t specify research that backs up any of the statements made. Without that, it’s only one person’s opinion, combined with a sample of comments that themselves aren’t proven to be representative of statistically acceptable results. That said, this isn’t a criticism of the author himself, but rather the fact that there isn’t any proof of what he’s written. Therefore, we’re left to believe that the experience of one person should be representative of everyone, which might not be accurate.

  • AlexK

    survivor bias
    @John Hunter: teams that do use unit tests do not need consultants as much as the teams which neglect unit testing. So naturally consultants’ perspective is skewed ;).

  • Charles Morissette

    Yeah, sure there
    People like you are why I use software that crashes, never tells me why and generally makes my life a living hell, aren’t you?

  • John

    Without numbers, this is just guesses on top of guesses
    Clearly, as evidenced by the titles of sections, there is a bias/hypothesis/dislike for unit tests. Drawing graphs that prove it with no underlying data is not of high value. Would have been better / less biased simply to suggest other things to consider when evaluating the overall model for your organization.
    -j

  • Malcolm Rasool

    There is more to it
    I assume that timeline only covers development phase – happy to be corrected. If that is so, how would the benefit compare if we extend the timeline to include maintenance phase – which is generally where the lion share of the cost incurs? And again if so, it’s a bit unfair comparison I would say, since one of the key reward of TDD is reaped during maintenance phase when change requests come thick and fast and one can play with exisiting code with lot less fear.

    I also find it unproven that TDD or unit test based solution can cause delay in getting ‘out’. the initial delay generally occurs during first test releases but by the time it’s done with stabilisation phase, TDD tends to get in front due to it’s ability to quick return on fleshing out the issues. So not sure where the opportunity loss supposed to occur. And finally, is the credibility loss any less serious or costly as a result of the buggy code?

  • Russ F

    I party agree but you missed the point
    That the biggest issue isn’t really the cost/benefit it’s the fact that people think unit tests are some magic dust to say “it works”.

    Also, assuming you write the tests afterwards is soo 20th century. At least get up to speed with the TDD bandwagon.

  • Salim Nair

    So many assertions, so few evidence
    What caught my eye in the article is its complete lack of any kind of data on any of the trends or overarching comments the author makes. Even the nice looking (though suspiciously well behaved) graphs kind of fool one into thinking that there is actual data behind it, but then realize that it is a just so story. As someone else pointed out, the arguments about unit test slowing down the rate of coding is presented as if it is a well-known fact.

    The same goes for the analysis of bugs, which shows a surprisingly simplified assumptions about bugs and a naive view of the inversion and other injection methods. Again, it shouldn’t be too hard to present some data to support this assumption.

    I guess, this is exactly what unit tests would have prevented – Making declarations without evidence!

  • Ron

    unit tests are good but !!
    i had developed briefly in .net and c# doing unit tests, code coverage etc but was not satisfied whether it covered the business requirement and it was what the user wanted. still we kept writing tests and changed code and tested again when users were not satisfied.
    until i came across bdd which looks promising, did not get chance to give it a go but will eventually.

  • Dave R

    What do you consider a unit test??
    The example you give that starts with “When enterprise applications were built 20 years ago with ” is flawed. Those are all compile time issues that cannot be caught with unit tests unless you are using a language that supports reflection and you are using it extensively. If true, I think you have other things to worry about!

    I think the bigger issue, thought, is that your idea of a unit test is different than mine and most other people I suspect. The purpose of a unit test is to test a single unit of code (eg. a class) in isolation. The example you give in you diagram is an integration test, not a unit test. Completely different animal. While they could be used in such a situation, unit tests are not intended for, nor are they good choices for, this type testing.

  • BuggyFunBunny

    It’s Still The Mantra
    While some have caviled in the comments as to what constitutes a unit test; is it a test of an atomic entity (function, method, class, package) or something larger, the fact remains that the OO/TDD folks with which I’m directly familiar take the position that OO languages can produce guaranteed correctness with unit tests. I don’t buy it, but there it is.

    Their argument is that with sufficiently small entities (classes and their methods), logic errors asymptotically approach zero, and unit tests remove syntax errors and such. I don’t buy it, but there it is. (What happens is that the flow of control among an infinite number of infinitely tiny code chunks leads to coding in the debugger, since the flow of control can’t be held in one’s noggin. See below.)

    The Agile meme fits in here, because it proports to do better development with infinite baby steps. Contrast with BDUF (see: Spolsky), which the Agile folks dismiss as so Grandfather’s Way of Developing. Testing, of all definitions, is required when using dynamic languages just because the “compiler” is necessarily dumb about syntax and semantics.

    While I don’t advocate returning to the days of COBOL/FORTRAN coding sheets and #2 pencils (and a pentel eraser), believing that full coverage tests are a replacement for thinking and analysis is the road to Hell. It’s easy to spot developers who prefer the code first think second approach: rather than spending most of their day thinking and typing in their favorite editor, they spend most of their day staring at the debugger wondering “How did I get here?” (yes, David Byrne).

    A debugger is no place to build code. If you’re doing that, you’re doing the wrong thing.

  • Anonymous coward

    IMO, this is all BS
    The first diagram is plain wrong. You get to the break-even a lot earlier – after shipping the first version of the app you’re already there. Which makes the savings when using unit tests huge, since usually only 10% of all effort goes into the initial version.

    As for bug counts … try writing unit tests to an existing codebase, and see how many bugs you find. Then compare the cost caused by some historical bug discovered by an end-user with the cost if would have taken to write a unit test for it. Then talk again.

    It’s indeed stupid to unit-test everything. UI is extremely difficult to unit-test. It makes not much sense to thoroughly unit-test CRUD code. But at least every bit of algorithm and business rules code should be unit-tested.

    For the complex parts, however, 100% coverage isn’t enough. It just means your unit tests exercise each line of code. However, when testing an algorithm, you usually need to throw at least a handful of different inputs at it, before you can confidently consider it correct. Which means you need to execute most lines of code repeatedly with distinct input. Which means coverage of 300-400% for the critical or difficult parts of your code.

  • Nick

    Don’t agree, but you are welcome to an opinion
    First of all, the number of times TDD has saved my bacon is unbelievable.

    Secondly, you try adding new features to an enterprise application with no unit tests.

    Based on experience I have to totally disagree with pretty much everything you say.

  • Kirill Osenkov

    – 1
    Sigh. I just hope non-technical managers don’t read this. It’s hard enough to ram the right way of doing things down their throats.

  • Tim Linquist

    Wow, you have no clue what you’re talking about
    “I have rarely seen a developer convince an end user that adding unit tests justifies delaying a much-anticipated feature just to prevent of a few potential bugs.”

    “Enter today’s improved integrated development environment (IDE) tools and managed memory runtimes, and many C/C++ lifesaving checks handled via unit testing became superfluous”

    I just picked out a few of the completely invalid assertions made in this post. Am I going to hear next the argument for QA & a compiler sufficing for identifying / resolving bugs, preventing regressions, producing maintainable, quality software? Wait, I already did!!

    This post actually has zero substance. Perhaps the graphs initially could have given a legitimate argument except the author backed them up with no real data. Oh and they are completely false in the real world where we have large applications that interface with many other applications that power large organizations that have to be maintained over time by engineers not QA teams and compilers. Engineers that understand how horrific it is to work on code that has little to no test coverage. Engineers who understand HOW MUCH IT ACTUALLY COSTS TO CONTINUE TO INVEST IN THIS CODE THAT IS A GIANT BALL OF MUD <http://en.wikipedia.org/wiki/Big_ball_of_mud&gt;.

    The worse part is that the author drew a hard line then backtracked then drew a hard line again. Are you against unit testing or for it because no meaningful evidence was provided for or against either side?

    Every reader please know this, unit tests are only one facet of producing quality, maintainable software. There are many other facets of testing. The author actually touched on two, integration testing and smoke testing (actually automated testing.) Others include functional testing (black box), load testing, regression testing, system testing, and more. That said, unit tests ALWAYS, ALWAYS return the most value for your investment. A good analogy is to think of the foundation of a home. You can’t build on top of a foundation with cracks. Well actually you can, that is what this author is suggesting you do by implying a cost / benefit analysis needs to be done before creating a culture of testing. Unit tests give you a solid foundation to branch out from. The other layers of testing are STILL fundamental. In the only decent illustration of this post, the systems diagram, the necessity of integration testing is outlined. Saying that suffices in terms of building a quality code base code base proven to work is like slapping paint over your cracked foundation and saying “it holds the house up so it must be fine right?”

    The truly sad part is that this article averaged four stars. I am the guy who is going to have to clean up all of your shit because some guy validated your poor habits that produce deficient code. I’m the guy who’s going to have to actually explain the benefit of TDD because you told some business person you were consulting with that it was an unnecessary investment. Please read some of the other comments on this post similar to mine. After, strike this post from your memory.

  • WBrewer

    Unit Testing

    @Tim Linquist
    You make some interesting points but I’m a bit puzzled by some of the things you say. I’ve had some very good experiences as a developer in major projects working with Unit Test teams, and am convinced that it has an essential role in development. However, I like Tom’s point in saying we should not rely only on myths or beliefs to determine the amount of development effort allocated to Unit Tests. We need, surely, to justify the benefits with verifiable facts. Also I like Tom’s point that we should target those unit-test resources where they can be most effective.

    I’m not sure who you’ll convince with your analogy of house-building, but most managers will dismiss this a ‘sales talk’ rather than science. To then bang the caps-lock and talk about ‘shit’ and ‘balls of mud’ might rally the less-experienced to your flag, but I doubt if it will cut any ice with seasoned IT professionals: Quite the reverse.

    Are you saying that TDD is synonymous with Unit testing? Are you also saying the Unit Testing always gives more benefit than any other form of testing? If so, do you have proof, or is this another ‘assertion’?

  • Stefan Hendriks

    Ball of mud
    @WBrewer, actually a Ball of Mud is just another term for spaghetti code or spagehetti code with meatballs.

    Lets just say this blog post has the same amount of hard data any other post has claiming unit testing is not worth the effort. I don’t think Unit testing should be considered as a seperate thing to do. It needs to be integral with your code, along with a TDD style of developing.

    Just like Rob Levine posted (3rd above), thats how we should see our code.

    We (Software Engineers) cannot deliver code, without it being tested. Code created without tests ‘driving the architecture’ will be guarenteed to deliver hard to test/more complex than needed code. Several studies done by larger companies (Google, Microsoft) have shown TDD (and unit testing) does work, and does actually save time and money.

    Read this post: http://blog.objectmentor.com/articles/2009/10/07/tdd-derangement-syndrome

    And you’ll find some resources backing up claims that TDD does work and save time. Please if you post blogs like this, post resources or back it up with data. Thank you!

  • Anonymous

    his resume
    I’m guessing he doesn’t have “TDD” on his resume…

  • Mike

    Pair with people that know WHY TDD works
    When I was young, someone asked me, “Do you like sushi?” I replied, “No.” He followed up with, “Have you ever tried it?”. My answer was “No.”

    At that very moment, I realized the stupidity and stubbornness demonstrated by those two answers.

    I had the same epiphany later in life when it came to TDD. Once I tried it for a while with people that understood the reasoning behind it, I would never go back.

    Just like if you try sushi for the first time at a bad restaurant, you will be scarred for life and will never have sushi again. If you try TDD pairing with people that do not know WHY it works, you won’t be fully bought in and will go back to antiquated ways.

  • Tim Linquist

    fair point
    @WBrewer The caps lock were to make the point clear. The “shit” was also purposefully used because that is what code without proper test coverage amounts to eventually.

    Yes one of the things I am saying is that unit testing returns the most value for the effort in most situations (yes there are always exceptions.) My main motive was to call out the lack of substance in this post. I also hoped the analogies I used would resonate with developers in terms of their own experiences with code lacking unit testing. My apologies if my language turned you off. I wasn’t stamping my feet angrily though. There is brevity in my comments.

  • Anonymous

    Coding like its 1999
    I did a double take when I read this article – like being back in 1999.
    Who writes “unit tests” to make sure there arn’t any bugs ?
    I use BDD in order to drive out the domain, in order to focus on the features that the stakeholders require. The fact that I get coverage is a bonus.
    I really think its time people stopped talking about “unit testing” and started talking about driving out the domain, in discovering the problem, in growing the system through repeatable, executable features.
    The comments about deployment are laughable – I can deploy a system in seconds, and have it live in minutes – if you can’t, then you have a problem with your deployment process. So fix it.
    As for integration ? That’s why we write anti-corruption layers and talk about bounded contexts … all of which can be “tested” by writing behavioural specs (in for example, Cucumber) which exercise the behaviour of the whole system.

    Have a read of Dan North, Liz Keogh to get up to speed on BDD, start using Cucumber and MSpec (or similar), and have a read of Eric Evans for DDD. And maybe check out Jez Humble in order to fix your deployment headaches !

  • Stefan Hendriks

    Ball of mud
    @WBrewer, actually a Ball of Mud is just another term for spaghetti code or spagehetti code with meatballs.

    Lets just say this blog post has the same amount of hard data any other post has claiming unit testing is not worth the effort. I don’t think Unit testing should be considered as a seperate thing to do. It needs to be integral with your code, along with a TDD style of developing.

    Just like Rob Levine posted (3rd above), thats how we should see our code.

    We (Software Engineers) cannot deliver code, without it being tested. Code created without tests ‘driving the architecture’ will be guarenteed to deliver hard to test/more complex than needed code. Several studies done by larger companies (Google, Microsoft) have shown TDD (and unit testing) does work, and does actually save time and money.

    Read this post: http://blog.objectmentor.com/articles/2009/10/07/tdd-derangement-syndrome

    And you’ll find some resources backing up claims that TDD does work and save time. Please if you post blogs like this, post resources or back it up with data. Thank you!

  • Anonymous

    Have you read “The Flawed Theory Behind Unit Testing”?
    http://www.dzone.com/links/r/the_flawed_theory_behind_unit_testing.html

  • Philip Schwarz

    Have you watched “The Deep Synergy Between Testability and Good Design”
    screencast of NDC 2010 talk: http://vimeo.com/15007792

  • Philip Schwarz

    Have you read “The Sensitivity Problem”
    http://butunclebob.com/ArticleS.UncleBob.TheSensitivityProblem

  • Dave Rooney

    Definition of Unit Tests
    Like most of the people here, I was writing test harnesses many years before XP brought unit testing to the forefront of development. When I first learned about XP in 2000, I jumped on the automated unit testing bandwagon right away, and haven’t looked back.

    However, I think part of the issue that you’re bringing up here has to do with the definition and/or the purpose of “unit tests”. You say:

    “Even the crudest test caught defects that were attributable to undeclared variables, mixed case method names, missing header file, calling functions without parenthesis, etc.”

    Those things aren’t caught by tests, those are caught by a compiler or linker. When I write a unit test for a function or method, my test is verifying that the behaviour is what I expect. A single test usually tests a single path through the function or, said better, a single thought. To be able to do that, you can’t have 100+ line methods.

    At one recent client, I was told that they already did a ton of unit testing. When I enquired further, I was told that one test could take a couple of weeks to write! When I write a unit test, it might take 5 minutes. Obviously, that term has become dangerously overloaded and means vastly different things to different people. Because of that, I’ve been using the term “microtest” coined by Mike Hill (http://www.helpinggeeksproduce.com/ ).

    Another issue that you don’t seem to touch on is that the expense of writing microtests is huge when it’s being applied to existing code that wasn’t written with tests already! Backfilling tests reveals the mass of spaghetti that is the dependencies that people create in their code when it’s written without microtests. Just the act of writing the tests will require you to clean up the code, but yes it does take time and money.

    However, when you have a system that’s tested manually, your productivity erodes over time until the cost of adding a new feature to the system outweighs the cost of simply rewriting it. While it’s something that’s done all the time, it’s bad business.

    Simply hacking some code together to get something out quickly may be a “get rich quick scheme”, but we all know how well those turn out in the long run. Unit or microtests represent an investment that does require you to slow down in the short term, but you will be paid off handsomely in the end.

    Consider them a get rich slow scheme.

  • Nimos

    Time scale
    I think the graphs are true in that there is a breakeven. Hello world is written faster (in most highlevel languages) if we don’t write a test for it first. But I believe that we will pass the breakeven point very soon after that. Which means that for any project bigger than a few lines we will spend basically all our time to the right, outside of those graphs. Hence we should go with the tdd approach from the start.

  • Tim Linquist

    A counter to this article
    I wrote up my thoughts on this article for a guest post on a friend’s blog. Hopefully some readers will read this article then read mine to use multiple perspectives to help shape their thoughts on this topic, http://dipperstove.com/development/software-testing-benefits.html.

    Enjoy. Happy coding.

  • Anonymous

    How many tests?
    In TDD you write a test, then write the procedure to pass the test. Are you finished if the test passes?
    If the procedure had one 16-bit integer parameter, I would create five tests for it. If the procedure had two 16-bit integer parameters, 25 tests are required. And so on. I test all the edge conditions (-32768, -1, 0, 1, 32767). For two parameters, that is five times five tests to check possible interactions.

  • Anonymous

    Saving money
    Unit testing possibly doesn’t save money during development but, before applications or maintenance goes into production, especially in the financial world, it is paramount.

  • Ebenezer Ikonne

    Writing unit test is a software development practice
    Writing unit (micro) tests should be seen as a tool (in a plethora of tools) for the design and coding of software systems. Depending on factors such as how strategic and how long the software will be around, you’ll need to make a decision as to whether the benefits of writing unit tests is worth it for the organization.

    A lot of people do not fully grasp the design benefits of designing code test-first such as low coupling with high cohesion that you are afforded and don’t make the investment to learn. Having no tests is much better than poorly written tests.

    Ultimately, I think this article has taken the right approach in looking at it from a “cost benefit analysis” angle. We need to do more of that. However, I think because there is so much code/experience that exists sans unit testing, I think its going to be a long while before it is practiced widely.

  • Anonymous

    Close but no cigar
    I’ve been in IT for 20 years, and guess what? There were bugs then and there are bugs now (using TDD)!

    I could hence draw a (flawed) conclusion and say that “therefore Unit testing does not reduce the bug ratio”, or “therefore Unit testing does not reduce the cost of software”.

    But the reality (from working on dozens of IT projects) is that software quality reflects developers quality, not methodology.

    You can “bend” poor software into shape by applying layer of fixes after layer of fixes, so that the end result passes all the tests and makes the QA manager happy, but internally it’s all become a big ball of mud.

    Basically, TDD is great for structuring your thoughts as you write software, and validating your design but if all you can think/write is crap software, unit tests will validate that too.

    My personal conclusion is that there is no correlation between software cost (taking into account technical debt) and TDD for poorly written software. But there is some positive correlation for good software because in that case TDD helps with refactoring and keeping technical debt down.

  • John Hunter

    survivor bias?
    @AlexK – Ah, but you see – I am a consultant.

  • timothyawiseman@gmail.com

    Unit tests are wonderful… most of the time.
    Very well said. I find that the vast majority of the time, Unit tests are well worth writing and they have helped me several times.

    But it is important to realize that they do come with costs, and in a business determining the ROI can be very significant. While Unit Tests are well worth the effort put into creating them most of the time, there are some limited exceptions, and that fact is not acknowledged enough.

  • GDJ

    A little too easy.
    This is a great article only because it’s so provocative and delightfully indefensible.

    I’m glad that my belief in the value of unit testing does not come from such thin arguments as this article.

    I feel the article to be flawed in that the 2 ‘myths’ are really only one idea: that of defect reduction. The Cost Savings angle says the savings is from defect reduction. So really, Tom’s talking about whether unit tests do or do not reduce defects. His argument about reducing defects seems preposterously thin.

    It does serve to point out that, like all things, some tests are better than others. A sound cost-benefit understanding applies to everything. Aiming for 100% coverage is likely to be poor on the cost-benefit scale.

    I believe that it was just fine for my team to not emphasize unit tests earlier in our product’s history. The cost of mistakes were lower, the ease of correcting them was greater, the application as a whole was better understood by a larger percentage of the team. The product was immature and changing very rapidly in relative terms. Changes were confidently made in short periods of time. Things change… I don’t dispute that there is a ‘break-even’ point. But if you are going to graph it, you better back it with hard, empirical quantitative data rather than incomplete, qualitative hand-waving.

    Unit Tests are a pillar of support for sustainable, low-waste, just-in-time, high-quality development. Efficient spending and a quality result are byproducts of this kind of development. If you don’t believe in that kind of development, then we can probably agree to disagree on unit tests right here and leave it at that.

    Let’s assume you do think that lean, value-oriented processes are for you. How do unit tests help us?

    1. Support Leaner Processes

    Unit Testing, and more specifically Test-Driven development, reduces the need to do as much upfront design of those ‘detailed’ things which could be decided later – i.e. during implementation.

    • Unit Testing (and Test-Driven Development) *replaces effort* normally invested in a design cycle.
    • Unit testing supports *deferral of effort* to the point it’s actually needed.

    Being able to defer time from a feature improves the ability to deliver the most valuable items sooner. Furthermore, it is possible we NEVER build the less-valuable items, so we never spend *any* time on things we really do not need.

    2. Support Confident, Rapid Change

    Unit Tests support confident change. Confident changes are made both faster and are generally more successful than changes made with significant uncertainty.

    Rather than deciding on API’s and variable and argument names during a ‘design cycle’, these can be implemented by the developers, and then changed quickly during a code review (if they even need it), with the comfort that the tests we wrote make sure that these ‘late’ changes don’t break anything.

    Similarly, when we often develop ‘on the fly’, some code comes out of our brains tightly coupled and procedural. This is natural – our brains are “Big Balls of Mud”. Unit Tests allow us to revisit, refine & refactor our code and be sure that it still works. This is another example of deferred design.

    When we think of ‘design’, the more expensive it is to change later (or the more costly a mistake), the earlier it should be decided confidently. The opposite of this is true too; the easier it is to change, the later in the cycle we could make that decision. Decisions take time and people to make, and they are, by nature, restrictive. Thus, the longer we can delay a decision, the longer we can operate with reduced restrictions – this lets us remain flexible for as long as possible. The fewer people that need to be involved in a decision (and still arrive at a conclusion we all accept), then the cheaper that decision is to make.

    3. Support Good Design & Code Quality

    The article refers to this already in the ‘Practices’ section but presents it as an afterthought perhaps to avoid the hellfire of responses such as the ones we are providing.

    Writing unit tests supports good design practices due to effects of necessary decoupling and simplification. You can’t unit test tightly coupled and complex code. I shouldn’t say can’t. But the tests become as hard to understanding and maintain as the code. As the article says – it forces the developer to think of themselves as if they were another developer – something I had always wanted at ‘design review’ time!

    Our experience is that the first write of the code is almost never the last time a developer needs to understand it and change it. It is almost never the same developer that is the one who needs to understand it later. Readability and Adaptability of code is paramount in our product.

    Designs and Design reviews as a means to foster simple, good quality code were well-intended, but largely ineffectual. We wanted rather designs to focus on the ‘big’ ideas (the ones that are expensive to change, or where mistakes would be costly). Code quality needs to come further downstream.

    4. Support Meaningful Documentation

    Unit Tests are, by necessity, a form of living documentation about what the code does. They are effective because as the code changes the Unit Tests must change as well – or they break. They basically tell the developer what the code does. Of course, the Method Name and the Javadoc should say it as well, but they are often wrong or incomplete (the ‘living documentation problem’). A successful unit test is not wrong until the requirements of the code change.

    It has long been acknowledged, however begrudgingly, that the only right answer as to how a feature in a large and complex application works is to check the code. Unit Test are the “questions” to which the Code “answers”.

    So What?

    I don’t know about anyone else who’s working on a large, complex, mature product that’s continually evolving and growing, but I see themes of Cost Savings and Defect Reduction all through here. The quantification of these will vary by the product and the people.

    But you know what else? Developers are by nature creators and innovators. If they believe the right thing is to unit test, they’ll find ways to make it easy, too.

  • Jeff Moden

    Ouch!
    Heh… wow. This article has more high speed and accute “reversals” in it than the best of slolumn courses. I’d sue for whiplash but the article appears to have done what was apparently intended… be a controversy that starts much discussion.

    I can’t speak of the front end world because I left that world 10 years ago to pursue my passion in the world of databases.

    For the database side of the world, there’s nothing worse than having the customer find a bug when they can least afford it. It really gives the company a black eye and word of that black eye travels like lightning simply because people do talk about problems with their peers including peers in other companies.

    I have to admit that I really don’t understand all of the controversy over “unit testing” because it just doesn’t take that long to “unit test” your own work. With a little extra knowledge on how to quickly gen test data, it doesn’t take that long to include a little performance testing, as well.