Test Driven Development White Paper

page owner: CamilleBell

This topic contains a whitepaper by Camille which will be developed here and published on the public site. Over time we will allow wiki feedback for registered users. This is a work in progress.

---

Test Driven Development

Overview of Test Driven Development: Lightweight Rigor

Agile Development is a mindset as much as it is a set of techniques. Key to the agile mindset is improving quality by maximizing the amount of work not done. While at first glance this sounds like a paradox, because it sounds like an invitation to sloppiness, properly done, it is both rigorous and lightweight.

Agile testing, specifically Test Driven Development, is an example of lightweight rigor.

Finding and fixing bugs is an incredible amount of work, much more work than writing code. Finding and fixing one line of code can take days when bugs are allowed to fester. In contrast, writing test code is an amount of work similar to writing the code itself, far less work than a bug hunt. Consider health as an analogy; preventing disease through cleanliness and vaccine is far easier than curing illness.

So the agile vaccine is to write tests instead of chase bugs.

Since writing tests for large amounts of code is much harder than for small amounts, agile developers write small amounts of code at a time. Since writing tests later is harder than writing tests earlier, agile developers write tests as early as possible – at the beginning.

The agile way is to write tests before writing any code.

Since writing large tests is harder than writing small tests, agile developers write small tests.

Since only a little bit of test is written first, then only a little bit of code can be written. More tests and more code are then added over time, and the code evolves.

A test that always passes isn’t meaningful. Unless one verifies that a test can fail, its passage proves nothing. So an agile developer always writes tests that fail the first time the test is run.

The test is designed so that, if it passes, it defines the desired functionality. The failing test should be a little test defining only a little bit of functionality. This is RED. Then the agile developer writes just enough code to make the test pass. When the code passes the test, this is GREEN. Over time, evolutionary code can need refactoring cleanup (such as fixing duplicate code, long methods, large classes, etc.) to keep it trim. With tests in place, it is safe to refactor the code to clean up and improve the code as long as no new functionality or complexity is added. This is known as REFACTOR. The whole cycle starts again to add a little more capability by first writing a new failing test: RED. Both source code and test code check-in should take place after every passed test, whether the source is new or refactored.

The trick to RED, GREEN, REFACTOR is to do development in baby steps: only a little bit more test, a little bit more code and a little bit more refactoring at a time.

Following the RED, GREEN, REFACTOR paradigm of Test Driven Development ensures that as far as humanly possible the code always works. Test Driven Development also ensures better architecture and design, because writing for testability enforces loose coupling.

Following the RED, GREEN, REFACTOR paradigm also means never writing code just because the code might someday be needed in the future or be a more ideal design. If functionality is needed in the future, then in the future a test is written for that functionality and then code is written to meet the need specified by the test. If a better design is later needed, then the code evolves toward that improved design through refactoring.

If RED, GREEN, REFACTOR is combined with customer prioritized story development, the benefits increase: the development team can guarantee that at any given point in time the product always works; the customer is always getting the most business value possible; and the code base and design always remains simple and easy to enhance.

The Test First pattern of agile development should be applied to both story testing and programmer testing. However, because stories encompass end to end user functionality, the granularity of story tests is much larger than programmer tests. Because user story scenarios are inherently larger, story tests go through fewer RED, GREEN, REFACTOR cycles, typically once an iteration. In contrast, the granularity of programmer tests should be tiny and most agile developers go though many RED, GREEN, REFACTOR cycles a day, while some of the very best agile developers have multiple cycles an hour.

Bugs That Slip Through

Ideally, if Test Driven Development is the Agile Development vaccine, then bugs shouldn’t be a problem that agile developers need to worry about. Unfortunately nothing in the real world is ideal. Even with vaccine available, the unvaccinated can become severely ill. Vaccines vary in effectiveness and some of the vaccinated may also become ill, though usually less ill than the unvaccinated.

The software bug disease can surface on a Test Driven Development project for many reasons. Often only the newest part of the code base is Test Driven and older code is bug ridden. Developers new to Agility and new to Test Driven Development take time to grow their agile skills. As many developers start with weak testing skills and growing testing skills can take time, bugs may slip through as developers grow their Test Driven Development skills. Finally, neither the process of Test Driven Development nor the human beings (even best agile ones) who develop software are perfect and bugs may sneak in. While bugs happen far less often with Test Driven Development, they do still occasionally happen and when bugs occur they must be handled.

Bugs are most commonly detected in Test Driven Development projects in one of two ways: automatically because pre-existing programmer or story tests turn RED and catch the bugs or because despite all tests being GREEN customers or developers find undetected bugs in the process daily use.

Bug Found With RED Turning Test: Fixing a bug found by a RED turning test is fairly easy, if automated tests are run after every code check-in or at least nightly as part of the automated build. The bug is almost certainly traceable to some recently checked-in code that broke the build. With a relatively small delta of code changes to investigate, finding and fixing the bug is usually straight forward. After finding the source of the bug, the developer locally makes a fix in the developer’s local environment and runs the RED test. If the formerly RED test turns GREEN and all the other tests remain GREEN, then the developer checks in the fixed code. The team then starts up new automated build and again runs all the tests. If all are GREEN, the bug is fixed.

Bug Without Pre-Existing RED/GREEN Test: Fixing a bug found by manual testing or product usage is much harder, because most of the process resembles a traditional bug hunt. However the first step in the bug fix is straight forward, because the first part of the problem is that no automated test exists (all GREEN tests when a bug exists is bad). To prove the bug is fixed and detect the bug if it reoccurs, the first step is always to create the RED automated test. As with new development, the RED test is designed so that, if it passed, it will define the desired functionality. The definition of that functionality usually comes from a bug report or user interview. A bug report should include the reported (buggy) results or behavior of the product and how that differs from the expected results or behavior. With that in hand, the developer writes a test for the expected results and runs the test. The test should go RED, because the bug still exists. If possible, the developer should write a RED Programmer Test of the appropriate level. If not, write a RED Story Test. If reasonable, write both programmer and story tests; both should turn RED before the bug is fixed and GREEN after.

Unfortunately after writing a RED test, finding the bug will often resemble a bug hunt. With a RED test in place, a code coverage tool may simplify the bug hunt, if the tool can trace the path of a single test. With individual test code coverage, the failing RED test can be used to identify the path taken by the failing test, and likely source of the problem.

Once the bug is found, the bug fix process is identical to the one above. After finding the source of the bug, the developer locally makes a fix in the developer’s local environment and runs the RED test. If the formally RED test turns GREEN and all the other tests remain GREEN, then the developer checks in the fixed code and test. The team then starts up a new automated build and again runs all the tests. If all are GREEN, the bug is fixed.

Feedback