The Term “Unit Test”

The phrase “unit test” means so many different things to different people, that we need some new terminology to describe the tests involved in test driven development (TDD).

I gave a presentation on unit testing, and there was some pushback on the “unit tests must run fast” rule. Michael Feathers says, “Unit tests run fast.  If they don’t run fast, they aren’t unit tests.” (p. 14) Someone said that they wrote a unit test that tested all possible input (all numbers in range), and that took a couple if seconds to run, but they wanted to fully test it. It tests the unit, so isn’t it a unit test?

In TDD, unit tests are quick tests that are written before the code. They are kept as long as the code they test exists. Every build runs every unit test in the project. They don’t have to be exhaustive. The aim in keeping them around is to tell immediately if something breaks. If they take too long to run, developers won’t run them every build, and their main advantage will be lost.

Exhaustive tests are different. They aim to ensure that you’re taking care of every possibility; to ensure the problem is completely solved. We want to keep them around and run them as regression tests, to find subtle errors that might creep in. They can take longer to run, and accordingly shouldn’t be run every build. We still want to run them regularly, maybe every check-in.

These are both important, but they are two different concepts. They are both unit tests, but they have different requirements and should be run at different times. So lets give them different names. We could name them based on when they are run: Build Tests and Check-in Tests. Or we could name them by description: Quick Tests and Exhaustive Tests. We can put the name in the class name (e.g., StringParserBuildTest), then the build process can run them at the two times. For example, “ant build” could run Build Tests, and “ant full” could run both.

It’s just a question of semantics, but semantics dictate the way we look at the world and the way we communicate. Maybe there’s some other terms which would work better, but if we stop calling everything “unit tests “, we can better discuss what we mean, and we can get better tested code.

Tags: terminology, unit test