Unit testing in the dim and distant past of the last century meant testing the code I was writing before it was integrated with others’ code. It was testing a chunk of software in isolation from the rest of the application, system or component code.
Unit testing in the JUnit era has slowly come to mean the testing of the smallest practical chunk of software. For mainstream object-oriented languages such as Java and C# this has come to mean testing the individual methods within a particular class. Everything else is considered too wide in scope to be a unit test and should be called functional or integration testing instead.
In turn, this idea of unit testing being the testing of the smallest chunk of software has led to rules appearing about what unit tests may not do:
- talk to the database
- communicate across the network
- touch the file system
- run at the same time as any of your other unit tests
- require special tweaks to your environment (such as editing config files)
All this sort of behaviour must be mocked or stubbed out.
One claim for writing unit tests this way, is that these tests precisely identify where a problem is when they fail, and indicate precisely which code is working when they pass. This reduces the time that would be spent diagnosing problems should a wider-scoped test failed. Of course, this assumes the unit tests will actually help diagnose the problem.
In fact, the overriding aim of these rules is to keep the tests fast so that they do not impede a developers productivity when run every few minutes or just before committing code to a source-control environment, or as part of a continual/continuous integration environment.
Tests must add value to a project: the benefit of being able to run a test must exceed the cost of writing and maintaining that test otherwise it is a form of waste that according to lean development principles should be eliminated from the process. There is a danger that developers produce a large number of trivial unit tests that comply with these rules but do not earn their keep.
Unfortunately, unit tests complying with these sorts of rules, by definition, are not useful as part of a frequent, regular or continual (‘continuous’) integration build. By definition, tests with wider scope are required here that test that components integrate with each other, that they work correctly together. Developers need to write different automated tests in addition to unit tests for their builds.
If the current popular definition of unit test is to remain:
- developers need to think about what unit tests to write
- or find a way of generating this kind of very basic unit tests
- developers need to think about what functional/integration tests are required for the frequent/regular/continual integration builds
- developers need to separate tests intended to be run frequently within an individual developer’s development environment, from those run as part of a continual integration build, and from those run as part of a regular (nightly or weekly) build.
