How to Decide on Unit Test Coverage

Testing is an integral part of any software development process and unit testing is one of the fastest and most reliable testing methods to ensure your code works properly. But despite their seeming simplicity, unit tests cause many questions: do I use unit testing correctly? Should I aim for 100% test coverage and what percentage is enough?

In this article, you’ll learn about all hidden gems of unit testing as well as about the sufficient percentage of code coverage and ways to measure it. Ready to up-level your unit testing game? Let’s get started.

Unit testing defined and explained

A unit test is a way of testing the functionality of the smallest components of the code that can be logically isolated in a system. These individual components can be a single function, class, or object that contains logic.
The main value of unit tests is that they reveal a large percentage of defects and help to make sure that the code works as intended before it gets to the deployment stage. As well, unit tests automate and speed up the testing process, reduce the complexity of bug detection, and increase test coverage percentage since attention is paid to each tested unit. 

It’s worth mentioning a test pyramid here. This is a testing methodology that was designed with automation in mind and that helps QA engineers fine-tune their testing process. As the name implies, the framework is designed in the form of a pyramid, where unit tests are the pyramid’s base. Then come integration tests and the vertex of a pyramid is E2E (end-to-end) tests. Why are unit tests placed in the pyramid’s base? Because this testing method is faster and easier than others and thus helps save time and improve the quality of software products. 

Main benefits of unit testing

Unit testing may seem like a tedious process since developers have to write these tests themselves. But in the long run, the benefits of unit tests for a business become obvious. Unit testing is the fastest and the most direct method of detecting and fixing bugs before they affect the underlying code. As for other significant benefits, they are:

• Rapid and simple testing process: any changes to the software entail additional costs and risks.  Modular testing makes it easier to make changes and helps avoid regressions, which results in a safe refactoring process. 
• Reduced costs: detection of bugs and errors early on leads to greater efficiency, reduced downtime, and lower costs. This, in turn, impacts the speed of the development process. 
• Simplified integration: in unit testing, separate parts of the program are tested first, and then the whole program is tested. Subsequent integration testing is therefore greatly simplified by testing individual modules first.
• Early detection of bugs: unit tests help identify issues at an early stage and eliminate them without impacting other parts of the code before moving on. Such issues can be errors in the programmer’s implementation or flaws in a module specification.
• A better understanding of functions: by analyzing how unit tests work, developers can have a better understanding of the functions covered by unit tests.

What is code coverage? 

As the project grows, it becomes more difficult to determine which code parts were tested and which were not. Code testability can be measured with the help of the code coverage metric.

Code coverage is a metric of software testing that determines the number of code lines successfully tested. This metric helps developers detect and eliminate “dead” code, as well as understand what part of the source code was tested. In addition, the metric checks for missed or uncovered test cases. 
Code coverage is usually measured with a specialized tool such as SonarQube, Clover, or CodeCover. Such tools will not only give you a percentage of code execution but will also allow you to drill down into the data and see exactly which lines of code were executed during testing.

Ways to measure code coverage 

The code coverage measurement tools listed above use several criteria to determine how the code was tested during the execution of tests. These criteria are:

• Function coverage: the tool calculates what functions in the source code are called and executed at least once.
• Statement coverage: the tool calculates the number of operators that were successfully tested in the source code.
• Path coverage: the tool calculates threads that contain a sequence of controls and conditions that have worked well at least once.
• Branch/decision coverage: the tool calculates decision control structures (e.g. loops) that were executed well.
• Conditions/expression coverage: the tool calculates logical expressions that were checked and executed both true and false according to test runs.

The code coverage of the executable program code is calculated by the formula: the number of code lines, covered by tests, divided by the total number of lines of code and multiplied by 100%. For example: if the tested software contains 100 lines of code and the number of lines of tested code is 50, the percentage of code coverage is 50%.

By now, you are probably thinking that 100% code coverage is what you should aim for since a higher code coverage equals better code quality. But that’s not really true. So what is the perfect percentage of code coverage to strive for? About that below.

Why 100% code coverage does not equal 100% quality

Even though many people believe that 100% code coverage equals perfect code, it’s not. The truth is, a 100% code coverage does not guarantee that the covered lines or branches of code were tested correctly – it only shows that they were executed by the test.  On the other hand, a low percentage of code coverage means that large areas of your product remain completely untested. Hence, the question arises: what percentage of test coverage should you really aim for?

There is no universal answer to that question since the required percentage of test coverage will depend on your business needs and the complexity of an application. As well, it’s recommended to cover with tests those critical parts of an app (or app’s logic) that users use the most. For example, the “About us” page is not highly important for the product and does not usually get many visits from users – hence, you may skip it while performing unit testing.

As for the coverage percentage, there are general recommendations such as 60% is “acceptable,” 75% is “commendable,” and 90% is “exemplary.” However, many developers prefer to follow the 80/20 rule, meaning 80% test coverage is quite sufficient and the remaining 20% can be tested later and upon necessity.

It is important to remember though that measuring code coverage is no substitute for good code analysis and good programming techniques. Instead of aiming for 100% code coverage, it will be better to analyze the code base and see what makes sense to try next. In general, you should set a reasonable coverage goal and aim for an even coverage percentage for all modules of your code. 

Note though that for critical systems (i.e. rocket science, healthcare, finances) the maximum code coverage is a necessity. Hence, such systems should be 100% error-free and 100% covered by tests.

When can you skip unit testing?

In general, it is recommended to implement unit tests on a regular base since they are so beneficial for the software. However, there are certain cases when unit tests do not work so well or don’t have to be applied:

• Limited time: sometimes writing a test takes more time than writing code. And since unit tests are useful for testing pure business logic within a particular function, it is worth thinking twice about implementing tests when the deadlines are tight.
• Integration and performance bugs: unit testing is unit-oriented and it is much more effective when combined with other testing techniques. This means that integration or system level errors will not be detected by unit tests. 
• Unstable system components: code that interacts with such parts of the system as ports or timers is quite difficult to test in an isolated environment. Unit tests can still be applied in this case but overall, it’s not really recommended.

Final thoughts

The main piece of advice that we can give here is to set your priorities and decide what code area is critical for your app’s performance so it can be tested foremost. After all, the primary goal is to make sure that the code meets the functional requirements. And if unit tests, for whatever reason, do not give you confidence in the quality of the product, you should find an alternative approach that makes the most sense for your project.