Source: https://www.appsierra.com/blog/mutation-testing
The modern frameworks allow building web applications of a high level of complexity and throughput with increased speed of the development. The key part of such applications is a complete test suite that ensures their stable functioning after each change. Although test coverage was traditionally used in order to assess the quality of the code and test suite, it has a number of drawbacks. Test coverage provides a proportion of the code that is executed during testing, but it does not indicate if tests are correct or check all aspects of a program’s logic. The alternative approach is to use automated mutation testing that analyzes the quality of the test suite by measuring its ability to detect and reject intentional bugs. Mutation testing is a method of assessing the quality of the test suite. It simulates the process of manual mutation testing, when a program is altered artificially, in order to create a set of illegal, faulty forms of code. Afterwards, test cases are executed until a mutated version of the code is detected. It allows assessing the quality of the test suite and determining its ability to expose logical errors. The key idea is that if some test case detects the intentional bug, the test suite is trustworthy and covers all important parts of the application. The automated tools for mutation testing analyze the source code and generate hundreds of altered versions that slightly change the logic of the program. All of these altered versions are tested with the test suite. If the test suite rejects a mutated code by failing any test, this mutation is considered killed. However, if none of the tests fail, this code is considered as survived. Mutation testing allows to check more aspects of the code than traditional code coverage. Modern frameworks have complicated the process of developing and testing web applications, thus requiring new approaches to ensure that all parts of the code are correctly tested. Web frameworks contain numerous controllers that route HTTP requests to appropriate parts of an application. The code of these controllers has to be thoroughly tested, since it is executed every time the application receives an HTTP request. Traditional test coverage tools can report that these controllers are tested, but they do not state whether any errors are possible during the request routing. Mutation testing allows us to detect such issues by altering the source code of the controllers and checking whether tests are able to reject such mutated code. This approach can be applied not only to the routing logic, but to all other parts of the framework, including database access, request validation logic, and other important components. Although mutation testing offers better insights than traditional code coverage, it is rarely applied due to its complexity. The main issue with mutation testing is that it is computationally expensive, as it requires testing hundreds of altered code versions. Modern tools address this issue by utilizing optimization techniques in order to reduce the number of tests that are executed. For instance, the mutation analysis can be performed only for the code that was recently committed, instead of the whole project. Mutation testing tools are incorporated into the continuous integration systems of many projects, thus allowing the mutation analysis to be performed regularly and automatically. The computational power of modern computers allows to execute mutation analysis even for large projects without additional optimization. In conclusion, mutation testing provides a better assessment of the code coverage than traditional approaches. It ensures that the test suite of a project is strong and tests all important aspects of the application. Mutation analysis is a valuable practice that should be used in all projects in order to ensure that their test suite does not have gaps and potential logical errors.
Sources
https://learn.microsoft.com/en-us/dotnet/core/testing/mutation-testing https://www.tricentis.com/learn/mutation-testing