For those that are familiar with TDD, you should be aware of what code coverage is. For those that don’t, do check out the paper written by Brian Marick “How to Misuse Code Coverage”!
From my perspective, code coverage measures certain quantity and not the quality of the test on the code. Quantity figure defined can be easily achieved without any quality. Imagine writing test that just assert whether an object is null will put those line of codes or sequence points visited. However, does this test really visit the complexity involved? If you have one test to cover a method and if you have five tests to cover the same method, the coverage is still the same. These five tests might be written to provide a more exhaustive test (Phil Haack has provided a good overview about the row test).
How many times are you thrown a code coverage figure to be achieved? I believe the easiest figure to be defined is 100%. But will a project be able to meet this figure realistically, given the resources? How many have written test for the sake of achieving this code coverage figure but without any quality. Probably is important for people who determine this figure to be achieved to make some thoughtful analysis of what is more important. Having read a couple of times that the 100% is unrealistic in many projects, so what make a good and realistic figure? How many have been given a figure of 80%? Was this figure pluck from the air? I often don't see how this figure is derived. To me more importantly, certain guidelines need to be defined what need to be achieved from the test suites rather that just a quantity figure.
The guidelines should define the quality of the test, what need to be tested and what is not (this fall outside the code coverage figure). Most often, tests need to cover the core business domain logic, to prove that they not only pass but all the complexity in business logic defined are tested. Does a facade or delegate without any logic need to have any test? My answer should be most likely NO. Why? Usually a façade or delegate intend is to expose certain methods, so how are you going to test this intend? Accessibility concern can be easily detected when a project is complied.
Therefore, do have a good understanding of what you trying to achieve instead of letting tools available to determine what you can achieve without any significant purpose. Code coverage tools will be useful to provide an overview on the code coverage for the project, but the quality of code and test suite most likely can only be achieved thru a set of standard and guidelines defined with code review. Of course, other tools such as NDepend which provide different types of code analysis are also available.
From my perspective, code coverage measures certain quantity and not the quality of the test on the code. Quantity figure defined can be easily achieved without any quality. Imagine writing test that just assert whether an object is null will put those line of codes or sequence points visited. However, does this test really visit the complexity involved? If you have one test to cover a method and if you have five tests to cover the same method, the coverage is still the same. These five tests might be written to provide a more exhaustive test (Phil Haack has provided a good overview about the row test).
How many times are you thrown a code coverage figure to be achieved? I believe the easiest figure to be defined is 100%. But will a project be able to meet this figure realistically, given the resources? How many have written test for the sake of achieving this code coverage figure but without any quality. Probably is important for people who determine this figure to be achieved to make some thoughtful analysis of what is more important. Having read a couple of times that the 100% is unrealistic in many projects, so what make a good and realistic figure? How many have been given a figure of 80%? Was this figure pluck from the air? I often don't see how this figure is derived. To me more importantly, certain guidelines need to be defined what need to be achieved from the test suites rather that just a quantity figure.
The guidelines should define the quality of the test, what need to be tested and what is not (this fall outside the code coverage figure). Most often, tests need to cover the core business domain logic, to prove that they not only pass but all the complexity in business logic defined are tested. Does a facade or delegate without any logic need to have any test? My answer should be most likely NO. Why? Usually a façade or delegate intend is to expose certain methods, so how are you going to test this intend? Accessibility concern can be easily detected when a project is complied.
Therefore, do have a good understanding of what you trying to achieve instead of letting tools available to determine what you can achieve without any significant purpose. Code coverage tools will be useful to provide an overview on the code coverage for the project, but the quality of code and test suite most likely can only be achieved thru a set of standard and guidelines defined with code review. Of course, other tools such as NDepend which provide different types of code analysis are also available.
No comments:
Post a Comment