Friday, February 22, 2008

Quantity of test must come with quality

For those that are familiar with TDD, you should be aware of what code coverage is. For those that don’t, do check out the paper written by Brian Marick “How to Misuse Code Coverage”!

From my perspective, code coverage measures certain quantity and not the quality of the test on the code. Quantity figure defined can be easily achieved without any quality. Imagine writing test that just assert whether an object is null will put those line of codes or sequence points visited. However, does this test really visit the complexity involved? If you have one test to cover a method and if you have five tests to cover the same method, the coverage is still the same. These five tests might be written to provide a more exhaustive test (Phil Haack has provided a good overview about the
row test).

How many times are you thrown a code coverage figure to be achieved? I believe the easiest figure to be defined is 100%. But will a project be able to meet this figure realistically, given the resources? How many have written test for the sake of achieving this code coverage figure but without any quality. Probably is important for people who determine this figure to be achieved to make some thoughtful analysis of what is more important. Having read a couple of times that the 100% is unrealistic in many projects, so what make a good and realistic figure? How many have been given a figure of 80%? Was this figure pluck from the air? I often don't see how this figure is derived. To me more importantly, certain guidelines need to be defined what need to be achieved from the test suites rather that just a quantity figure.

The guidelines should define the quality of the test, what need to be tested and what is not (this fall outside the code coverage figure). Most often, tests need to cover the core business domain logic, to prove that they not only pass but all the complexity in business logic defined are tested. Does a facade or delegate without any logic need to have any test? My answer should be most likely NO. Why? Usually a façade or delegate intend is to expose certain methods, so how are you going to test this intend? Accessibility concern can be easily detected when a project is complied.

Therefore, do have a good understanding of what you trying to achieve instead of letting tools available to determine what you can achieve without any significant purpose. Code coverage tools will be useful to provide an overview on the code coverage for the project, but the quality of code and test suite most likely can only be achieved thru a set of standard and guidelines defined with code review. Of course, other tools such as
NDepend which provide different types of code analysis are also available.

Thursday, February 7, 2008

Version control for database?

Having a database under version control have it own advantages and disadvantages. K.Scott Allen recently has written a five part series on database version control. I think his article provide a very good background on getting the database under version control. Jeff Atwood also highlighted that the database are very often neglected in most project. To my understanding these articles seems to have assume that all database should be kept under version control. I think the first question should be whether version control for a database is required? The following are a few questions to ask yourself before deciding on whether you should have your database kept under version control:


  1. How many people have the right to make changes to the database?
  2. Is there any problem with the development and testing with a constant change made to the database?
  3. How many environments do you have from development to production?
  4. How is the deployment for the changes to the other environment carried out?
  5. Have scripts not meant to be deployed, been deployed during deployment?
  6. Any review done to the changes made to the database?
Application with big and multiple databases usually have more that one single person having the right to change any part of the database and this can be worrisome, as things can get out of hand easily. Version control will be able to help in keeping track what the changes are made over time. This is very useful for everyone involve in the project, from the developer to the DBA as everyone will be able to track the changes made. By using a shared database, how often and badly does it affects other developers working on the same project? If the impact is unbearable or consuming too much time and effort, then probably keeping the development database separately might be good. However, there is trade off as keeping database in sync is not going to be an easy task without proper process and tool.

Having too many environments such as development, multiple testing environment and production is going to make deployment an uphill task. With each environment in the different phase of a project will make things even more complicated. Therefore, with version control of the database, the different version or phase at the point of time can be easily identified. This will perhaps make the change management easier to manage and prevent wrong version from been release to production. If you find yourself in the mess described, my advice is to go ahead and get your database under version control.

Imagine with a click on the checked in log, you can easily track all the changes done to a database. This is the advantages that not only allows you to just track changes, but it also allow you to review the changes done and any wrong doing. From experience, the process of keeping a database under version control is complicated and can be tedious. If the process is not well thought enough and people not having a clear picture of the version control, result will be the above issue coming back haunting on you again. Therefore, the initially phase in adapting to new process can be difficult, but over the long run the result will obvious.