Source control with continuous integration process

Recently, while reading the book on Continuous Integration Improving Software Quality and Reduce Risk, I chance upon the following practices:

Commit code frequently. Try to make small changes and commit each task when is completed. Waiting more than a day or so to commit code to the version control repository makes integration time-consuming and may prevent developers from being able to use the latest changes. The longer you wait to integrate with others, the more difficult your integration will prove to be. Developers do not want to commit their code until it is “perfect”.

Yes, commit code frequently is very important especially in a CI (Continuous Integration) environment. But how frequent is frequent? Above suggested more than a day is too long and should day be used as the benchmark? In usual practice, task often assigned can be further breakdown into many smaller units. Does that mean we can commit this small unit each time is completed? Yes, but that will probably result in codes committed every hour. Other than that, usually codes are developed in the smallest unit before you work on the bigger units which depend on the smaller unit. Therefore, if there is no other part of the project having dependency on this small unit, then there isn’t really any advantage to commit the codes to allow other to make use of them.

From experience, people working on very closely related task often encountered conflict when attempting to merge codes. So even merging codes is a problem, not to say about integration. Therefore, closely related task assigned to different developers have to properly manage. The management of such process is very important.

Using source control is highly recommended, but I noticed that some common practice on committing, updating of source control are seldom discussed. Was it because everyone knows about these practices? I doubt so, with my recently experience working on a project with 6 developers of some with more than 5 years experience.

1. One single developer can be committing codes up to 5 times in a span of 2 min. What happen here? These people are committing codes not from highest trunk level but from each folder where changes are made. Imagine you happen to get an update in between the 5 commit? Build fail as probably your update did not capture the last 2 commit which the first 3 commit have a dependency on the last 2.

2. Every time I get an update of the source, my build fail. And usually, this is not due to a single fault where someone makes a mistake, but very often 3-5 mistakes are found that have cause the build to fail. What happen here? Partly due to point 1 and codes are not fully committed. Committing one folder but not the other which they have a dependency.

3. Test fail. Developer are not getting an update and running test locally to ensure latest version merged and integrated seamlessly with their changes.

The consequence of such ill disciplines resulted in many man hours lost. I spend an average of half to an hour of time going thru the error and hunting down the person causing the build to fail each time I get an update. While trying to get to the bottom of this problem, I realize that the Cruise Control never had a single successful build for a week with up to an average of 40 build a day. Digging deeper and after much questioning, I realized that the some project have been removed from the build due to some testing causing the whole build process to hang.

Not to give up, I added those projects in and wanted to see how where the problem is. One amazing problem found was a test actually open up a notepad and cause the whole build to hang there. Next, a test in one of the project keeps causing an error on NANT which need human intervention to go into the build server to close the error prompt. Failing to do that, result in NANT time out and each build just stalled there. Worst still, no one make any noise about this issue.

Therefore in order to minimize such occurrence again, I have come out with the following guideline while using the source control and continuous integration:

1. Get an update before working on any task.
2. Work on the task with necessary test executed successfully.
3. Get an update again before any commit.
4. Run all test again to ensure that you task or changes integrate with the latest version without any problem.
5. Commit your task in one revision.
6. Check that build on the build server run successfully.
7. Fix or rollback any error reported immediately.

Lastly, the book mentioned above is a great for anyone that are interested to implement the continuous integration process or enhance their existing process. The book covered the database, source code, test and deployment continuous integration. Imagine without this process and having to encounter similar problem above will be a nightmare of getting things right.

Version control for database?

Having a database under version control have it own advantages and disadvantages. K.Scott Allen recently has written a five part series on database version control. I think his article provide a very good background on getting the database under version control. Jeff Atwood also highlighted that the database are very often neglected in most project. To my understanding these articles seems to have assume that all database should be kept under version control. I think the first question should be whether version control for a database is required? The following are a few questions to ask yourself before deciding on whether you should have your database kept under version control:

1. How many people have the right to make changes to the database?
2. Is there any problem with the development and testing with a constant change made to the database?
3. How many environments do you have from development to production?
4. How is the deployment for the changes to the other environment carried out?
5. Have scripts not meant to be deployed, been deployed during deployment?
6. Any review done to the changes made to the database?
   

Application with big and multiple databases usually have more that one single person having the right to change any part of the database and this can be worrisome, as things can get out of hand easily. Version control will be able to help in keeping track what the changes are made over time. This is very useful for everyone involve in the project, from the developer to the DBA as everyone will be able to track the changes made. By using a shared database, how often and badly does it affects other developers working on the same project? If the impact is unbearable or consuming too much time and effort, then probably keeping the development database separately might be good. However, there is trade off as keeping database in sync is not going to be an easy task without proper process and tool.

Having too many environments such as development, multiple testing environment and production is going to make deployment an uphill task. With each environment in the different phase of a project will make things even more complicated. Therefore, with version control of the database, the different version or phase at the point of time can be easily identified. This will perhaps make the change management easier to manage and prevent wrong version from been release to production. If you find yourself in the mess described, my advice is to go ahead and get your database under version control.

Imagine with a click on the checked in log, you can easily track all the changes done to a database. This is the advantages that not only allows you to just track changes, but it also allow you to review the changes done and any wrong doing. From experience, the process of keeping a database under version control is complicated and can be tedious. If the process is not well thought enough and people not having a clear picture of the version control, result will be the above issue coming back haunting on you again. Therefore, the initially phase in adapting to new process can be difficult, but over the long run the result will obvious.