Home / Posts tagged "TeamCity"

TeamCity: Improve Build Times with Compressed Artifacts

cross-post from http://blogs.jetbrains.com/teamcity/2013/06/04/improve-build-times-with-compressed-artifacts/

Do you work with build chains and artifact dependencies? Do you wonder why it takes so much more build time when your artifacts get bigger and bigger?

You probably have the same potential for improvement as I had in my current project. Read on to learn how you can get your builds faster and waste less time waiting for your builds to complete. A fast feedback loop is of major importance, especially in agile software development.

The issue

In the project I’m currently working on, we have a fairly complex build process with many artifact dependencies between the various stages in our build chain. At some time we felt the build started slowing down. In the next retrospective meeting my team decided that I should analyze the build process and implement improvements to get us back to fast build times.

In this post I’ll only write about how I implemented the improvements. Let’s have a look at our build chain first:

BuildChain

In the above diagram you see the 4 stages in our build chain: build, test, installer, publish. You can see that the artifacts of the Integration-build (.NET) are used in four dependent builds: 3 test runs and the installer. These artifacts are about 1.2 GB in size and consist of thousands of dll’s and other small files. Now, keep in mind that the TeamCity system has to copy those artifacts 5 times per build (e.g. 6 GB in small files). Why 5? – You got it. Once from the Integration-build (.NET) build agent to the TeamCity server (artifact publishing) and 4 times to distribute them to the 3 test run and the create installer agents (artifact resolving).

Remember what you learned about reading and writing very small files from and to a hard disk? It is very slow compared to writing one large file with the same total size. Let’s have some numbers to see the impact on our build times: Resolving the build artifacts takes about 20 minutes for each of the 4 builds. So, it takes 80 minutes of CPU time just to resolve the artifacts in our build, which is triggered on every commit. As we are running these 4 builds in parallel, we actually “only” lose 20 minutes with artifact resolving, but it’s still too much. And it causes a higher CPU consumption on our virtualized server environment.

How can we improve and get our build faster?
Continue reading

Continuous Integration with Sql Compact 4.0

In our project we are using Sql Compact 4.0 for testing the nhibernate mappings and our queries. In production environment we use Sql Server 2008 Express R2, therefore we thought that Sql Compact 4.0 is as close as it can get for a file database. Recently we moved our continuous integration system from cruisecontrol.net to jetbrain’s teamcity 6. The teamcity agent is installed on a Windows 2003 R2 x64 server. But our unit tests accessing sql compact 4.0 didn’t work on teamcity! Let’s look closer what happened and how we solved it.

Continue reading