I’ve said before; I’m kinda the tools-guy at work. Never get a human to do a computer’s job, you know? With that in mind, I’m working on a library of custom rake tasks for making a build automation dev’s life easier. That’s only half the story, though - the ability to make a build in one click is useful, and all, but it’s far more useful when there’s a continuous integration process doing it for you. Checking out the latest code, building it, running tests against it, packaging it, deploying it (I was particularly inspired when I read IMVU’s doing the impossible 50 times a day…!) - all of that good stuff that, without computers, would fail miserably because humans just aren’t good at doing mind-numbingly repetitive tasks from checklists consistently well.

So I’ll explain a few of the design decisions I made when working on rake-dotnet.

Firstly, hopefully the aim to keep the Rakefile as short as possible is understood and well-received. I wanted it to be as by-convention as possible, because honestly, writing a build-script for the 80% case is surely pretty much a solved problem by now; conceptually, at least. It’s often making time to do it that’s the hard part, right?

Secondly: Why isn’t the Rakefile in the root, next to the solution file, so I can just type rake and have stuff happen? I need to cd build; rake before I see stuff! I did this because of the way TeamCity (TC) and subversion (svn) work together. In TeamCity you set up a Version Control System (VCS) root. The simplest thing is one pointed at the trunk of your codebase, so it gets everything. You can customise the VCS roots by telling TC to only check out paths matching some rules you tell it, or omit externals, and some other tricks. However, it doesn’t check out lazily, it checks out greedily (as at 4.0.2 anyhow). Say you set up the VCS root {svn}/trunk and a checkout rule to skip {svn}/trunk/foo - it’ll check out everything from {svn}/trunk and then, afterwards, delete the foo directory. This takes time - at my place, we’ve got a modest project, but sadly a slow svn server - the checkout takes more time than the do-work part of the build.

A bit of background. For this project, we have one svn repo for the code that we write, and one for the 3rd-party libraries, tools, code, and other artifacts that we depend on to produce a build. This keeps the size of checkouts smaller for remote staff over the VPN (which is also not the most robust), and backups quicker (since we make fewer changes to the 3rdparty repo, we back it up less frequently, especially since it’s not complicated to reconstruct because we’ve got commit emails of what changes happened to it). We also only have the one NCover license, which is installed on just one TC build-agent - this means our setup can’t just run all the build-tasks from beginning to end, since the other two agents don’t have an NCover license. So, TeamCity isn’t set up in the simplest-way-that-works. A build project has a number of build configs:

  • Continuous (in Debug configuration) (this will generate AssemblyInfo.cs files, compile, run unit-tests, harvest the artifacts, and publish them to the CI server for later configs’ use)
  • Coverage
  • FxCop
  • TeamCity Duplicate Finder
  • Continuous (in Release configuration)

It would be easy to see that later we’d also want:

  • Integration Tests
  • NDepend
  • Deploy to test environment
  • UI Test
  • Deploy to production
  • Notify clients of new functionality drop :-)
  • etc

As I say, running end-to-end in each build-configuration on each agent is not only not possible, it’s wasteful. It’s also wasteful to hammer our feeble little svn server for a full checkout of stuff we don’t need during subsequent stages (we don’t need the source-code if we’ve got a convenient zip file of all the build output already, right?). Fail fast. So, how? VCS roots.

What all of this effort lets us have is a config up front that does the basics (compile, unit-tests-that-are-fast, package, etc), then pushes that up to the CI server; the process then branches out into parallel-building by taking advantage of the fact that TeamCity gives you 3 agents for free. Each config pulls down the DLLs etc that the first one generated, and runs off them. The subsequent configs will run in parallel, then the Continuous-Release one will run if all three of the Order-2 ones pass.

Now, if only I could get the git plugin to work. No joy, so far. Git renders some of this irrelevant, because its checkout speed is so much faster that svn’s (in our environment, at any rate). Another day…