Keeping 'CI' feedback time low by putting developer builds first

More and more people are now adopting some kind of Continuous Integration system, where their full suite of tests and verifications are run on every commit with the aim of giving fast feedback to developers. As this adoption grows, and more and more tools are created to help with it, feedback time for developers can actually increase rather than decrease. This can often happen in larger organisations where a dedicated Release Engineering team exists and who write a lot of product builds rather than the development team taking this on themselves.

Builds are now being written in such a way where developers have no choice but to check-in/commit & push to check whether their code compiles and passes various tests. Now you might say that’s a good thing and developers can then get on with writing more code for a new feature, after all that’s what they are paid for. But not necessarily. Context switching is a massive productivity killer. It might all be ok if all tests pass and none of the checks on code quality fail, in which case everyone is happy and life goes on. But what if the build fails for one reason or another? That developer then has to park what they are now doing, remember what they were doing previously and try to work out what went wrong. Not too much of a problem? Add more developers, more code, more checks and it starts to become a real nightmare.

So how can we help this? By making sure the build can be run on a developer's machine. Unit tests and, ideally, acceptance tests should be runnable within 3-4 minutes, so they can be run every 10 minutes or so. Unit Tests should be even quicker so they can be run repeatedly when refactoring. Once there is some code that is ready to be checked in a longer build can then be run, that perhaps takes 10 minutes, while developers grab a coffee or play some table tennis.

But what’s the point? Forcing tests to be run in a CI environment increases the time it takes to get the results. Ok the tests might take as much as 25% less time to execute due to more power or parallelisation, but what about the extra time it takes to wait for the CI server to figure out there’s been a check-in; wait for a machine to be available; pull down the code from the server? By the time we’ve done all that developers could have run the tests, seen a failure, fixed the code, run the tests again and then checked in. And all this time the CI build is still green so other developers aren’t blocked pushing more commits themselves.

Sounds easy, right? Ok, so I agree sometimes its easier said than done. But it is well worth bearing in mind when adding builds to a CI server. There are some things to watch out for. One big problem can come when the CI build is vastly different from the developer’s build. If a portion of the build is going to be run in both environments they need to be 100% identical, otherwise this leads to broken builds and developers claiming ‘It works on my machine’. An easy way to avoid this problem is to forget about CI to begin with. Write a build that can be run on any developer box and then, without changing the build, configure the CI server to run the same build. Extra checks that are CI specific can then be tacked on to the end.

So how do you keep 'CI' feedback low? Empower and encourage developers to run the full build on their own machine rather than forcing commits to be pushed in order to run them in the CI environment; keep the CI environment as identical as possible to developers' environments; & developers - write your own builds so they are written to work for you.