What Is Continuous Integration?
DevOps drives development today. And one of the most important aspects of DevOps is Continuous Integration.
What Is Continuous Integration?
Continuous Integration is the practice of automating the build and testing of code every time a change is made — and committing that code back to a central repository.
Continuous Integration applies to every developer's code. And it's important for the quality of the codebase. You can do this by breaking up the tasks required for this into small chunks which can be performed frequently.
Continuous Integration and Continuous Delivery
The term Continuous Integration (CI) and Continuous Delivery (CD) are often used together.
CI is used during the build and test phase. CD is used once changes are committed. The ultimate aim of CD is to always have validated and verified code in the shared repository. Validated and verified code has passed testing and met the requirements of coding standards.
So, CI helps to achieve CD.
Why Continuous Integration in Software Development?
Continuous Integration (CI) accelerates software development.
Without Continuous Integration…
It’s a Waiting Game
Developers write the code for the tasks assigned to them. As development proceeds through this phase, developers may commit their changes to the repository. But they usually don’t update their working development code.
Once all the code has been developed, there is typically an integration phase. This brings together all the changes from individual developers. The aim is to provide a testable build of the software.
Code Conflicts Emerge
Developers may have created conflicting code. Each developer’s private copy of the code works perfectly. But when code is merged together, there’s a conflict. They each made changes incompatible with each other’s changes.
The code then needs to be refactored. This can introduce still more conflicts, making it a never-ending task. And in extreme cases, the effort to resolve conflicts is greater than the effort spent on the task in the first place.
You can resolve these conflicts by only allowing one developer to commit to the repository at a time. And when code is committed to the repository, any issues must be resolved immediately. This results in a repository with no conflicts — but at a cost.
Giving one developer exclusive access to the repository can create a significant bottleneck. It is also hard to enforce. And it’s hard to manage with a team larger than a few developers. It’s impossible with a geographically distributed team.
Continuous Integration and Your Release Process
The solution is to stop segmenting development into phases and start using Continuous Integration.
Each stage itself consists of integration, testing, verification and deployment — and they are performed every time a developer commits a change to the repository. So, every change that is integrated, tested, and verified — and each test that is passed — brings the commit closer to being a release candidate.
Using Continuous Integration means the integration burden is split into smaller, more manageable tasks. Big integration problems – so called “Integration Hell” – never get the chance to arise. Problems are dealt with individually, early on, before they turn into big problems.
Test Before Commit
Test Before Commit is a methodology to ensure only good, working, tested code makes it into the repository. It’s therefore a prerequisite of CD. The idea is that only code worthy of release enters the repository. So, all testing of the code must be done before the code is committed.
You can do this by setting up a staging repository to act alongside the real repository.
So, when a developer checks some code into the repository, it goes into the staging repository. The CI server checks the code out of the staging repository and builds and tests the code. If the code passes all the tests, then the commit is transferred to the real repository.
If the code does not pass all the tests, then the commit remains in the staging repository. The developer is notified. And they can take can correct the issue.
Continuous Integration Systems
Automated testing is key to CI. Once the test phases have been automated, the CI system can run the appropriate tests and take action on the results. So, a CI system can be thought of as a sophisticated scheduler, launching compilation, and testing programs.
The simple core requirements of a CI system are the ability to automatically:
- Detect code has been committed (either by monitoring the repository for commits or accepting some external stimulus – usually from the repository).
- Checkout code from the repository on to a machine equipped with all the necessary build and testing tools available.
- Build the code.
- Run all the tests on the resulting executable.
- Report the results of the build and test.
This can be accomplished with some scripting and command line tools. Many companies start off with this type of setup. However, administering these scripts becomes a job in itself, drawing the developers away from their proper task of writing code.
Using Nightly Builds for CI
Many organizations use nightly builds for CI. This is where the codebase is checked out of the repository at the end of the working day, built, and tested. And the results from this build are examined the next morning.
But there are problems with this approach.
There’s a time lag between checking code in and seeing the results. A developer may check in some code in the morning which breaks the build. The developer will not know about it until the next day. By that time, they may have moved on to a different task.
The builds and testing are generally controlled by custom scripts. These require regular maintenance — and they’re often subject to bugs. And it’s difficult to reuse scripts.
And the output of these nightly builds is typically only available to developers (and possibly team leaders). Higher level management would probably like to know what tests the code is passing or failing. And they would want to know if the number of tests passed today was greater than the number of tests passed last week.
Using a CI system is the simplest solution to these problems.
How Continuous Integration Works With CI Systems
- Developers check code into the version control system’s staging repository.
- The VCS notifies the CI server that a commit has occurred. (Or the CI server polls the repository periodically looking for commits.)
- The CI server starts the build process on a build server.
- The code containing the latest commit is checked out of the repository into a local workspace on the build server.
- The code is built and then tested.
- Important results are reported back to the CI server, along with any important files that need to be retained.
- The CI server sets the final result of the build.
- If the build met the success criteria, then code is transferred to the real repository. If the build failed, the code does not get checked in to the real repository (which only contains working, tested, and verified code).
- The CI server notifies any parties who have registered interest in the build. They can then log into the CI server to view the status plus any additional information.