Continuous integration is a software development practice that is generally used by engineering teams to integrate the work they have been doing. This type of integration merges the code and helps in identifying the bugs that appear during the process. The development teams take care of the feedback immediately and pushes back the code to the mainline for further integration.

The main aim of implementing Continuous Integration is to reduce the tension around Integration, improve visibility, speed, and efficiency. Continuous Integration is one of the popular practices in Extreme Programming (XP) and when followed with other XP practices such as TDD, Refactoring etc can set you up on the pathway to practicing Continuous Delivery. One of the common setbacks of shared development models is, every developer connects to the same copy of the production database on the same server. This may lead to developers overwriting each other’s modifications. Continuous Integration helps in avoiding one developer’s work-in-progress from breaking into another developer’s copy. The deployment process becomes easy and fast as the developer makes sure that the application on the mainline is always ready.

History of Continuous Integration 

Before the advent of CI, the task of merging the codes and testing them was difficult, time-consuming, and expensive as each code must be merged separately to check the features, and the results were not always favorable. With the advent of CI, the task of testing and merging became a simple, fast, and easy process. The software developers were able to save their energy, resources and focus on value addition.

The term Continuous Integration was first used by Grady Booch in 1994. Later, in 1997, Kent Beck and Ron Jeffries had brought the concept of Continuous Integration into practice while they were working on Chrysler Comprehensive Compensation System Project. They invented a framework called Extreme programming (XP), which included CI as a practice and advocated integrating the build as many times as needed.

In a publication, Beck has highlighted continuous integration, he mainly talked about face-to-face communication using technology. Beck has published the first-ever book on continuous integration in 1999. Cruise Control, the first open-source continuous integration tool, was released in 2001. With the growth of the Hudson CI, the practice of continuous integration became more popular, later it got replaced by Jenkins in 2011 after a dispute with Oracle which claimed rights to the name.

When Grady invented it, it was not intended to use it for building the entire application. But later, with the increase in complexities of business procedures and coding systems, the process of automating and building the entire application became mandatory. Automating the testing and integration process improved software readiness while practicing version control and maintaining mainline helped to keep the code clean.

Back To Contents

Getting started with Continuous Integration

Continuous Integration practice is used, only when the code is checked in and is ready for integration. The working of continuous integration is a stepwise process, and each step is preferably automated. Build automation, automated integration, version control, and automated testing are essentials of continuous integration practice. Software Integration is a long and unpredictable process but following certain principles and taking the help of appropriate tools can help you get started with CI more confidently. Briefly, the continuous integration workflow would look like this.

Maintaining a single source repository

When multiple developers and testers are involved, keeping track of the build, and orchestrating it at the time of deployments can get tricky. Source Code Management tools, configuration management, version control systems, repositories – call it anything but having a source code repository system improves code re-usability, minimizes messiness, and gives you control over the endless modifications.

Pro tip: Just because your mainline allows you, don’t abuse/overuse the branch creation. Try and have one mainline unless the project really demands for more.

Automate your build

Compiling your code, moving the files around, and loading the schemas into databases can be tedious each time you try to get your code into the mainline. Whether you work with UNIX, Java, or .Net, there are features to generate artifacts and you can build and launch your system using a single command. It is a necessary pre-condition for Continuous Integration.

Pro tip: Depending on your requirement, chose On-demand, Scheduled, or Triggered Automation.

Automated Self-Testing

In the rise of Test-Driven Development (TDD) practice, techniques like self-testing code have gained popularity and rightly so. This practice involves writing comprehensive automated tests along with the functional software. In the end, you will be able to test your code with fewer commands and it will cast light on the bugs if there are any.

Pro tip: Hiring developers will get you code but grooming Full Stack developers will produce value.

Automated Deployments

Since the production environments could be slightly (sometimes even drastically) different from the developing environments, shipping the code from one to another can break its functionality. In Agile development, you’ll have to repeat the process at least once a day. To avoid ‘It’s your code’ / ‘No, it’s your machine’ disputes between Dev and Ops, automate your deployments. It saves time, reduces errors, and improves consistency.

Pro tip: Always set up a distinct build for each test suite and create a dominos effect by triggering each test case with the one before it. It is the easiest way to identify the problem and avoid shipping irregularities into the Prod environment.

Fix the broken builds instantly

If you have a habit of prioritizing tasks, nothing and absolutely nothing should be written above ‘fixing the broken code on the mainline’. The entire concept of Continuous Integration is about knowing that you're always developing on a known durable base.

Pro tip: Unless the root cause for the breakage is absolutely known and obvious, always take the mainline back to the last known good configuration and take the detected problem to the dev table to debug.

Bring in a Digital Twin of your Production Environment

This is another way to make deployments less stressful and more predictable. Not all, but most of the software upgrades are not reversible. So, cloning your production environment for testing can help you verify the components are working the way they are intended to let you fix everything that may pose a hurdle at the time of deployment.

Pro tip: Duplicating your production environment for testing is only an extra step that is recommended. It is not a mandatory practice. You can totally skip this if your production environment is expensive to clone.

Maintain the build together

When the promise is to have daily commits, there are always new builds in making. But only in an ideal world, the mainline stays healthy throughout the day. In reality, developers not updating the mainline before each commit or difference of environments between the developer systems can cause the build to break. So, it is advised to commit the regular builds on an integration machine first and only if this integration build succeeds, the commit should be considered done.

Pro tip: This can be executed manually or with the help of a Continuous Integration server. Although, the manual way looks simpler, adopting a CI server would give you an edge to practice Automation.

Write tests as stories

User stories are the most important part of the framework. Involving the developers while writing the user stories is the best way and it will also increase the quality of the test cases. Documenting the bugs after fixing them will help us avoid the reoccurrence of the bugs.

Back To Contents

Best Practices to achieve Continuous Integration

• Fail faster to fail cheaper
• Commit at least once a day
• Never ever walk past the broken builds
• Automate your delivery lifecycle as much as you can
• Define a tight Release, Rollback and Rollforward strategy
• In a race between Quality and Speed, Security wins
• Reliability is everyone’s responsibility

Back To Contents

Benefits of CI

Enough has been written on the benefits of Continuous Integration. But, here are a few: 

Back To Contents

Continuous Integration Tools

After the advent of Continuous Integration, many tools have come up. The tools automate the process and make the task easier and help in keeping track of the program. Some of the best available continuous integration tools are

1. Jenkins

   One of the most popular Continuous Integration tools and helps in building, deploying, and automating any project. Jenkins helps to orchestrate a chain of actions to achieve Continuous Integration with an automated approach. It is an open-source software that is gaining popularity across the world and has a vast and active community with over 300k installations globally and a rich plugin ecosystem.

2. CircleCI

   CircleCI allows you to automate the build, test, and deployment of your software. It is available as a SaaS offering and also can run on your own infrastructure. This software integrates with multiple version control tools and creates a pipeline as soon as you commit code. From running automated tests in a VM, notifying the team, to automating deployment; CircleCI is known for improving performance, giving the development teams control and flexibility.

3. TeamCity

   It is a Java-based Continuous Integration server that offers flexibility to work with different SDLC workflows. TeamCity optimizes the code integration cycle and makes sure that you never get broken code in the repository. This CI server can run parallel builds simultaneously on different environments and platforms. It is available in a Freemium model.

4. Bamboo

   Bamboo is a CI-CD server from Atlassian which will stabilize and tie your builds, tests, and releases in a single workflow. This tool helps the teams to build, test, deploy and connect. In addition to offering end-to-end visibility into release implementation, Bamboo’s flexibility lets the developers focus on writing code than integrating with other tools.

Back To Contents

Challenges and Solutions in adoption of Continuous Integration

1. Lack of Environments

   In most cases, testing teams have limited access to the production environment for testing the software. Multiple developers and testers could be simultaneously committing code to the same continuous integration server and performing parallel testing. As different tests would require different environments, running them all in the same environment could cause wrong indication of failed tests. Hence, the testing ability gets compromised.

   Creating dedicated testing environments in the cloud that can be used on-demand can be a solution for this challenge.

2. Ownership Challenges

   Software Bugs can be identified at any stage of your release cycle. The longer it takes us to detect the bug, the harder and costlier it is to fix those bugs. That is the very reason it is strongly recommended to have a well-documented Version Control in place. That way, each bug can be sent back to the developer’s table and they can fix it and push it back to the assigned owner of the pipeline. This improves ownership and promotes feedback-driven improvement of the pipeline.

   There are many open-source CI-CD orchestration tools available in the market that can track the changes, detect bugs, and inform the respective personnel in time to minimize the damage.

3. Massive applications and multiple pipelines

   As the digital business models are growing, and the complexity of applications is increasing; it is natural for enterprises to have multiple large-scale applications in different levels of production. As the code repositories are mounting up, it certainly becomes a task to analyze issues and dig for a root cause rapidly to match the expectations of delivery.

   This is what we do to overcome such situations. We can employ CI-CD templates for all the project teams across the organization but break down the massive projects into smaller modules. This simplifies the reporting process, speeds up the recovery and ensures delivery is in-line with business expectations.

4. Implementation discipline

   A poorly planned adoption of any methodology creates more complexity than doing any good. A successful continuous integration practice requires a proper integration between different components, tools, and environments. Every step including check-ins by the developers, adding of test cases for better coverage, error reporting, build creation, everything should be done regularly and monitored thoroughly. Implementation of Continuous Integration is not a one-time process. It is a systematic drill that needs to be practiced as long as you’d want it to reap results.

   A flawed Continuous Integration pipeline slows down the entire process which will cost you more than not having a continuous integration pipeline at all.

5. Cost and Resource Management

   Although continuous integration aids the speed and continuity that is demanded by DevOps, one should be cautious with easily consumable public clouds which can easily blow up the budgets. It is a good time to remember that the underlined objective of CI-CD is Sustainability – in terms of code, infrastructure, and resources. Businesses should pull themselves out of ‘at any cost’ attitude.

   Kickstart your cost control by scheduling the non-production resources and signing up for pay-as-you-use model. Analyze the usage patterns of your development teams from day 1 and benchmark your instances to optimize your expenditure.

Back To Contents

Wrap up

Continuous Integration is one of the most popular XP practices for software development across the globe. There is no by-the-book recipe for implementing it nor a one-size-fits-all solution. Start with automating your build, establish source control and monitoring, and you’ll find your wind. It is a process of continuous exploration, learning, and implementation. But the bright side is the availability of various tools including open-source ones to get you started on the right foot.

Are you planning to implement Engineering practices from Extreme Programming? or looking to improve your DevOps maturity? Write to our team of DevOps Experts [email protected]