An introduction to Continuous Integration (CI) and Continuous Delivery (CD) pipelines for software testers

"Great testers who have a broad knowledge of how systems and automation work, even if they do not write the code for them, can add so much value through holistic reviews. Testers are often referred to as the glue that holds teams together, and a CI / CD world is no exception."

Introduction

When testers hear the phrase "CI / CD pipeline," they tend to react in one of two ways. For those who have worked closely with pipelines or are automation specialists, they hear it as an opportunity. But for those who are not automation specialists, it can be scary. I’ve heard people say or write things like: 

• I don’t really know anything about them other than it is automation.
• As a manual tester I’ve nothing to contribute to pipelines.
• I don’t write code so I don't need to know anything about them. 
• There is a specific role for doing that so I’m not involved.

CI / CD pipelines are certainly part of automation, but there’s so much more to it than that. So in this article I’m going to share: what they are; why all testers should understand how they work; and why they need to care. So let us start at the beginning with what they are. 

What are "CI" and "CD"?

Continuous Integration (CI) 

This is a process that runs automated checks each time a developer commits their changes to a code repository. Generally, when using CI, developers will make small changes and commit more often. This way they get fast feedback that their change hasn’t unintendedly broken any key components. 

Continuous Delivery / Deployment (CD) 

These are two related but distinct methods. 

• In continuous delivery, the team builds, tests and delivers code to internal environments such as development or testing. Continuous delivery is most often used by organisations that do not have regulatory or other constraints, so you probably won’t see it in a financial environment. 
• Continuous deployment is the next step in the process. Code that has passed all the tests is sent to production. 

To summarise, integration is part of the development process and delivery happens after the code is complete. 

What are CI / CD pipelines?

CI / CI pipelines defined

Each organisation that uses a CI / CD pipeline has its own definition of the term. Here are some brief explanations from leading CI / CD pipeline platforms:

• A CI / CD pipeline is a series of automated steps that helps software teams deliver code faster, safer and more reliably - CircleCI 
• A CI / CD pipeline is an automated process utilised by software development teams to streamline the creation, testing, and deployment of applications - GitLab 
• A CI / CD pipeline automates your software delivery process. The pipeline builds code, runs tests (CI), and safely deploys a new version of the application (CD) - Semaphore 

What are the benefits? 

Automating the movement of code from one environment to another can remove a number of manual steps in the process of software development. But that is just one aspect. Here are a number of other benefits that organisations have reported. 

• Faster releases and reduced time to release to production. It takes less time to build, check, and deploy the code. 
• Better code quality. Since code is tested as soon as it is committed, any mistakes or errors are found quickly. 
• Fast feedback. Similar to the point above, developers get feedback immediately, but as the code moves to production quicker, it can lead to faster feedback from customers or clients too. 
• Less downtime. Urgent fixes have a quicker route to production, reducing downtime, which also makes it easier to recover from or address incidents. 
• Supports DevOps. Organisations that bring development and operations closer together also use CI / CD more often. 

This is not an exhaustive list, but it does cover the main benefits for organisations that use pipelines. 

Types of tests commonly run in CI / CD pipelines

Visualisation from Tech Target article CI / CD pipelines explained: Everything you need to know

Pipelines can include several different types of tests. Here are some examples.  

• Unit tests. Focused on individual functions or methods, these tests validate the building blocks of the code. Typically, these are the quickest automated tests to run. They help validate logic and help reassure developers when they refactor areas of the code base. 
• Regression tests. Focused on the effects of code changes, these are usually built up over time to check fundamental functionality. They can also include tests created after bugs or issues are found. They offer a level of confidence in the releases. 
• Integration tests. These tests verify that modules or components work together as expected. The components can include databases, API’s, microservices, and so on. These tests are particularly good at identifying dependency problems or data flow issues. 
• Performance tests. Depending on the application or system under test, these tests may explore scale, response times, degradation, and more. Essentially they look for anything that might slow down the system and can be useful in helping optimise performance. 
• API tests. Generally focused on validating endpoint response logic, these tests can also cover data integrity. They can be important when microservices are used. 
• Security tests. Automated security tests are good ways of ensuring all the low-hanging fruit of potential security issues is covered. While automated penetration tests and vulnerability scans can never replace a skilled human hacker, they can ensure the human is focused on the most important testing. They can also be a regulatory requirement in some industries. 
• End to end tests. Generally focused on real-world user journeys, these can often be the flakiest and most costly tests. That said, when designed well, they add another level of confidence that the main functions are working as expected. They can also be looked at as the highest form of integration test, since they look at whole system functionality. 

This list is not exhaustive and different teams may use some of these tests in different ways. 

If you work in an environment that uses pipelines, it is important for you to know what the pipeline is covering and how it covers it, so you can complement and enhance the testing. Simply knowing there is automation is very far removed from understanding what it is and what it does. 

Why CI / CD pipelines matter to testers

As you can see, CI / CD pipelines are nothing to be scared of. As testers, even those of us who are not automation specialists, we can contribute to them in various ways. We should care and gain understanding of these practices and processes, since they can have real benefits for testers in the right environment. Your contribution to tests in pipelines will not only increase quality, but it will also strengthen your reputation and value. 

Here are some of the top benefits for testers (and the organisations that use CI / CD pipelines). 

• Enhanced exploratory testing. As we are well aware, automated testing can only take us so far. Good CI / CD practices allow testers to focus on exploratory testing, looking closely at the application or system. They can focus on those scenarios that are complex and difficult to automate. 
• System quality characteristics / Non-functional testing. With many functional tests automated, testers can focus on areas such as security, performance and accessibility. They can really utilise their critical thinking skills to dig into areas such as testability. Does our logging and monitoring give us the level of observability we need to respond quickly to issues? There are many ways system quality can be improved given the time and focus. 
• Better quality pipeline tests through reviews. While the tester may not be the one writing the automation tests, they can still contribute by reviewing and suggesting them. Are we missing any critical tests that can be automated? Are we testing consistently? Can we remove any bloat in the test collection? Testers can make objective and valuable contributions to the success of automation suites by understanding what exists at the moment and using their skills to improve on what's already there. 

Great testers who have a broad knowledge of how systems and automation work, even if they do not write the code for them, can add so much value through holistic reviews. Testers are often referred to as the glue that holds teams together, and a CI / CD world is no exception. 

For more information

• CI / CD and delivery pipeline ask me anything - Abby Bangser 
• Tests Your Pipeline Might be Missing - Gene Gotimer
• Continuous testing- Gary Fleming 
• Revisited: The Tester’s Survival Guide to Joining a Continuous Delivery Project - Amy Phillips 
• How to Plan and Define Your Continuous Deployment Pipeline - Patxi Gortázar