Tests Your Pipeline Might be Missing - Gene Gotimer

Developing a delivery pipeline means more than just adding automated deploys to the development cycle. To be successful, tests of all types must be incorporated throughout the process in order to be sure that problems aren’t slipping through. Most pipelines include unit tests, functional tests, and acceptance tests, but those aren’t always enough.

I’ll present some types of testing you might not have considered, or at least might not have considered the importance of. Some tests will address code quality, others code security, and some the health and security of the pipeline itself.

This talk is aimed at people that are trying to build confidence in their software delivery pipeline, whether it is automated or not. I’ll talk about specific tools we use to supplement our pipeline testing. I won’t get into how to use each tool-- this is more of a series of teasers to encourage people to look into the tools, and even letting them know what types of tools and testing opportunities are out there.

Takeaways

• The pipeline offers a lot of opportunities to do tests that you might not have done if you had to set aside an explicit block of time to do them.
• The pipeline is about building confidence that the software is a viable candidate for production. Or realising as early as you can that it isn’t.
• Do just enough of each type of testing at each step in the delivery pipeline to determine if further testing is justified.
• Do the most expensive tests last. Those are often the manual or subjective ones.
• Don’t forget the infrastructure. The pipeline needs to be tested just like the software does.