"Taking a quality engineering approach gives the testers on your team the opportunity to do more. And when you give them the room to do more, they will identify and solve interesting and complex problems. By solving these problems they improve the experience of everyone else working on the product."
You may have heard the term "quality engineer" before. What is a quality engineer? And how is it different from the other job titles used to describe people who work in software testing and quality?
In this article I’ll share my high-level definitions of various job titles. I'll then describe a quality engineer, and why it’s my preferred title for people who do anything related to "testing."
I'll also share some examples of projects delivered by my quality engineering teams that, while some may consider them outside the scope of typical "quality" or "testing" work, have been of great value to our organisations.
Testing job titles you've probably heard of before
Quick caveat: this article is simply my opinion and perception of what a person with the given job titles are likely to do day to day. I realise this will vary greatly between organisations.
Tester
- Someone that undertakes testing, likely without the use of automated checking or testing tools
- Often is involved only in the dedicated test phase of the software development life cycle (SDLC)
- Often hears: “why wasn’t this caught in testing?” when defects show up in production
Quality assurance or QA
- Another word for "tester"
- May begin work before the SDLC's traditional "test phase" begins, thanks to the increased popularity of Agile processes and the holistic testing model
- Still considered "the bottleneck" holding up release to production
- Is often charged with "assuring quality" on their own
Test automation engineer
- Someone that spends the majority of their time writing, updating, and maintaining test automation frameworks
- Often hears: “why wasn’t this caught by our automated tests?” when defects show up in production
Software development engineer in test (SDET)
- Does work similar to that of test automation engineers
- May also be involved in pipeline creation and other "development" tasks that contribute to improving developer experience
- Primarily focussed on testing-related software development tasks
Quality coach
- Provides guidance and support so that everyone on the team owns and improves quality practices
- Facilitates change, educates people on new ways of working, and is a passionate advocate for quality
- Unlike the other roles mentioned above, likely to be more "hands-off" when it comes to testing and development activities
Quality engineering: A holistic approach
A good quality engineer combines the best traits of all the previous roles. Quality engineers balance manual exploratory testing with creating and maintaining the most valuable test automation. They also provide coaching, guidance, and support to enable a shift-left approach to testing and quality.
Most importantly, quality engineers take a holistic approach to quality and testing. This enables them to ensure that quality is at the heart of the entire software development lifecycle (SDLC), and, as much as possible, at the heart of what the organisation does.
Yes, I really did just say a good quality engineer does what five people in other roles might do. But they aren't the only ones responsible for each of those tasks. A quality engineer is, first and foremost, an enabler. Sure, they dive in and get their hands into code when it makes sense for them to do so. But they also set up their teams for success by ensuring that they themselves aren't indispensable to the quality effort. When they are out sick or on vacation, things can continue without them or the need for someone to cover.
Why "quality engineer" and not some other title? I think it is the most accurate one to describe the set of skills necessary for the role. They are engineers that specialise in quality just as we have specialist engineers in data, front-end, back-end, full-stack, security, and so on. Like other specialists, they undertake work within that specialty while they support others to learn and do better work in their own areas.
Adopting a quality engineering approach: My experience and recommendations
What does "quality" mean?
I often talk about "quality" as improving the experience of everyone involved with a product: developers, product owners, customer support, customers themselves, and beyond. The role of a quality engineer is not only to identify the pain points experienced by this broad demographic of stakeholders, but also to propose and implement solutions to improve their experience.
However, you can't change to a quality engineering approach overnight. Getting to a stage where your quality and testing folk can work as quality engineers requires an investment of time and energy.
Quality engineering is a culture
You can change job titles and job descriptions. However, to succeed at quality engineering, it takes the right culture.
As a first step, the people on your quality team need to buy into the improvements that can be made, the journey the team will take to get there, and their places in it. For every organisation this transition will look different, though there are themes common to all transitions.
Implementing a true quality engineering culture also requires getting the rest of the organisation on board. In my experience this cannot be a big-bang, sudden change. Like many large-scale changes, talking openly about the desired end-state before you begin the transition is a good idea. After the concept has taken hold, you should plan and take small, deliberate steps to work towards the goal. This helps to build confidence in the approach as small changes begin to add up to big improvements.
The skillset of the quality team will likely change over time. The quality engineers-in-training should become involved in areas outside their usual realms of expertise. For example, they can learn to build pipelines and tools that reduce manual task repetition and provide early feedback. With the essential busywork automated, they can focus on learning, exploration, and critical thinking about improving processes. Exploratory testing is a great way to encourage systems thinking, as is suggesting and designing ways to improve test coverage.
Our team's move to quality engineering
I've helped lead a transformation to a quality engineering approach. The interesting part of that journey is that at first we didn't think of it as a transition to quality engineering. The initial motivation for the change was a shift from a fortnightly deployment cadence to multiple times per day. However, what we had to do to achieve the improved release cadence was just as much about a shift towards quality engineering as it was anything else.
The improvements we made required taking a holistic view of quality, looking at how we could improve processes in all phases of the software development lifecycle. On our team, the shift of mindset meant that the quality engineers-in-training were encouraged to suggest process improvements outside of their usual testing activities. Quality engineers brought together people across disciplines to collaborate on small, iterative improvements: ways of working, building pipelines, and so on. All of this gradually moved the company towards the ultimate goal of more frequent AND more reliable releases.
Below I describe some of the improvements we saw as we moved toward a true quality engineering approach.
Some examples of quality engineering wins
These projects aren't traditional "testing" or "quality" initiatives. But they helped our team deliver better products.
Collect and report DevOps Research and Development (DORA) metrics
We introduced collection and reporting of DORA metrics. This was an iterative process that started with basic manual data collection and reporting. Over time, as we saw that the information was valuable, we automated metrics collection and dashboarding. We set it up to make metrics available for each of our repositories, allowing each team to monitor their own performance.
Set up ephemeral test environments for quick test results
Ephemeral test environments are short-lived, temporary infrastructures for deploying an application or service. To create these environments, our team worked with a platform team to create the infrastructure-as-code and associated pipelines.
With this capability, teams could test their code early and often. A development engineer could make code changes, then raise a pull request. The pull request would trigger the deployment of the full application to an ephemeral environment, where a suite of automated checks would run and report results. After the pull request was merged or closed, the environment was deleted.
Provide testing as a service (TAAS)
We built workflows on top of the ephemeral test environments to provide a centralised, easy-to-use way of running the full suite of automated tests. The workflows were designed so that even someone unfamiliar with them could execute the tests they needed in the environment they needed, obtaining clear results at the end. Everything was available in a centralised location with guardrails so that you could run tests at all levels of environment, including production, while protecting you from doing anything that could be damaging.
Deploy smaller releases more frequently
Our quality teams improved processes and ways of working across engineering that ultimately enabled more frequent and higher-quality deployments. We contributed directly to pipeline improvements, advocated for improved developer branching practices, and conducted experiments to build confidence in proposed changes.
Improve the incident management process
Our quality folk had always been close to our incident management: supporting root cause analysis, testing patch releases, and so on.
So when we were looking to improve our incident management processes, bringing on more people to support it, the quality engineers played a huge part. We ran "war games" with support to simulate real-world incidents and follow the full incident management process. This gave people a hands-on opportunity to learn and understand not only the process, but also how best to work with our observability platform to understand the issue at hand.
Build a close working relationship with support
We learned to work even more closely with our technical support staff. Among the process improvements were:
- Enabling support to raise "urgent" severity incidents for speedy investigation
- Giving support a separate channel for less urgent queries that would still benefit from engineering insight
- Setting up a regular meeting for core engineering and support leadership to identify and address important customer-facing issues
To wrap up
Taking a quality engineering approach gives the testers on your team the opportunity to do more. And when you give them the room to do more, they will identify and solve interesting and complex problems. By solving these problems, they improve the experience of everyone else working on the product.
When everyone's experience is improved, the quality of your product improves, resulting in happier customers and employees. What's not to like?
For more information
- Adding Value Outside Automation, Steven Burton
- Get Out Of The Testing Game, Bill Matthews
- Community Stories: Collaborating With Distributed Teams on Culture Transformation and Good Practices, Iram Arshad
