Achieving Quality And Speed: Tips For QA Leaders In Delivering Exceptional Products

As a QA professional, I have been fortunate to work with various individuals and companies in the tech ecosystem. Through these experiences, I have gained valuable insights from (in my opinion) some of the top software engineers in the Greek software engineering community. 

In this article, I would like to share a few tips from my personal experience that may benefit QA leadership endeavours in other organisations. These principles have provided me with a solid foundation and guiding principles throughout my journey so far, enabling me to approach testing with confidence.

Clear QA Vision

As a QA leader, it is essential to convey a clear and concise vision so that everyone understands the testing perspective and goals. The QA engineers play a crucial role in the quality of releases and in identifying issues early in the testing process. By involving QA throughout the entire software development lifecycle, the risk of releasing software with defects can be mitigated and also facilitate faster release cycles for teams. 

Below I share an example from my recent experience; the following mission and vision statements were presented to the leadership team of the engineering department and were subsequently shared across all teams. 

Mission: What is the purpose of QA?

We in QA play a crucial role in supporting the product teams within the engineering department. We provide expertise in testing methodologies, conducting comprehensive functional and non-functional tests. Our primary goal is to proactively assess the quality of our products, ensuring that they meet customer needs as outlined in the requirements and service level agreements. By leveraging automation tools and frameworks, we aim to instill confidence in every deliverable and ensure that our customers and users can accomplish their tasks effectively and easily using our software.

Vision: What does the world look like when you’ve achieved your mission?

We will be able to release our products and services fast, often, and with confidence.

Having a well-defined direction for QA engineers is crucial for maintaining focus and achieving success. This direction serves as a guiding light, allowing your team to gain expertise and confidence across different business domains. By doing so, you can reduce uncertainties in the release process and make steady progress toward achieving continuous deployment.

To attain widespread understanding and alignment within the organization, this direction must be clear and easily comprehensible by everyone, from management to each contributor. Moreover, it eliminates distractions that may arise from misconceptions or confusion about the nature of software testing and establishes a transparent direction for your QA engineers, creating an environment conducive to focused and productive work. 

Transparent Test Strategies

To assure that your company's software products meet the requirements and function as expected, it is crucial to have a well-defined and transparent test strategy in place to avoid missing any critical testing layers or types. 

Implementing a test strategy demands substantial effort from us as leaders in quality assurance. In my career it was not an uncommon challenge to encounter situations where leadership misunderstood our role, assuming that anything labeled "QA" and "test" fell solely under our purview. Another challenge I have encountered is that testing is not purely a technical matter. While quality-related metrics like code coverage and testing coverage can be easily measured (with the help of tools in low-level testing layers), they are not the only considerations and goals in a test strategy. There are complex concepts involved in the upper testing layers as we get closer to the users’ perspective, which from our side requires experience with the product and the ability to bring test scenarios to the QA engineer's mind. This cannot be easily quantified, especially when discussing concepts like exploratory testing and other test techniques such as heuristics to set patterns for designing and choosing what kind of tests to perform.

To effectively carry out this approach, one thing that has helped me is to differentiate between accountability and responsibility within the test strategy. It is crucial to emphasize shared accountability for quality across the organization, and that each member understands their role and responsibilities. So, I ensure that  specific responsibilities are allocated, such as:

• Unit, integration, and contract tests for back-end, front-end, and mobile engineers
• Data testing for data engineers
• System, end-to-end, UI tests, and exploratory testing for QA engineers 

Additionally, incorporating non-functional tests like performance and security checks based on project requirements further enhances the testing efforts. Providing for each of these types of tests ensures a comprehensive and collaborative test strategy to achieve high-quality software deliverables since each testing type and layer needs to be fulfilled by different roles with adequate expertise.

It is also crucial to engage the product team in processes like user acceptance testing (UAT). I find it very effective to incorporate the testing approach into the definition of “done” for each deliverable so that user stories directly reflect the testing perspective as a precondition to completing the deliverable. While some testing types may not always be applicable (since we work in small increments), this transparency about what will and will not be worked on is vital. Finally, before each sprint review, I encourage product owners to join the team in reviewing deliverables and collectively confirming that the acceptance criteria have been met, covering all the testing types as outlined in the definition of “done.”

However, it's crucial to keep in mind that a single communication and documented process alone may not always be enough. While it can be challenging, we as QA leaders must do regular follow-ups and exhibit unwavering determination. Over time, this understanding will gradually take hold. As an ancient Greek aphorism wisely states, "Repetition is the mother of learning." This process of repeating and reinforcing is essential to ensuring that everyone comprehends and faithfully fulfills their roles within the test strategy.

Moving forward with the test strategy, to optimize test coverage, collaboration between QA engineers and back-end, front-end, and mobile software engineers is strongly recommended.

For example, to avoid duplicating tests in different testing layers, and to ensure that each layer of the testing pyramid is well covered, I encourage the QA engineers I supervise to participate actively during our technical analysis meetings. In these sessions, we collaborate with the software engineers to understand the features being developed and the associated risks. By doing so, we can prioritize the writing of test cases early on, which aligns our testing efforts with the development timelines. This proactive approach not only reduces delays but also helps in identifying any last-minute issues.

To provide early feedback on the product's quality, I also recommend using the same continuous integration and deployment (CI / CD) tool as the back-end and front-end engineering teams do. For instance, in our teams, we collaborate with the back-end and front-end teams to integrate our API and UI tests into their CI / CD pipelines. This allows us to run our tests as part of the continuous integration and deployment process. Using the same CI / CD tooling fosters better engagement from our peers since they can easily access and review our test reports. This enables faster identification and resolution of any issues, ultimately improving the overall product quality.

Common Bug Reporting Process

Establishing a streamlined bug reporting process across the company is crucial to safeguard the effective identification and resolution of issues and also to ascertain that everyone works from the same definition of “bug.” I have frequently come across situations where incidents, requirement gaps, or even ideas for new features were mistakenly referred to as bugs. 

To address this issue, I provided a standardized bug reporting template and documentation that guides users in filling out a comprehensive bug report. This template includes essential information, such as the environment in which the bug was encountered (post-release or pre-release), the severity of the bug based on its impact, and detailed information within the bug report. In our project management tool, we have applied the common template shown below. It is readily available whenever someone needs to create a bug ticket.

Assigning the bug to the appropriate person, such as the responsible developer, product manager, product owner, or engineering manager, is also essential. Keeping the bug report updated with new information and linking it to any other existing tickets is also important. Although it is recommended to create tickets for all bugs, exceptions can be made for quick fixes or trivial issues, provided that this decision is effectively communicated within the team.

At my current company, with the help of our business intelligence department, we fetch data from all our projects into a data visualization tool. Within that tool, we have created useful dashboards that are shared with the engineering managers and the rest of the team to track important metrics such as mean time to resolve a bug. See graphic below.

Identifying bugs in pre-release environments offers valuable insights into the quality of deliverables throughout the development process. It highlights areas that require improvement and emphasizes the need for comprehensive testing by the developers before tasks are handed over to the QA engineers. Likewise, a significant number of post-release bugs indicates potential customer impact and underscores the criticality of increasing investments in testing efforts.

Prioritizing Testing Efforts Based On Risk

QA leaders should prioritize testing efforts based on risk to balance the need for thorough testing with the need for speed. By focusing on the most critical areas of the application, QA engineers can ensure that good software testing is applied at all phases of development, no matter what the person’s role is. By “good software testing,” I don’t mean functional, non-functional, and automated checking alone. I am also talking about deep learning about the products which comes from analyzing the requirements, creating mind maps for testing flows, and other processes that help people within our teams explore and experiment with our products and not simply write tests based on given specifications. 

Drawing from my personal experience, when I started my career journey as a software support specialist, I had the opportunity to closely engage with end-users of our product. This hands-on experience sparked a profound reflection on the trajectory of my career as a QA engineer. I began questioning whether we were delivering a product that genuinely empowered users to seamlessly accomplish their tasks. This realization led me to shift my focus from the number of tests conducted to identifying those tests that would identify potential risks to the product that could impede desired outcomes. To excel as a QA engineer in this new approach, it became imperative for me to cultivate a comprehensive understanding of the system I was testing. I committed myself to familiarize myself with the intricacies of the product, including its features, functionalities, and the intended user experience. Just like a gamer who masterfully knows every button combination and its impact, I endeavored to become the most knowledgeable user of the system, capable of answering any question and understanding the implications of each action. This level of expertise allowed me to experiment, anticipate potential outcomes, and devise effective test plans. 

What’s more, I recognized the importance of sharing this mindset with my QA engineers when I had a leadership role later in my career. I encouraged them to invest time and effort in learning every aspect of the product. By adopting this mindset, we fostered a culture of deep expertise within our team. We strived to go beyond surface-level testing and instead to comprehend fully the product under examination. By embracing this approach, we gained a comprehensive understanding of the product, enabling us to identify critical risks and prioritize testing accordingly. 

On the other hand, continuing with risk-based testing, there are cases where there is not enough time for testing. Throughout my career, I have been a part of teams that operate in short development cycles, employing Agile methodologies such as Scrum or kanban. Our projects often involve delivering minimally viable products, introducing new products or microservices, conducting trials for potential clients, creating proofs of concept, or implementing a refactoring of a core service to reduce technical friction. Each of these scenarios is unique, and it is crucial to recognize that the test scenarios should be aligned with the associated risks.

In a recent situation, I encountered a team facing "urgent" testing requirements for a project. Upon analyzing the situation, it became evident that we needed to develop and test a newly created service to meet the specific business needs of a potential client during a trial phase. Considering the presence of competitors in the trial and the looming deadline, it became clear that investing significant time and effort in designing test cases for every possible customer interaction with our service, as well as writing extensive automated checks, would be impractical and out of context. This was primarily because the trial had a specific scope and did not even require a minimum viable product. 

Instead, our main focus shifted towards acting swiftly by thoroughly reviewing the documentation to fully understand the requirements and then to validate the expected functionality, taking into account potential edge cases that could impact the trial. When I shared this testing approach with the team, we realized that most of the test scenarios could be adequately covered by unit tests. This was because the development plan’s scope was hammered down to provide a simple back-end service for parsing files and obtaining the desired output file without any user interface. As a result, we avoided unnecessary testing efforts and instead chose to observe the development process closely until the service was delivered, while also coordinating with the product owner to oversee user acceptance testing from the client's perspective.

This experience highlighted the importance of creating clear, easily followed processes and recognizing when certain actions may not be necessary. It is crucial to avoid wasting time and effort on low-risk situations. By adopting a risk-based testing approach and aligning our efforts accordingly, we were able to address the trial requirements efficiently without unnecessary testing overhead.

Shift-Left Testing Efforts

QA leaders should encourage all QA engineers to work in parallel with back-end, front-end, and mobile engineers by preparing test cases early on and running tests proactively to align testing efforts with development timelines. This can help reduce delays and prevent issues from going unnoticed until the last minute. A way to do that is to prepare test scenarios and the structure of the respective automated tests in parallel with the engineers’ efforts in preparing the first versions of the deliverables. This way, when the first deliverables are ready for functional or end to end testing, the QA engineers will be ready to run their tests.

In addition, incorporating API and UI tests into the back-end and front-end engineers’ SDLC can provide early feedback on the quality of the product. This helps ensure that issues are caught early and enables faster resolution. As I previously suggested, something that helped our teams significantly was to use the same CI / CD tool as the engineering team and add a specific job that runs end-to-end tests whenever a pull request is opened for the base branch, running the tests in the respective test environment.

In my team, we have implemented a custom plugin that facilitates the execution of tests on the creation of pull requests. This plugin allows us to publish our test scenarios from the test automation project to our test management tool. Additionally, our test management tool is integrated with our project management tool. And we also notify the respective team's chat channel regarding the test run result.

Consequently, when the tests for a pull request are executed, the team is promptly notified about the test run status. The test results are made visible to the back-end or front-end engineer who initiated the pull request, since we use the same CI / CD tool. If the tests fail, it leads to the blocking check in their pull request. Moreover, the product owner, who authored the corresponding user story, is also notified through the test management tool and project management tool integration.

It is also extremely important to emphasize QA engineers’ involvement in the whole SDLC. As QA leaders, you should set an expectation that the QA engineers will participate in the entire software development process, from ideation to delivery. This promotes ownership, improves quality from the outset, and encourages collaboration. Engaging QA engineers from the start enhances individual skills and also helps to improve the quality of automated tests.

QA leaders should avoid keeping the QA engineers separate from the product team. One way this separation can manifest is to have QA engineers start testing only after the technical analysis has been refined, as this can lead to an increased risk of releasing bugs that would be expensive to fix later. Instead, encourage a culture of exploration and ownership where everyone is responsible for testing. By doing so, QA engineers will feel more connected to the engineering teams, leading to better collaboration and faster feedback loops.

Technical Growth And QA Mindset

Lastly, as a QA leader, it is of utmost importance to prioritize technical development and foster a mindset that emphasizes the pursuit of quality throughout the organization. While test automation is indeed a valuable tool, it should not be the sole focus of the testing process. It is crucial to acknowledge and promote the cultivation of other essential skills, including analytical and logical thinking, curiosity, and technical expertise. By doing so, the QA engineers will possess the necessary abilities to conceive testing scenarios, design and prepare tests, execute them, evaluate and monitor the results, debug any issues, and continuously enhance the testing process.

To cultivate a culture of continuous learning and growth, QA leaders should encourage the implementation of learning frameworks so that everyone is equipped to tackle any testing challenges that may arise. It is beneficial to establish a personal development framework among team members, providing them with the necessary support and resources to enhance their skills, achieve personal and career objectives, and contribute to the overall success of the organization. 

We should apply this principle to ourselves as well, as it's important to lead by example. One valuable piece of advice that has greatly aided me over the years is to stay connected with leaders outside of our organization. By learning from them and engaging in Lean coffee sessions to discuss concerns, we can continue to grow personally. Everyone needs to have a mentor outside their immediate network who can assist in their growth.

Summary

As QA leaders, our pivotal role lies in ensuring that our products consistently meet the highest standards of quality. To accomplish this, it is essential to prioritize the development of a comprehensive test strategy that encompasses all crucial testing layers while avoiding redundant efforts. By closely collaborating with our team's tech leads and developers, establishing standardized processes, encouraging early participation in technical meetings, and fostering a culture of continuous learning and growth, we equip our teams with the necessary skills and inspire them to achieve remarkable outcomes.

For Further Information

• Growing Test Managers, Eric Proegler
• How I Stopped Worrying And Learned To Love Management, Ben Kelly
• Managing People With Autism, Eva Podbrdská