User Driven Test Reports, Based on Feedback and Feelings

  • Locked
Lewis Prescott's profile
Lewis Prescott

QA Lead

User Driven Test Reports, Based on Feedback and Feelings image
Talk Description

I'm going to talk about what changes we made to our test reports to change the perspective. During my 7 year career working for startups, charities and fashion retailers, I've seen many different styles of test reports!

Some benefits we gained from this, we got feedback earlier by inviting users to our sprint demos and also improved rollout of new features by including operations data.

By the end of my presentation, you will know how to add user based metrics such as user feedback comments to your test reports. I’ll walk through the process we used to come up with the user based metrics to include in our test reports.

Let me give you an example of what I mean.

What does a test report really tell you?

What do you mean the RAG status is amber?!

The most recent example is from a charity. I naively thought people looked beyond the red, amber, and green status, turns out if it wasn’t red the report wasn’t read by key decision makers.

I soon realised there was no clear understanding of what the risk was related to test cases executed and bugs found during development. It’s not the responsibility of the stakeholders to understand the intricacies of your test plan and testing process. 

Often qualitative metrics are more powerful in relation to risk and feelings of confidence.

How to create user driven reports

User data makes test reports more relatable for the readers. In my current role at a startup, I ran a workshop to understand from my testers what they think we should report on, this is what they came up with.

The user experience team performs user testing on designs, so we include their feedback in our test reports. The users worked on the requirements, so we included their feedback in our test reports.

Bugs that weren’t going to be fixed before release became modifications to requirements by adopting a zero bug policy. When the code gets deployed to production, the test report isn’t important anymore. So we put the production issues in our test report. 

All these changes meant the report was user centred and risk could be interpreted by the feedback and feeling of the users.

  1. Involve everyone who holds information to the user
  2. Use production data such as live issues, support requests, and user feedback surveys to outline how your bug fix impacts the users
  3. Remove testing related terms from your reports such as bugs and test cases.

Are you going to add more user related data to your test report?

What you’ll learn

By the end of this talk, you'll be able to:

  • Why stakeholders don't understand the risks of traditional test reports
  • Where you can get user data from for your reports
  • How you can remove bugs from reports with zero bug policy
Lewis Prescott's profile'

Lewis Prescott

QA Lead

I'm an experienced QA Lead at Cera Care (one of Europe’s fastest-growing companies), having worked across different industries including Healthcare, Non-profit, Retail and PropTech. I am also a course author on Test Automation University & Udemy, sharing my knowledge is a passion of mine.
Suggested Content
ReTestBash UK 2022: Live Q&A with Marie Cruz
The Power of Storytelling as a Tester
TestBash UK - More Than a Multi-Track Conference
Too Many Bugs in Production - What Are We Going to Do?
TestRail Jira EPIC Integration
Defining Story Completion As A Software Tester
Explore MoT
Episode One: The Companion
A free monthly virtual software testing community gathering
Bug Reporting 101
A quick course on raising bugs

Tags

  • testbash-uk
  • test-reporting