I'm going to talk about what changes we made to our test reports to change the perspective. During my 7 year career working for startups, charities and fashion retailers, I've seen many different styles of test reports!

Some benefits we gained from this, we got feedback earlier by inviting users to our sprint demos and also improved rollout of new features by including operations data.

By the end of my presentation, you will know how to add user based metrics such as user feedback comments to your test reports. I’ll walk through the process we used to come up with the user based metrics to include in our test reports.

Let me give you an example of what I mean.

What does a test report really tell you?

What do you mean the RAG status is amber?!

The most recent example is from a charity. I naively thought people looked beyond the red, amber, and green status, turns out if it wasn’t red the report wasn’t read by key decision makers.

I soon realised there was no clear understanding of what the risk was related to test cases executed and bugs found during development. It’s not the responsibility of the stakeholders to understand the intricacies of your test plan and testing process. 

Often qualitative metrics are more powerful in relation to risk and feelings of confidence.

How to create user driven reports

User data makes test reports more relatable for the readers. In my current role at a startup, I ran a workshop to understand from my testers what they think we should report on, this is what they came up with.

The user experience team performs user testing on designs, so we include their feedback in our test reports. The users worked on the requirements, so we included their feedback in our test reports.

Bugs that weren’t going to be fixed before release became modifications to requirements by adopting a zero bug policy. When the code gets deployed to production, the test report isn’t important anymore. So we put the production issues in our test report. 

All these changes meant the report was user centred and risk could be interpreted by the feedback and feeling of the users.

1. Involve everyone who holds information to the user
2. Use production data such as live issues, support requests, and user feedback surveys to outline how your bug fix impacts the users
3. Remove testing related terms from your reports such as bugs and test cases.

Are you going to add more user related data to your test report?