In our project, the business requirement definition starts quite early, then the Product Owner (PO) brings the requirement(s) back to the team and the domain teams start the feature implementation. The teams work independently of each other most of the time. They test and deploy their changes with the automated CI/CD pipeline continuously. It works well with a mature application where the application is live and all we need to do is to add some minor new features. But if we want to bring a completely new application to production, it can get far more complicated. This is because each team is only focusing on their small pieces of the puzzle, most likely in isolation. In order to increase the quality of the whole application, there will usually be a need to test the end to end flow to make sure all of the pieces work well with each other. Especially to evaluate if it meets customer needs from the end customer point of view. But how can we test the end to end flow if we have no idea what the stories/features are? Is it even possible for end to end testers to get deep enough into each feature that is implemented by 10 domain teams? As we all know, the end to end testing phase isn’t normally very long and it can often be shortened due to the delay of the feature implementation.
When time is limited and the test scope is big, how can we set our focus, prioritise our testing work and set clear testing goals?
Some time ago, I got to know the TestSphere card from a testing conference and also read several articles about the possible usage of the cards. Since our teams were somehow familiar with the “outside-in” heuristic of risk-based testing from James Bach but not everyone had sufficient testing know-how to answer “how to test” to mitigate the risk, we decided to use the TestSphere cards to structure and support our workshop. The workshop was a brainstorming session with developers, POs and UX from the different domain teams. The goal of the workshop was to have a shared understanding between QA, developer, PO and project manager on the testing goals, scope and quality expectations.
How we organised the workshop
Our risk stroming workshop worked like this:
Phase 1: Determine the quality aspects that are the most important for your application. → Why we test
Phase 2: Determine the associated risk for each quality aspect. → What to test
Phase 3: Discuss how to mitigate the risks. → How to test
Due to COVID-19, we were not able to run this workshop as an onsite, physical, face to face workshop. Therefore, we had to use a collaboration tool called Miro to support our workshop. Miro allows the participants to collaborate with each other. In addition to that, Zoom was used as the primary communication tool. For preparation, we also scanned and uploaded the TestSphere cards into the Miro board, so they could move the cards around in the Miro board.
For each of the phases, we divided the participants into small groups and assigned them into different breakout rooms (a pretty awesome feature of Zoom), so they could discuss without any interruptions from other groups. When the pre-agreed time ran out, we brought everyone back to the main room to present their ideas and results.
Our miro board looked like this:
How we handled the result of the workshop
After the workshop,we didn’t want to lose any of the results of our 2 hour discussion, so we documented the results on a confluence page. To have it more structured, we clustered the risks into different risk categories. Since the time was quite tight, we did not have enough time to discuss everything. Here is a short example:
In order to present the results in a straightforward way to other participants, we abandoned the sometimes tedious and long, test strategy document and summarized all the important inputs into a very colorful one page test plan.This means it is quite short and the color easily catches people’s attention. :-)
Here is an example of the test plan:
Lessons Learnt and FeedbackÂ
The result of the workshop provided a very good basis to set up our test strategy/test plan. The document is no longer only owned by the QA team, but it is a shared product from the whole development team.Â
A shared understanding is established between all the participants. Especially the discussion of the quality aspects helped the project to set the focus and priorities.Â
Based on the results of the discussion, we as QA can better organise our time and test effort to focus on the most important stuff.Â
This new format is quite interesting for the participants, especially in combination with the TestSphere cards. It is the first time for our project team to discuss the quality specific risks as a group.
It is also a good way to promote the quality mindset and gain the testing knowhow for non-testers. TestSphere cards provide very detailed explanations and examples of each quality aspect and test method, through reading the card and discussion in the workshop, DEV and POs now have better insight into different test aspects and testing methods/techniques.Â
However, there are also couple of things that we would like to improve for future:
Challenge | Solution for Further Improvement |
---|---|
Time constraint: Since the workshop was remote, it is very common that people’s focus and cognitive process will go down slowly. We decided to limit the workshop time to 2 hours. Compared to the scope of our discussion, the planned time (2 hours) was not enough. | Reduce the scope of the Application Under Test to focus on a feature or even a user story. |
Focus of the workshop: The discussion was somehow “superficial”. Most of the participants mentioned the usabilities, but there are a lot of underlying functional and technical risks that were not touched on at all. | Set a clear focus for the workshop (either functional or technical) in advance. Note: We do not recommend including both the business functional risks with the technical risks in one workshop. Ideally the technical risks should be addressed in another separate meeting (with a different discussion group). A well-prepared sequence diagram can serve as the basis for technical discussion. |
Lack of business involvement: It was very unfortunate that a lot of business process risks were left out. | Inform business users in advance and get their involvement for the next time. |
Digitalisation of TestSphere card: We had a lot of discussions about how to organise this remote workshop, especially about how to distribute the TestSphere cards. In the end, we decided to use the Miro board and upload the cards as pictures onto the board. It took us a lot of time just to prepare the board. | If your budget allows, there are some tools available to use, for example https://riskstormingonline.com/; if your budget is tied, then making a board/confluence page as template and sharing it with other projects teams would be a good idea. |
Tooling: Miro is a relatively new tool for our company, so some of the participants are not used to using the board to interact with each other. | Inform participates in advance about the new tools. It would be a good idea to send out the link of the tool to the participants in advance, so they can check the tools in advance, eg: create an account, try to login.. |
Reduced fun factor, since we can not physically distribute the cards to each person and play the card as a game. | Integrating some small games in the workshop to make it a bit fun. :-)Â |
Missing direct feedback from the groups. | Join different breakout rooms during the discussion sessions to hear participants’ direct feedback and answer questions if needed. |
In general, we found this approach quite interesting compared to the traditional risk storming workshop. Despite all the difficulties of organising the remote workshop, we had a relatively successful session. Some working items were identified together with the project team. The biggest benefit of this approach is that the test strategy is not only a working document for the testing team anymore, it is shared with the whole project team. Hopefully this article can provide you with some ideas about applying TestSphere cards in a remote workshop. What is your experience of conducting the risk storming workshop remotely? Do you have any other good ideas?
Reference :
https://www.ministryoftesting.com/dojo/series/testsphere
Try Risk Storming Online
Enable your team to carry out successful Risk Analysis sessions with RiskStorming Online
About The Author
Shuqi comes from China and has been living and working in Germany since 2009, she worked previously as a SAP consultant and fell into testing by accident in 2014. Since then, she has been passionately working as an agile quality advocate for almost seven years.Â
Her main focus is to apply a context-driven agile test practice to help the agile teams to reach their goals. Rather than only focus on testing, she is a fan of a holistic approach to integrate quality into each part of the development lifecycle. She also deeply believes that helping the teams to establish an agile mindset is much more important than just showing them how a particular tool works.