By Simon Tomes
As an exploratory tester, there’s no doubt that finding problems is important. But exploratory testing is way more than that. We know this and sometimes we need to hold the mirror up to remind ourselves. Take a step back and remember, you’re not just a bug magnet!
Exploratory testing gives us the opportunity to contribute to a product in a big way. We evaluate what’s actually in front of us instead of what people are saying it’s supposed to be and hoping it does. This is an essential part of building products our customers want.
This collection of tips has helped me throughout my career as a testing professional. It’s the list I wish I’d had when I started and now I pass it on to you in the hope that it might be of some benefit.
Far Beyond The Application
1. Exploratory testing is everywhere
Don't fixate on working applications. Explore a wireframe. Explore a mockup. Explore databases, systems diagrams, APIs, acceptance criteria, ideas, processes, feature files, assumptions, the UI, specifications. Go and explore anything you can gather useful information from!
2. Embrace the experimentation mindset
Exploratory testing is an experiment mindset. Channel your curiosity and amplify your investigative approach. Seek out truth and discover answers to questions about risks and assumptions. Testing is intimately connected to the same experimentation mindset used by the startup community.
3. Identify a current context
Using a one-size-fits-all approach to exploratory testing won’t cut it. What is your current context? Consider more than your overall testing goals and product goals, such as your team’s inclination to collaborate and experiment. Understand historical business decisions and your company’s current focus. All this can influence the type of session reports you’d like to produce.
4. Challenge assumptions about your role
“Just keep releasing or we won't hit our targets!"
screamed the angry sales manager. I’m just a tester, should I have to deal with this? Colleagues don't always make it easy, but as testers, we can be more. I'd seen, at a previous company, we could go faster by slowing down! As odd as it sounds I knew it would work, and it did.
Look outside your expectations of the role of a tester. In my case, I took time away from testing to present a two week release cycle process to the company, and it was absolutely the right thing to do. What can you do to help your team, when you look beyond your usual daily tasks? For example, you could pair with a designer to explore mockups, collaborate on story writing with a product owner or help a developer investigate a problem on their local machine.
5. Read Explore It! by Elisabeth Hendrickson
Elisabeth Hendrickson’s Explore It! is the go-to book for exploratory testing. It has everything you need to become incredible at exploratory testing.
6. Consider how exploratory testing fits into a strategic testing model
An exploratory approach benefits from test charters, Session-Based Test Management and note-taking. Dan Ashby provides a useful model for how this all fits into the bigger picture.
Be Prepared To Explore
7. Define a goal for your session
Give your exploratory session purpose. Become a charter-generating machine. Too refined and you’ll restrict an opportunity to uncover a useful array of information. Go too wide, lose focus, and you’ll miss important discoveries. With practice, you’ll be setting goals that feel “just right”. Elisabeth Hendrickson provides an excellent template:
Charter Definition: Explore <target> with <resources> to discover <information>
Example: Explore the shopping basket with the basket API to discover security vulnerabilities
8. Frame your tests so they address risks
What are some important risks? Try taking the 5 minute challenge. How do you investigate them? Stemming test ideas from risks, and answering questions related to those risks via exploration is a great way to talk with management about testing.
9. Don’t let requirements hold you ransom
“Well this thing I’ve discovered isn’t related to the acceptance criteria so I don’t need to capture it”.
Your requirements, often written up as user stories and referred to as acceptance criteria, shouldn’t hold you ransom. I’d often fall into the trap of just testing against the requirements. Yet I knew I was missing an opportunity to discover something important. Requirements became an oracle to trigger test ideas and conversation, not the key influencer of where to place my exploratory testing effort.
10. Automate tasks to support your exploratory testing effort
Embrace the Automation in Testing mindset. Use automation to enhance how you explore, not to replace any kind of exploratory testing. Why create user data manually when you, or a developer, could put together a script?
11. Heuristics trigger test ideas
Submit an empty field, use just one character and then too many characters! It took me a while to discover that I use many heuristics to guide my testing. Define a set of heuristics before exploration and refer to them to trigger ideas. The incredible Test Heuristics Cheat Sheet is a treasure trove. Heuristics help you de-focus when you’re in too deep.
12. Timebox your exploration
Time provides focus. It’s easy to keep exploring, to just find one more thing. A timeboxed session helps you stop, take a step back and get the most out of a debrief. It won't be long until you can run another timeboxed session.
I recommend no more than 90 minutes per session. The more you capture the more likely you are to overwhelm your audience. Respect your own time and you’ll respect your audiences. Combine a short exploration time with a clear goal and you’ve got one powerful exploration combo.
13. Try out different notepads
Always have a physical or digital notepad to hand. Every conversation is an opportunity to explore and capture learnings. For successful session-based testing I believe a note-taking tool is essential.
Capture With Intent
14. Write notes like you’re telling someone a story
You’re in the middle of an exploratory session. What’s happening right now? What are you thinking? What are you actually doing? You want to engage with your audience and help them empathise with the experience of using the product. I’ve learnt a lot by understanding how journalism is written. Try this: Imagine you’re a reporter visiting a newly opened water park. Your readership want to know, is it any good? You need to engage them, empathise with them but also help them understand, is it for them?
15. Categorise your discoveries
Give meaning and guidance to your notes by categorising your discoveries. I’ve had great success labelling my notes as a Problem, Question, Idea or Praise (the PQIP method). Iconography and colour are powerful communication tools, use them to enhance your discoveries.
16. Use video to describe a bug
A video will save you the hassle of listing steps to recreate and explaining what actually happened. Use a one-line caption to support the video. Remove ambiguity; show, don’t tell. My go-to video capture tool is LICEcap.
17. Ask a question
Questions help you frame your exploration. Use them to seek answers during your session. Ask questions if you’re unsure of something. Questions remove assumptions. Clarify your understanding with a question.
18. Spark collaboration with your ideas
Ideas are a powerful thing. We come up with tonnes whilst exploring. Document those ideas and introduce them as part of your debrief. Give yourself the opportunity to help shape a product.
19. Give praise more often than not
Appreciate what's in front of you instead of always looking for what isn't right. Step into the shoes of someone else when exploring – not only your customers but the people who contributed to the item you're exploring. Share your appreciation with those individuals. This act of kindness builds relationships with your team and beyond.
20. Use comparable products to form an opinion
We use products all the time. What is it about them that you find useful? What don’t you like about them? Use comparable products to guide your assessment of a problem. They’ll also help you spark ideas for testing, development, or feature enhancements.
21. Tag items off-charter
It’s ok to deviate from the goal of your exploratory session. Tag those items as off-charter. Track this over time to get a feel for how useful it is to stay on charter. Those off-charter items can feed your next charters. Use off-charter items to start conversations about risk.
22. Look under the hood
If exploring a web application, look under the hood with the browser’s console and view your application logs. You’ll discover stacks of useful information.
23. Take a break
Sometimes you’re in too deep and your brain overthinks what it would like to achieve. Stop the timer! Go get some fresh air or a glass of water. Observe those thoughts with some distance, let them drift past and come back fresh. It’s incredible what a short break can do.
Share A Great Testing Story
24. Review your notes before sharing
Read through your notes at least once before sharing. Have you answered everything you set out to? Do you need to create more charters/goals/missions for unanswered questions? Will your notes likely make sense to the target audience? Make session notes easy to read and referenceable for later use.
25. Get to the point but don’t censor yourself
There’s a fine line between capturing absolutely everything and getting to the point. The art of preparing to debrief lies in prioritising the information you’d like to share with a recipient. Consider what’s important to them, your team, and the wider business. Tailor your message accordingly. Start with a summary and allow the recipient to easily consume the detail.
26. Debrief early and often
You’ve just finished a timeboxed session. You’ve reviewed and edited your notes and captured a useful summary. Now what? Don’t hesitate to share with your target audience. Lead with a question: “<session goal> is ready to debrief. When’s a good time for us to get together to run through my learnings?”.
Inspire Yourself To Inspire Others
27. Pair with another tester
Learn how another tester investigates. Try their exploratory testing approach. Collaborate on developing an approach together. We can learn so much from our colleagues.
28. Contribute to the exploratory testing discussion
There’s continual chat on the #exploratorytesting channel on the Ministry of Testing Slack. And useful topics on The Club. Have you considered contributing? It doesn’t matter if you’ve been a tester for five minutes or five years, you can contribute to the conversation.
29. Invest in a growing library of learning material
Pay for a pro subscription to the Ministry of Testing Dojo and it’ll open up a treasure trove of useful resources.
30. Write a blog post about your experiences
Putting into words what you actually do – and learn – helps you reflect on your approach. You’ll end up researching around the topic to help support your post. And you’ll discover amazing material that’s already out there, like Marcel Gehlen’s Exploratory Testing Pathway.
A Journey Of Learning
There is an incredible opportunity to frame exploratory testing as a learning experience. It isn’t something we should take for granted or devalue. There is much to discover and even more potential to push the approach and mindset forward. Take a leap and share with others, spark collaboration and let’s continue to evolve the exploratory testing practice.
Let’s encourage each other to keep exploring!
- A Tutorial in Exploratory Testing by Cem Kaner
- Exploratory Testing 3.0 by James Bach
- Session Based Test Management – The Mindmap by Ministry of Testing
- Charter Template for Exploratory and Session Testing by Melissa Eaden
- 99 Second Introduction to Testing Notes by Dan Ashby and Mark Winteringham
- Exploratory Testing in an Agile Context by Elisabeth Hendrickson
Simon is on a mission to move the exploratory testing craft forward. He believes in simplicity, creativity and collaboration. Simon is co-founder of Qeek, the team behind TestBuddy – a session-based testing tool for exploratory testers. He shares ideas, thoughts and learnings on his blog. You can find him on Twitter, LinkedIn and the Ministry of Testing Slack.