Software Tools For Easy, Effective Exploratory Testing
By Mirza Sisic
Exploratory testing (which will be referred to below at times as ET) can be seen by some as an entirely free-form activity. While this is partially true, exploratory testing also benefits from having a bit of structure and focus.
In exploratory testing, we do not create test cases with predictable outcomes (expected results). This is because we are trying to explore the unknown and thereby gain a deeper insight into the system we’re testing.
Exploratory testing sessions should have a short testing charter. The charter determines the time allocated for our session as well as which part of the system we will be covering. For optimal return on investment, we should focus on a specific part of the application to be able to explore it thoroughly.
Using the right tools can help us make our exploratory testing sessions more productive and easier to execute. For a bit more details on ET, you can check out this great article from Atlassian’s blog.
Why Use Software Tools For Exploratory Testing?
Tools can aid us greatly in our exploratory testing sessions. The Guru99 tutorial site has a nice definition of these tools: “Exploratory testing tools are the programs that help testers to check the system on the fly. These tools allow you to perform test design and execution simultaneously without much planning. Exploratory testing tools also help to automate the testing process, collaborate with teams and create reports with ease.”.
Many of these tools not only help us to be more efficient but also provide us with the ability to create reports based on our exploratory sessions. Other benefits are the improved ability to learn more about the system, which in turn can give us a clearer picture on what to automate and what not to automate.
How Exactly Can Tools Help In Exploratory Testing?
Session Management And Recording
The simplest and perhaps the most critical component of exploratory testing is note taking. Writing down observations, questions, and concerns allows us to reflect and gain valuable insights from our explorations.
Session related tools can help us with:
jotting down our testing charters
writing down ideas and observations
making screen recordings with optional audio
importing and exporting our exploratory testing sessions (in different formats) so we can use them for reporting purposes.
A pen and paper or a simple program like Notepad can be of use here. There are other more specific tools tailored towards exploratory testing, in the form of browser extensions (mostly suited for web-based or UI exploratory testing) all the way to full fledged desktop, mobile and web applications. Some of the most widely used tools are.
ET browser extensions:
Exploratory Testing - browser extension to manage your ET sessions with simple minimal UI. It allows bug reporting, note-taking, screenshotting, URL tracking, saving and importing sessions as well exporting your session as a report in any of several formats. A good choice if you’re just starting out with ET tools, as it’s very intuitive and simple to use.
SpiraCapture - enables you to record your ET sessions, organize them, capture keyboard strokes, detect changes in the URL, and capture network errors and dynamic HTML previewing in local testing sessions. It can be used as a standalone free extension, or, for Inflectra customers, it can integrate with their test management products, such as SpiraTest, SpiraTeam, or SpiraPlan.
Test & Feedback This extension allows you to take notes, create bugs, tasks, send feedback, create test cases, and add attachments. It integrates well with Azure Test Plans or with Team Foundation Server 2015 or newer. The extension is free regardless if you use it in standalone or connected mode.
TestRail - A fully fledged test management solution that supports creation of ET test session test types where you can add pass parameters for your session, state a mission and goals for your charter (it’s possible to configure additional fields if needed). You can add comments, attachments and link with your bug tracker using defect IDs
Testivator - notetaking for testers available for iOS and Android. It offers a clean, minimalistic UI enabling you to create test charters, add notes during ET sessions, take screencasts and screenshots, and do basic reporting. You can use their open API to connect your existing ET client app and store data in Testivator.
XRAYExploratory Testing - Slick looking desktop app for Windows, Mac and Linux. It offers native support for integration with Xray and Jira. You can export ET session reports in PDF, do video / audio recording, make screenshots, and time your test charter. My favorite feature is the ability to select different types of comments: Comment, Problem, Suggestion, Idea, Question, Concern and Positive.
Rapid Reporter - explanatory note taking app, The UI looks a bit outdated (if you’re nostalgic about the old Windows look and feel this might make you feel at home) but if your needs are simple it does the job. The tool is standalone and requires no installation: you can carry the exe file on your USB thumb drive with you. Always stays on top of the other opened apps on your desktop, during active sessions. Notes get stored in CSV format.
Exploring different combinations for input forms can be made easier with tools. Many tools for the purpose can be instrumented with predefined values which cover the most common pain points. In this way you can do more thorough negative testing by practically applying equivalence partition and boundary value analysis techniques.
BugMagnet extension is a fine example of one such tool. With it we can instantly populate input fields with different text sizes, various characters, both valid and invalid URL and email combinations, and a whole lot more. After you install the extension, just click on the form (after installing the extension) and select the input from the context menu.
Tools in this category also allow us to do penetration and security testing. You can experiment with SQL injection using tools such SQLMap, jSQL Injection, Blisqy and many others. The above mentioned tools allow us to easily explore how our SUT handles SQL Injections, JS Injections, Cross Site Scripting (XSS), HTML Parsing, handling broken HTML, and so on. You don’t need to be a security testing specialist to check for the most common exploits.
API testing is becoming increasingly important, since the API-First approach is being implemented by more companies all the time. For software testing, knowing how to test an API (Application Programming Interface) can help gain deeper insights into the system and also save a lot of time. Exploring an API can help us find potential security vulnerabilities and learn what type of data different systems are consuming (and sending). Aside from that, knowledge of APIs used by our system under test can help us improve our automated UI checks as well by hitting the API directly instead of simulating all of the UI interactions.
Examples of API exploration tools include:
Network tab. From the web browser’s developer tools, we can see the requests made and check details about the JSON request/response body. By checking the Response and Preview details in the network tab, especially the Headers area, we can find the response status codes, request URL, HTTP method used, headers, and so on.
HTTP clients, such as Postman, allow you to do all of this. I use Postman every day to do API testing, a bit of monitoring, and collaboration with the team. It supports sharing of collections and I run a few automated smoke suites tests there as well. Other HTTP client tools are SoapUI, Insomnia, and many others. They help with making HTTP requests and finding out the same info as the Network tab in the browser but also provide us with more control. We can use these tools for testing API, API development, monitoring and troubleshooting.
Monitoring And Network Data Capture
Tools that monitor and capture network traffic and data such as the Developer Tools in our web browser, proxies like Fiddler and Charls, or HTTPS clients like Postman can make our exploratory testing more technical and give us useful and relevant data. Command-line tools like curl are often helpful here as well.
Additionally for dealing with test data, we can make use of SQL or noSQL DB queries to set up preconditions for our ET sessions and to drill deeper into our findings as well.
Let’s suppose we need to check new functionality that allows the customer to save their preferred payment method. We need to make sure that many payment options are available. We could use SQL to create test data: good test data for the application is expected and “bad” test data which is not expected by the application. Examples of such tools would be SQL Data Generator and the Faker library.
Visualizing The System Under Test
Mind-mapping tools help us in visualizing our goals and can be used to gain understanding of architecture of the system we’re exploring.
We can also use mind maps to visually express any exploratory heuristic we are using such as SFDPO (Structure, Function, Data, Platform, and Operations).
Examples of mind-mapping tools include:
Miro - paid tool with a lot of integration options. It supports online whiteboard and other team collaboration activities, and is much more popular now with the advent of remote work.
Xmind - paid tool with a lot of built in options, from mind-mapping to visual brainstorming. Web, mobile and desktop app versions are available.
MindUp - feature rich online mind mapping tool available in a free version. Supports custom keyboard shortcuts, file exports in multiple formats (PDF, PPT, etc.) and integrates with cloud storage options like Mindup Cloud and Google Drive.
MindMeister - online mind-mapping and visualization tool. Supports simplified project management functionality and creating presentations. Free plan is limited to three mind-maps, while other plans are subscription based with reasonable prices.
User Interface Exploration
These can help us in inspecting the HTML and CSS if we’re testing a web app, manipulating the DOM, and generating common input combinations.
Examples of UI exploration tools include:
Bug Magnet- browser extension very useful for testing inputs. You can apply equivalence partitioning and boundary value analysis techniques. For instance, with it you can check for the most common valid and invalid email combinations and see how well our system is handling the validation of such input.
Developer Tools in the browser:
The Elements tab, useful for checking the DOM (Document Object Model)
The Console section, where you can check for console errors and warnings
The Application tab can give us insight on session tokens. By editing the token’s expiration, you can check if you get logged out once your token is no longer valid. You can also see the cookies in use. Finally, you can clear the cache selectively depending on the app being used.
The Toggle Device toolbar allows us to simulate different screen size and devices so we can check how responsive our website is
The Developer Tools can be enhanced with a browser extension. For example, when testing single page apps using the Virtual DOM, we can install browser extensions for React, Angular, Vue, etc.
To Sum Up…
Exploratory testing can provide our product with great benefit if done “the right way.” Capturing notes and observations give us insights about the pain points in our system and give us a good idea on what is worth automating and what’s not. Employing adequate tools in our ET improves our learning opportunities.
To make sure we are using the most suitable tools for our exploratory testing needs, researching the available tools is the first step. But the most important thing is trial and error - trying out different tools to find out which ones sit best with us. To make sure we are always using optimal tools, you can make trying out new tools a regular activity.
Remain curious, and happy explorations!
It’s October 2021 and the Ministry of Testing is running their 30 Days of Testing tools challenge. Personally, I love challenges like these: seeing how other testers from the community view and use testing tools is quite valuable. And as an added benefit you get to learn about some new tools as well.
Mirza has always been a technology geek, helping friends and family with computer-related issues. Started originally in tech support in 2014 and moved to software testing in 2017, and has been there since. Mirza worked as a freelance web developer for a while as well. When he’s not sharing memes online Mirza is usually learning new things, writing posts for his blog, and being an active member of the testing community.
Mirza has always been a technology geek, helping friends and family with computer-related issues. Started originally in tech support and moved to software testing and has been there since. Worked as a freelance web developer for a while as well. A casual RPG gamer and a Sci-fi fan. When he’s not sharing memes online Mirza is usually learning new things, taking part in the testing community, or writing posts for his blog.