Systematic Exploratory Testing With YATTIE

Exploratory testing is a popular software testing style that involves simultaneous learning about the product, designing tests, and executing them. Unlike scripted testing, exploratory testing emphasizes discovery, learning, and flexibility. 

Although exploratory testing is a highly creative and effective way to find bugs, it is often confused with non-systematic and unstructured work. In this article, we'll explore YATTIE, an open-source tool that enables testers to add structure to their exploratory testing and make it systematic.

What is YATTIE?

• YATTIE is an open-source exploratory testing assistant developed by David Acosta.
• It's a desktop-based tool that enables testers to create, manage and document their exploratory testing sessions systematically and efficiently.
• It aims to remove the friction from the exploratory testing process: YATTIE supports the entire process of exploratory testing, from gathering evidence to generating test reports.
• YATTIE is cross-platform and supports Windows, Mac, and Linux.
• You can download it from the YATTIE GitHub repository.

YATTIE offers many useful features such as:

Test charter creation: YATTIE provides a simple interface for creating test charters, which outline testing objectives, detailed scope, time limit, and any preconditions. Test charters act as guidelines for testers to ensure that they are testing the intended areas and are not missing anything crucial.

Note-taking: YATTIE provides a note-taking interface where testers can record their thoughts via text or voice note about anything that they encounter during the session, in any of these categories:

• Comment
• Problem
• Suggestion
• Idea
• Question
• Concern
• Positives

Notes can also be organized using tags, and testers can easily search and filter notes to find specific information.

Recording evidence: YATTIE allows testers to record screenshots and videos during testing. Testers can capture and annotate screenshots, add comments, and highlight issues to provide better context for the team. YATTIE also records the tester's actions as they use the software, providing a video recording of the testing session.

A timeline view of your testing: YATTIE provides a timeline view that displays all key testing activities in chronological order. This feature is especially helpful for those who want to do retrospectives on their work or debrief it with others. Additionally, it provides a clear and concise overview of your testing session.

Personal Experiences With YATTIE

I have always been a fan of exploratory testing and used to take my exploratory session notes using mindmap tools. However, mindmap tools are not always sufficient when it comes to organizing your exploratory testing sessions into distinct charters or capturing video recordings of the issue in a mindmap. In short, I was looking for an exploratory testing assistant.

I first discovered YATTIE through the Ministry of Testing Community. I started with the beta version 0.2 and immediately liked some of its features such as test charters, quick tests, note-taking capabilities, and support for checklists. I soon started to use it for capturing notes during my test sessions at work.

How Does It Help Me?

There are a lot of benefits to using YATTIE for testing. YATTIE helps me to stay organized and document my testing activities systematically. Here are some key benefits that I got from YATTIE:

1. Free testing assistant: YATTIE is available to use for free and can act as your testing assistant that helps to organize and streamline my testing efforts. It also allows me to strategize and plan my subsequent testing sessions. I use YATTIE as my virtual testing assistant.
2. One for many: YATTIE is a versatile tool that can be used for different types of testing needs, including session-based testing, note-taking, evidence recording, reporting, adding checklists, and much more. Before YATTIE, I used to use different tools for screenshots, video recording, note-taking, mind mapping, test reporting, and so on. Now, I simply use YATTIE for all these needs.
3. Organized test notes: YATTIE helps me keep my test notes organized and easily accessible, even when working on test sessions spanning multiple days. It supports pausing your test sessions and resuming them when you want to start testing again. The overall experience for test charters and different note-taking options make it easy to capture my test notes. 
4. Simplified reporting: The ability to showcase your testing work is often a big challenge. For automated tests, it’s easy to showcase them through automation test reports. However, for exploratory testing, I used to create them myself, a time-consuming and error-prone process. This is one area where YATTIE has helped me the most. YATTIE provides me with an attractive PDF report which contains:
   • Test charter details
   • Test environment details
   • Test notes (timeline view) 

Challenges With Using YATTIE:

As YATTIE is still in its beta stages, I have faced some challenges with using the tool:

1. Minor bugs: As of today, YATTIE is still in beta version and undergoing active development, which at times results in some bugs. YATTIE needs more testing. 
2. Mindmap note-taking: Although YATTIE offers notetaking via a mindmap, this feature is still under development and may not be easy to use, or may not work as expected.
3. Limited report formats: YATTIE only supports exporting reports in PDF format, which may not be editable or easy to customize. This is a challenge if you want to create a customized version or share findings in other formats like Microsoft Word or HTML.
4. Limited documentation: The documentation is still in its early stages. However, with this article, I hope to make the user experience better for future new users.

Conclusion

Systematic exploratory testing is critical for efficient, effective, and comprehensive testing. YATTIE is an open-source tool that enables systematic exploratory testing and provides valuable features such as test charter creation, note-taking, evidence recording, etc.

My team’s experience with YATTIE has been quite positive, and we are also contributing to it by suggesting feature requests and raising product bugs. I would highly recommend it to anyone looking for a tool to support their exploratory testing activities.

As YATTIE is an open-source tool for the testing community, I encourage fellow testers to try YATTIE for their exploratory testing needs and contribute to it. Community contributions will certainly help us to improve and enhance the tool further. 

For More Information

• Yattie Online Community
• Explore Galore! 30 Tips to Supercharge your Exploratory Testing Efforts, by Simon Tomes
• How to Explore with Intent - Exploratory Testing Self-Management, by Maaret Pyhäjärvi
• Discussions: Reporting Your Exploratory Testing, with Suman Bala, Niranjani Manoharan, Monica Arzani, and Laveena Ramchandani