Onboarding new testers means that they must learn your product. Considerations of how to do that depend a lot on your context: your product, your industry, your teams, your processes and so on. Here are some ideas on how to enable your new testers to better learn your product, and how to make that product easier to learn than it might otherwise be.
Approaches
Lectures And Workshops
Lectures are a way to introduce ideas surrounding the product, such as domain-specific knowledge, the industry, the market, and what the product does in a general sense. You can use them to introduce diagrams of complicated concepts like tech stacks and system models.
Workshops are a good way to include some expert-led interactivity. Usually this involves a new tester and a laptop, and guiding them in how to do simple tasks using the product. You can instruct multiple testers at once this way, with only one expert, but keep in mind that there will be more setup to do and more questions to answer.
Having useful background information will mean that testers can learn more effectively. A series of talks on your product will help set them up to do better exploration themselves.
Paired Learning
Having a tester experienced with your product who understands the domain and industry can be invaluable when a new tester is learning. They can field questions, give product and testing tips, direct the new tester to useful resources, make introductions with other testers and departments, and walk them through how the product works in an interactive way with immediate feedback. It also helps the new tester to become familiar with their co-workers, and settle in faster. There’s lots of ways to approach this, but sparing a human to help teach is very often worth the cost of their time to your organisation.
Exploratory Assistance
Allowing testers to interact with the product is a great way to facilitate learning about it. One temptation is to do this with overly-specific test cases, but I find this to be very restrictive on a tester’s freedom to learn and adapt their current knowledge. Test cases are also brittle because of their specificity, and hard to maintain in terms of keeping current with the product.
Instead I offer charters, checklists, and challenges!
Charters are a mission statement for testing. Usually no more than a few sentences long, they are open-ended and do not attempt to fully prescribe how a tester interacts with a product or what they should observe.
An example charter might read “Explore how to create, read, update and delete users from the User Management tab.” You can add a time limit to give a reasonable place to stop, for example a half hour. Some testers may be able to write their own charters, and guide more of their own learning, while others can be given charters written by others. They are freeform, and allow a tester to learn in their own way.
Challenges are basically short, output-focused charters. “As an admin, create a user” is one example of a challenge, something you want a tester to work out how to do. You can use these to create guides for learning each part of your product or process, and adapt them to the needs of each new tester. You don’t have to specify how the challenge should be done. Instead, you are allowing the tester to take responsibility to figure it out, look it up, or ask questions.
Both charters and challenges can be organised into checklists. Or, they can be combined: for example, a charter giving an overview could be coupled with a checklist of specific things to do to help a new tester navigate a large task. Charters and challenges can be used in different ways, say to use when pairing with another person, and then the pair can work through the list together.
When a tester is exploring the product, have them make notes on questions they want to ask, areas of the product or process that confused them, interesting findings, possible problems they found, risks they discovered, and tests (or kinds of tests) they might want to run. The notes can be valuable not just to the new tester but also to other testers and people in other roles They can be incorporated into a risk catalogue or used to improve existing testing. A fresh look at your product is valuable, and you may consider using onboarding as an opportunity to use that value.
It’s worth considering which materials a new tester might want as they are exploring your product with a charter or attempting to complete a challenge. Manuals, documentation, specifications, user stories, and knowledgeable people are all possibilities.
Materials And Resources
When a tester is learning a new product it’s helpful to have access to information about it. This can include manuals and diagrams, coverage outlines, input and data examples, risk catalogues, site maps, user data, tutorials, installation guides, requirements documents, technical design documents, specifications and many other useful oracles. Consider what might be helpful and see that new testers know about them and have access to them if they need it. A helper who can answer questions, even asynchronously, is an additional option.
Factors Affecting Product Learnability
Available Time
The more time you have to dedicate to learning a product the easier it is to learn. But you may have to strike a balance between time and resources dedicated to making a tester more knowledgeable and having them actually perform testing on the product. The more they learn the better they will be, but you need to consider what must be learned through applying resources like time and what can be learned through testing. Don’t consider learning time to be wasteful, though, because it can yield important testing results: questions that lead to finding problems in the product; process improvements; building risk catalogues; and so forth.Â
Domain Knowledge
Not all domains are created equally learnable. For example, mobile games can make for much easier products to learn than stock trading software. One reason is domain knowledge. I can make useful observations about a mobile game because I know what a mobile and a game are, I know what they’re designed to do, how they’re made available, ways in which they make money and so on because I’ve interacted with them before, like many other people. Stock trading requires an understanding of long and short positions, buy, sell, bid, ask, spreads, limit orders and a bunch of terms only understood to people who trade stocks. A stock trader who has never touched a mobile game before will have the inverse problem, of course. The point is that to make useful observations to learn your product the tester needs a basic understanding of the domain. Complex and obscure domains make learning much more difficult.
While you cannot control the world in which your product exists, you can make certain assumptions about what it might take to learn it. Consider what you need to teach, who will teach it, and what form that will take. Consider what someone needs to know to understand why your domain exists, and why your product fixes a problem in that domain. See if your product can be broken down into parts, so that each part has required learning, and how specific it will need to be. Consider the seniority, experience, and skill of the tester who will learn it.
Upcoming Or Newly Introduced Product Changes
Introducing unknowns by creating, updating, or removing part of a product can mean that a new tester has to re-learn an existing part of the product or has wasted time on learning a defunct one. If you can predict a change, say by knowing that there’s an upcoming project to remove a section of your product, then new testers would benefit from knowing that and your onboarding materials might have to be updated to reflect this.
Having a flexible onboarding process with few high-level materials (instead of many overly specific ones) will help here, as it can be adapted to changes in your product. Consideration of how much your product changes over time or between use cases is important in this regard. Review your existing training materials, and if you know there are important changes coming up or that just occurred, communicate the change to your new tester so they can prioritise their work or you can prioritise it for them.
Product Complexity And Usability
A simple product whose scope is small is easier to learn. A product with fewer inputs and outputs is easier to learn. A product that is hard to use is also hard to learn. Some products are built specifically for use only by particular domain experts, not by the general public. So usability, accessibility, and charisma may be lacking, making the product harder for most people to interact with. If your product presents a wholly new problem domain to a newcomer then they will need extra help in understanding it.
You cannot control the complexity and usability of your product, at least in the short term, but you can consider it when assigning resources for learning. In addition, some parts of your product may be particularly complex or hard to use, requiring extra training time and resources.
Knowledgeable People At Your Organisation
Having people available to answer questions is important. You may need to dedicate some hours per week for some experienced team members to pair with the new testers, present a talk or workshop, or just sit down with some paper and draw some explanatory diagrams. Obviously this has a cost in terms of time to the organisation, but the loss of hiring an expensive new tester who can’t use the product or understand how it works makes such an investment worthwhile in most cases.
A new tester will very likely get stuck during the learning process and need someone to go to. I recommend assigning someone to take responsibility either to answer the question or find someone who can do so. This is a great opportunity to introduce your new testers to others throughout the company to help them network and become familiar with different teams and departments! Make people available, and make it part of their job to be helpful and to answer questions.
Also make the new tester feel comfortable and free to ask questions of others. There’s no point in making people available if the new tester won’t go to them for help. Make sure you make actual introductions in person or via communications channels. People need to know who the new tester is, and the new tester needs to know who they are and what they do. Give them contacts, help them network, and this communication will be paid back to you for their entire time with your company.
Product And Tool Availability
A new software tester is going to require a lot of tools and resources that existing testers take for granted. They will need a working copy of your software, a platform on which to run it, test tools, test environments, test data, and IDEs, to name but some. A common concern is having access, in terms of logins and permissions, for drives, servers, environments, tools, bug tracking software, version control, software licences, and databases. Let them know that they can ask for anything they don't have, to cover the concerns you didn't think of ahead of time, and consider including anything that they request in your future onboarding materials.
The product becomes easier to learn when there are fewer blockers preventing a tester from interacting with it. Consider what your testers use every day, and ask them what you believe a new tester might need. Take this list and use it to make your onboarding process more efficient, but also more empathetic. You will be able to better predict the resources you will need, such as the time it will take for someone to set up an environment or obtain a new licence, and reduce the pressure on your new tester by getting them started faster and happier. This also gives a good first impression and helps to set expectations. It may be that some of the setup would best be done by the new tester to help them learn the system, and you need to strike a balance between a fast, smooth experience and an educational one. Remember that while a new tester is learning your product they are also learning your tools and processes.
Tester Skills And Knowledge
The ability of a tester to explore and design tests will depend on their proficiency and experience, and will affect how swiftly they can learn the product.If they have domain knowledge, or experience with the platforms and technologies you use, they can leverage that to learn quickly.
Consider the individual tester’s needs when you design their onboarding. A new tester, or one unfamiliar with exploratory skills, will struggle more when learning the product and may need additional help. If they lack a background in your field they may need assistance with that.
Observability And Controllability
Being able to examine the product, its current state, and what it is doing is important. Determining what a product actually does as we interact with it allows us to learn faster. Similarly we need to interact with the system to make useful observations about it.
Try to give your testers as much access as you can allow. For example, consider their ability to see the state of the database. If they want to see how your product stores data for new users, for example, they will need access to the database and the tools set up to do so. They may need administrative access in your product to access the user data, even if they don’t go directly into the database, to see the users they’ve created. Make it clear to new testers that they need to ask for access to things that you haven’t considered, so they have a route to make the product more observable without you.
Another concern is state control. A common case for testers is to put the product into a known state. If a tester can reset a product back to a known state, for example by loading in a valid database or restarting a container, they’re more likely to experiment with the product and have more confidence to explore it. It’s also useful for setting up specific scenarios with particular settings or types of users. Try to make sure testers can do all these tasks themselves or with minimal help from others.
Final Thoughts
The complexity of software and its relationship to the world, and the variability of context, and of people who test, mean that onboarding is rarely a cookie-cutter affair. It needs to be adaptable and work for each person.Â
One of the most important considerations is encouragement and support, and getting feedback so that you can adapt your approach. Avoid having people sitting around waiting for permission to learn or access to the test environment by considering who is learning this product and what they need to learn it thoroughly. Consider how much responsibility you’re willing to give them, the tools they need to exercise it, and remove things that block their learning.
Your efforts will be repaid by a happy, knowledgeable, fully set up and well-connected tester with the confidence and understanding to do great testing.
For Further Information
- Onboarding Testers: Growing Your New Hire, AJ Larson
- A Journey: Introducing Colleagues To Exploratory Testing, Nicola Lindgren
