What's a bug report?Firstly, a bug, commonly called a defect or problem, is anything that is perceived to threaten the value of the software. A bug report, is the information about the bugs that we find that we collate and supply to the product owner, stakeholders, or developers. Some means of documenting and communicating our defects within the application that we're testing or the software.  Can you give some examples of things that you'll find in bug reports?As part of bug reports, we will have, usually, a a summary of what the bug is, a detailed description with some environment information, attachments of screenshots or videos of the bug, and log files possibly too. And you would also have a priority and severity for the bug. What's the value of a bug report?Logging a bug allows us to track the progress of that bug. It also may be some time before the bug is addressed depending on its severity and priority. So logging the bug means that we won't forget about it. And it's a good starting point for developers to be able to fix an issue. What are the pitfalls?Counting bug reports as a means to determine quality can be misleading. Quality is subjective, and one person's bug is another person's feature. And also just focusing on writing a bug report can have adverse effects on having the collaborative and meaningful conversations about the bug by developers and stakeholders as well. And also badly written bug reports can also damage your reputation as a tester.

What is exploratory testing? (ET) Exploratory testing is an approach towards testing. It involves concurrently learning, designing, and executing, and recording tests, at the same time rather than specifying all your tests upfront before execution, and it has the purpose of uncovering information related to risks. Do you have any examples? I could do things like explore different types of data that could be submitted via a web form or explore the rendering of a web page across multiple browsers or explore how a feature works depending on different user privileges. What's the value of utilizing an exploratory testing approach? Exploratory testing empowers, testers to use their skills to discover information freely. ET can be structured to help keep the tester focused on specific risks, and it can help, a tester to reveal more information that a scripted test approach wouldn't uncover as you're not biased by the test case steps. Are there any pitfalls? Exploratory testing is an approach that requires a lot of skill and practice. A good exploratory tester needs to know how to focus and defocus when testing, And note taking is important to help communicate what a tester has done whilst not hampering them, but this is hard to do.

Why is influencing relevant to testing?So influence is the skill to affect something to your ideal. So influence skills are important for a tester as the topic of testing is generally misunderstood within other areas of software development. So you need to be able to influence people in order to be able to teach them about good testing practices.  Do you have any examples? So the rule of reciprocation is a great influence technique where if you do a positive favor for someone then they will be subconsciously obliged to return the favor. You've got rejection and retreat which is an influence technique that's made possible by the contrast principle where if you feel an idea that you want to pitch will be flat out rejected, pitch another idea that seems less appealing so that it gets rejected. And the second idea, the real idea you want to pitch, seems much more appealing.  What's the value?So influence skills can definitely help you overcome difficult situations as a tester. Many people have many misunderstandings of testing such as when you hear things like automate all the testing or testing is just checking requirements. Influencing techniques can help open the door to communicate to those people to teach them about what testing is and what good testing looks like.  Are there any pitfalls?Influence techniques can sometimes be confused with negative manipulation activities. Some people confuse things like bribery and guilt tripping as influence, but they're not. It's important to remember that sometimes influence techniques don't work, and it's also important to remember that you may actually be trying to influence incorrect messages based on your own misconceptions. So be aware of your own biases in the ideas that you're trying to promote.

What is session based test management?Session based test management or SBTM and it's a method for measuring and managing exploratory testing. It's a management tool to help others see how a tester is progressing in their exploratory testing work, and it empowers testers to store test notes and useful metrics for other team members to use without sacrificing the quality of the testing. And how does it work?A tester will create a charter and add an estimate of length to it, small, medium, or large. The tester executes the charter as an exploratory test session, and once complete they'll fill in a report that contains test notes, the actual time spent, and metrics on the percentage of time spent on test design and execution, bug investigation and reporting, and session setup. And then at the end the tester debriefs to another tester to ensure that they're happy with what they've tested and reported.  What's the value of SBTM? The metrics can reveal how a tester is getting on. For example, how much time they're actually testing versus setting up an environment, and charters can be calculated with estimated lengths to give an estimate of time to complete testing.  Any pitfalls? You and your team need to be disciplined when it comes to running debriefs. Missing them could affect the quality of the reports. Also, SBTM is not a widely accepted or known approach, so it requires some effort towards selling it to to stakeholders or managers.

What are acceptance criteria?Acceptance criteria are a list of expectations that a user will have, when using a feature. They are written in short sentences to define a specific behavior of a feature, and they are commonly added to user stories to add more description. Do you have any examples?Examples could be, the widget should have a text box and character count. Entering characters to reduce the character count could be another example. Or the third one is when character count is at zero, no more characters can be added to the text box. What's the value?It gives the team a boundary on what they should and shouldn't be delivering in a feature, and acceptance criteria, can help with discussion and collaboration of features. And acceptance criteria can help testers generate test ideas and identify risks. Are there any pitfalls?Acceptance criteria can help stem testing ideas, but they are not tests themselves. And vague acceptance criteria can potentially lead to misunderstandings, and bugs in the system too.

What are user stories? User stories are a tool that are used by, agile teams to capture a description of a feature. They follow a template of as a I want so that, and they are typically put on a card or stored in a tool like Jira.  Do you have any examples?An example could be, as a user, I want to change my email address so that I can keep receiving notifications to my new email address. Or as an admin, I want to edit records so that I can keep my records up to date.  What's the value?They help facilitate discussions about feature within your team. They can be used to track progress of the development of a feature, and, it helps teams determine their workload.  Are there any pitfalls? It's important that your user stories have clear value to a user. Vague user stories make it harder to develop and test.

What are software testing notes?Software testing notes are used for recording information during different testing activities, whether it's testing an idea or a product. And testing notes can come in many forms. Some are more suitable for certain activities than others. And notes can be collated for future re reporting, or sharing information with your team. Do you have any examples of software testing notes?Mind maps with multiple nodes to demonstrate different ways you have tested the product, and to show more information about your testing, test ideas and discoveries and problems, or diagrams to demonstrate behavior during the requirements of planning sessions is a good example, or tools like RapidReporter or JiraCapture, to notate your exploratory testing in a chronological text based format. What's the value?Good testing notes can stem more test ideas. They can help you track different activities beyond executing tests, such as session setup and bug investigation. They can also be used as a trigger to remind you of what testing you have done during an exploratory testing session. They can be used as testing evidence displaying coverage and risks that have been investigated. What are the pitfalls?Writing too much detail in your testing notes can be a drag on your time and may affect what information you're taking in. And writing vague notes with too little detail will result in a lack of understanding in the future. And sticking with one note taking approach may bias the way that you test to so experiment with different note taking approaches. 

What is an oracle?An oracle in testing is a source of knowledge that informs your testing, and that information can be used to inform or determine your testing or to determine if something is a defect or not. Oracles can sometimes inform you of other oracles, and oracles are like a spidey sense when you get the feeling that something doesn't quite seem right here. Cool.  Do you have any examples? You have comparable products. So does the application have to work in a way that is in line with similar products? Statutes. Does the application have to work in a way that conforms to laws and regulations. Claims, does the application have to work according to requirements or specifications. Are there any marketing or sales claims made regarding the software, or past experiences, so relating to things that you have experienced in the past to inform you of the software that you're testing?  What's the value? They can be used as a source, for backing up your book reports, and, also, they can inform your test design and what might take priority in your testing.  What are the pitfalls?Oracles are flawed. Using one oracle will return only one piece of the puzzle. You need to use many oracles to gain a fuller picture. And now we know about oracles.

What is technical testing?Technical testing is an approach where testers use technical skills such as programming, scripting, and development practices to inform their testing. This approach can be used to help testers reveal in-depth information of a product that typically less technical based testing would not reveal. Technical testing is more of a mindset than a set of prescribed activities, focusing on how technical knowledge can help me as a tester. What technical testing examples are there?Using browser tools to learn technical details about my product and manipulate my product in unusual ways Pairing with a developer as they build the product to test their ideas and the code as it's created, I could use a HTTP proxy tool to monitor traffic between product components. What's the value?The technical knowledge of your product can help you generate new test ideas and inform your test strategy. Technical knowledge can help communicate with developers by providing detailed evidence of bugs or having technical conversations about new features. You can use tools to create complex tests to discover interesting and complex bugs that sometimes have severe consequences. Are there any pitfalls?Whilst understanding how to code can help you with technical testing, Technical testing is not solely about coding. There are tools that can be used that don't require programming skills. Also, focusing too much on the technical activities in testing can become a distraction with testers wasting time on setting up or building tools. 

What is scripted testing?Scripted testing is an approach to testing that is aimed at confirming that a product meets expectations set from requirements. Scripted testing typically uses test cases to aid the comparison between the expectations and the product itself. Scripted testing is sometimes referred to as checking to emphasize that scripted testing does not cover investigative testing activities. And scripted testing is usually reported in a quantitative manner following the pass or fail output of whether the expectation is met or not.  Got any examples?So an example of a test case could be step one, browse to the search screen. Step two, add search criteria. Step three, click on the search button. The expected result is that the relevant search results relating to the search criteria should be displayed in the results table, and the table should be paginated to ten results per page.  What's the value of scripted testing?So scripted testing could be a good starting point to follow steps to help understand how to operate the software and how it's expected to operate. Scripted testing can easily be repeated for change detection. If a previously passed expectation is now being met, we can assume something has changed. Scripted testing can also stem other testing activities that complement scripted testing in line with the whole testing picture. What pitfalls exist with scripted testing? So an overreliance on scripted testing means that you will miss potential bugs and information. Scripted testing is also expensive. It takes time to create and maintain scripted tests, and scripted testing alone does not constitute the whole of testing. There are many misconceptions about this.

What is lateral and critical thinking?Lateral thinking is where we try to think creatively about an idea, requirement, or piece of information from different angles to uncover more information about variables and perspectives relating to that initial information. And critical thinking is where we critically assess the information that we have, thinking about risks, assumptions, and problems relating to that information that we are assessing.  Do you have any examples? So thinking laterally about dice, it can have between four and a hundred and twenty sides. The number of the dice can be displayed as dots, digits, or even as written text on the dice. The dice can be made from plastic, wood, or many different kinds of metal. And thinking critically about the fact that we need the dice to play the game, we can ask questions like, well, why do we need the dice? Would our random number generator, be suitable instead?  And what's the value?So lateral and critical thinking skills help us to uncover test ideas. If we are testing the idea of the software early in the SDLC process, then our lateral and critical thinking skills will help us dispel assumptions and uncover more information to help us prevent problems and risks further down the line.  And what are the pitfalls?So many people aren't familiar with lateral and critical thinking and don't realize the importance of these skills for testers. And many companies don't fully understand the investigative side of testing. And even with strong lateral and critical thinking skills, it's important to remember, you can't think of every permutation or each variable.

What is testability? Testability is a measure of how easy it is for testing to happen in a given context. Testibility is a measure of many drivers such as people, product, vision, process, time, resources, and problem. These drivers can have a positive effect on our testing, making it easier to test, or they can have a negative effect to making it more difficult to test. Interesting.  Do you have any examples?The team's attitudes toward testing can either mean testing is prioritised and supported, or it's an afterthought. A product with many accessible layers such as APIs means we can be more targeted with our testing and automation. Access to testing tools could be restricted due to budget or IT policies. This impacts the types of testing we'll be able to do.  What's the value of knowing about testability?Knowing about testability informs our testing approaches and our strategies. It provides clear framing for the testing we do and don't do, which helps communicate that work to others, improving testability often delivers more value to the team than doing more testing activities.  And what are some of the pitfalls around testability?Testability is fluid. Therefore, teams need to periodically review it to ensure approaches are fit for purpose. As testability is an assessment of contact drivers, it can often be biased by those doing the measuring. So it's important to include the whole team.

What are models? A model is an abstract representation of a thing. So that could be a process or architectural design or or thinking or a structure, and models can be used to help people to understand concepts or complexity of that thing. And they can be created in many different formats too. So whiteboard drawings, logical flow diagrams, maps, like main maps or even written text or even verbally described.  Do you have any examples? There are some common process models that you may be familiar with, like an agile sprint cycle or the agile testing quadrants, for example, or the many different automation triangles. There's some other business specific models that people might be familiar with, which are business process diagrams or risk maps.  What's the value of a model?So models can help provide information. They can help understand a problem space. They can also trigger test ideas and models stem more creative thinking as well.  What are the pitfalls? Where all models are flawed. A map of the UK, showing roads, is gonna be different to showing weather. For example, so a specific model won't have a specific purpose, but it can be used for many different purposes. And models are based upon perspective as well, meaning that assumptions are made that baked into the models.

What is agile?Agile is a methodology for developing software and agile approaches allow teams to take an iterative approach towards building a product.  What is the Agile Manifesto?So there's four key things in agile manifesto. They are individuals and interactions over processes and tools, working software over comprehensive documentation. Customer collaboration over contract negotiation and responding to change over following a plan. What's the value of Agile?So agile encourages collaboration between different team members. For testers, that means that we engage with our teams earlier in the project. And teams can regularly feed back to users to ensure that they are on the right path. And another big benefit agile is agility being able to adapt to change. What are the pitfalls? Agile is a principle and not just a series of activities such as planning and a retrospective. And it requires a team to be mature and regularly communicate and collaborate with one another. And a team not making any noise as an agile.

In software development, requirements are things that the business or users want to see in their product. They are ideas for new features formed into artefacts which state expectations of how the software should work.Requirements usually come in the form of short descriptive sentences that detail rules, And a specification is a set of different requirements that will describe a feature, or a product. Requirements can help provide context for what the team needs to deliver, and we can use requirements and specifications to guide our testing activities too. Requirements are explicit instructions based upon implicit ideas, meaning that the author's intention might not be communicated well. Requirements can be misunderstood by the team when reading them and assumptions can be made. Software testers should be aware that they may need to test around the requirements, meaning they shouldn't just confirm the requirement is true. You can never think of every permutation or variable of the software and how it will be will be used when creating the requirements. So explore it to the testing beyond the requirements as required.Requirements examples: Online shoppers should be able to add multiple products to a shopping basket. The user should be able to remove products from the shopping basket. The shopping basket should show the total price for the products plus shipping costs to the user. 

Software testing is many processes such as learning, experimenting through hypotheses, testing design, test execution, and collecting information about products, projects, processes, and ideas.Software testing is focused on identifying risks and exploring the product or project to discover if those risks are real and a threat. Software testing offers a feedback loop for providing information that's been uncovered or for confirming whether expectations on how the software should work are being met or not. Software testing gives us an increased understanding for the team. The more we test, the more we know about the product. We uncover a fuller determination of the product's actual quality, and it can challenge the team's assumptions on what is to be created or what has been created. It can also help facilitate collaboration and communication between team members. There are many misconceptions that software testing is an easy thing to do, when it's not. Software testing is a craft that requires skill, insight and passion for learning. It can also be very difficult to describe the purpose and value of testing. It requires practice. Examples of software testing: Asking questions about the ideas relating to new software features Discover how the product will solve users problems Discover any risks surrounding the ideas Exploring the product and executing tests to discover information Giving feedback to the team or to the stakeholders to make an informed decision about the next steps to take with their product 

Communication is a key skill that's important for software testing professionals to have. We need to communicate with project members and management to talk to them about great testing practices. We need to talk to our team and stakeholders about testing that we've conducted, communicating our testing notes, and reporting on our progress. Communication can take many forms too, from text based, which is emails or chat apps, or verbal communication, which can be face to face or via a call, or visual communication as well using models and diagrams. Understanding communication and the different forms it can come in helps us to communicate more effectively and helps people to understand the messages that we're trying to communicate. Becoming an expert communicator will help you grow your reputation as being a great tester as well. Being able to explain what and why your testing goes a long way. Not being able to fluently talk about testing or about testing in general means that many problems and challenges you face as a test may go unresolved.Being unaware of different communication methods will hinder your effectiveness in communicating to some people. Communication examples: We can use whiteboards to sketch out and model test ideas or test strategies while in a team meeting We might send emails to provide text based status updates on the progress of our testing We can verbally communicate test ideas and discuss the designs and risks of the software when pairing and mobbing with developers. 

Test automation is the process of taking repeatable testing actions carried out by a human and setting up a machine to carry it out instead. Test automation may come in the form of off the shelf tools or tools that have been developed by the team.In testing, the purpose of automation is to help support a tester or the team. We can think of automation as a testing tool. Using the right tool can mean increased productivity as repeated activities are sped up. A good tool can be used repeatedly with the same output each time, and there's little risk of human error. The output of certain tools can be used to stem new text testing activities. Overreliance on test automation can mean you have gaps in your testing and feedback. It's important to remember problems first, tools second. Using a tool because it's popular can lead to problems, so it's best to choose a tool that is specific to your problem. Test automation comes with a cost to set up and maintain.Test automation examples: You could go for simulating user behavior on a browser with a tool like WebDriver.  Rapidly generating test data using Bug Magnet or Pearl Clip.  Creating HTTP requests with Postman. 

Test ideas are hypotheses that testers generate to reveal information. And they usually come in the form of a question, and test ideas are informed by information we have about the product and perceived risks. Executing actions based upon a test idea to help reveal information.Test ideas can help trigger other test ideas as well. And recording a test idea and notes can provide evidence of testing carried out too. Test ideas can bias you. The more you test, the more likely you are, to be biased towards a specific area or type of testing. Test ideas sometimes don't come naturally at all. It takes skill to help generate test ideas. Test idea examples:  An example would be, what if I pressed that button a hundred times really fast?  Or what if I turned off that microservice?  Or what would happen if I enter a a date in a different format to my region. 

A test charter is a sentence that focuses on an exploratory testing session. Each test charter is based on a risk that has been identified within a specific area that has to be focused on. You can create test charters using templates. One such template could be to explore target with resources to discover information. Or you could try a mission to discover information. Test charters templates give much more structure to an exploratory tester in revealing information for a specific risk. It also allows an exploratory tester to determine whether they're going off charter and determine if that's acceptable. A group of test charters can help define the body of work for a tester to carry out on a user story or feature. Test charters need to be just the right level of abstract and specific to allow a space to explore but not to get lost in their exploration. These are required skills for identifying risks. They are fallible in that they focus on a specific risk, so missing charters can affect your level of coverage. Test Charter examples:  Explore a login form using different credentials to discover how secure the login form is.  My mission is to test the user interface to discover accessibility issues.  Explore the new user web form with various ASCII two five five characters to discover risks and problems surrounding data handling. 

In this adaptation of “The Pen Game” lateral thinking game, the challenge enables the audience to test “the system”, giving them control of testing their ideas.

Continuous testing throughout the SDLC - Dan Ashby

Learn more about the essentials of Software Testing and the models that can help you.

Dan Ashby joined us at ReTestBash UK 2022 for a live Q&A session after his talk Evolving Our Testing: Assessing Quality Throughout The SDLC and we had some very interesting questions and answers.

Watch "Evolving Our Testing: Assessing Quality Throughout The SDLC" with Dan Ashby from TestBash UK 2022

Start your journey into software development and testing by learning what it's all about

"Power of Models" from TestBash San Francisco 2018

Discover tips from a manager on gaining budget spending approval

Use an interview questioning model to find great testers

Learn how to deal with people's misconceptions about testing