Lessons Learned From 20 Years Of Software Testing

Lessons Learned From 20 Years Of Software Testing

Explore the personal journey and collective wisdom of a seasoned software tester

"Assume the best of your colleagues. If you come across something that's less than perfect, remember that they, like you, are human and don't have all possible information available to them at all times. We can strive for quality and be kind at the same time." 

Who I Am And Why I'm Writing This

I started out in software testing as a 37-year-old with experience ranging from farm work to driving wagons. Along the way, I audited British Standards in Quality, Health and Safety and Environment. Little did I know that some twenty years later I would have found a career in testing, rather than just another job. I’m not saying I didn’t enjoy those other jobs, but I’d never really thought of myself as having a career until I met the wonderful testing community. Ministry of Testing in particular, opened up to me a world of brilliant humans and knowledge I might otherwise never have found. 

So here, in no particular order, is a collection of lessons I have learned through those twenty (mostly) wonderful years. I invite you to consider, ignore, argue against, or agree with them. If you have your own lessons, please share them on the Ministry of Testing Club. I’d love to hear them. 

And Now, The Lessons…

It Depends: Context, As Always, Is Key 

“Well it depends on…”, is the answer to every testing question ever asked. I say that because context is the key to understanding, and every testing scenario needs a context to help it make sense. 

Consider the list of questions below and note that every answer you think of will contain assumptions, since they are asked without context. So how do I test a log-in page, not how do I test a specific log-in page? 

How do I test a log-in page? 

  • It depends on whether you are on a desktop or mobile device. 
  • It depends on whether you use a mouse, a keyboard, a touch screen, or a screen reader. 
  • It depends on which operating system you're running. 
  • It depends on the fields present on the log-in page and, if any are present, what you can or cannot enter into them. 
  • It depends on the information the URL gives you. 
  • It depends on what happens if you click submit without filling anything in. 

I could go on but hopefully you get the point. 

If you keep the principle "it depends" in mind, you'll ask further questions. You'll find yourself digging deeper and questioning your initial reactions, biases, and assumptions. 

So remind yourself that whatever the question, scenario or situation, understanding the context in which it arises is of the utmost importance. 

Continuous Learning Is Vital 

You can learn anything from anyone and from everywhere you go, and everything you see or hear. No matter your experience in testing, you always have something to learn. Sometimes it will be a new technique, and sometimes it will be more in-depth knowledge. Every encounter, every new person and situation you encounter, old or new, can teach you something. At every conference I’ve been to, I’ve learned as much from the conversations I’ve had with people I might otherwise never have interacted with as I did from the conference talks and activities themselves. 

Just like we can’t test all the things, we can’t know all the things either. This is where continuous learning comes in and helps you gather more knowledge as you go, helping your testing, your career, and your future. Through continuous learning I not only have deep knowledge of testing, Agile methodologies and digital accessibility, but I also gained some knowledge in other areas, from performance to security. That breadth of learning has helped me in so many ways over the years, particularly in creating a learning path for apprentice testers. 

You Are Paid To Think, So Think! 

It took me most of my first decade of testing to understand what a thought worker really was. We all take thinking for granted 99 percent of the time. It’s like breathing. 

But when you are paid to think, you need to learn all about thinking and then actually apply those thought techniques to your work. There are so many techniques we can use. Just to name a few: heuristics and mnemonics, critical and systems thinking, or methods like the six thinking hats. Thinking Fast and Slow by Daniel Kahneman is a great book that explains how to think deliberately and effectively. 

I like to say that 'testing is a task without end.' The reason I like to say that is because there’s always more to think about, areas to consider. Look at Ministry of Testing’s Test Heuristics Cheat Sheet to see how many different things we need to think about. Whether we decide to act on those thoughts or decide (or have it decided for us) that it is time to move on, or that the risk is sufficiently low, we always need the skill and mindset to be able to think deeply about testing. 

Find A Thriving Community Of Testing Practice

There are some brilliant test communities out there. I have a special affinity for Ministry of Testing (MoT) since that’s the one I found first and where I found the most support. A community can not only help you learn and support your journey but also open up or make you aware of opportunities. 

A while back, thanks to a sudden hiring freeze on account of the COVID-19 lockdown, I had a job offer withdrawn. That rendered me technically unemployed, and at such an uncertain time for the world it was quite a frightening and daunting prospect. Fortunately the great MoT community gave me support and found me opportunities. I did my first webinar and first online talk, gave training remotely, and was pointed to several job opportunities. I was fortunate to receive a couple of job offers and accepted one just five weeks later. Without the power of the community, I’m not sure how I would have handled that situation. 

In 2020 and again in 2021, MoT stepped up to keep the community going during lockdown with TestBash Home. That was such a cathartic event for me personally. I hadn’t realised just how much I'd missed connecting with the wider community directly and it was quite an emotional day for me. 

So find and get involved in testing communities. Like me, you might just make lifelong friends and have your career saved. 

Learn How To Give Helpful Critique  

People talk about the testing mindset, but it is very rarely explained. In my opinion there is a fine line between being critical in a shallow way such as, ‘it doesn’t work, doesn’t do this or that’ and critiquing an idea or product in a helpful way by suggesting ways to improve the system. That fine line contains some of the most important parts of humanity like communication, empathy, and intent. 

How you explain your thoughts can be damaging in a number of ways. Whatever your intent, undue harshness of expression when giving feedback or writing a bug report can be damaging to your relationship with the receiver as well as your personal reputation. Imagine if you had written the code yourself. How would you like to hear feedback? How would you give feedback to someone who's just starting out in their profession?

How you think about the person or group you explain to can change your tone, body language, and language choice. This is where empathy comes into play. Norm Kerth, author of Project Retrospectives: A Handbook for Team Review created a ‘Prime Directive’ for retrospectives, but I think it can apply to all aspects of our professional interactions. 

“Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.”

Negative Testing Doesn’t Require A Negative Mindset 

As testers we have lots to consider when approaching products and systems. Generally my first response is curiosity. At some point we will consider risk and the 'sad path,' but that doesn’t mean it always has to be negative activity. 

There are many aspects of negative testing that can actually make you happy or smile. Here are a couple of examples. 

  • Graceful degradation done well generally makes me happy. 
  • Precisely worded, helpful, and appropriately placed error messages increase accessibility, which is always a good thing. 
  • Good handling of interrupted connections gives the opportunity to have an offline option, accompanied by a helpful message. When a connection is again available, the application should pick up where it left off with no loss of data. Sounds like good quality and user experience to me. 

I like to think like this Classic Fisher Price Advert baby when encountering new things. I use my curiosity, consideration of risk, and a number of other influences to find the positive in negative testing. 

Promote Mobile-First And Universal / Inclusive Design 

Mobile-first design forces you to consider all different sizes of devices and how to ensure your responsive design adapts to them. Considering mobile device usage also means you have to focus on the most essential features and content and put them front and centre. This focus benefits all users, including those with cognitive impairments, by potentially reducing cognitive load and enhancing the overall user experience.

A definition of universal design from the National Disability Authority says, “Universal Design is the design and composition of an environment so that it can be accessed, understood and used to the greatest extent possible by all people regardless of their age, size, ability or disability.” Considering all the different ways people may use your software means it can be more inclusive without the need for add-ons or adaptations. As technology advances, a universal or inclusive design is likely to continue to be more usable with minimal updates. 

Combining those two approaches means software will be accessible and adaptable no matter how, or on what kind of device, a person interacts with it. 

Being Comfortable With Public Speaking And Storytelling Can Help Your Career

Often testers take the lead in discussions, demos, and conversations about product features and quality.  We tell stories to explain what we have found, what we feel needs addressing, and why that’s important. Telling stories can help to explain complicated concepts and ideas and adds context to our findings. That’s one of the reasons we like heuristics, mnemonics, and examples to work with when testing.

Being the lead in demos and other team meetings naturally lends itself to storytelling, which in turn gives you some of the skills for public speaking. I’ve been very fortunate to speak at many meetups and conferences, both in the UK and abroad. However, I’m not a natural public speaker. I was very shy as a child, and a teacher once pushed me into a public speaking competition. It was a horrible experience and one I was keen not to repeat.

However, the storytelling I did on the job as a tester gave me confidence that, when I wanted to talk to people about accessibility, I'd be able to carry it off. I’ve been doing public speaking for over a decade, and it's gone well, but it's still a challenge. I still get nervous and need breathing exercises to get up on stage, even for 99-second talks! 

So even if public speaking seems unattainable for you right now, I guarantee that if you have been a tester for a while, you already have some of the skills required to become a public speaker. 

Capture And Celebrate Your Successes 

Find a way to record your successes no matter how big or small. Doing this helps in many ways: sessions with your line manager of course, not to mention your own self esteem and mental health. It is also excellent for providing evidence for raises and promotions at end-of-year reviews. 

If, like me, you sometimes focus on what went wrong or was less than perfect much more than what went right, then recording your successes as you go can be really helpful. When I first started reviewing my successes, I did it only once a month. I found that I missed a lot that way. Instead, try to capture your wins as they happen, when they are fresh in your mind. That way you can do yourself justice. 

The logical next step is to celebrate those successes. Whether that is in a review at work, on social media, or as an addition to your CV, that is up to you. If you do something awesome but you don’t share it, you are doing yourself and your career a disservice. 

Develop A Habit Of Empathy 

As I mentioned above, testers have to be critics of the product or system they are reviewing. That does not mean we have to be critical of the person who made the decisions or wrote the code. 

A long time ago, while I was training to be an auditor, a wise trainer said, ‘The best way to create a rapport with people is to tell them that you want to catch them in, not out.’ The point of that expression is to be honest and open about what you want and need so you can build trust. As for our users, we need to consider all possible uses and users, as neither you nor I can possibly speak for the needs of all of humanity. Not everyone will do the same thing or even use the same small set of variations. 

There’s a reason I say that testing is a task without end. That’s because there are so many variations, as described in the ‘it depends’ section above. It costs nothing to be kind. It doesn’t make you less professional, far from it. It can make you more successful in building trust across your team. 

Assume the best of your colleagues. If you come across something that's less than perfect, remember that they, like you, are human and don't have all possible information available to them at all times. We can strive for quality and be kind at the same time. 

Don’t Dismiss Your Life Experience 

To me this is especially important for those wanting to learn to be testers or those new who have come from other roles. I think all testers bring life experience with them to the work they do. That's one of the reasons why having a diverse set of testers is much better than ones from similar backgrounds. 

Many of my prior experiences influenced me as a tester. One standout is being trained as an auditor. Being able to look at a process and understand all the touchpoints and steps, then working through that documented process and assessing if reality is aligned to it, was a great background to build on when I became a tester. 

Another part of my background that came in handy in testing was capturing knowledge to create guides and training material when I was a team lead in customer support and processing. That has helped in so many ways: creating how-to guides, designing a testing apprenticeship, and giving workshops and talks. I'm so grateful for all the different paths I travelled before coming to testing.  

Helping Others Helps You

Along the way in my testing career, I’ve been a mentor, a trainer, a public speaker, and I've written articles too. I've also created a software testing apprenticeship curriculum and content for it. I assumed these roles not just to share my thoughts and ideas but also to help others learn and grow. I’ve had so many benefits from doing those things: meeting people I might otherwise never have met, financial reward, reputation building, and mental and emotional well-being (confidence and self esteem). 

You may feel that you have nothing ‘new’ to share with the community, and I’ve even had people say that to me directly. But there’s something you need to remember. You are unique through your personal experiences and no one else has had exactly the same journey or same experience as you. You have stories to tell and knowledge to share from your approach to testing, a fun bug found, or something unique to you. Sharing makes us all stronger and supports our continuous learning journey. 

So help yourself by helping others. The Ministry of Testing Club is a great place to start contributing by answering the questions asked by the community. 

Sharing through social media isn’t the only way you can aid people in becoming stronger. There’s mentoring, or simply supporting those around you. I saw a great thought on LinkedIn: “Be the person you needed 10 years ago. The old you of 10 years ago needed someone. Someone with a set of skills, someone with experience, someone with understanding, someone with values.” Alyzande Renard MSc MBCS.

I think it is great to think that way, since we have all had people help us on our journeys. It could be in a formal setting, or just a conversation with an explanation. Everyone needs help, so please try to make others stronger where you can. 

As Scott Kenyon said in review comments while helping me with this article. 

“Don’t be amazing on your own, but be amazing with everyone around you.”

Testing Experience Doesn't Mean You Know More Than Others

I’ve encountered testers of all levels, with zero to 40 years experience in software testing. One thing has become very clear to me. The number of years someone has been doing something does not necessarily mean that their knowledge and experience are what you might expect. 

There are many facts and many more opinions on how to develop and test software. Doing it for a long time doesn’t automatically equate to having good knowledge of testing. Often a tester with one year of experience can have as much or more insight as someone who's been in the field for ten years. People brand new to testing have shared insights with me that taught me something. So don’t underestimate a new tester or overestimate an established one. 

Take Care Of Yourself 

It is easy to focus on work and your career to the exclusion of everything else, especially in the early stages. But please don’t sell yourself short by forgoing a good work-life balance. 

I, like many, am highly motivated, working long hours either for work, learning, or for extras (side hustles). And I have sometimes lost focus, overcommitted, or simply put too much pressure on myself to get things done. 

Be kind to yourself, talk to yourself with empathy and love. Allow yourself to say no, do it tomorrow, or pause if you need to. I have had some hard periods in my life and career with my own mental health issues. I’m still learning about myself and trying to learn what works for me. After working for 25 years outside of testing and then for 20 years as a tester, I still don’t have all the answers, so please do try to take care and look after yourself. 

And finally, an adage that took me far too long to learn…

Know The Value Of Your Contribution And Get Paid For It

I know that sounds mercenary and selfish, but you only get one career and one life. Make the most of it. It is important that you know your value and are prepared to walk away if your employer doesn't value your contributions by paying you adequately. 

I’ve always been one to collect evidence as I go. And I have used that evidence to ask if there is any appetite for a pay increase at the halfway point of the year. I’ve never demanded a pay rise. I also use salary market data to back up my proposals. Your salary affects not only your current situation but also your future level of comfort in retirement. So the better your salary, the more you save for later. 

Sometimes this means walking away. After over a decade at one company, I tendered my resignation even though I didn't have a new job yet. But I gave three months' notice so that I would have plenty of time to find a new role. It was what I had to do in that situation. 

Quick Tips

  • Work the TESTING problem, not the situation, pressure or time constraints. Don’t let outside influences stop you from doing a great job. 
  • Everyone learns differently, so share information in multiple ways to accommodate visual, auditory, kinesthetic, and reading / writing learners. 
  • It is OK not to know something. There are no ‘stupid’ questions, or, as I prefer to say, there are no questions too simple or obvious. Questions lead to clarity of understanding for all.
  • It is your job if it helps. Saying 'it’s not my job' might be technically correct, but it could keep you from suggesting positive change, just for starters. Yes, there are role definitions. But sometimes you just need to do something, even if it's not usually yours to do or it is a little outside your comfort zone. 
  • Reading code can be a huge advantage. Testers absolutely do not need to know how to code, but it is so useful to be able to read and understand how code works. 
  • What goes on behind the screen is fascinating. Every part of software hiding behind the F12 button (browser developer tools) is interesting and gives you insights into bugs you might otherwise not have observed. The same applies to APIs, databases… in fact, the whole system. 

To Wrap Up

Whether you agree or get any benefit from one, all, or none of these lessons, they are what I will take forward with me through the rest of my career. And you can rest assured, this list is not exhaustive, and I will no doubt add to it. Continuous learning isn’t an option; it most definitely is a must! 

For More Information

Ady Stokes's profile
Ady Stokes

Quality Engineering Architect

@A11y_Ady on Twitter (X). Passionate about accessibility, exploring and testing as part of the creation and development of software. I help teams build better software and I strongly believe in collaborative methods and using different thought techniques and people perspectives to look at things from many angles. Accessibility is about inclusion, not just disability. In my career I’ve been a Director. Test, BI and Logistics Manager. Tester, Test Engineer, QA, Site Lead Tester, Quality Engineering Architect and any other value adding role required at the time. I have also taught, coached and mentored people throughout my career. My career highlight is creating the Software Tester Apprenticeship for the Coders Guild and training people to get their first role in IT through government sponsored free training courses based on my apprenticeship. I have my own blog at The Big Test sharing my thoughts, occasional poetry and my Periodic Table of Testing, a visual heuristic showing the breadth of the testing universe.

The Skill of Continuous Testing - Jitesh Gosai
Why I Lost My Job As a Test Manager and What I Learnt As a Result
Owning Your Craft and Gaining Control by Learning Technical Skills
Software Testing News – Issue 468: The First Step
Your Weekly Testing News - Issue 429
Your Weekly Testing News - Issue 448
Explore MoT
TestBash Brighton 2024
Thu, 12 Sep 2024, 9:00 AM
We’re shaking things up and bringing TestBash back to Brighton on September 12th and 13th, 2024.
MoT Advanced Certificate in Test Automation
Ascend to leadership roles by mastering strategic skills in automation strategy creation, planning and execution