A Personal Course Review & Practical Advice For Software Testers 

By Adrian ‘Ady’ Stokes 

These are my recent experiences with the ISQI Certified Agile Testing course, referred to by many as the ‘CAT course’.  With the course details and structure, I’ll include some practical information and advice should you be considering taking it. I conclude with my thoughts about those who I feel may benefit from the course and perhaps more importantly, who I think may not. 

I feel I should begin by saying that I am neither an advocate for or against certifications in general which is discussed in detail elsewhere on Ministry of Testing in The Club, Are Software Certifications Worth It?  I realise there are folk out there that have very strong feelings for and against testing courses of this nature and certification in general but that is not the purpose of this article.  

Background, Why Was I There? 

I spent a week taking the Certified Agile Tester course sponsored by my employer.  There are many reasons why a company, particularly one who has a large number of clients, might want their testers to be certified in some way.  It is a selling point that the systems being used have been tested by ‘certified testers’. And for some clients, these types of things can sometimes be required to allow companies to be part of the conversation to win business.  For other companies, it can be an enhancement to their reputation to use the fact they have certified testers as a selling point. The course is on offer in various locations for a cost in the thousands, which is not a problem for larger companies but might be if you are funding yourself.  

Background, Me

A long time ago I completed the ISEB Foundation Tester Course a year or so after I began my ‘official’ testing career having had some user/business acceptance testing experience first.  While the course put some names to things I was already doing, I came away feeling I’d not particularly added much to my skills or toolbox. That four-day course was laboured with forced language and an instructor who kept emphasising specific details as ‘important’ which it quickly became apparent were ‘clues’ to passing the exam.  I did and with a decent mark but I did not enjoy the exam as I felt very pressured to pass. Which leads me to what I feel is an important aspect. 

The Emotional Impact

I feel I have to address the impact of being asked to validate through an exam, a decade of working in agile environments as a tester.  It was stressful! I’m not going to go into a lot of detail about my imposter syndrome firing at the prospect of having virtually my entire career scrutinised by a stranger on their terms and under their rules.  I felt anxiety in the weeks leading up to the course, throughout and on the day of the exam. So enough about me. What did the course look like? 

The Certified Agile Tester Course

The Week’s Agenda 

• Day 1: Agile Methods and Process 
• Day 2: Planning 
• Day 3: Testing 
• Day 4: Teams 
• Day 5: Exams 

How My Day-to-Day Looked 

This may vary in timings based on your instructor or course provider. With the exception of Friday when the exams filled the whole day, each day had a very similar structure.  After our introduction to Agile Methods, we had a ‘stand up’ which then happened each of the other days. Mornings were filled with learning, or in my case, refreshing agile knowledge and the afternoons with exercises which developed into more practical exam focused practice as the week passed.  We were also encouraged to spend time at home reviewing, as the very thick course manual of 300 plus pages and 15 sections had plenty of sample questions included.  

The irony of being given a very thick document which quite quickly describes how big up-front documentation is a bad thing was not lost on me.  

How You Are Judged

There are two exams and an ongoing ‘Soft Skills’ assessment through the course.  The two exams are worth 45% each and 10% for the soft skills. To pass and be ‘certified’ you need at least 50% in each of the three parts and over 65% in total.  

The Soft Skills Assessment

There are 5 areas the course instructors are looking for. They wanted to see you to show the requisite skills in:  

• Being willing to take on challenging activities 
• Planning work and actively contributing to delivery and goals 
• Trusting other team members and being willing to share and mentor 
• Accepting common responsibility for success (and failure) 
• Communicating clearly in both written and verbal style

This felt like the easy part of the course, even though I was worried about the impact my questioning of a large proportion of the course might have had on my final grade. Often the materials and class assignments felt too far removed from reality to be effective in real life situations.  

The Two Exams

The Practical Exam 

The practical exam was two-hours, in which session sheets were worth 40 points and you were assessed on how you created and followed a sprint plan which included:

• Timings: a brief outline of how long you plan on spending for each section
• A task board: for example, a Kanban board with work in progress limits 
• Test charters which required:
  • tests 
  • defects found 
  • references 
  • any retests 
  • Absence of references could cost points, e.g. DEFECT01
• A retrospective: where it was important to get the key points and highlight at least one improvement action. 

The plan, progress board, and retrospective were worth 5 points each so it was worth ensuring they were correctly completed. Things like defect references and succinct defect reports also received points so a simple check of each defect for a reference and brief report were worth doing.  As was checking for retests as they were an important part of further ‘code drops’ and score points. 

The Written Exam

The written exam was a two-and-a-half-hour essay-based exam worth 100 points spread over 10 short questions worth 4 points each, and 3 longer scenario-based questions at 20 points each.  It was suggested by the instructor to give longer questions half an hour each leaving an hour for the 10 shorter ones, or about 6 minutes each.  

I found that doing two of the short questions, then a long question helped me with timings. This also had the added bonus of ensuring I might miss only one or two shorter questions rather than a longer one if I ran out of time. This tactic worked well for me, but a colleague taking the same test followed the instructor's original recommendation and said it worked for them. 

My Experience Of The Course

Day 1: Agile Methods & Process

Day one began with an overview of the course, what was wrong with traditional methods like waterfall and the V model. The day progressed onto agile methodologies with a focus on Scrum and the concept of ‘ideal’ Scrum.   

The Methods and Process section contained a lot of useful information about how to transition to working with agile methodologies. It also had pointers for those beginning the transition process, and ideas for those which had less experience with Agile. It was an enjoyable refresher for me, even though I have worked on Agile projects for over a decade. 

The day finished up with some Lego related exercises to help gain clarity around user stories. While these were fun, they were so far removed from reality I struggled to see how they would help someone who wasn’t already experienced. With no real guidance on what the learning focus was, the Lego activity led to some frustration.  

Unrealistic Expectations 

We had a new team for the Lego activity and were expected to almost instantly come up with an estimating concept. Our first planning session happened without a vision, guidance, or any other sprint zero type activities that you would normally expect at the beginning of a project. It was difficult for all involved, but we did manage to make decisions as a team and deliver something.  

Towards the end of the Lego activity, a picture of what the finished building should look like was shown to our team.  In a ‘specification by example’ mould, the goal and expectations became much clearer.  If the point was to highlight that building (software) is hard without clear goals and cooperation, it worked. It felt like a strange way of conveying the lesson to me, but again, to someone without experience already it could potentially be very effective. 

Day 2: Planning

The second day was the hardest day, by design, and quite pivotal in terms of my approach.  Following the morning overview of release planning, task boards, test strategies, estimation and burn down charts, the afternoon exercises were constructed to bring failure after failure. The significant time restrictions and lack of guidance while also working on team building made it hard for me and the rest of the attendees as all the normal reactions and methods of addressing a situation like this didn’t feel to be available.  

To offer some examples, without describing a play-by-play of the whole afternoon; in our sprints at my work, we have an hour for story refinement and an hour for planning.  Story refinement allows everyone to have a shared understanding of each story, having discussed how we will implement it, the acceptance criteria, how we will test and so on.  On the course, we had no time for refinement and a very short window for planning. So when we began our testing it felt uncoordinated. 

It took time to get my head around the whole thing, and come to terms with what was required, and then work out a way to cope. To pass the course, focusing on what was required, ‘in their opinion’ to pass the exam was key. In the way of many different courses, an open mind and desire to learn what they had to teach was key to getting the most out of the course. 

I don’t want to give the impression that the course was not doable or valuable, but I would have liked a lot more information before taking the course.  Understanding that the lessons of failure and finding ways to mitigate that failure in advance would have made the experience less frustrating.  

Day 3: Testing

Day three was another enjoyable morning and frustrating afternoon with the added bonus of a breakthrough.  Very late in the day, it dawned on me I was being overly literal and still leaning too much on my experience, which meant everything I was doing was far too granular for the purposes of the course exercises. This meant I felt I had no hope of passing the course doing what I was doing. In practical terms, this meant not putting tasks on my task board that ‘made sense to me’ but were short and very ‘to the point’. This turned my testing tasks into, ‘check UI’, ‘check database’ and ‘retest’. Not really very useful but it seemed to make our product owner happy.  I can only surmise this was because this approach made it more likely to complete the exam in the time allowed. The focus of the course was not on the testing, but testing management and application in Scrum methodology.  

Day 4: Teams

On day four, the morning was less enjoyable, mainly because of the exam day looming over the horizon. It was all about practising the exams but in much less time. I believe the thinking was if you get ‘somewhere’ with less time, the longer 2-hour exam would feel a bit easier.  Spoiler, it didn’t! But it was valuable practise. We gained an understanding of the type of questions and time pressures we would feel the next day. It allowed us more time on the system under test and to get familiar with the tool we were using to examine the database. This was definitely useful for the exams. 

At the end of the day, we were advised of the results of the instructor’s assessment of soft skills. The good news, as the instructor put it, was that we had all passed this part. The bad news, we could take the exams! 

Day 5: Exam Day

Day five, the day I’d been dreading. I was never very comfortable in exams at school or in further education and much preferred discussing subjects. Unfortunately, the exams required specific answers and ways of working.  

The Practical

I’m not going into massive detail here as I don’t think it would be appropriate and I was so glad when it was over! The exam invigilator was nice and did their best to make us feel at ease. The practical exam was based on the same system as our practice exercises. This gave us something familiar to work with for the exam. I found that focusing on finding a bug then moving on helped my timing rather than any representation of an in depth investigation of the system. There was very little room for anything but the primary goals due to the time constraint.  

The exam was to plan then test the system. We were told there were three ‘code drops’ available on request. When we had done our initial testing subsequent code drops required us to have a ‘retest’ task as part of our plans. Even with this light touch, I completed the exam with a couple of minutes to spare, which were spent checking my work.  

The Written 

Rather than a series of multiple choice questions, there was a series of scenario based questions on Agile itself and on testing in an Agile environment. I found it useful to focus on the key words and phrases they were looking for rather than expand on the answers too much. Even though I took that approach, I barely completed the questions in the time frame. 

For both the practical and the written, time flew by and I felt really pressured. I would have liked an extra half-hour for each one, which might have been fairer. On the bright side, we all had a great retrospective in the pub following the exams. There was a general feeling that both exams were very hard and everyone felt the same kind of pressure.    

Course Smells

Very much like ‘code smells’, these are the things that didn’t feel quite right about the course. These are my opinions, and you may not see them the way I do, but I wanted to highlight the things that stood out to me:

• The course material was presented in a very large document. This seems very anti-Agile. 
• My experience of being an Agile tester felt like a hindrance to successfully completing the course.  
• Focus on almost absolute minimum feedback due to time restrictions, rather than a ‘proper’ investigation to gather useful information and actionable insights was the theme of the lessons taught. If your focus is on becoming a better tester this may not be for you.  
• The documentation mostly used gender-neutral language but I suspect the document was rewritten at some point as I found a couple of instances where the product owner was referred to as ‘he’. 
• The manual was copyright © 2016 and the slide deck used was copyright © 2014. This seemed odd if they were advocating responding to rapid change. 

My Main Takeaways

It is a good course to learn the Scrum version of Agile, or as a refresher, albeit an expensive one. I did take some exercise ideas back to work but nothing really earth shattering. 

Recommendations For ISQI Certified Agile Testing Course 

My Course Outcome 

I passed with 77% and have now had the last decade of my testing career validated by a stranger or strangers, through scribbles on pieces of paper, against an ‘ideal’ set of answers.  I am certified… until I’m not? There’s an expiry date on my certificate which is very strange as the website Certified Agile Tester website has the comment, “Will I need to recertify after a certain period? No, the “Certified Agile Tester” has lifelong validity.”  

Recommendations Based On Context 

Rather than a blanket yes or no recommendation, I think this is very much context related. Each of the three scenarios below involves knowing if you have the funds and time available to take the course. The course is expensive, especially if you are paying for it yourself. It could be a worthwhile investment in the right context. If you are supported by your company, you have little to lose by taking the course. 

Already Experienced Agile Testers

For someone in a similar position to myself, with lots of real-world experience testing in an Agile environment, I don’t think there is a value return for the cost of the course. If you are a job seeker or looking to improve your CV then this could be something that gets you through the door for an interview. You can then let your individual value and experience shine through. 

Already Experienced Testers 

If you have the test experience but not in an Agile environment then this could be a course to consider. The morning sessions with the theory of Agile, its history, and an overview of Scrum methodology were well structured and, on my course, very well taught. The abstract of the afternoon exercises gave a sense of working on Agile teams but for the most part, was a discovery of how to produce the artefacts the assessors were looking for.  Those artefacts do have value in the right context but as mentioned were not really useful for me. If you are an experienced tester, taking the course would need careful consideration as the course has its challenges for those with both broad and practical experience of Agile and Scrum. 

New Testers

If you are new to both testing and Agile then I’d have to say this probably isn’t for you until you have at least a basic understanding of testing under your belt. The exams involve practical testing of a web application and back end database so if you have a basic understanding of both those and applicable test techniques I believe you could find value in this course. If I had to put an arbitrary time period on it I’d say if you have between 6-months to a years’ experience, including web and database experience or research into those test techniques, then you could take quite a lot of value away from the course. 

References

• International Software Testing Qualifications Board  
• Test Bytes summary of the top 10 certifications available in 2019 
• Article with recommendations based on your level of experience 
• Waterfall Software Development information 
• Article on the V-model 

Author Bio

Call me Ady! With an audit and management systems background, I came to testing in 2002 and found both a flare and a passion. With a thirst for learning and continually improving I'm always looking for ways to share information from my blog to a monthly newsletter and global quality initiatives at work. I'm a great believer in community and help the great one in Leeds by helping organise the MoT Meetup. I believe accessibility is fundamental to applications and greatly undervalued causing many problems for lots of users. I'm trying to spread the message that accessibility isn't about disability, it’s about inclusion. 

@cricketrulz | https://www.thebigtesttheory.com/

You May Also Be Interested In:

• 99 Second Introduction to Agile
• 30 Things Every New Software Tester Should Learn
• Providing Value to Agile Ceremonies as a Tester with Melissa Tondi
• Context-driven Agile Tester with Dragan Spiridonov