This article is not based on any company or individual but is based mostly on accumulated experiences of testers within the software industry obtained through discussion (often over beer and chips).
My cat isn’t much of a software tester and yet is a far better software tester than many humans I meet. When I sit down with her and discuss the importance of automated testing, continuous integration, early testing, benchmarks and metrics she looks up at me with her little whiskery face and meow’s. I think in cat speech this roughly translates to “yes but where’s the tuna”, because Lilly knows nothing about testing software. What makes her a better tester than many humans though, is that she sticks to what she’s good at, eating tuna and hunting mice (of the furry kind), and makes no assumptions regarding the testing of software.
Sadly in the software world many humans fail to do the same. In fact it seems that everyone and anyone is an expert software tester and their ignorance of the subject is just as good as any studying or practical experience any test engineer might have. If you have never experienced this then you’re a very lucky individual, however I suspect for many testers this is the rule.
What I’m referring to is Dunning-Kruger effect, which to quote Charles Darwin is the phenomenon whereby “ignorance more frequently begets confidence than does knowledge” .
The Dunning-Kruger effect is named after Justin Kruger and David Dunning who studied this phenomena and published their results in the 1999 article “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments” .
This phenomenon tends to affect us testers in three different ways.
- Our interaction with non-test staff.
- Our interaction with managers.
Interaction with Staff
This can vary, from someone who just about knows nothing more than how to use an XBox to someone with a PhD in astro physics. The XBox user may, since they have used computers, come to the conclusion that they understand the testing of computers. If we worked in isolation we could in theory simply and imperiously ignore them, however testing often is a co-operative task done with input from other teams and individuals, so simply ignoring them may not help us complete the tasks we need to do.
A customer support team may insist that a tester spend their time on user acceptance testing (a subject they understand) rather than unit testing (a subject they may not understand). It may be that the tester-individual is known as an expert on a particular subject (which leads to the logical fallacy that expertise in one subject must mean substantial knowledge in another).
I tend to see this as a structural problem that only management can resolve. Ultimately if the tester is to be responsible for creating the process for testing software, they must be the ones to decide how to implement the process. Those who speak loudest and show the most confidence are not necessarily qualified to make the best decisions.
For example a person may be an excellent driver, but they are not necessarily the best person to decide the process for testing a car that is being designed to go into production (because individual components should be tested well before the driver takes it for a spin around the track).
Interaction with Managers
This may be your line manager, your manager’s manager or another manager who has a great deal of influence on the project; a project manager for example.
The reason I’ve separated management from non-test staff (even though they might be non-testers), is that they may have a great deal of influence on the current or past testing processes. It’s unlikely to be an equal discussion since any knowledge you might have on the subject may well not be equal to the authority they have within the organisation. This can be frustrating but there is not much you can do about it. The best you can do is clearly outline your proposals for a process and the reasons for your proposals with the respective parties who have authority on aspects of a project. Your proposals will either be accepted or not.
Naturally it’s possible that we may actually not understand a technology or process ourselves, and according to Dunning-Kruger we may not be aware of this! The problem is that other individuals need to have faith in the knowledge we have. If our confidence is misplaced our mistakes will cause other people to lose faith in our knowledge and skills. Given the breadth and level of the testing domain it’s impossible to become a walking encyclopaedia on the subject, but it’s not really necessary. As testers it’s not only the knowledge that we contain within our heads that’s important, but also the knowledge we have access to; other testers, online resources, books and forums, so that in the event we come across something we do not know about we can go away, research the topic and come back with informed answers. The more informed we are the better and more accurately we are able to influence the two categories I discussed above.
This final category I feel is the most important category because it’s the category we can most heavily influence, ourselves.
The other finding in Dunning-Kruger’s study was that the more training an individual had within a specific domain or subject matter area, the better they were able to judge their level of understanding.
“Participants’ estimates of their ability and test performance depended on whether they had received training.”
For us testers this means that by increasing our knowledge of a subject we can make a more accurate judgment of our knowledge of the subject, rather than falling into the trap of thinking we understand something we don’t. This means continual training and development, which really means, going to courses, reading about the subject and being involved in the community.
 Darwin, C , “The Descent of Man, and Selection in Relation to Sex”, New York University Press, 2009.
 Kruger, J, Dunning D, “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments”, Psychology, 2009, 1, 30-46
About the Author
Melvin Burton is a Software tester for Android and iOS platforms, originally from an Electronics background before drifting into the library industry, now spends his time trying to organise, categories and measure software.
Melvin spends his free time juggling far too many hobbies, reading about those hobbies and adding new hobbies to juggle with.