MindMap: Testing and Checking

The concept of Testing vs Checking was originally discussed and written about by Michael Bolton.  It’s hard to believe it was back 2009.  The topic seems to keep coming up in conversations, which is why I thought I’d attempt creating a MindMap on the topic.

I’ve called it ‘Testing AND Checking’, instead of ‘Testing Vs Checking’ as upon reading through blog posts and comments it felt right to me that checking is a part of testing and is not necessarily a separate discipline.  As testers we all (probably) do testing and checking, but recognising when we are testing or checking is a useful exercise.

Perhaps others will think differently…let the conversation or debate continue!

MindMap is below, followed by a downloable ZIP file and a text list.

Enjoy!

Testing and Checking

Download – ZIP file includes PDF, png, text, Opml, Freemind and SimpleMind+ formats.

Testing and Checking

Checking

  • Doesn’t mean the product works
  • Help us know that the product works within expectations
  • Can ‘pass’ but still have serious problems
  • Automated tests are checks
  • Need specifications
  • Helps answer – does the system do what it is supposed to do?
  • Great programmers do alot of checking

Is

  • Confirmation
  • Verification
  • Validation
  • Making sure the program doesn’t fail
  • Verifying beliefs we expect to be true

Checks

Are machine decidable:

  • Yes
  • No
  • True
  • False
  • Pass
  • Fail

Testing

Is

  • Exploration
  • Learning
  • Finding new information
  • Discovery
  • Investigation
  • Evaluating
  • Finding problems
  • Searching for extents and limitations
  • Driven by questions
  • Finding problems not covered in checks

Tests

  • Have open ended results
  • Asks if there is a problem here?
  • Can be about finding out if checks have been good enough
  • Problems found from testing can be used to create checks to stop them appearing in the future
  • Do not require specifications
  • Involves some checking

The Debate

  • Why do we need to define the difference?
  • Does more terminology just cause more confusion?
  • What problem is this distinction solving?
  • Should it not be ‘versus’ it should be Testing AND Checking?

Useful Sources:

Websites we came across upon our research:

You might be interested in... 1 Day Python for Testers Course

MoT Courses-06

Tags: , ,

4 Responses to “MindMap: Testing and Checking”

  1. ChrisJuly 30, 2012 at 10:53 am #

    Why do we need to define the difference?
    Because there is a difference to be defined, combined with a common confusion between the two.

    Does more terminology just cause more confusion?
    We make the terminology we need, otherwise we’d have no domain language at all. The terminology is to demarcate between the two so we can have a reasonable conversation about it and know what we’re talking about. It’s to raise our conciousness about the subject of exploratory, scripting and automation so when we mention them we carry with it a tacit understanding of what they mean.

    What problem is this distinction solving?
    The problem of replacing testing with limited checks and the problem of understanding the limitations of automation and scripts (and their power)

    Should it not be ‘versus’ it should be Testing AND Checking?
    Yes, if appropriate. Once people understand the difference between testing and checking they can make educated decisions about which to use, and when. I use both – automatic checking is fantastic for smoking potential testable problems quickly, and for checking the state of the database before and after important processes. But without knowing that “automated testing” very often equates to “automatic checking” how can testers, management and the business make good decisions about where to invest their time and money?

  2. Jeff LucasAugust 1, 2012 at 2:26 pm #

    Very good mindmap! My only concern is the presentation of “absolutes” when concerned with testing. What you list here are some of the characteristics of -effective- testing.

    For example, testing does not require specifications, but it does require a structured set of oracles to be effective. If the person performing the test does not have a thorough understanding of the oracles that apply in the context of the test, then it is not possible to report valid defects.

    Another primary difference is that a check can be repeated with the expectation of getting the same result (under the same conditions). A test (e.g. based on a charter) may get different results based on a change in the tester’s inspiration, imagination, or understanding. That is why a tester may revisit a charter more than once during the course of a product development cycle based on new risks uncovered.

  3. Josh EastburnAugust 28, 2012 at 6:04 pm #

    In a recent keynote at CAST 2012, Elisabeth Hendrickson revealed that it was actually she who first made this observation publicly (sort of). Just a bit of pedantry on my part, but a great talk: http://www.youtube.com/watch?v=hANmIC9JqUA&feature=g-user-u

  4. Jesper L. OttosenFebruary 4, 2013 at 8:54 pm #

    Add: http://context-driven-testing.com/?p=69
    The Insapience of Anti-Automationism by Cem Kaner