TDD for Testers

What does TDD mean for me, the Tester?

I have some experience of working as a Tester in development teams practicing TDD and I hadn’t really thought about what impact TDD had on me until someone actually asked me:

Test Driven Development (TDD) is a predominately Programmer driven practice, so what do you as a Tester think about it?

What a great question!

My immediate answer was that TDD has enabled me to gain an understanding of the software and associated tests so that I can add more value in the design decisions, highlight business risks and potential impacts to the GUI / Acceptance tests.

This article tries to explain in more detail how I work to provide value in the TDD environment.

Who are the different players in this game?

TDD is a practice adopted the eXtreme Programming community and as such I have taken my terminology from XP Roles in a development team:

  • Programmer
  • Tester
  • Customer

What do I think TDD is?

I thought I had best start with what I believe TDD is. I’m certainly not the world’s authority on TDD so I will keep my description brief. The idea behind TDD is that you write a failing test, write only the necessary product code to make the test pass and then you refactor the code to be cleaner, more efficient or adhere to some coding standard. This is commonly known as the TDD Cycle (red, green, refactor)

TDD is primarily about the design of the code; the resulting tests are a nice side effect that we Testers can use to our advantage, so long as we don’t succumb to their illusion of 100% safety and confidence.

The TDD cycle provides Programmers the fast feedback they require in order to make informed decisions about the design of the code. Vastly reduced feedback loops are also core principles of Continuous Integration (CI) and Continuous Delivery (CD) automation frameworks. Testers can and should be making use of this fast feedback loop – it helps prevents code from “being thrown over the wall” and helps to enable CI and CD.

The TDD cycle is perceived as beginning with the creation of a failing test. I feel this is misleading as my observations have led me to believe that adding the test isn’t the first step in the cycle; first we need to think about what the failing test should be.

How can Testers be involved in TDD if they are not writing code?

Predominantly, Programmers are concerned about the code design and Testers are concerned about the behaviour of the software. But the lines between the two concerns are very blurred – Testers can and should think about design, Programmers can and should think about behavior.

Testers are striving for testability in the design of the code. According to James Bach, the 2 primary areas Testers should be considering with regards to Testability are controllability and observability: Can we put the system into the state we require and then observe how the system is behaving? How easy is it to get the application into the different states required for the different tests? How will we know what the application is doing at a given point in time – will logging be implemented? What will that logging look like?

When talking to Programmers about the software, I like to use sequence diagrams (adopted from UML) to map out the requests and responses to different parts of the system. This enables us both to understand the flows through and different states of the system that in turn should help us in work out what tests are required where.

Here is an example of a typical web application:

sequence diagram

Image courtesy of blog.quent.in created using websequencediagrams.com

My knowledge of the system and the Programmers in-depth knowledge at the component level compliment each other to help prevent unnecessary duplication of tests, or better still the prevention of invalid tests. We both understand the test strategy.

Here is a contrived example of a collaborative test strategy conversation between Paul (Programmer), Terry (Tester) and Carl (Customer)

Paul: “Hey Carl, Terry, have you got a sec?

Terry: “Yep”

Carl: “Sure”

Paul: “Great, pull up a chair. Our system makes requests to other systems outside of our application. Currently when those external, 3rd party systems are unresponsive or unavailable, we display error messages on the UI for each error. From a technical point of view this isn’t great – the UI shouldn’t be aware of these different errors.
Carl, are you happy for a generic error that indicates a 3rd party service is unavailable to be displayed on the UI?”

Carl: (after some deliberation)” Yep, I’d be happy with the generic error message.

Terry: “OK. We have a range of tests that check those error messages in the UI – are those tests still going to be valid?”

Paul: “No. We will catch the error in the backend service and throw the generic error message.”

Terry: “OK. What can I assert on in the UI then?”

Paul: “Do you really need to assert in the UI? We will be making an assertion on the error in our backend service.”

Terry: “So the assertion is there that you sent the error, but we do not know that the error has been displayed on the UI. I’d like to keep a test at the UI level to check that the error message is being displayed and that the user has a follow on journey from the error message.”

Paul: “Good call on the follow on journey I hadn’t thought about that. Carl, would you like the follow on journey to be the same as it is currently?”

Carl: “If possible, yes please”

Paul: “OK. It will take a bit more work but we should be able to do that. We will need to update the logging so we can see the new generic error being thrown – we can let you know what the changes are shortly.”

Terry: “Great, so I should be able to refactor one of the error scenarios to account for the generic error and remove the now surplus error scenarios. How can I consistently trigger the error scenario for this test?”

Paul: “The process of triggering the error should be the same as now, we’re just handling the errors differently. The existing mock of the 3rd party service should still be valid, but we’ll be checking it when we’re writing our unit tests so we’ll let you know. Everyone happy?”

Terry: “Yep!”

Carl: “Yep!”

Paul: ”Excellent! Lets catch up in a couple of hours to see where we’re up to”

The TDD Cycle in action

Let’s walk through the TDD cycle to demonstrate to you how I focus my testing.

TDD Cycle

Image adapted from Uxebu.com

Write a failing test

Programmers begin their TDD cycle writing code and listening to the feedback from the unit tests.

I start to think more deeply about my test ideas and the different edge cases I’d like to explore. I may start to write some scenarios that could later be executed through automation.

As the design evolves, more questions and problems will crop up which need answering.

For example, Programmers may come to me with a specific scenario I’ll need to cover higher up the stack as they tried to cover the scenario with a unit test, but the test proved inadequate.

Likewise, my edge cases may require design changes or tweaks.

Write enough code to make the test pass

The Programmers let me know when they have some code & tests to commit. We walk through the code and associated tests (predominantly “unit-integration” tests), discuss the acceptance criteria coverage and then I’m let loose to explore the software on the Programmers machines.

I try out a few of those edges cases I was considering and some mutation testing to check the validity of the tests. There is also the opportunity to show the software to the Customer to check we are meeting expectations.

This feedback cycle of failing test, code demo then commit is typically hours rather than days.

Depending on the progress of the software, I might demonstrate my testing of system to the Programmers as well as asking for help with automating some of my test scenarios.

Admittedly, I can never know for sure if the tests were written before the production code (in true TDD style), but ultimately I have trust in the Programmers and there is, after all of the steps above, tested code.

Refactor

The Programmers improve the quality of the software. I continue my exploring.

From pairing with the Programmers, I know what tests already exist, how the parts interact and any potential pain points the Programmer had whilst writing the code and tests.

I take the information gleaned from the pairing session to compliment and build upon my test charters for the exploratory session.

In the refactor phase we may also review the testability and test coverage to ensure it is still efficient and effective.

For example, the Programmers may think that we can prove a requirement with a unit test rather than a UI scenario. We talk about the intent of the test as well as the pros and cons of pushing the test down the stack. What happens to the test is a group decision, not a uni-lateral decision.

The information about the design and behaviour of the software is shared both ways. Nothing is hidden and raising bugs becomes a far more lightweight exercise.

Closing the loop

I think of the development team’s close collaboration during the TDD cycle like stitches in clothing, where the stitches are like interactions and articles of clothing are the relationships. Closer stitches result in increased strength of the bond. With fewer stitches comes the likelihood of the clothing falling apart sooner.

When it comes to deciding whether a feature is “done”, the development team and Business stakeholders have feedback from numerous sources, not just the Testers.

The whole idea behind Testers “signing off” the code or being the gatekeepers to quality disappears. The whole team knows the state and behaviour of the software so all have an opinion on whether it is worthy of being released to the Production environment.

Here is the same image of the TDD cycle as displayed above, but I have added some examples of the types of activities Testers can get involved with throughout the cycle:

TDD Cycle Enhanced

Tester-enhanced TDD cycle

What if your team doesn’t practice TDD?

The Programmers may not call their development practice TDD, nor write their test firsts, but hopefully they are having discussions about the design of their code and you as a Tester can certainly get involved here.

Just because a development team are not using tests to drive out the design of the code does not mean they are not thinking about design. The tests are a nice byproduct.

If you and your Programmers are not creating a testing framework around the software to make it “self testing”, then close collaboration with the Programmers upfront will help with the defect prevention.

Now what?

Here are a few hints and tips that have helped me work closely with Programmers and get more involved with the design of the software.

Context Driven Testing

The situation drives the approach to testing.

I left a waterfall / staged delivery organization to join an XP development team practicing TDD. My previous approaches to testing were not applicable.

I needed to inspect and adapt in order to provide valuable testing in the new organization. The Context Driven Testing community really helped me to grow and understand what kind of Tester and team player I was.

Awareness of the Programmers craft

Programmers are generally proud of their work. Take some time to learn a bit about what the Programmers do, why they do it and how they got to where they are today.

You’ll be surprised how this interest soon gets reciprocated.

Be a domain expert

Own the domain you are testing in!

Be aware of who is going to be using your software and what problems they may have that this software is trying to solve. The closer we can represent our users, the better our testing can be.

A great book that helped me in domain and design related conversations was Eric Evan’s “Domain Driven Design”. A key point that is driven throughout the book is the use of ubiquitous language. A lot of ambiguity lives where business language is translated into “dev speak”. Understanding your domain, applying critical thinking and promoting ubiquitous language can help to drive out this ambiguity.

Demonstrate your value

My love for testing and close collaboration with other development team members didn’t just happen overnight.

It has taken some hard work and a fair amount of cakes, but through critical thinking, passion and proving myself on numerous projects I can now demonstrate what value I can offer in Programmer centric environment.

If I can do it, so can you.

You are a Tester in a software development team. You have an opinion on the design of your software; you just might not know it yet…

About the author

Duncan Nisbet is a tester who believes development teams & the larger business can be smarter at  working together. He coaches Testers, Programmers & Business folk on how they can help each other communicate & collaborate in order to deliver software which will actually help to solve the problem. His efforts are focussed both during the everyday development or in team workshops which he facilitates. Follow him on his blog or via Twitter @DuncNisbet


Edit: After reading this article, Richard Bradshaw (@friendlytester) tweeted a nice & concise response:

And also created a great doodle which I feel adds real value to the article, so here it is:

TDD Doodle

You might be interested in... The Dojo

Dojo Adverts-05

Tags: , ,

25 Responses to “TDD for Testers”

  1. DavidAugust 18, 2014 at 1:42 am #

    Great informative post. I ranted about something similar before, but more geared towards improving UI/Selenium automation workflow:

    http://autumnator.wordpress.com/2013/08/02/pseudo-test-driven-development-with-selenium-and-page-object-model-for-developers-and-qa/

    • DuncsAugust 18, 2014 at 8:28 pm #

      Hi David,

      Thanks for your feedback & link to related article!

      Duncs

  2. Chris (kinofrost)August 18, 2014 at 11:40 am #

    I really like this post! It’s nice to have a refreshing look at the social side of TDD integration, benefits to the tester, and some advice on helping the team get the best out of it. Having a set of rejection checks in place driving better behaviour is a hard thing to argue with. TDD isn’t testing, but it’s a nice place from which to start it!

    • DuncsAugust 18, 2014 at 8:29 pm #

      Thanks Chris, appreciate the feedback!

      Duncs

    • DuncsAugust 18, 2014 at 8:48 pm #

      “TDD isn’t testing” is such a great & important statement its worth pulling out by itself – good shout 🙂

  3. Rafael BattestiAugust 18, 2014 at 12:24 pm #

    Great post! What I liked the most is your idea of testers demonstrating their value in a programmer centric environment. In the project team I work, we’d been having heated discussions within the team and with the client to try and demonstrate the value of the work of testers, until we decided to do just as you said so, get closer to the programmers and show the work using, by now, requirements engineering, business analysis and other tools a tester can use.

    • DuncsAugust 18, 2014 at 8:32 pm #

      That’s great news Rafael!

      Do you have any examples of how you demonstrated the value of Testers?

      How does your new way of working compare to your previous way of working?

      Thanks for your comment,

      Duncs

  4. Peter KarasAugust 18, 2014 at 1:24 pm #

    Great article.
    It’s interesting how the development effort often focusses on the development team, but its actually the QA effort, including the feedback to the customer that makes or breaks the project.
    The best (most productive teams) I’ve worked on are those where the developers write code and do accceptance tests. Keeps the focus on meeting the customer requirements and coding to the minimum required.

    • DuncsAugust 18, 2014 at 8:36 pm #

      Hi Peter, thanks for the feedback, appreciated!

      Could you expand on how its the QA effort that makes/breaks the project please?

      One example I’m thinking of where Devs/Programmers could break the project – the team stops writing code, QA had no hand in it.

      Duncs

      • DavidAugust 18, 2014 at 9:35 pm #

        Duncs,

        On Peter’s comment, what I’ve seen related to that is the following:

        * planning effort, at least initially anyways is always from dev work and never accounting estimates (or accurate estimates from actual QA) for QA work until later in the stage of planning process, and thus project work is mis-estimated to begin with

        * those who do the planning (project managers, product managers, business analysist, developers) fail to anticipate the heavy QA workload for the project even though they’ve seen it in past projects, so they should estimate on the bigger ballpark end rather than smaller to have buffer for things. And when they do anticipate for it, it ends up as being too costly for such resourcing (# QA resources, time required, automation effort, etc.) and they end up reworking things to try and fit the schedule.

        * the planning team (project managers, product managers) fail to include QA in planning process until way late, and that ends up hurting the project from a quality perspective & release timeline because we didn’t get adequate time to pre-plan in advance.

        • DuncsAugust 24, 2014 at 8:35 am #

          Planning is a common theme in that response David & it seems I’m turning into my dad – one of his favourite sayings is “fail to prepare, prepare to fail”.

          I’ve seen those behaviours before as well & the impact of poor planning.

          The prime example project which comes to mind is when all the planning had been “completed” without any input from the test team. The assumption was we would be using Off The Shelf products so only integration testing would be needed.

          Guess what, a whole lot more was needed.

          When we talked through our test cases with the client, their response was “they’re great, when do we get to see the rest of them?”

          By the time I left the project, the 100 or so cases written had swollen to >1000 & we still hadn’t seen the software. I have no idea what the final count was.

          Definitely working with TDD has really helped to mitigate that kind of analysis paralysis

      • Peter KarasAugust 19, 2014 at 7:31 am #

        Hi Duncs / David
        QA making or breaking a project…
        Yes – I was thinking along the lines of Davids comments.
        I also had Acceptance testing (or System / UAT Testing) in mind. Developers can work really hard and come up with technically excellent work but if it fails to meet acceptance criteria its no good to anyone. On my current project the ‘test first’ approach often means we develop a technical solution that is simple and good quality, but not the most sophisticated. For example read / write to a file and not a database. As a developer I naturally want to use the latest tools and create something thats perfect, but the QA reminds me that I have to work within the limits of the requirement. It really makes the difference between releasing working software or forever refining a product that will never be used.

        • DuncsAugust 24, 2014 at 8:45 am #

          Nice response Peter – good to tie the 2 different aspects of programming & testing together.

          So does the QA element in your organisation help you to remain in the problem space before leaping to a solution so that you all understand the requirements.

          It’s good for me to understand what you meant by making or breaking the project – I’ll be honest, I originally thought that the QA in your organisation are the quality police, or gatekeepers to quality with the None Shall Pass mentality to defects 🙂

          • Peter KarasAugust 25, 2014 at 5:15 pm #

            Hi Duncs,
            Thats right. I was thinking about development being kept on track by TDD and QA process. I think getting the developers to write tests (unit or acceptance) or to work along side testers to jointly deliver the requirement really helps everyone keep focussed.
            Cheers
            Peter

          • DuncsAugust 25, 2014 at 5:18 pm #

            Nice summary Peter, thanks!

  5. Richard BradshawAugust 19, 2014 at 10:41 am #

    Hey Duncan,

    Great article, as Martin said on Twitter, its very much a transcript of the conversation we had in a bar at CAST2014. I was thinking about the comment Chris (kinofrost) made, about TDD isn’t Testing. Gave me food for thought of my commute this morning and ended up producing this little doodle, I agree, but working in a team as you described gives you some great artefacts for testing.

    “TDD isn’t testing, but provides some great artefacts to support Testing”

    Diagram can be viewed here. https://drive.google.com/file/d/0B11AaEUdgzxgT2RBNWEwR29DWkU/edit?usp=sharing

    • DuncsAugust 24, 2014 at 8:48 am #

      Hey Richard, thanks for the comment & great doodle – it really adds to the conversation & I’d love to read a write up about it…

      The conversation at CAST sounds like one I would have enjoyed being part of 🙂

      Were there any other thoughts that might be useful takeaways for Testers working with TDD?

  6. Stephen BlowerAugust 19, 2014 at 11:14 am #

    Great article Duncs, as Richard mentions this was a discussion we were involved in at CAST2014 and before that discussion my opinions were fairly negative due to only having a basic understanding and believing that it was just extra work for no real benefit.

    However after those discussions I’ve now become to understand it’s usefulness in driving design and understanding of the requirement rather than having shallow agreement with various parties due to not delving deeper.

    • DuncsAugust 24, 2014 at 8:51 am #

      Thanks Stephen, appreciate the feedback.

      Really interesting to hear what journey you’ve been on regarding your views on TDD.

      Where there many other arguments against TDD in that conversation at CAST? I’d be interested in hearing them.

  7. Avaneesh DubeyAugust 19, 2014 at 4:02 pm #

    Hi,
    TDD and the related question that was posed to you, represents a trend that specialist testers (who don’t or can’t program) and developers (who don’t like testing) need to worry about.
    The future belongs to engineers who can do both.
    In fact, this will go even further. Programming is increasingly becoming a small part of what a product developer does. A lot of time and effort goes into understanding customers and their needs.
    Finally, I must complement you on describing TDD well.
    Avaneesh.

    • DuncsAugust 24, 2014 at 9:33 am #

      Hi Avaneesh, thanks for adding your voice to conversation & for the feedback, its greatly appreciated.

      Not sure if its relevant, but the question was posed to me by someone who is neither a Tester or a Programmer.

      WRT “The future belongs to engineers who can do both.” I’m not so sure about that statement. There is a much wider discussion around Testers learning to program (& I guess Programmers learning to test) but that isn’t the focus of this article.

      I will say on this topic that I feel there is a need for specialists, such as Testers & Programmers, but their roles & responsibilities are changing.

      For example – where Programmers previously would have to wait for the requirements before generating the specifications & in turn, the Testers would have to wait for both requirements & specifications before writing test cases. The situation in Agile & TDD shops now is that Testers & Programmers are now more accountable for requirements gathering, generation & feedback.

      There are certain tasks & challenges that need a specialised focus, not just a generalised focus.

      Those who like to program & those who like to test are wired differently with different motivation. I definitely believe that there is room on the development team for both.

  8. Ana SalinasAugust 19, 2014 at 8:14 pm #

    Very interested topic, this is interested way to explain how testers and development can work together to get a better product by using TDD
    Thanks to share your experience
    AS

    • DuncsAugust 24, 2014 at 9:34 am #

      No worries, thanks for your feedback Ana, its greatly appreciated!

      Duncs

  9. DevaAugust 20, 2014 at 4:59 am #

    Very nice article.. awareness on TDD is a must for all the Testers…

    • DuncsAugust 24, 2014 at 9:36 am #

      Thanks Deva, appreciate you taking the time to comment.

      “awareness of TDD is a must for all the Testers…”?

      Can you think of an example where Testers do not need to be aware of TDD?