Breakdown Your Costly E2e Tests into Efficient Low Level Tests Through Hermetic Testing
10th October 2023
-
Locked
Jaswanth Manigundan
Staff Quality Coach, Culture Amp
Talk Description
End-to-end tests are really good testing solutions that provide greater confidence in the system that it tests. They are also the most commonly used form of tests when it comes to test automation. It is also pretty straightforward to get started with them. But as software systems grow and become complex, your tests also become complex. Most of all, they become… costly, very soon.
So should we ditch e2e tests altogether?Â
Well not really. In my opinion, e2e tests provide a good level of smoke test coverage. You can test out your core workflows and make sure they are not affected. But they aren’t the most efficient when it comes to feedback times. Any kind of errors it might catch comes at the last point of your development workflow. Ie. after all of your services have been deployed to a dev or staging environment. By that time it is a bit too late. The unplanned work for product teams becomes tenfold.Â
What's the solution then?
So instead of relying heavily on end-to-end tests, we now take a systematic approach to achieving the same level of confidence, but by taking a step down. We now try to break down our e2e tests into low-level tests. And to achieve this, we try to use the concept of hermetic testing.Â
What is hermetic testing?
A hermetic test is a test that is completely self-sufficient. It is fully independent, therefore every time a test is run, you are absolutely sure that a failure is a real failure, and has nothing to do with a dependency going wrong.
In order to use hermetic testing to achieve what we want, we first split our e2e into meaningful chunks. The most common breakdown is to split it into frontend and backend systems.Â
We then run our tests within the context of the front end and back end independently. We then add additional levels of tests to make sure we make up for skipping the e2e tests like contract tests, API tests, visual tests and improving existing unit testing coverage.Â
Now we know what journey we went through in our organisation, in this talk I will be presenting the strategy and implementation of this concept. I will:
- Showcase how we broke down our system into piecesÂ
- Introduced different levels of testing in lower levels.Â
- Reused most of the e2e testing work we did
- Demonstrate through real-world examples on how it can be done, step by step. Â
What you’ll learn
By the end of this talk, you'll be able to:
- Describe the complications of e2e tests as your software grows in complexity
- Explain what a hermetic test
- Follow how to breakdown e2e tests in to more efficient low level tests
- Discuss real world examples of how this has worked in different organisations
Jaswanth Manigundan
Staff Quality Coach, Culture Amp
I’m Jas from Melbourne. I currently work at Culture Amp as a Staff Quality Coach, specialising in test automation and testability.
I have worked in the mobile app development space for almost a decade, where I saw mobile devices and operating systems get better and better by the day. My fascination towards mobile devices made me start my professional career as a mobile app developer. But my quest to build better apps and write code that is much more testable lead me to transition into a test automation engineer specialising in mobile.
Since then, I’ve gradually explored a wide variety of test automation frameworks and have consulted for some of the leading software teams in Australia to build efficient test automation suites for both mobile and web apps.
I have spoken about my test automation journey in a variety of testing conferences like the Ministry of Testing’s Testbashes, Automation Guild (by TestGuild) and Saucelabs’ Saucecon.
I now work at Culture Amp alongside the legendary Anne-Marie Charrett helping build a passionate quality enablement team.
I currently work with test automation frameworks (functionality, contract and load) for APIs, web apps, micro-services and event sourced systems.
Latest obsession… Observability within test frameworks.
Suggested Content
With a combination of SAST, SCA, and QA, we help developers identify vulnerabilities in applications and remediate them rapidly. Get your free trial today!
Explore MoT
Thu, 12 Sep 2024, 9:00 AM
We’re shaking things up and bringing TestBash back to Brighton on September 12th and 13th, 2024.
Ascend to leadership roles by mastering strategic skills in automation strategy creation, planning and execution
Tags
-
e2e-end-to-end-testing
-
automation
