Cameron Fairburn

Achievements

Club Explorer
Glossary Contributor

Certificates

404 Certificate Not Found
Level up your software testing and quality engineering skills with the credibility of a Ministry of Testing certification.

Activity

This badge is awarded to members who contribute a new term or an alternative definition to the software testing glossary. image
This badge is awarded to members who contribute a new term or an alternative definition to the software testing glossary.
<a href="https://www.ministryoftesting.com/software-testing-glossary/hermetic-testing" rel="noopener nofollow">Hermetic Testing</a> image
Hermetic Testing
Cameron Fairburn
Cameron Fairburn
contributed:
<div>Hermetic testing to me is a method for ensuring that tests run consistently and offer a consistent value. If we apply the idea of Hermetic Testing to only the data element of a tests requirements, we can understand what this means.<br><br>
</div><div>Imagine there are three ways to handle data in our testing:<br><br>
</div><ol>
<li>Our tests can create their own data and then test that data</li>
<li>Our tests could expect data to already exist, and choose something valid at random</li>
<li>Our tests can expect specific data to already exist, and use the exact same data everytime they run</li>
</ol><div>The third bullet point is how I imagine Hermetic Testing. Test-001 expects [item]to already exist, it navigates directly to the UID of [item] and performs the test.<br><br>
</div><div>This is a more scientific method of testing when compared to 1. and 2., because they inherently have risk built into them.<br><br>
</div><ol>
<li>In <strong>approach 1</strong>, if the data creation step fails, the test fails without even reaching the point it’s meant to test.</li>
<li>In <strong>approach 2</strong>, if the assumed data isn’t present or changes unexpectedly, the test again fails for reasons unrelated to its actual assertions.</li>
<li>In <strong>approach 3</strong> (Hermetic Testing), the framework <strong>pre-seeds</strong> the required data in a known, stable state. As a result, tests <em>should</em> always run the same and <em>in theory</em>only fail when there’s a real bug.</li>
</ol><div>
<br>This, to me, is the purpose of Hermetic Testing: to build a regression pack that executes <em>exactly</em> the same way, <em>every single time</em>, so that when a test fails, we can trust that something <em>really</em> broke.<br><br>
</div> image
Definitions of Hermetic Testing
This badge is awarded to members who have signed up as a Club member. image
This badge is awarded to members who have signed up as a Club member.

Contributions

Hermetic Testing image
  • Jas Manigundan's profile
Hermetic testing to me is a method for ensuring that tests run consistently and offer a consistent value. If we apply the idea of Hermetic Testing to only the data element of a tests requirements, we can understand what this means. Imagine there are three ways to handle data in our testing: Our tests can create their own data and then test that data Our tests could expect data to already exist, and choose something valid at random Our tests can expect specific data to already exist, and use the exact same data everytime they run The third bullet point is how I imagine Hermetic Testing. Test-001 expects [item]to already exist, it navigates directly to the UID of [item] and performs the test. This is a more scientific method of testing when compared to 1. and 2., because they inherently have risk built into them. In approach 1, if the data creation step fails, the test fails without even reaching the point it’s meant to test. In approach 2, if the assumed data isn’t present or changes unexpectedly, the test again fails for reasons unrelated to its actual assertions. In approach 3 (Hermetic Testing), the framework pre-seeds the required data in a known, stable state. As a result, tests should always run the same and in theoryonly fail when there’s a real bug. This, to me, is the purpose of Hermetic Testing: to build a regression pack that executes exactly the same way, every single time, so that when a test fails, we can trust that something really broke.
Login or sign up to create your own MoT page.
Subscribe to our newsletter
We'll keep you up to date on all the testing trends.