The dictionary definition of reliability is a result or outcome consistently good in quality or performance; something which is able to be trusted.Â
When thinking about automated test outcomes it is important to remember the difference between reliability and validity. A test can be reliable without being valid (passes repeatedly without a strict assertion) however, a test cannot be valid unless it is reliable (a test which inconsistently times out or a feature which produces different outcomes). Another way to think of it, is that a test can give a consistent, poor/failing result but it cannot give a good result unless it is consistent.
When thinking about automated test outcomes it is important to remember the difference between reliability and validity. A test can be reliable without being valid (passes repeatedly without a strict assertion) however, a test cannot be valid unless it is reliable (a test which inconsistently times out or a feature which produces different outcomes). Another way to think of it, is that a test can give a consistent, poor/failing result but it cannot give a good result unless it is consistent.
