Test data driven development

I have been leading a team of software engineers for six years focusing on the delivery of SaaS software in the cloud on the SalesForce platform. My team doesn't have any testers or any individual responsible for the quality of the code. The features we deliver are reporting and analytics based, so all require data to already exist before the feature is discoverable. 

Due to the above and not wanting to train engineers to become testers, I developed a methodology similar to TDD but as I am delivering analytics features, the data in the environment alters how the system performs, it is the data that needs to be specified to define the scenarios the analytics should support.

When new requirements are received, a review is performed to identify if the existing test data is sufficient to cover the new scenarios. Depending on the complexity of the analytics requested, a spreadsheet is generated detailing the inputs and the outputs of the data which would demonstrate these features.

The data is added to the developer environment before development of these new features starts, and any manual testing performed to prove the new features uses that new data. All the developer needs to do is perform a comparison between the values calculated by the feature and the data in the spreadsheet (as with any reporting or analytics feature the most important part is the accuracy of the data). Once validated, automated tests are written which again expect that same test data detailed in the spreadsheet. These tests are then run on a daily basis against the develop branch, to ensure no regression occurs.

This approach has resulted in fewer than 20 customer defects raised over six years and no testers.

This definition of data alongside detailed requirements ensures that new reporting and analytics features are 'exercised' by a thorough spread of 'use cases' meaning that there isn't a dependency on developers to understand the way the system will be used by its users or consider positive and negative scenarios as they were included in the data model spreadsheet. 

• Explain how Test Data Driven Development adapts TDD principles for analytics and reporting features by using data specification instead of traditional test case design
• Recognise the project types and team structures where a data-first quality approach is more suitable than conventional testing roles
• Create a spreadsheet-based input/output data model from new requirements that defines expected system behaviour before development begins
• Describe how pre-populated test data can guide development, support manual validation, and form the basis of automated regression tests
• Assess how defining use cases and scenarios within a data model reduces dependence on developer judgment and supports long-term regression prevention