Testing in Production

The practice of running tests, gathering feedback, and validating product behaviour in the live environment with real users, rather than relying solely on pre-release environments. In fast-moving teams, particularly startups, this is sometimes the only practical way to observe how a feature behaves under real-world conditions: real traffic, real data, real devices, and real user behaviour that a test environment can never fully replicate. It sits on a spectrum from deliberately staged experiments (feature flags, canary releases) to the more informal practice of shipping something and watching what happens. 

For example: rolling out a new onboarding flow to five percent of users before expanding it; or monitoring error rates and support tickets in the hours after a release as a form of post-deployment validation.

Definition: Testing in production (TiP) is the practice of testing software in its live environment, with real users, real data and real infrastructure. It suits large-scale software services, where the provider controls deployment and can monitor the data centre. TiP covers a family of methodologies, from ramped deployment and A/B experiments to synthetic tests, load tests and destructive testing in production.

So what? A test lab can only approximate how real users behave and how complex data centres fail. TiP finds problems a lab cannot, while monitoring, fast fixes and exposure control keep the risk to users low.

Examples: Running the same automated tests in a test lab and against the live cloud service. Releasing a change to 1% of users and watching error logs before rolling it out further.