Definition: A controlled test flight is a testing in production methodology where the new version of a service runs alongside the existing one. Users are randomly assigned to each, without knowing it, so the quality of the new code can be compared fairly against the old. It may lead to deployment and can form part of a ramped deployment.
So what? Comparing new and old code under the same real conditions shows whether a change makes things better or worse. It's a variation of controlled online experimentation, focused on quality rather than user experience.
Examples: Comparing error logs, load and memory use between users on the new and old versions of a service. Combining a test flight with an A/B experiment so both quality and user impact are measured.
