From Script Writer to Test Strategist

Amna Zafar | Senior QA Engineer | ISTQB Certified 

Writing test scripts is one of the most time-consuming parts of a QA engineer’s job, and one of the least talked about. We talk a lot about framework architecture, CI/CD integration, and coverage metrics, but far less about the hours spent writing test cases line by line, turning acceptance criteria into structured automation scripts before any actual testing has started.

At a high-traffic B2B marketplace platform where I led QA on a flagship retail application, this was a real constraint. The release flow was brutal, multiple feature releases per month with high business priority and tight timelines. The automation suite was mature, but keeping up with new feature scripts ate up the time I needed for exploratory and integration testing (writing instead of testing).

In mid-2023, I started playing around with Artificial Intelligence AI to address this. What started as an experiment became a workflow — and that changed how I thought about my role.​

The biggest mistake most QA engineers make when using AI for scripting is treating it as a shortcut, paste in the user story and ask for a test suite. That results in generic, low-quality output that takes almost as long to fix as writing from scratch. What I developed was different: it uses AI as a first-draft partner within a structured prompting framework, not as an autonomous generator.

Stage 1: Context loading:
Before generating any script, I load the AI with all technical information: the API contract or endpoint specification, the feature data model, edge cases known from previous releases, and the syntax of the framework we use (RestAssured with Cucumber BDD here). Quality of output depends on quality of context, so a vague prompt gives a vague script; a precise, technically loaded prompt gives something close to usable.

A prompt that worked well looked like this: “Using RestAssured and Cucumber BDD in Java, write a test suite for a checkout API endpoint. Endpoint accepts [specific parameters], returns [specific response structure], and must handle [specific edge cases].” Use our existing step definition pattern [example]. Happy path and failure scenarios too.” ​

Stage 2: Draft generation and structural review
The AI draft is never used directly. The first thing I check is structure: does the generated code follow framework conventions? Are the step definitions reusable instead of duplicated? Is naming consistent with our suite? At this stage I’m not running tests,  I’m reading them like a code reviewer would, looking for architectural problems before they become runtime problems. This structural review takes 15–20 mins vs. 2–3 hours to write the initial draft from scratch (that is the time saving, not that AI did it, but compressed the blank-page phase into something I can review and reshape).

Stage 3: Validation versus real behavior
Once the structure is sound, I run scripts in a test environment and compare actual API responses with expected values in generated tests. AI-generated scripts often have plausible-but-wrong assertions, they infer expected values from context rather than knowing them. This validation step is mandatory: every assertion gets checked against real data before commit. This is also where the domain knowledge is irreplaceable. The AI doesn’t know that a field behaves differently for wholesale buyers than retail buyers, or that an edge case was introduced in a previous release. QA engineer does. The validation step is where that knowledge gets applied. ​

Stage 4:  Integration and monitoring
Validated scripts are integrated into the CI/CD pipeline (Jenkins here) and monitored through the first three release cycles for false positives and flakiness. AI-generated scripts tend to be slightly more brittle on dynamic data than hand-crafted ones (they assert on specific values when pattern matching would be more durable). I built a small internal checklist to identify and harden these cases after integration.

The measurable effects that I used over time are:

• Test script authoring time dropped by about 60–70% per feature.

• Time freed up was redirected into deeper exploratory testing, integration testing, and performance analysis
• Sprint delivery on flagship retail application had zero post-deployment defects during that time. • Production defects across platform fell by up to 90%, and release cycles accelerated by 50% (combination of automation coverage increase + faster CI/CD feedback loops).

The less measurable but just as important outcome. I stopped thinking of myself primarily as a script writer and started thinking of myself as a test strategist. AI writes the mechanical first draft, the QA engineer writes everything else.

What it is not, I want to be clear about what this approach isn’t because I’ve seen AI-in-QA talks where people go way too far in claiming what is possible. This doesn’t replace test design thinking. It doesn’t catch requirements gaps, if the acceptance criteria are incorrect, the AI will write perfect tests for the wrong behavior. It doesn’t know your system’s history, failure modes, or business logic that lives in your product team's heads. All of that still has to come from the QA engineer. What it does is compress the mechanical work of translating well-understood requirements into structured code, giving the QA engineer more time on the work that actually requires a human.

Applicability and next steps
The workflow I described is framework agnostic in principle (I used RestAssured and Cucumber, but the same staging approach works with Selenium, Cypress, Playwright, etc., where the QA engineer can provide syntax examples in the prompt). I am currently exploring adaptations of this methodology for health technology contexts, where the prompting framework needs to account for other constraints: GDPR compliance, NHS data standards, patient safety implications of false negatives, and regulatory audit trail requirements. The core methodology remains the same, though the context-loading stage is richer, and the validation stage is more rigorous.
​

If you are a QA engineer considering this approach, then start by running one low-risk feature through the four stages end to end before integrating it into your main delivery process. It only works if you keep the validation discipline, without Stage 3, you introduce AI-generated assumptions into your production test suite (which is worse than writing tests yourself).