Two MoTacon attendees are on the left. The MoTaacon logo is in the center, and to the right a prompt to Get Your Ticket.

Jas Manigundan | Comments

Displaying 1 comment
Goldens for your evals image
Goldens for your evals
Hi Swathika, sharing some context from my experience over the last year. Hope it helps. - Evaluation Scenarios - Start small with a "golden dataset" (1–3 examples per scenario). It aligns the team on "what good looks like" without becoming a maintenance burden. - Shifting Left: Integrate test-case preparation directly into user stories and acceptance criteria. Leaving this until after launch is incredibly painful and doesn't scale. The whole team should own this and contribute. - Golden data as part of Requirements - 100% yes. In an indeterministic LLM world, building before defining your expected responses makes no sense. - Metrics (50 scenarios / 80%) - Numbers alone are meaningless. An 80% score is a failure if the missing 20% covers your core features. Assign weightage to critical scenarios, start small, and iterate. It depends on the domain you are in - for example, bias might have more weightage than another factor for certain industries like HR.
June 29th
Jas Manigundan profile image
Jas Manigundan
Lead Automation Engineer, Culture Amp
Award Star

I am a lead test automation engineer working as a part of the QE practice at Culture Amp led by the legendary Anne-Marie Charret.

I'm a developer turned tester. Even though writing code to build applications is so much fun, my now best friend showed me a few years ago that helping write good quality apps is much more fun. I jumped into testing from being a developer and have never looked back. I'm very passionate about native testing frameworks for web, backend, infra and mobile platforms

CI/CD is something I am very passionate about. On the web side of things, AWS CDK has been quite intriguing. I have been an advocate of Infra as Code since working on it. Devops is something that is quite exciting for me, especially given it is always evolving.

I have been speaking at a few international conferences on testing and CI/CD. Saucecon'21, Ministry of Testing's Testbash and UI testing week events to name a few.

It has been so much fun building testing frameworks for a wide range of platforms including mobile, web, infrastructure and microservices.

Please feel free to reach out to chat anything testing.

47
stars
14
badges
0
certs
7
followers
5
following
Open To
Write
Teach
Speak
Mentor
Subscribe to our newsletter