James Pearce
James Pearce
Senior Quality Engineer
Quality strategist specialising in artificial intelligence testing, driven by a passion for solving complex problems with creativity and logic.
馃巶 MoTaBirthday | May 24, 2024
Open To
Write
Mentor
Teach
Speak
On the 15th of January 2026 myself and Ujjwal Kumar Singh joined a call and pressed record. We were curious to see what would happen. A few weeks later I鈥檇 done the same with Neil Taylor and Clare ...
6 Jul
A collection of what I snapped on the day of Leading With AI.I turned them into a Gif with a Claude Gif Maker Skill.
24 Jun
I went to LWAI in London on Friday last week, really enjoyed Nicks talk on Given AI - When steps are scripted - Then humans have controlToday I've been trying to implement some of these ideas creat...
23 Jun
AI has changed everything but there are key aspects of our work like shifting left and right and risk based assessments that remain as relevant (if not more) than before. There is a lot of uncertai...
19 Jun
When a system improvises like a jazz soloist, "different" stops meaning "wrong"
3 Jun
Sometimes the most effective guardrail isn't the most sophisticated one
21 May
James Pearce shares a practical framework for testing AI products by making them more verifiable, measuring risk clearly, and learning safely from production.
10 Jul
Excessive agency
Excessive Agency is the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from an LLM, regardless of what is causing the LLM to malfunction. Common triggers include:
hallucination/confabulation caused by poorly-engineered benign prompts, or just a poorly-performing model;
direct/indirect prompt injection from a malicious user, an earlier invocation of a malicious/compromised extension, or (in multi-agent/collaborative systems) a malicious/compromised peer agent.
The root cause of Excessive Agency is typically one or more of:
excessive functionality;
excessive permissions;
excessive autonomy.
Excessive Agency can lead to a broad range of impacts across the confidentiality, integrity and availability spectrum, and is dependent on which systems an LLM-based app is able to interact with.
Panic-Driven Development (PDD)
Panic Driven Development (PDD)The abandonment of good development practices by a company or individual in order to react to a perceived external threat or future risk.
Example: "We started a PDD cycle because other companies are using AI to get ahead of us."
Guardrails
AI guardrails are about:
Protecting the user from bad outputs or bad actions
Protecting the AI or LLM itself from even doing those actions in the first place
Protecting the companies that are deploying these AIs or LLMs or models
Moment MoTivator
This badge is awarded to those that have been tagged in 10 Moments.
Badge Award
2 days ago
Moment-um Builder
This badge is awarded to those that have been tagged in their first Moment.
Badge Award
2 days ago
Guardrails & Evals: Keeping Your AI Skills Honest
A free event by MoT AI Chapter
Chapter Event
4 days ago
