Guardrails and evals people can trust

In the session "Guardrails & Evals: Keeping Your AI Skills Honest"  from  Maneesh Maddala in the AI Chapter

Writing Guardrails:
1. Make it testable: "A rule without a test is a suggestion. Pair every guardrail with an eval." 
2. Use a real failure: "Build guardrails from failures you have seen, not hypothetical ones." 
3. Make enforcement predictable: "When the same violation sometimes blocks and sometimes passes, people cannot trust the rule." 

Writing Evals:
1. Match cost to evidence: "Run deterministic checks on every commit. Use an LLM judge only when the answer needs judgment." 
2. Include one attack attempt: "Test at least one prompt injection or adversarial request." 
3. Measure repeated runs: (This box is highlighted with an orange border) "pass@k = one success in k tries. pass^k = every run passes (needed for guardrails)"