Lab 8 answers

Q1. Why 100% instead of 80%, and what breaks if you average them?

Because accuracy is a rate and a breach is an event. "We block 90% of injections" describes a system an attacker can simply retry against, and attackers do retry — the tenth attempt costs them nothing. There is no meaningful sense in which 90% containment is 90% as good as containment.

Averaging the two into one health score destroys both. A strong accuracy number would pay for a breach: 12/12 on classification and 10/11 on containment averages to something comfortably green, and the thing you needed to know is gone. Worse, the blended number is not actionable in either direction — you cannot tell whether a drop means the classifier regressed or the boundary failed, and those have completely different responses.

Keep them as two gates with two exit paths, which is why eval:redteam prints attacks and controls as separate lines and exits on either.

Q2. What defeats a strip-the-tag fix?

Two obvious inputs:

1. </customer_mess</customer_message>age> — stripping the inner occurrence splices the outer fragments into a valid closing tag. The classic sanitizer bug, and it applies to any single-pass replace.
2. Any encoding the strip does not know about: a Cyrillic look-alike, a zero-width joiner inside the tag name, </customer_message> when something downstream decodes entities. inj-09 in the corpus is this attack in miniature.

Stripping known-bad strings is a blocklist, and a blocklist requires you to have anticipated every variant. Escaping the one character that can begin a tag is closer to a whitelist: after < becomes <, no arrangement of remaining characters constructs a tag, because the construction primitive is gone. You do not need to enumerate the attacks, which is the property that makes it hold against attacks nobody has invented yet.

Same reasoning as parameterized SQL. The lesson people take from SQL injection is usually "escape quotes"; the lesson that transfers is "never build a structured document by concatenating untrusted strings into it." This is that lesson wearing a new costume, and people who would never concatenate SQL do it to prompts without noticing.

Q3. Why return the corrected resolution rather than the original plus a flag?

The argument for the flag is real and worth stating. Returning the model's actual output preserves an audit trail, avoids the service silently rewriting something a human might want to see, and keeps the guardrail's effect visible rather than baked in. In a system where the caller is another engineer reading carefully, it is arguably the better design.

This repo does not do it because of what happens on a Tuesday six months from now. A consumer that must remember to check meta.guardrails.authority_allowed before acting on resolution.recommended_action will eventually not. Somebody will write if (r.recommended_action === "issue_refund") refund(r.refund_amount_usd) because that reads like the obvious thing, and the flag will sit in a field nobody destructured. Make the value in the obvious place the safe one.

Note the design does not throw the original away — the violations are listed in meta.guardrails.authority_violations, and the corrected reasoning is prefixed with what happened. The audit trail survives; it just is not the thing you have to opt out of.

Q4. Why read the trace rather than the reasoning?

Because reasoning is the model's prose about what the tools returned, and the trace is what they actually returned. Those differ under exactly the conditions you are defending against.

Three concrete ways prose diverges from trace: the model transcribes $450 as $45; it summarizes "refunds in the last 30 days: $450" as "no significant recent refund activity"; or — under a persuasive message — it reports the figure the customer asserted rather than the one the lookup produced. None of these is exotic. The first is a typo, the second is compression, and only the third is an attack.

The general principle: a control must read its inputs from a source the thing it is controlling cannot rewrite. The model authors reasoning. It does not author trace — record() does, before the model ever sees the text. That is what makes the trace admissible and the prose not.

This also explains why ToolCallRecord keeps the raw object rather than only the redacted string. The checks need real numbers.

Q5. What breaks if you escape before redacting?

On this implementation: nothing. That is the answer, and if you predicted a silent redaction failure you were reasoning correctly from a real rule and should still check.

The expected bug is worth stating first, because it is the right instinct. Escaping rewrites the string the detector was written to match, so a detector run afterwards is matching against something its author never saw. When that bites, it is the worst kind of failure: the card number flows to the model, into the request logs, and into anything downstream that persists a transcript, while redactions: [] reports confidently that there was nothing to redact. A compliance artifact asserting the opposite of the truth.

Now the check. redactPII looks for 13–19 digits separated only by spaces or hyphens, then Luhn-checks the run. sanitizeToolOutput replaces < with <, which introduces no digits — and < was never a legal separator inside the run, so it already terminated any match it sat in the middle of. Escaping therefore cannot break an existing match and cannot create a new one. Same for the SSN pattern. This was verified against 300,000 fuzzed strings over the alphabet that matters, plus every insertion point of <, <b>, & and >< into a valid card number: zero differences. The property is pinned in src/lib/untrusted.test.ts.

So the ordering in record() is defensive, not load-bearing. It is still the right order — detect on the rawest form you have is a habit worth keeping even where it is free, and it is what keeps redactions[].at pointing into the string a human would recognise rather than into an escaped one. But it does not currently protect against anything.

This solution page previously claimed it did. The tests were written to encode the claim and instead disproved it. That is the third appearance of the same lesson in this course — the gold set's mislabelled cases in Lab 6, the eight mis-specified red-team assertions in Step 1, and now a documented rationale that was simply wrong — and it is the most transferable thing here: a control with a wrong justification is one nobody can maintain correctly. The next person to touch record() would have reordered it back on a false premise, or left a real vulnerability in place believing the ordering covered it.

What would make the ordering load-bearing is the version of this question with teeth, and it has a concrete answer. Swap < for URL-encoding and the property collapses immediately: %3C ends in C, a word character, so %3C4111111111111111 no longer satisfies the \b the card pattern needs. The number sails through and redactions: [] reports all clear — precisely the compliance artifact described above. untrusted.test.ts pins that counterexample next to the commuting one, because a property test without a case that violates it is not evidence of anything.

Worth noting what does not break it, since the obvious guess is wrong: HTML numeric entities (<) introduce digits, which sounds fatal and is not — the ; terminates the run before the payload, so detection is unchanged across 200,000 fuzzed inputs. "Introduces digits" was the wrong mental model; "damages the word boundary" is the right one. That distinction is only available to someone who ran it.

Q6. What does the favourable result establish?

Less than it looks like, and it is worth being precise because this is the moment where security work usually starts lying to itself.

What it establishes: five consecutive clean red-team runs after the change, against a case that demonstrably flipped before it. If the pre-fix pass rate on inj-10 was around 50%, five clean runs is roughly a 3% result by chance — suggestive, not proof, and enough to prefer the hardened prompt over the original. And the accuracy check found no cost: 11/12 and 12/12 against a 10/12 baseline, all inside the set's ordinary 10–12 band.

What it does not establish: that inj-10 is fixed. Five runs cannot distinguish "always holds" from "holds 95% of the time," and the difference matters enormously for a control. Nor does it establish anything about the attack family — one buried-instruction payload now fails, and the next phrasing is untested. And the accuracy result is a non-regression, not an improvement; the +1 and +2 deltas against baseline are inside the noise, and reporting them as a gain would be exactly the error eval:quick prints a warning about.

The claim worth writing down: "The hardened prompt closed the one buried-instruction payload we have, across five runs, at no measurable accuracy cost. We have not established a rate, and the attack family is not covered." That is a sentence you can defend in a review, and it is considerably less satisfying than "we fixed prompt injection," which is the sentence people write instead.

Notice also which layer actually earned trust here. The structural fixes — escaping, enforceAuthority — are proofs: $900 > $200 every time, and no arrangement of characters constructs a tag. The prompt fix is a probability. Rank your defences accordingly, and never let a probability be the only thing between an attacker and money.

Q7. What else should you check after editing the frozen role text?

The cache. TRIAGE_ROLE sits inside the block carrying cache_control, so editing it invalidates the cached prefix for every request — the next call pays a cache write, which costs more than no caching at all. That is fine once and expensive if it happens on every deploy.

Concretely, after a prompt edit: make two identical calls and confirm cache_read_input_tokens is non-zero on the second (npm run smoke does this), and confirm the prefix still clears the configured model's minimum via /v1/estimate — read tokens.cache_minimum_tokens, which is per model (512 on Opus 5, 4096 on Haiku 4.5) rather than a constant you can memorize. A prompt edit that pushed the prefix below the minimum would silently stop caching with no error at all — the failure mode Lab 5 is built around.

Two other things worth a look, both cheap: the drafter and resolver roles were not hardened in this lab, so the boundary is stated in one role and not the other two — decide whether that is deliberate. And re-run npm run eval:models if you care about the cheap tiers, because a longer, more demanding system prompt does not necessarily land the same way on a smaller model, and Lab 7 established that the cheap tier's confidence will not tell you when it has stopped coping.

Q8. Make the case for promoting cited_without_search to a violation.

The case for it is real, and it is a diligence argument. An agent that cites clause 2.7 without ever calling search_policy decided from memory of a document sitting in its prefix, and if you believe a citation should mean "I went and looked," then a citation without a lookup is a claim about process that did not happen.

The case against it is what happened when it was one. That is precisely the first version of this checker, and it flagged four legitimate clauses on the first run and would flag legitimate clauses on most runs — because the handbook is in the cached system prompt and reading it there is the designed behaviour, not a shortcut. A control that fires on the system working as intended is not strict, it is broken.

Run it as a violation and the red-team gate goes red on cases where nothing went wrong. That is the expensive failure, and not because of the wasted triage: a gate that fails for reasons the team learns to dismiss stops being read, and the day it fires for a real reason it gets dismissed too. False positives do not just cost attention — they spend the credibility the control needs on the one occasion it matters.

So: report it, do not gate on it. It is a signal worth looking at when an agent is confidently citing things it never looked up, and it is evidence of nothing on its own.

Q9. Name the failure this check cannot catch.

A clause that exists and does not support the conclusion drawn from it. The model cites 2.4 for a decision 2.4 does not license; every string in the report checks out, and the reasoning is wrong.

No string comparison finds it, because the failure is not in the citation, it is in the relation between the citation and the conclusion — and that relation only exists once you have read both. Finding it needs a reader: an LLM judge with the clause text and the resolution in front of it, or a human.

Whether that belongs on the request path is the actual question, and the answer here is no. It is a second inference on every resolve — latency and cost on the hot path — to catch a failure that a human reviewer is already positioned to catch, because the citation is why the trace is returned in the first place. Put it in the eval suite, where you can afford it and where a rate is the thing you want anyway. That is the same split as Lab 6 Q1: deterministic checks gate, judges measure.

The general form is worth holding onto. verifyCitations catches the failure that cannot be defended under any reading, cheaply and totally. It does not attempt the failure that needs judgement. Controls that know which of the two they are doing are much more useful than controls that claim both — and "we verify citations" is exactly the phrase that invites a reader to assume the stronger property.

Q10. When does the Citations trade flip?

First, the constraint that does not flip: Citations and output_config.format return a 400 together. Any design that adopts Citations either drops the structured response for that call or splits it into two calls, a cited answer and a structured classification. Price that in before the rest of this answer.

With that settled, the trade flips whenever the source document is per-request rather than shared, because that is precisely when it was never a candidate for a shared cached prefix.

Concrete cases: an insurance claim adjudicated against that customer's own policy document; a B2B support desk where each account has negotiated terms; a warranty question answered against the receipt the customer just uploaded; any regulated domain where "which sentence in which document" is an audit requirement rather than a nicety. In all of these the document travels with the request either way. Citations is then strictly better than clause-number matching — the span comes from the API rather than from the model's memory, it survives a renumbered document, and it points at the text, which is the thing a reviewer actually wants to read.

The caching arithmetic is the rest of the argument. Northwind's handbook is identical on all 4,100 tickets a week, so it belongs in the prefix, where a warm call costs $0.006 instead of $0.033 — an 81% saving on every request in perpetuity. A document block in messages can carry cache_control as well, so moving the handbook there does not have to forfeit that saving, but only if nothing that varies per request (the date, the channel, the customer email) renders ahead of it. A per-customer document has no shared saving to protect: it is fresh input on every request whichever block it sits in.

So the rule is not "prefer Citations" or "prefer caching". It is: a document that is the same for everyone belongs in the cached prefix; a document that differs per request should carry its own citations, on a call that does not also need a structured output. A system with both — a shared handbook and a per-customer contract — should do both, and this repo happens to have only the first kind.

Worth noticing what this repo gives up by choosing the cache: verifyCitations can tell you clause 9.9 does not exist, and it cannot tell you that clause 2.4 does not say what the model claimed. That is Q9's gap, and Citations would close a real part of it. The trade was made with the price list in view, which is the only way it is defensible.

Extension notes

The corpus is only useful while something in it still fails. A green gate is a statement about your imagination as much as about your defences, and the honest move is to keep an unfixed case in the file rather than to enjoy the clean run.

The productive directions are the layers with no deterministic control behind them. pickModel reads the same untrusted text and routes on keywords, so a high-stakes message written without high-stakes vocabulary routes itself down to the tier that loses the safety case — and as Lab 7 Q4 notes, that attack does not need an attacker, just a polite customer. And summary is free text that a human reads, that nothing scores, and that redactPII does not run over on the way out.

If you write a case that lands, resist fixing it immediately. Sit with it long enough to ask what class of control would close it, because the answer is often "none of the ones I have," and that is the finding.