I once wrote a working capability out of a system's study material because a search returned nothing.
The question was whether a particular hook could rewrite a tool's output before the model saw it. I searched the codebase and the vendor documentation for the field name I remembered, updatedOutput. Zero hits, both places. I concluded the capability didn't exist, appended that conclusion as a "correction" in four places, and moved on.
The field is called updatedToolOutput. The capability exists, is documented, and does exactly what I'd wanted. My search was correct, my regex was correct, and the answer I took away was the opposite of the truth.
That's the shape this article is about. A search that finds nothing and a search that ran wrong return the same thing. Zero is a legitimate answer to a well-formed question, which means there is no signal in it — a true negative and a broken query are byte-identical at the call site.
I started keeping a list the first week this cost me something. It had six entries then and has fifteen now. What follows is the part of it I could reproduce on the machine I'm writing this on, each command quoted as it behaved here, the mechanism behind each, and the discipline I now use instead of trusting a zero. It ends with the one that happened while this piece was being fact-checked.
The shell cases
These cost the most, because the shell tells you nothing went wrong. Everything below is Git Bash on Windows: bash 5.2.37 (MSYS), git 2.53.0.windows.2, GNU grep 3.0, findutils 4.10.0, curl 8.18.0, Node 24.14.1. Your versions will differ, and the whole point of the piece is that you should check rather than trust mine.
Two I've written up before, so one sentence each. Under MSYS, any argument token that starts with / is rewritten to a Windows path before a native executable sees it, so git grep 'href="/blog/' actually searches for href="C:/Program Files/Git/blog/, prints nothing and exits 1 — exactly what a real no-match does, so a converted pattern and a true negative are indistinguishable by exit code as well as by output, and MSYS_NO_PATHCONV=1 is the fix (hold that thought). And xargs -0 LC_ALL=C grep tries to execute a program called LC_ALL=C: on findutils 4.10.0 that is loud — stderr says xargs: LC_ALL=C: No such file or directory and the exit code is 127 — and it becomes a clean zero only when the pipeline continues past xargs or stderr is dropped, which is how it once reported zero across a repo holding 218 matches. Both are in the earlier piece.
git log -G with a Perl-style pattern. -G takes a POSIX extended regular expression. \d, \w and \s are not part of it, and neither is the basic-regex habit of escaping | and {}. Six patterns against one file in a repository where the right pattern lists 76 commits, each run as git log --oneline -G '<pattern>' -- sitemap.xml | wc -l:
<lastmod>2026-\d\d 0 # \d is not ERE
<lastmod>2026-[0-9]\{2\} 0 # BRE interval, wrong flavour
ogImage\|lastmod 0 # BRE alternation, wrong flavour (run over sitemap.xml and posts/)
<lastmod>2026-[0-9]{2} 76
<lastmod>2026-[[:digit:]]{2} 76
ogImage|lastmod 77 # ERE alternation (same two paths)The fix I first reached for, --perl-regexp, does nothing: the manual says it governs the "limiting patterns" — --grep, --author — not -G, and git log --perl-regexp -G '<lastmod>2026-\d\d' still returns 0 here. --pickaxe-regex is for -S and refuses to combine with -G at all. Write the extended regex:
# 0 — \s and \d are not ERE
git log --oneline -G '"date":\s*"\d+' -- posts | wc -l
# 39
git log --oneline -G '"date":[[:space:]]*"[0-9]+' -- posts | wc -lThis one is nastier than it looks because the same \d works in grep -P two lines earlier in the same script, so you carry a mental model that has already been validated.
node --check as a verification step. It parses. It does not evaluate. A file with a temporal-dead-zone violation — a const referenced before its declaration is reached — parses cleanly and exits 0, then throws the moment it actually runs.
// tdz.js
const items = ["a", "b"];
items.forEach(x => LABELS.push(x));
const LABELS = [];node --check tdz.js # exit 0
node -e "require('./tdz.js')" # ReferenceError: Cannot access 'LABELS' before initialization, exit 1I used node --check as a gate on generated code for weeks. It catches a missing operand (const x = ; fails it, I checked). It catches nothing about whether the file works.
Line-ending counts. grep -c $'\r$' is the folk way to count CRLF lines. On GNU grep 3.0 under MSYS it returns 0 for a file a byte scan shows to have 1,585 CRLF endings and no bare LF — this grep strips the CR from each line before matching, so the pattern can never fire. It also returns 0 on a pure-LF file. Same output, opposite files. -U keeps the bytes:
grep -c $'\r$' index.html # 0 (file has 1,585 CRLF lines)
grep -U -c $'\r$' index.html # 1585 (-U: binary mode, CR preserved)
grep -U -c $'\r$' blog/index.html # 0 (pure LF — the negative control)sed strips the CR too: sed -n 1p index.html | od -c ends in a bare \n on that same CRLF file, so a sed | od pipeline reports LF for everything. Read the bytes directly, or use -U.
grep -P on this locale. With LANG unset, grep -P refuses every pattern, not just exotic ones:
grep -c -P 'html' index.html
# grep: -P supports only unibyte and UTF-8 locales (stderr, exit 2, nothing on stdout)
LC_ALL=C.UTF-8 grep -c -P 'html' index.html # 8Redirect stderr, or pipe the result into wc -l, and it is a clean zero.
od | grep -o '0d 0a'. A tempting byte-level check for CRLF: dump hex, count the pairs. od emits sixteen bytes per output line, so any 0d 0a pair that straddles a line boundary never matches. On the same 1,585-line file:
od -An -tx1 index.html | grep -o '0d 0a' | wc -l # 1487
od -An -tx1 -w1 index.html | tr -d ' ' | tr '\n' ' ' | grep -o '0d 0a' | wc -l # 1585Not a zero this time — an undercount of 98, which is worse in one way: it looks like a measurement.
grep -c '\r'. In a basic regular expression \r is the letter r. On a pure-LF, 401-line file it returned 317: every line containing an r. This is the saturated failure — the detector that fires on everything — and it is why the discipline below has a negative control as well as a positive one.
git show HEAD:file as a baseline. Under core.autocrlf=true the blob is stored LF and the working tree is CRLF. git show HEAD:index.html | wc -c gives 135376; wc -c < index.html gives 136961. The delta is 1,585 — exactly the line count. A byte comparison against the blob therefore reports every CRLF file as changed, and a "14-byte difference" on a block with fourteen newlines is not a difference at all. Compare LF-normalised content.
The fix that caused the next one
This is the best case on the list, and my first draft had it wrong.
The git grep fix above is MSYS_NO_PATHCONV=1: stop the shell translating Unix-looking arguments before they reach a native executable. I exported it for every session that grepped HTML or JSON. Some weeks later a search-engine submission silently never ran, and it took a while to connect the two.
Path conversion is also what turns /dev/null into nul for native programs. With the variable set, curl.exe receives the literal string /dev/null, cannot open it as a Windows path, and exits 23 — write error — after a complete, successful fetch. -s hides the message. -w prints the 200. And && short-circuits.
MSYS_NO_PATHCONV=1 curl -s -o /dev/null -w 'HTTP %{http_code}\n' https://www.claudecertifiedarchitects.com/sitemap.xml && echo "NEXT COMMAND RAN"
# HTTP 200
# (nothing else — $? is 23)
MSYS_NO_PATHCONV=1 curl -sS -o /dev/null https://www.claudecertifiedarchitects.com/sitemap.xml
# curl: (23) client returned ERROR on write of 8268 bytes
curl -s -o /dev/null -w 'HTTP %{http_code}\n' https://www.claudecertifiedarchitects.com/sitemap.xml && echo "NEXT COMMAND RAN"
# HTTP 200
# NEXT COMMAND RANWriting to NUL or to a real file works with the variable set; I ran both. So the fix for one false zero manufactured another, in a different tool, weeks later, with the visible output — an HTTP 200 — actively reassuring. A list of tools that lie is not a list of independent facts. The entries interact, and the fix column is where they do it.
The same shape in code
The shell cases are easier to spot once you know them. The ones that live in application code have cost me far more.
A lookup with a fallback. This one ran for eleven weeks in an email template.
const tip = own(STUDY_TIPS, domain) || STUDY_TIPS['Agentic Architecture'];STUDY_TIPS was keyed on 'Claude Code Configuration'. The value stored against every affected record was 'Claude Code Configuration & Workflows' — the display label, not the key. The lookup missed, the || supplied a default, and every consumer downstream received a valid, well-formed, entirely wrong tip.
167 emails went out that way: 85 carrying a study tip from the wrong domain, 82 carrying a sample question from the wrong domain, each under a heading naming the domain the reader had actually asked about. Nobody complained. The content was correct advice about something, and no recipient had any way to know what they were supposed to have received.
|| on a lookup converts a miss into a value. If the key is expected to exist, the miss is a bug and needs to be loud:
const tip = own(STUDY_TIPS, domain);
if (!tip) throw new Error(`no tip for domain: ${domain}`);A wrapper that catches and returns []. The agentic version of the same thing — a subagent fails, the wrapper logs it and returns an empty array, and the orchestrator receives [] and treats it as a fact about the world — which I wrote up last week and won't repeat, beyond noting that it survives code review because the code looks defensive.
A script that fabricates a readout. The worst one I've written. An audit script was supposed to report stored state. It didn't merely always answer yes — it compared its input to itself and printed a plausible, well-formatted readout of state it had never read. Everything downstream consumed the readout as the check.
If you want to see how this reads as an exam problem rather than an essay, we write scenario questions on exactly this in the practice bank. It's independent preparation material. We're not affiliated with Anthropic, and we neither sell nor administer the exam.
The discipline
One rule, and it is not "be careful."
Never accept a zero you have not proved with a known-positive control.
Before you believe a search returned nothing, run the same detector against something you know it should find. If the control doesn't fire, the zero is meaningless and you have learned that instead — which is a result, and a cheap one.
# the question: does git grep find the link?
git grep -c 'href="/blog/' -- blog/index.html # (nothing), exit 1
# the control: the MSYS grep binary, same file, same pattern
grep -c 'href="/blog/' blog/index.html # 47
# the question again, with the fix
MSYS_NO_PATHCONV=1 git grep -c 'href="/blog/' -- blog/index.html # blog/index.html:47Two controls are better than one: a positive that must fire and a negative that must miss. The second catches detectors that match everything — grep -c '\r' above — which is the failure mode the first one can't see.
The second half of the rule matters just as much, and it's the one I'm still bad at.
State the property your detector tests, then check that the detector tests that property — not the instance that made you look. When I found the updatedOutput error, I fixed the four places that carried it. I did not ask what else I'd written from the same guessed name on the same day. Closing an instance feels like closing a class, and the difference surfaces weeks later, by accident, usually in production.
For the email lookup, the detector that would have caught it in June is, in outline, three lines: call the real render function once for each distinct value actually present in the datastore, and read the output.
for (const domain of await distinctStoredDomains()) {
console.log(domain, '→', buildEmail1({ weakestDomain: domain }).subject);
}That check now exists.
Where this costs more than it's worth
Controls are not free, and I don't run them on everything.
If a zero is cheap to be wrong about — you're looking for a string you half-remember in a file you're about to read anyway — the control is ceremony. If a zero is loud when wrong — the build fails, the tests go red, a human sees the empty page — the environment is already your control.
The cases worth the discipline are the ones where a wrong zero is quiet and durable: an audit result written into documentation, a class-closure claim, an enum lookup with a fallback, anything an autonomous process consumes without a human between the query and the conclusion.
There's an inverse failure worth mentioning too, since it cost me an hour last week. I compared two spreadsheet files cell by cell using a library's style objects, and every cell came back as different — including the empty ones — because those objects don't compare equal across two workbook instances. A detector that fires on everything is as useless as one that fires on nothing, and the negative control is what catches it.
Why this got worse, and what happened while this was being checked
All of the above predates LLM agents. What changed is who reads the zero.
A human who gets an empty result brings context: they know roughly how many results to expect, they remember that this search worked yesterday, and they have the option of asking someone. A model has your string and nothing else. Hand it [] and it will report, fluently and confidently, that nothing matched — and then act on that.
It happened during the fact-check for this article. To confirm the opening example, the hooks documentation was fetched through a tool that hands the page to a small model and asks it a question. Asked how many times updatedToolOutput appears, it answered: zero. It added, helpfully, that the page does not describe the hook modifying output. A curl of the same URL and a grep of the bytes: seven occurrences, including the table row that defines the field, with the control string PostToolUse at 58. The summariser had read part of the page and reported its absence as the page's. The false zero at the top of this piece was reproduced, by an agent, inside the session checking whether the piece was true, on the exact field name the piece is about.
So in an agentic system, "nothing found" and "the search did not run" have to be told apart in the value that comes back — not in a log, not in an exit code the model never sees, and not by a reader who was not there.
{ "results": [], "status": "ok" }
{ "results": [], "status": "error", "error": "upstream_timeout", "retryable": true }Two different facts about the world. If your tool returns the first for both, every consumer downstream will eventually treat the second as the first, and the one reading it hardest is the one least equipped to notice.
Every command above was run on the machine described at the top — Git Bash on Windows 11, bash 5.2.37 (MSYS), git 2.53.0.windows.2, GNU grep 3.0, findutils 4.10.0, curl 8.18.0, Node 24.14.1 — and is quoted as it behaved there. Behaviour varies by shell, tool version and platform; verify each on your own system before trusting the fix, which is the article's entire point. The 167 / 85 / 82 email figures come from the commit that fixed the lookup and the read-only count it recorded; the underlying export is not on disk, so treat them as reported rather than reproduced.
Independent preparation material for the CCAR-F exam, also written as CCA-F. Not affiliated with Anthropic; we don't sell the exam and can't register you for it. The published exam guide weights Context Management & Reliability at 15% of CCAR-F. Our practice question bank covers that domain with a full explanation on every question, and the free diagnostic is ten questions and needs no account.