Context is burned by tool output, not by conversation

When my agent session hit the context ceiling, my first assumption was that I'd talked too much. Two months of back-and-forth, thousands of exchanges, and that adds up.

I sorted the transcript entries by size to see which conversations were the expensive ones. None of them were conversations.

awk '{print length($0), NR}' session.jsonl | sort -rn | head -10

1.3 MB   a single tool result
1.2 MB   a single tool result
1.2 MB   a single tool result
1.2 MB   a single tool result
1.2 MB   a single tool result

Every one of the ten largest entries was the output of one command. A 1.2 MB result is roughly 300,000 tokens, close to a third of a million-token window, produced by a command I ran once and probably skimmed.

Why one command costs so much more than it looks

The thing that makes this expensive is not the moment it happens. It's that the transcript is the context, and the context is re-sent with every subsequent request.

Run a command that dumps 300,000 tokens on a Tuesday, and you don't pay for it once. You pay for it on every request for the rest of that session, until compaction eventually summarises it away, and compaction is itself expensive, and triggered earlier because of what you did on Tuesday.

A conversational turn is a few hundred tokens. You would need a thousand of them to equal one careless cat.

The commands that did it

I went back and looked at what those entries actually were. They fell into four groups, and none of them were unreasonable things to run:

a full-file read of a large generated file
a search whose pattern matched most of a codebase
a build log from a failing multi-package build
a directory listing that recursed into a dependency tree

The shape they share is that the useful part was small and the output was not. I wanted one error out of the build log. I wanted to know whether a symbol appeared, not to see all 4,000 occurrences. In every case the information I needed was a few hundred bytes and the transcript got a megabyte.

That's the whole problem in one sentence, and it means the fix isn't discipline about which commands to run. It's about where the output lands.

What I do now

The rule I settled on: anything that might be large goes to a file, and I read the part I need.

# instead of this
grep -rn "handleSubmit" .
 
# this
grep -rn "handleSubmit" . > /tmp/hits.txt; wc -l /tmp/hits.txt
head -30 /tmp/hits.txt

The file stays on disk. It costs nothing per request. If I need more of it later, it's still there, and reading thirty more lines costs thirty lines rather than the whole thing again.

The same shape applies broadly:

pnpm build > /tmp/build.log 2>&1; tail -40 /tmp/build.log
curl -s "$url" > /tmp/resp.json; jq '.items | length' /tmp/resp.json

There's a related habit that matters as much: ask for the count before the content. wc -l on a search result tells you whether you want to look at it, and costs one line. I now do that reflexively for anything I can't predict the size of.

What the redirect does not buy

Everything above is a claim about one axis: what the transcript carries. It is not a claim about how long anything takes, and the two are easy to blur when the advice is shaped like a performance tip.

Someone else timed both forms of the same search for me, on the same corpus:

grep -rn <pattern> .                    9.2s
grep -rn <pattern> . > /tmp/hits.txt    9.4s
git grep <pattern>                      0.02s

The redirect does not shrink the wall clock. The command does the same work either way; the only thing that moved is where the bytes went. I would not read the two tenths as a cost either, since that is four runs, and the same search on a cold file cache took 58.9 seconds against roughly nine once it was warm. Cache state moves this number by two orders of magnitude more than the redirect does.

The third line is not a redirect at all. It is a different program answering a narrower question against an index git already maintains, which is the actual way to make a search faster. Worth separating from the advice in this post, because "send it to a file" and "use a tool that doesn't have to walk the tree" solve different problems and only one of them is about context.

The one that surprised me

The largest single entry in the transcript was not a build log or a search. It was a file read of something I had generated myself: a metadata file the indexer writes, four megabytes of JSON describing every file it knows about.

I had opened it to check one field.

This is worse than the accidental cases because it was deliberate. I knew the file was large. I opened it anyway, because opening a file to look at it is such a basic operation that it doesn't feel like it has a cost. In a text editor it doesn't. Here it cost more than a week of conversation.

The version of that operation I use now:

# what shape is this
python3 -c "import json;d=json.load(open('meta.json'));print(type(d),list(d)[:5])"
 
# just the field
python3 -c "import json;print(json.load(open('meta.json'))['model_name'])"

Two commands, both under a hundred bytes of output, answering what a four-megabyte read was supposed to answer.

It's the same principle as the aggregate-first rule but it applies to reading, not just running. A file read is a command whose output size is the file size, and it's easy to forget that because reading feels passive.

What this looks like as a habit

I've ended up with three questions I ask before running anything whose output I can't predict:

Can the command answer the question instead of providing material for it? grep -c instead of grep. jq '. | length' instead of the document. A count is one line and often it's all you needed.

If I need the content, does it need to be in the transcript? Redirect to a file, read a slice. The file remains available for the rest of the session at zero ongoing cost.

Am I about to read something I generated? Build output, lock files, indexes, logs, caches. These are the largest files in most repositories and the least useful to have in full, because they're machine-written and their interesting parts are tiny.

None of this is sophisticated. What made it necessary was understanding that the cost is not paid once, and I did not understand that until I sorted a transcript by entry size and found ten commands accounting for more than everything I had deliberately done.

Aggregate first

The stronger version is to make the command itself return the answer rather than the material for the answer.

# 40,000 lines to answer "which files"
grep -rn "deprecated" .
 
# 12 lines, same question
grep -rl "deprecated" . | sed 's|.*/||' | sort | uniq -c | sort -rn

Both answer "where is this used." One of them costs three hundred times more than the other, and the expensive one contains the same information plus a lot of surrounding lines I was going to skip.

This is not new advice. It's how you'd write a shell pipeline anyway. What's new is the cost function. In a terminal, verbose output is free and you scroll past it. In an agent transcript it's charged on every subsequent request.

Some of this belongs in the tooling

Not all of it is the operator's job, and the better agent runtimes have started to take the obvious half.

Mine now intercepts output above a threshold and writes it to disk instead of into the transcript:

Output too large (413.2 KB). Full output saved to:
  /tmp/.../tool-results/b7n6vnmxz.txt
 
Preview (first 2 KB):
  ...

That single behaviour removes the worst case entirely. The command still ran, the data is still available, and the transcript got two kilobytes instead of four hundred.

It's worth checking whether your setup does this, because the failure is silent if it doesn't. A tool that dumps everything into context looks identical to one that truncates, right up until the session slows down two weeks later and you have no idea why.

Where it doesn't help is the middle range. A 200 KB result is under most thresholds and still costs around 50,000 tokens, which is 5% of a million-token window for one command. The interception saves you from catastrophe; the habits are for everything below that line.

Where the ceiling actually was

Once I had this in place I went back through the compaction log for the session that died. Fifty compactions, every one triggered at the ceiling.

If I remove the ten largest entries from that transcript, ten commands out of two months of work, it drops by about 12 MB, which is on the order of three million tokens. Enough to have avoided several compactions on its own.

I want to be careful with that number. It's arithmetic on entry sizes, not a re-run of the session, and compaction changes what's carried forward in ways that don't reduce to simple subtraction. But the direction is not in doubt: a handful of commands accounted for a disproportionate share of everything that session ever had to carry.

Two months of conversation was not the problem. Ten commands were.

The part I'd tell someone starting out

I had thought of context as a budget that fills up gradually as you work, the way a disk fills. It isn't. It fills in jumps, and the jumps come from tool output.

That changes what you watch. Watching your conversation length is watching the wrong variable. It moves slowly and predictably. What you want to notice is the moment you're about to run something whose output size you cannot predict, because that is where the entire budget goes.

The tell, in my transcript, was that I could not remember running any of the ten most expensive commands. They weren't decisions. They were reflexes, a cat here and an unfiltered grep there, and each one cost more than every deliberate thing I did that week.