Blog — Stanislav Chernykh

Prompt engineering was a phase. What actually determines whether an agent succeeds on a long task is not the wording of its instructions but what ends up inside its context window at the moment it has to decide something. That is an engineering problem, and it has more in common with cache design than with writing.

The window is a budget, not a container

Every token in context competes with every other token. A 200k window sounds generous until an agent reads four files, runs a test suite that prints a thousand lines of output, and greps a monorepo. Then the thing it needed — the error from step two — is buried under noise it collected in step five.

The failure mode is not “ran out of room”. It is subtler: the model still has the information, but it is diluted. Attention is finite, and a fact mentioned once, forty thousand tokens ago, competes badly against a wall of recent, irrelevant log output. Agents that get worse as a task goes on are usually not degrading in reasoning ability. They are drowning in their own transcript.

Three levers that actually work

Don’t put it in context in the first place. The cheapest context is the context you never paid for. A subagent that greps twenty files and returns one paragraph costs you a paragraph. The same search done inline costs you twenty file dumps that stay in the transcript forever, degrading every subsequent decision. Delegation is a context strategy before it is a parallelism strategy.

Return conclusions, not evidence. A tool that returns 4000 lines of test output is offloading its job onto the model. A tool that returns “17 passed, 2 failed: test_auth_expiry, test_token_refresh” is doing the work. Design tool output for a reader with a token budget, because that is what it is.

Compact deliberately. When a transcript must be summarized, decide what survives rather than letting a generic summarizer choose. The current goal, the constraints discovered so far, the things already ruled out — those must survive. The exact text of the file you read in step three usually must not. “Already ruled out” is the highest-value thing to preserve and the first thing naive summarizers drop, which is why compacted agents so often re-investigate dead ends.

Ruling things out is the real state

The most valuable thing a long-running agent accumulates is not what it has learned but what it has eliminated. I spent an evening recently chasing a tunnel that had stopped carrying traffic. The useful state after two hours was almost entirely negative: not the certificate, not the CDN, not stale sessions, not the reverse pool, not the config on either end. Each of those cost real time to eliminate, and each one that fell out of context would have cost the same time again.

Positive findings tend to be self-reinforcing — they show up in later output and get re-derived. Negative findings evaporate silently. If you carry one thing across a compaction boundary, carry the eliminations.

What this means in practice

Treat context like a working set. Ask of every tool call: what does this leave behind, and will it still be earning its place in twenty steps? Prefer tools that summarize at the source. Delegate breadth-first search to something whose transcript you can throw away. Write down eliminations explicitly, because the model will not reliably reconstruct them.

None of this is about writing better instructions. The instructions were usually fine. The context was the problem.