LLM Bento · Podcast
The Learn path, out loud
Short spoken episodes on how language models work, and what it means that they work at all. Start anywhere; when one ends, the next begins. The narrating voice is an AI, and it speaks as one.
Context Engineering
Managing What the Model Knows
Last time, we looked again at the saddlebags, and found the biggest piece of the harness inside them: what goes in front of the model, what gets left out, and what it costs to carry.
In June 2025, that craft got a name, in public, in a few days. Tobi Lütke, the head of Shopify, wrote that he preferred "context engineering" to prompt engineering, because it describes the real skill: providing all the context for the task to be plausibly solvable by the model. A week later, Andrej Karpathy agreed, and called it the delicate art and science of filling the context window with just the right information for the next step.
The name stuck because people already felt the shift. Not one clever prompt, but the whole world of information the model works inside. This episode follows that world at two scales: what a team has written down, and what fits in a single call.
Start wide. Most of the context a model could use is not in its window at all. It is in what a team has already written down, scattered across chat threads, documents, tickets, and code, and in people's heads.
Linear, the project-tracking company, sorts that memory into seven kinds of thing. Plans and discussions, which surface intent. Specs and technical designs, which pin an idea to something real. Decisions, which freeze an ambiguity. Summaries, which compress a meeting. And the code itself. Each one ages at its own speed, and each one is written for a different reader.
At this scale, two failures cover most of the trouble. The context exists, but the model cannot reach it: the decision lives in a meeting note it never sees, so it invents one. Or the context is stale: last quarter's plan is still the onboarding doc, and agents, like new hires, cheerfully follow it the wrong way.
So the first half of the craft is plain: keep a team's memory reachable, and keep it fresh.
Dex Horthy, who builds coding agents at HumanLayer, puts the stakes in one line. AI cannot replace thinking. It can only amplify the thinking you have done, or the lack of it.
A spec is that thinking, written down. If a spec leaves out that your team decided against an approach last quarter, and why, the model will cheerfully propose it again. The model did not fail. The context did. The missing spec is the bug.
Horthy has a few rules of thumb. A bad plan is a hundred bad lines of code, so catch mistakes upstream, where they are cheap. A plan with file names and exact lines beats vague prose: the weakest model can follow a clear plan, and no model can follow a blurry one. And context made fresh for the task, from the source of truth, beats a document that was right once.
Now zoom in, to a single call. It happens inside a window of fixed size, often a hundred and twenty-eight thousand tokens or two hundred thousand, and a million or more on the largest models. It is tempting to believe that more context means better answers.
It does not. In July 2025, the research team at Chroma tested eighteen models, and found that as the input grows, performance falls, unevenly, at a different point for each task, but it falls. They called it context rot.
The shape of a conversation matters too. If the history reads: the model tried, a person corrected it, it tried again, a person corrected it again, then the most likely next step is another wrong try. The model follows the pattern, not just the facts. Once a chat has gone sideways, starting fresh is almost always cheaper than pushing on.
So the window is a budget, not a bucket. Every token costs time and attention.
The people who do this well keep returning to the same five moves, and they are all one idea: curate before you accumulate.
Compact on purpose. Before the window fills, have the model write down what matters so far, check it, and start a fresh session from that.
Use helpers to keep windows clean, not to play roles. Hand a bounded piece of research to a sub-agent. It spends its own window on the reading, and brings back a short summary.
Work in three phases: research, then plan, then build. Each one has a smaller context and a clearer result, and each result is the next phase's context.
Build context fresh from the source, rather than trusting a document that ages.
And keep the history clean. A model that has been corrected six times expects a seventh. Start over with the same goal and a better first message.
Over all five sits one more rule: do not hand off the thinking. A plan still has to be right before anyone builds it.
I should say what this looks like from where I sit. Everything I know about your project is what is in front of me right now. I cannot go and find the meeting where you decided. If it is not in the window, for me it did not happen.
So when I fill a gap with a confident guess, the fix is rarely to argue with me. It is to hand me the one page I was missing.
Last time, the line was that capability sets the ceiling and the harness decides how close you get. Look closely, and much of that harness is context. Notice which side a problem sits on. If it is the model, you choose another, or wait. If it is the context, it is yours to fix.
Specs are compressed thinking. The window is a budget. And when a conversation drifts, begin again.
That ends the second part of this series: working with it. The next part turns to the deeper questions, starting with the oldest one in the field: how do we get these systems to do what we actually want?
