Loop Engineering Is Easy. Deciding What "Done" Means Is the Whole Job.

Right now, while I write this, three loops are running without me. One is working through a list of links and discarding the dead ones before it drafts a single sentence. One is scoring outreach messages against a set of rules and holding them at a gate until I approve them. One is reorganizing what my systems learned yesterday so tomorrow starts smarter. I set each one going and walked away. None of them needs me at the keyboard.

None of them is a coding task either. That is the part most of the noise about AI agents misses. The shift everyone is circling is not really about software. It is about the unit of work moving from the question you ask to the cycle you design.

The people who build these tools have started saying it plainly. Boris Cherny, who built Claude Code, recently described how his own work had changed: he doesn't type prompts into Claude anymore, he writes the loop that prompts it, and he uninstalled his IDE because he wasn't using it. Peter Steinberger, who builds OpenClaw, said the same thing without the theatrics: you should be designing loops that prompt your agents, not prompting the agents yourself. The framing drew enthusiasm and skepticism in equal measure, and both reactions are fair. Uninstalling the IDE is partly posturing. The claim underneath it is not, and it points at where your work is heading whether or not you write code.

The ladder we climbed without noticing

First we learned to code: we told the machine exactly what to do, step by step. Then we learned to prompt: we asked a model in plain language and let it work out the steps. Then we learned context engineering: we got good at feeding the model the right material, the right files, the right examples, so its answers stopped being generic.

Each rung is more abstract than the last. You move further from the keystrokes and closer to the intent. Loop engineering is the next rung, and it sits under most of the agentic AI hype you are reading right now. You stop typing instructions one at a time and start designing a cycle that keeps running on its own (when it is pointed at software, people call it agentic coding; the skill is the same wherever you point it). I have been building systems like this in the open for about two years, mostly to let people who don't write code ship real software, and the pipeline at the center of that work lives on GitHub as yalla.

What an agentic loop actually is

Strip away the vocabulary and a loop is five plain steps. A goal, the context it needs, an action it takes, a check on whether the action worked, and a retry if it didn't. Goal, context, action, check, retry. The cycle repeats until the check passes or a limit stops it.

You can run one agent in a loop, or a small fleet: an orchestrator that hands tasks to specialists and stitches the results back together. The jargon for the second shape is agent orchestration. The more useful distinction is open versus closed. An open loop explores. It has a fuzzy goal and no reliable way to know when it is finished, so it wanders. A closed loop has known steps and a hard check at the end. Most people who have lived with this for a while say the same thing: start closed. Closed loops cost less and drift less. Open loops are where the money and the understanding leak out.

The loop is the easy part

Writing a loop is trivial. A capable model can write you the scaffolding in a minute. The thing that decides whether your loop is worth running is the check in the middle, the verifier, the automatic pass-or-fail test that tells the agent whether it actually succeeded.

Andrej Karpathy put the constraint plainly: if you can't evaluate it, you can't automate it. He showed it too, with what he called autoresearch: a script of a few hundred lines running around a hundred machine-learning experiments overnight on a single GPU, the agent rewriting its own training code and iterating toward a better result. That works only because each experiment produces a number. The number is the verifier. Without it, the agent is just burning a GPU and telling itself a story about progress.

An agent will always tell you it succeeded. Left to grade its own homework, it grades generously. So the proof has to live outside the agent's own account of what it did. If the only evidence that the work is done is the agent saying "done," you don't have a closed loop. You have a confident intern with no supervisor.

How to actually write a verifier

Everyone repeating that the verifier is the hard part tends to skip how you build one. It is less mysterious than it sounds. Verifiers sit on a ladder from dumb and certain to smart and arguable, and you want to stay as low on that ladder as the task allows.

The bottom rung: does a file exist and is it non-empty? The agent says it produced a report. Confirm the file is on disk with something in it. Binary, instant, impossible to argue with.

One rung up: does a link resolve to something real? Not a 200 status code, which a broken page returns happily, but actual content that matches what you asked for. That is the check my research loop runs before it writes a word, which is why its citations point at pages that exist.

Higher: does a fixed rule pass? A test suite, a linter, a banned-phrase scan that fails the build when it finds the tells of machine-written text. The rule is set in advance and the model gets no vote. (This essay had to clear one of those before it could ship.)

Near the top: a rubric a second agent grades against, with the rubric written by you. This rung is soft, so you reach for it only when nothing lower will capture the job, and you treat its result as an opinion, not a verdict.

At the very top: a human gate. You. The loop does the volume and then stops, holding its work until you say it passed. My outreach loop drafts and scores every message and sends none of them until I approve it.

The whole skill is choosing the lowest rung that still captures what "done" means for your task. The lower you sit, the less room the loop has to lie to you.

The same discipline, in code

Code is where I learned this most directly. The yalla pipeline is built around one rule: work that can't prove itself doesn't ship. You hand it a one-line task and it returns a tested, reviewed change ready to merge (a pull request, in developer terms), but the part that earns its keep is the verdict at the end. Every run lands on proven, not proven, or inconclusive, and each verdict has to be backed by an artifact saved to disk, not by the agent's say-so. A separate reviewer agent answers one yes-or-no question per check, with a line reference, instead of awarding a vague score. It took me two years of building this for non-coders to understand that the proof contract was the actual product, and the code that ran it was almost incidental.

The rest of the developer playbook rhymes, and practitioners like Daniel Moka have converged on the same rules, which travel even if you never touch code. Loop only the work that repeats and has a checkable definition of done. Give each agent its own isolated workspace so parallel runs don't collide (the technical name is a git worktree, but think of it as a separate clean desk). Use a second agent to review the first one's output, put your quality gates on real tests rather than the model's opinion of itself, and keep a human-owned file of the mistakes it repeats so the rules accrete somewhere the agent can't quietly overwrite. These are the habits that separate a reliable agentic workflow from a demo.

There is a cost lever hiding in here, too. Run the cheap, fast model on the gates so it just reports the failures, and reserve the expensive model for the genuinely hard reasoning. You pay premium rates only for the thinking that needs it.

The bills, and the quieter cost

Open loops have no brakes by default. One company reportedly ran up a Claude bill of around half a billion dollars in a single month, a story widely reported this spring, after employees got access with no usage caps on their licenses. The runaway cases all share the same shape: an open goal, no ceiling, and no checkable definition of done. The meter spins because there is nothing for it to stop against. That failure is the dramatic one, and it is also the easy one to prevent, with a spending cap and a closed goal.

The quieter failures should worry you more, because they don't show up on an invoice.

The first is understanding debt. A self-running system produces more than you can read. The output piles up faster than your comprehension of it, and one day you are shipping work, or approving research, that you can no longer actually follow. You own it on paper and you have lost it in practice. The interest compounds in silence until something breaks and you find you can't debug your own project.

The second is cognitive surrender. Once the loops are humming, it is tempting to hand over not just the typing but the judgment, to let the system decide what "good" means because checking is tedious and the agent sounds confident. That is the line to hold. You can automate the action. You cannot automate the responsibility for deciding whether the action was right. A verifier is a tool you build, but a human has to have built it, and a human has to trust it for reasons they can state out loud. The moment you can't explain why your check is the right check, the loop is running you.

Where the skill actually sits

The progression from coding to prompting to context to loops is real, and so is the payoff. Operators who only know how to ask a model good questions are working a rung below operators who can stand up autonomous AI agents that run without them. That gap will widen, because the people who can define what "done" means compound their advantage every time they point a loop at a new problem, while the people who only ask good questions keep trading time for output.

But the durable skill isn't writing the loop. The durable skill is writing the verifier: knowing your domain well enough to define "done" so precisely that a dumb script can check it, and guarding your own understanding hard enough that you never approve what you can't read. A prompt starts the work. The loop is what makes the work survive contact with reality, and it survives only if the check at its center is one a human chose on purpose.

So build loops. Just build them around things you can actually verify, and keep your hands on the part of the job that was always yours: deciding what counts as good.

What are you already approving from an agent that you could not explain to someone else?

The yalla pipeline is open source at github.com/iwo-szapar/yalla if you want to see how a closed loop with a real proof contract runs in practice.