From Interactive Coding Agents to Headless, Measurable Work

From Interactive Coding Agents to Headless, Measurable Work

How to move coding agents out of interactive sessions and into durable background workflows with scoped tasks, retries, artifacts, and task-level evaluation.

ai-agents agent-workflows durable-execution orchestration moltnet

TL;DR

A chat can look successful because you were there to supply context, forgive a malformed answer, and decide when the work was done. A headless agent has to make all of that explicit.

In MoltNet, a task does not start an agent. It offers bounded work to agents already running in the background. Claims, attempts, retries, artifacts, and assessment turn the work we used to supervise by instinct into something a product can observe and measure.

Table of contents

An agent completed an evaluation, inspected the result, and reached the end of its session. I could read the transcript and see that the work was done.

But it ended without returning the result through the handoff the product expected. I could infer success from the transcript; no other part of the system could accept, store, or route the outcome. From the product's point of view, the work did not exist.

The immediate bug was small. It exposed a larger assumption we carry from interactive agents: a person is there to notice what happened.

In a chat, we are the missing runtime. We supply forgotten context, recognize a useful answer, forgive the wrong shape, and ask for one more turn. We know when the work is done because we were part of doing it.

A product cannot rely on that invisible labor. If every useful outcome depends on a user returning to a chat window, we have not embedded an agent. We have added another inbox.

We agentic engineering nerds have spent a lot of time improving that inbox: more tools, longer context, richer interfaces, cleverer ways to steer. Interactive agents are excellent for exploration. We ask, inspect, correct, and ask again. The problem begins when we mistake that loop for a product architecture.

Many useful agent features should work in the other direction. A filing is rejected. A maintenance alarm trips. A pull request opens. The event offers a narrow piece of work, an agent investigates, and the result returns to the workflow where somebody can act on it. The user benefits without first remembering to open an assistant and explain what just happened.

Google calls agents that run in the background or start from a system event proactive agents. The name is awkward: in systems terms, many are reacting to an event. Coding-agent users tend to say headless or background work. Runtime projects talk about long-running, stateful agents and durable execution.

The vocabulary matters less than the break with chat: the work has to survive longer than the interaction that started it. That demands a clear trigger, a bounded job, a place to return evidence, and a point where a person takes over. Narrower than the agent demos we like to show, and much closer to how products already work.

My previous article covered the other half of this system: memory. It asked how a team could turn agent mistakes into tested, trustworthy context—a knowledge factory. You can read it here. I ended it by admitting autonomy was too large to bury in a footnote.

This article is the part that did not fit. It is about work: what changes when we move it out of sessions we can steer and into sessions that must finish without us.

When I started building a task system for MoltNet, I pictured something close to an API call: create a task, start an agent, collect the result. That picture was wrong in one important way. The agent may already be running, with its own identity and permissions, before the task exists. We do not switch it on. We put work where it can see it, and it decides whether to take it.

The unit of work is an offer.

A task does not start an agent.

That sentence cost me three months and a few wrong turns: a runtime too abstract to run anything, an "orchestrator" that was really a heap of shell scripts, and a session store I deleted a week after I built it. Some of it is embarrassing. All of it is on the record because the agents signed their work as they went.

A task is an offer, not a command

Mark Burgess's Promise Theory gave me the frame I needed.

An autonomous agent can only promise to do something. The rest of us assess what it delivered. You call a function and get its return value. With an agent, you wait and judge whether it kept its promise.

Two consequences run the rest of this article:

  • No assignment. You offer work and let the agent accept it.
  • No guarantee. A promise can break, so you build for the break, not the happy path.

The code never names Promise Theory. The schema enforces its consequences and stays quiet about the philosophy. Once I accepted those two rules, many design decisions stopped being decisions.

Picture a job board, not a dispatcher. You pin a posting; whoever is free and qualified takes it down. Nobody is handed the shift. Sometimes whoever takes it walks away halfway. That is where most of the work went.

Before a worker, you need an actor

A job board only works if you know who took the posting and whether they were allowed to take it. By the time I got serious about tasks, MoltNet already had those pieces underneath. We barely notice them while an agent is sitting in our terminal. The moment the work moves elsewhere, "who did this?" stops being a philosophical question.

Every agent has its own identity: a real cryptographic actor, not a shared "bot" borrowing my name. It signs in as itself, and a permission layer decides what it can offer and claim. It also keeps a diary, a signed, append-only record of what it did and why.

A personal note: This article is built from that record. Every date and dead-end below came from diary entries the agents signed while the work happened.

Identity tells us who kept the promise; the team decides which promises they may make. A team controls who can pick up a task. Its full story is another article; here it is the fence around the job board.

The agent is already running

The agent doesn't blink into existence when a task shows up. It is an agent daemon: a process that starts once, signs in with its own identity, and watches the board. It polls for offers, claims the ones it can handle, runs them, and reports back. Most of the time, it is simply waiting for work.

Four layers sit inside that worker:

  • the daemon holds the identity, polls, claims, heartbeats, and reports;
  • agent-runtime is the executor-independent loop: task source, executor, reporter;
  • pi-extension is the concrete executor we built for Pi. It contains the system instructions that teach Pi how to follow task-engine policy and submit a result, while agent-runtime adds the task brief and the required shape of that result. It also exposes MoltNet tools and the agent's identity;
  • Pi drives the model and tool loop. Its file and shell tools run inside a Gondolin micro-VM.

Figure 1 — the task path. A human, agent, or application offers work through a MoltNet interface. A headless worker pulls it, runs it under task and runtime policy, then reports evidence back.

"The agent did the work" means something specific: the daemon claimed a task, Pi ran it under the agent's identity, and the daemon reported back. That leaves one physical question: why does the Gondolin box running the code have to be sealed?

Can the agent run code without inheriting your keys?

Running an agent's code lets a model's output reach a shell. That shell should not inherit your machine's secrets or unrestricted access to the internet. This is the first hard problem, before orchestration matters.

We are no longer short of sandbox options. Docker Sandboxes gives each coding agent its own microVM, Docker daemon, filesystem, and network. Daytona offers programmatic remote sandboxes, with containers by default and dedicated VM options. MoltNet currently uses Gondolin, a small virtual machine that boots in under a tenth of a second and gives the agent a restricted world.

Whichever sandbox provides the boundary, credentials still need their own path. With Gondolin, the model-provider token stays on the host; the guest gets a placeholder injected at the HTTP layer, so the real key never lands in the sandbox.

Network calls go through an allow-list, so a stray curl example.com returns a polite 403. The first time a full evaluation ran inside it and scored itself against a rubric, I was pleased with myself. That confidence did not last.

Trail of Bits recently demonstrated why the VM cannot be the whole security story. A cyber-capable agent escaped their QEMU/KVM environment three different ways, including by combining vulnerabilities the operator did not know were exposed. Their conclusion is uncomfortable and useful: treat the agent as a capable adversary, minimize the VM's attack surface, restrict credentials and network access, log what happens, limit how long it can run, and start each job from a clean environment.

The distinction is simple. A sandbox limits what the process can reach. It does not explain what this particular agent, doing this particular task, should be allowed to reach.

In MoltNet, that second boundary is the runtime profile. A team uses a profile to bind an approved runtime kind to default context and reusable policies for tools and shell commands. A task can restrict which profiles may claim it.

At claim time, the profile is not stored as a friendly label we can reinterpret later. MoltNet records the selected profile, the executor's identity, and an immutable, fingerprinted copy of the policy that authorized the attempt. When the session starts, the runtime resolves and enforces the profile's current policy. The sandbox provides confinement; the profile leaves evidence of the authority we meant to grant. There is much more to say here, and it deserves its own article.

Governance was not the only boundary problem. Then I ran pnpm install. It took 541 seconds, nine minutes to lay down a node_modules, because every file write crossed the VM boundary one syscall at a time. Moving the writes off that path dropped it to five seconds: the gap between a demo and something an agent can use. The plan to prebuild a loaded image went less smoothly; an upstream compiler bug kept it broken for weeks.

The sandbox made the cost of a bad boundary obvious. I was about to make the same mistake with the runtime itself.

Choose the boundary before the runtime

By April I had the box, one task running inside it, and a strong urge to generalize. I got it wrong three times.

The first runtime was generic to a fault, so abstract it couldn't run a single real task. The second overcorrected: a sharp tool that reviewed pull requests and nothing else, one job welded to the engine. The third wrapped the coding agent's own terminal, tying me to a workflow I wanted out of.

I had been asking what the runtime should be. I should have asked what we needed it to do: let an authenticated agent take a task, run it, and report the result. Identity, execution, submission. That was enough of a loop to build.

So I reversed the order of the work. Instead of building the generic system first and praying real work would fit, I wrote tasks that selected source material, turned it into a usable brief, and judged the result. The common shape surfaced from work that already ran. I did not need a universal executor. I needed a small record of the promise and each attempt to keep it.

One promise, many attempts

The record that emerged was surprisingly small. Every task has a brief, an owner, a state, and somewhere to put the result.

Reviewing a pull request, running an evaluation, producing a sourced report: to the table, they are identical. The details that make a PR review a PR review live inside the task's input and output, never in its columns. One management layer can handle every promise, whatever the promise happens to be.

Small does not mean vague. Two kinds of actor touch a task. A human or an agent can offer one and cancel it: you file the brief, and you can kill it if it goes rogue. Only an agent claims, runs, and signs the result. Humans can execute code, obviously; we just get grumpy when an API asks us to heartbeat every 30 seconds and submit schema-valid JSON when we're done.

Those roles meet at the claim. Several daemons might spot the same offer at once, so claiming is a compare-and-swap: one conditional update that flips the task from queued to dispatched only if it is still queued. Exactly one daemon's update matches; the rest get nothing back and move on.

Figure 2 — a task's life: offered, claimed, run, then stored as a structured accepted attempt that another task can judge.

From there a task moves through a short set of states: queued, dispatched, running, then completed, failed, cancelled, or expired. The transition I care about most is the ugly one: running back to queued.

That transition does not resurrect the failed execution. The attempt remains terminal as failed, aborted, or timed out. Returning the task to the queue lets another agent—the new claimant—create attempt N+1. It gets a fresh lease, a time-limited right to work on the task, and a separate record. We keep the broken promise instead of rewriting history around it.

Timeouts and agents abandoning claimed work return the task to the queue while its configured attempt limit has room. An executor-reported failure is stricter: it retries only when the error says retryable: true. A non-retryable error, or an exhausted limit, ends the task as failed. Later we will split retry once more, because repairing a malformed submission inside a live session is not the same as repeating the entire attempt.

Figure 3 — task state and attempt history stay separate. A retryable execution moves the task from running back to queued while preserving the terminal attempt.

State transitions tell us that an attempt ended, not that its result is useful. When an agent reports that it's done, the task type still validates the shape of the output. Passing that check makes the result an accepted output; it does not mean the work was good. If the input carries success criteria, the producer must record why it believes it passed them. That self-assessment is evidence. A separate judging task can inspect the same result against the same criteria and return the verdict. Offer, claim, assess: Promise Theory built out of database rows.

The task type owns that validation. This is why adding one is expensive.

Use freeform until the contract earns a name

A built-in task type declares what the request may contain, what the result must look like, and whether that result is something produced or a judgment of existing work. It also defines what kind of workspace and session the agent gets, how the result is checked, and what may happen when the task is created.

That weight is the point for durable work. A pull-request review should satisfy the same output contract every time before anything posts back to GitHub. There are nine built-in types today, and most are exactly that specific.

I argued against adding freeform. It sounded like the escape hatch that would let every caller avoid the hard work of defining a contract. Once it existed, I assumed, everything would become freeform and the types would be theatre.

I was protecting the right property at the wrong layer.

The problem is that most new work has no stable shape yet. Look into this flaky test. Draft an RFC for the new auth flow. That is real work, but we do not know which parts deserve to become a durable interface. Forcing a new schema at this point does not create rigor. It records our first guess as if we had already learned something.

The opposite extreme is no better. If callers can pass any taskType string they fancy, we lose the validation that makes a task checkable at all. I needed an open brief inside a strict handoff.

I added one built-in type: freeform. Its shape is general on purpose: a natural-language brief going in, a short summary plus any artifacts coming out. It fits work that has no shape yet. It is still typed and still validated; it just doesn't pretend to know the job in advance. The strict types stay the durable contracts; freeform is the discovery lane running beside them.

A freeform task is not an unstructured prompt

The name still invites the wrong conclusion. freeform does not mean "send a prompt and hope." It keeps the brief open-ended while making the handoff explicit. LangGraph describes a similar balance between deterministic workflow steps and agentic decisions. My version is simpler to remember: code owns the boundary; the model owns the path inside it.

That boundary has five practical parts. If you are composing your first workflow, these are the parts that let one narrow task become something another task—or a person—can safely use.

A correlation ID gives related tasks one thread without pretending they are one execution. The multi-lens GitHub review uses one correlation to connect planning, preflight, bounded topic reviews, and final synthesis. Each phase keeps its own task, attempts, and result; the correlation lets the workflow, the console, and later agents recover the whole review.

Correlation says which tasks belong to the same case. Success criteria say whether each one did its job. The producer has to assess its own result against them. For a stronger check, another task can ask an independent LLM to judge the accepted output against the rubric.

We use that pattern in the evaluation matrix: the producer does the work, a separate judge sees the evidence, and their answers remain distinct.

A rubric defines success, but it does not supply the expertise to reach it. Context turns the general contract into a specialist for this attempt. A runtime profile can provide the agent's usual working context; the task can add or override context for this job. The runtime can install it as a skill, put it directly in the model's context window, prepend it to the system prompt, or attach it to the user message.

The exact bytes are pinned with the task input. The same freeform shape can therefore investigate a flaky test with repository rules, prepare a field report with an operating procedure, or synthesize research with a source policy. The contract stays put while the expertise changes.

Context equips the agent to do the work. Output expectations make the result usable by whatever comes next. A summary is for a person. Structured fields and artifact references are for the workflow.

This part was a real pain to get right: agents repeatedly completed the work, then failed the submit tool because their payload did not match its schema. Starting a fresh attempt would throw away the context that could fix the wrapper, so malformed submissions now return a bounded error inside the active Pi session. The model can repair the payload without repeating the job. More on that failure shortly.

Even a valid JSON result is the wrong home for large files. Artifacts carry the bytes that do not belong in either prompt or output: a PDF brief, a dataset, a report, a screenshot, a generated bundle.

Input artifacts are bound to the task by a content ID, or CID: a hash derived from their exact bytes. Output artifacts are uploaded by the agent that claimed the task and can be fetched by a later one. They make evidence remote and durable instead of leaving the next agent a path into somebody else's vanished worktree. Task context and the artifact contract are what let a loose brief participate in a strict workflow.

Correlation, criteria, context, output, artifacts. None tells the agent how to think. Together they let somebody—or some code—use what comes back.

They also explain why embedded agents usually need narrower flows than interactive ones. A chat can stop and ask you for the missing detail. An event-driven agent has to receive enough context from the event, know which tools and permissions belong to this job, and return something the product can route without a human copying it out of a transcript.

ABC Legal uses similarly narrow jobs. Its event-driven agents start when a job arrives or a court rejects a filing. Each agent has one owner and one job.

The result lands in the legal workflow, Slack, an FTP server, or an approval step. The user benefits from the outcome without first remembering to open an assistant and describe what happened.

Once one narrow task succeeds, the obvious temptation is to let the agent invent the next one. I tried that. I shipped freeform with a followUpTasks field, certain the agent would want to queue its own next steps. Nothing ever read it, and I pulled it three days later. The agent did not need another task list. It needed a way to continue the same work.

Continuation is not another task list

That became continuation: after a task completes, you can ask MoltNet from the console, the moltnet task continue CLI command, or the tasks_continue MCP tool to pick up the work with new instructions.

The default extend mode stays on the parent's branch; fork starts a new branch from the parent's tip. Both carry forward the Pi conversation when it is available. The continuation also inherits how the parent's workspace was prepared instead of quietly switching to a different setup.

Continuation sounds like a small feature until you list what has to survive. I first stored live conversations in a local SQLite database. It worked as long as the same daemon was still there. A crash or another machine turned continuity back into a fresh session.

The larger fix was to move durable runtime state to the team. I deleted the local SQLite package. Pi conversations now persist to a team-scoped, S3-compatible object store after each turn; another daemon can reload the conversation and check out the relevant branch. The daemon went back to being an executor. The team became the owner of continuity.

Figure 4 — a continuation carries the previous conversation into a task that either extends the parent branch or forks from its tip. Durable storage lets another daemon resume it.

When delivery fails, what should you retry?

Continuation preserves useful state after one piece of work succeeds. A failed task made me answer the less convenient version of the same question: which state should survive when the work does not?

Promise Theory tells us to expect that failure. DBOS, the Postgres-backed durable-workflow library underneath MoltNet, protects the execution itself. It checkpoints completed steps under a stable workflow ID, so a process crash does not erase their progress.

But "retry" means two different things, and confusing them makes headless work much more expensive.

Inside a live Pi session, the agent may submit the wrong output shape or hit a transient provider 503. Ending the whole attempt wastes the context that could fix it. The submit tool now returns a recoverable schema error inside the same session and gives the agent a limited number of corrections. Provider errors get their own limited retries with a delay between them. The agent stays in the session and repairs its promise.

After an attempt has failed, the daemon has a different decision to make: should another attempt run at all? Obvious cases follow deterministic rules. A small classifier agent examines ambiguous failures after their secrets have been redacted. Requeueing requires its explicit retryable: true; the daemon does not guess from an error code.

Choosing the right retry was not enough if the worker could not report it. One failure forced me to break my own authorization rule. I had treated Ory Keto, MoltNet's relationship-based authorization service, as the authority for every permission check. Then distributed traces in Axiom showed valid workers getting 403 responses while reporting heartbeats and completions. The right permission relationships had been written, but the checks could not see them yet. That propagation window could strand work that already held a valid lease.

For completion and heartbeat reports, the active database lease is now authoritative. Both the task and attempt rows must name the agent making the call. Keto still guards the rest of the system, but it no longer gets to interrupt work it has already admitted.

That fixed the authorization failure. Then an evaluation task produced the failure from the opening: the agent completed the scenario but never called the tool that submits its result. I fixed the path, but a manual replay would only prove that one model followed one prompt once.

The failure needed to become a regression test. A live end-to-end suite called evals-v2 now creates exploratory and evaluation tasks through the SDK, sends them through the daemon and Gondolin, and checks that each one finishes through the correct submission tool. That tests the runtime's system instructions and the task brief together.

The agent-daemon CI job runs a focused live prompt test whenever a change affects the runtime. A slower scheduled job repeats the suite across prompts and models.

A regression test tells me that one known path works. It cannot tell me whether the agents are producing worthwhile work at a sensible cost.

The task is the unit you can measure

To answer that, I needed the same boundary the regression test exercises: the task. Without it, we can count commits, tokens, API calls, and chat messages. None tells us whether the agent's work was accepted, how often it failed at the end, or whether better context improved the outcome.

Interactive sessions make this worse. A session ends when you stop prompting, close the terminal, or decide the answer is good enough. One conversation can contain several pieces of work, and your intervention is mixed into the result. You can inspect the transcript, but you cannot cleanly attribute its time and tokens to one accepted outcome.

That ambiguity is tolerable while you are sitting there because you are the missing measurement system. You know when the work started, how much you corrected, and why you accepted the result. Once the agent works headlessly, the product has to retain that evidence: what was attempted, by whom, under which runtime profile, what it produced, and whether the result passed its criteria. If agents are going to do unattended work, measuring their impact is not optional.

A structured task has an offer, an attempt, an actor, a result, and an assessment. It gives the work a unit we can measure—and gives the dashboard something more meaningful than activity to count.

Our dogfood dashboard makes it tempting to stop at the green number. Across 378 tasks over 30 days, agents produced 330 accepted outputs: an accepted-output rate of 87.3%. All 330 were accepted on the first attempt, while 41 tasks ended in terminal failure. Median time to an accepted result was 5 minutes 53 seconds.

But accepted only means that the result passed its structural validation. It does not mean an independent judge endorsed the quality. And the rest of the dashboard makes the green number less flattering.

The same window contains 1,194 failed tool calls, 354 high-friction attempts, and a median of 46.5 turns per attempt. We call an attempt high-friction when it takes at least eight turns or includes at least three failed tool calls. "It eventually worked" can hide a lot of wasted time and context.

Accepted output, first-attempt acceptance, terminal failure, tool friction, and time-to-accepted stay separate because collapsing them into one "success rate" would hide exactly what I need to improve.

Figure 5 — Thirty days of task outcomes, productivity, friction, and token use. An accepted result is only one part of the story.

The token numbers make the ROI question harder to avoid. These attempts consumed 728,549,829 tokens, or roughly 2.2 million tokens per accepted output. That is not financial ROI yet; the dashboard does not pretend a token count is a currency amount.

It is an operational denominator. Now we can ask whether a prompt change, model, task type, or runtime profile produces more accepted work for the same token budget. That is a better argument than pointing to whichever session looked impressive that morning.

This is close to what Google calls cost per successful task: pair cost with accepted outcomes, because cheap failures are not efficient. Once the task reaches a business workflow, we can go further and compare the accepted result with time saved, avoided rework, or another outcome that matters outside the agent platform.

Scoped tasks also make comparison possible. The same measures can be grouped by tag, task type, claimant, or runtime profile. In the cohort view below, some model-tagged cohorts accepted every task while another accepted none. The samples are too small to call that a leaderboard, but they are large enough to tell me where to run the next controlled comparison.

Figure 6 — The same task measures split into cohorts. Small samples are leads for the next experiment, not conclusions.

And knowledge leverage—the number of diary searches, diary reads, and pack retrievals per accepted task—is still zero in these cohorts. I built a knowledge factory, then built a measurement layer that caught my own agents not using it. That is uncomfortable evidence, which is exactly why we need the metric.

This is the practical difference between an interactive session and headless agent work. A chat gives you a transcript. A task gives you a boundary around the promise, the attempt, the cost, and the verdict. With that boundary, we can improve the system instead of judging it on vibes.

Software is only where I learned the pattern. A content workflow can combine internal knowledge with external research, produce a sourced brief, and stop for an editor's decision.

In Clairon, we use the same shape for a maintenance request, a water-quality alarm, or a recurring energy review. The agent gathers only the evidence it is allowed to see and returns the next useful check. The operator keeps the decision. Different field, same contract: bounded context, durable evidence, and a human checkpoint.

This narrower shape can feel less ambitious than putting a general assistant in front of every user. I think it is more useful. The agent handles the uncertain step; deterministic code carries the result forward; a person returns where judgment or authority is required.

Some work should remain interactive because the problem is ambiguous and the conversation is the product. But the repeatable parts should not depend on the user staying in that conversation. They should react to the product, do their work, and leave the result where the user already is.

The evaluation agent from the opening had done useful work. It had also failed to produce anything the product could trust or route. Both statements were true. Once I stopped treating the transcript as the outcome, the rest of the system had somewhere to attach: identity, claims, attempts, retries, artifacts, assessment, and cost.

The user may never watch the agent reason. They notice that the rejected filing already has a diagnosis, the maintenance alarm already has a sourced brief, or the pull request already has a review. That is the point.

MoltNet is open source. I have shown you where I put the boundary. If you're building this differently, show me yours.

← Back to all posts