Design the turn, not the demo

A user sent "check the dates" and, ten seconds later, "also compare the refund rules." The first model call was still running. The second message synchronized before the first reply committed, so a naive history reload placed both user messages before that reply. The system appeared to answer both, yet the second work item remained pending and later produced a duplicate comparison.

The bug was not in the model. The system had never defined which messages belonged to a turn.

A turn is a transaction-sized idea

pending messages
      |
      v
claim exact IDs -----> context snapshot
      |                      |
      v                      v
logical turn ----------> model loop
      |                      |
      v                      v
attempts <----------- tool operations
      |
      v
reply + statuses commit together

A turn starts with an explicit claim and ends with an explicit durable outcome. Conversation history is supporting context. It is not the work queue.

Terms for turn design

• Claim: The database transition that assigns pending inputs to an attempt.
• Pending set: The exact request IDs included in a turn.
• Logical turn: One user-visible unit of work that may have several automatic attempts.
• Attempt: One execution of a logical turn.
• Injected input: A synthetic user input from a schedule, event, or approval.
• Terminal outcome: Completed, failed, paused, or exhausted.
• Tool iteration: One provider response followed by zero or more tool results.

Bind the final user input to claimed rows

Loading the latest conversation history at execution time is necessary but insufficient. Messages can arrive after the claim. Build context in two parts:

1. history up to the turn boundary, excluding claimed messages;
2. the claimed messages appended as the final user input.

If several messages were batched, preserve their order and timestamps in a small wrapper:

function renderPending(
  rows: Array<{ createdAt: number; content: string }>
): string {
  if (rows.length === 1) return rows[0].content;
  const lines = rows.map(
    (row) =>
      `[${new Date(row.createdAt).toISOString().slice(11, 16)}] ${row.content}`
  );
  return `## Unanswered messages\n\n${lines.join("\n\n")}`;
}

The timestamps help with phrases such as "that one" when messages arrive close together. They also make the batching visible to the model without inventing separate roles.

New messages that arrive during execution stay pending. The worker processes them after the current turn commits. This rule is less magical than continuously refreshing context, and far easier to reason about.

Give retries a stable identity

An automatic retry is another attempt at the same logical turn. A later user click on "resume" is usually a new logical turn because the user has observed the previous outcome and may supply new context.

Derive a logical-turn ID from stable claimed work:

import { createHash } from "node:crypto";

export function logicalTurnId(ids: readonly string[]): string {
  return createHash("sha256")
    .update([...ids].sort().join("\n"))
    .digest("hex");
}

Keep a random run ID for each attempt. The pair supports two questions:

• Did the same failure repeat during automatic retry?
• Did the failure recur across later user-visible turns?

Those are different signals. Repeated failures inside one retry loop may show a transient provider problem. The same structural failure across several logical turns often indicates a bad tool contract or missing capability.

Bound the loop in several dimensions

An iteration limit alone is weak. A turn can consume a large prompt on every iteration and stay under the count. A cost limit alone may fail when a provider omits prices. A wall-clock limit alone lets a fast loop create many side effects.

Use several independent bounds:

type TurnBudget = {
  maxIterations: number;
  maxToolCalls: number;
  maxCostUsd: number;
  deadlineMs: number;
};

function shouldStop(
  used: { iterations: number; tools: number; costUsd: number; elapsedMs: number },
  budget: TurnBudget
) {
  if (used.iterations >= budget.maxIterations) return "iterations";
  if (used.tools >= budget.maxToolCalls) return "tool_calls";
  if (used.costUsd >= budget.maxCostUsd) return "cost";
  if (used.elapsedMs >= budget.deadlineMs) return "deadline";
  return null;
}

When a soft threshold is crossed, compact older tool exchanges before the next request. When a hard threshold is crossed, pause before another tool call. Keep completed changes and tell the user how to resume.

Pausing is often better than failing. A long coding turn may have already produced a valid draft. Throwing away that state because the next read would exceed the budget punishes useful work.

Completion needs a durable definition

A provider returning text does not mean the turn completed. The text may follow a failed tool call, a malformed protocol response, or an exhausted loop.

Define terminal outcomes in application terms:

• Completed: The model produced an acceptable final response and required commits succeeded.
• Failed: The turn cannot proceed safely and the user receives a visible error.
• Paused: Durable partial work exists, but a budget or approval stops further execution.
• Exhausted: The model used the iteration allowance without reaching a final result.

Record the final model, attempt number, iteration count, tool-call count, tokens, cost, and fallback count. These values explain behavior later without storing private prompts.

Decisions and alternatives

Batch rapid messages for a short fixed window. Calling the model for every keystroke-like follow-up wastes money and produces crossed replies. A debounce of roughly one second is often enough. A long debounce makes the interface feel unresponsive, so keep it bounded.

Serialize within a conversation. Parallel turns can work for independent jobs, but ordinary chat contains references to previous replies. Parallel execution turns ordering into guesswork.

Do not merge messages that arrive mid-turn. Dynamic context refresh sounds helpful. Most provider APIs do not support it cleanly, and tool decisions already made cannot account for the new input. Queue it.

Use explicit provider loops. Convenience runners reduce code, but production needs control over unknown tools, malformed arguments, compaction, fallback, and exhaustion. An explicit loop earns its extra lines.

Stop retries after side effects. If a tool changed the world, replaying the entire turn is unsafe. Post a visible failure with the completed operation record. The next turn can inspect state and continue.

Failure patterns

The model may repeat the same invalid tool call unchanged. Detect a canonical signature of tool name plus normalized arguments. After a small repeat count, return a strategy-change instruction or stop the turn.

A provider may return a tool name that is not registered. Treat it as a protocol failure, not a process crash. Record the model and iteration because model-specific routing may be the cause.

A fallback model may receive a reconstructed prompt that drops tool results. Preserve the same transcript when moving to another compatible model. If protocol families differ, do not pretend replay is safe.

A scheduled turn may inherit interactive assumptions. It has no pending user row and perhaps no open browser. Represent its instruction as injected input, give it a stable logical ID derived from the schedule occurrence, and select tools for isolated execution.

Turn review checklist

• A turn claims exact request IDs.
• Claimed messages become the final user input.
• Messages arriving during execution remain pending.
• Automatic attempts share a logical-turn ID.
• Each attempt has its own run ID and attempt number.
• Conversations serialize turns.
• Iteration, tool, cost, and time budgets are independent.
• Soft limits compact; hard limits pause before another side effect.
• Unknown tools and malformed arguments become recorded protocol failures.
• Retry policy changes once any side-effecting tool succeeds.
• Completion, failure, pause, and exhaustion are distinct durable outcomes.

The boundaries in Draw the trust boundaries constrain authority; a well-defined turn constrains time and input. The next chapter builds the durable path through browser writes, server commits, crashes, and restarts. Continue with Outboxes before agents.