Why 'do this for all fifty' stalls partway through, and the loop that doesn't

You handed the assistant a list of fifty clients and one instruction to apply to each of them. It stopped at around a dozen, with no error and no explanation, just fewer results than records you gave it.

Why rewording won't fix this

A single prompt holding fifty records competes for the same limited attention and output budget a single long document does, and past some size a model starts summarizing or quietly dropping records rather than finishing the list, with nothing in its answer flagging that it quit early. One call also means one failure point: if record 13 trips something odd in the model's response, there's no log of which one failed and no way to retry just that one. Wording the instruction more forcefully doesn't change either limit, because neither is about how clearly the task was stated.

Call the model once per record

Loop over the records in code, and call the model once per record instead of once for the whole batch. Each call is small enough to get the model's full attention, and each one succeeds or fails on its own instead of dragging the other forty-nine down with it.

Fifty calls, not one. A failure on client 13 never touches the other 49.

function draftRenewalReminders(clients) {   // clients: [{ id, policyNumber, renewalDate }]
  const drafted = [];
  const failed = [];

  for (const client of clients) {
    try {
      const draft = callModel(buildPrompt(client));
      drafted.push({ id: client.id, draft });
    } catch (err) {
      failed.push({ id: client.id, reason: err.message });
    }
  }

  return { drafted, failed };   // every client accounted for, one way or the other
}

What happens when one record fails

Wrap each call so a single record's failure doesn't take down the batch, and keep a list of which ids failed and why instead of letting them disappear into a gap in the output. That failed list is what you retry, not the whole book.

It's also the first thing worth checking once a run finishes: a batch with forty-seven drafted and three failed is a different situation than a batch with forty-seven drafted and three never attempted, and only the failed list tells you which one happened.

The same ceiling, a different shape

This is the same attention limit the playbook on reading a long document describes for pages, applied to records instead: a model's attention across a long input thins out well before the input runs out, no matter how the instruction is worded. Splitting by record instead of by page is the same fix wearing different clothes.

See also: Missed a clause in a long document

What a loop costs you

Fifty calls take longer and cost more than hoping one big call handles the whole book, and that trade is real. It buys a version that actually reaches every record and names, by id, the ones it couldn't. A single call that silently stops at twelve never tells you anything was missed at all, so the extra cost is buying information you didn't have before, not just extra calls.

And if these fifty drafts are headed out to clients once they're ready, finishing the loop is only half of it. The playbook on a human approval step covers the staged review those drafts still need before anything sends.

See also: Draft that went out before anyone looked at it

What stays manual

  • Deciding what happens to a client missing a required field before the loop reaches them.
  • Spot-checking a sample of the drafts, not all fifty, before anything moves forward.
  • Any judgement call that spans the whole book, like which clients to prioritize first.