DEV Community: Taylor Lin

Admission Control for Free-Tier Coding Agents: A Glossary, a Decision Tree, and Four Worked Leaves

Taylor Lin — Mon, 09 Nov 2026 20:54:53 +0000

Illustrative scenario, not a case study. You have a backlog of forty tasks and a hosted coding agent that runs on a free tier. Someone suggests sending the whole backlog through it over a weekend. Nothing in this post describes a real deployment, and none of the code below was executed — it is a harness you can adapt, drawn to show where the decision points are.

The useful question is not "how many tokens do we have left?" It is "which tasks deserve to be sent at all?" A free allowance is a budget for experiments, and budgets are spent fastest on tasks you cannot check.

Glossary: six terms worth fixing before you argue

Term	Working definition	Why it changes the decision
Reversal cost	Work needed to undo the change: `git revert`, a data migration, a credential rotation, a customer email	Anything above "revert a commit" disqualifies the task from unsupervised runs
Verification latency	Wall-clock time from "agent finished" to "a machine says pass/fail"	If it exceeds a few minutes, you will skip it, and an unverified pass is noise
Blast radius	Set of files, services, or users a wrong answer can touch	Bounded by a file allowlist, not by good intentions
Context dependency	How much private or tacit knowledge the task needs that you cannot paste into a prompt	High dependency turns into guesswork with confident prose
Deliverable contract	The exact command and the exact expected outcome that defines "done"	Without one, "done" becomes a human opinion
Task card	A small machine-readable record: goal, base commit, allowlist, acceptance command	Makes admission checkable instead of conversational

The tree: four questions, asked in order

Ask these in sequence and stop at the first hard stop. The order matters — reversibility first, because it is the cheapest thing to establish and the most expensive thing to get wrong.

Is the change reversible without a data migration or credential rotation? No → Leaf D.
Can a machine decide pass/fail in a couple of minutes with a command you already trust? No → Leaf C.
Can you bound the scope with a file allowlist and one acceptance command? No → shrink the task; if it still will not fit → Leaf C.
Does the task need context you cannot paste (private docs, tribal knowledge, production state)? Yes → Leaf C. Otherwise, if a failing test can be written first → Leaf B, else → Leaf A.

Four leaves. Each one gets a concrete artifact.

Leaf A — admit as-is, with an allowlist guard

The task is reversible, checkable, and bounded. The artifact you need is not a better prompt; it is a guard that fails the run when the diff escapes the agreed scope.

Task card first (illustrative):

# task.yml — illustrative task card, not a real ticket
id: T-1042
goal: add a retry cap to the webhook sender
base: origin/main
allow:
  - "src/webhooks/*.py"
  - "tests/webhooks/*.py"
acceptance: "pytest tests/webhooks -q"

Then the guard (illustrative, unexecuted):

# check_scope.py — illustrative; adapt before trusting it.
import subprocess, sys, fnmatch, yaml  # PyYAML

card = yaml.safe_load(open("task.yml"))
changed = subprocess.check_output(
    ["git", "diff", "--name-only", f"{card['base']}...HEAD"], text=True
).split()

outside = [f for f in changed
           if not any(fnmatch.fnmatch(f, p) for p in card["allow"])]
if outside:
    print("BLOCKED: files outside allowlist:")
    for f in outside:
        print("  -", f)
    sys.exit(1)
print(f"OK: {len(changed)} changed files, all inside allowlist")

Two honest checks of the guard itself: run it on a branch that only touches allowed paths and expect exit code 0; add a stray file such as README.md and expect exit code 1. If both hold, the guard does what the comment claims.

Leaf B — admit with a contract written first

Here you can verify, but only if the check exists before the agent starts. Write the failing test yourself, commit it, then hand over. This converts a subjective review into an exit code.

# tests/webhooks/test_retry_cap.py — illustrative; written before the change,
# expected to fail until the retry cap exists.
import pytest
from src.webhooks.sender import send_with_retries

def test_retry_attempts_are_bounded(monkeypatch):
    calls = {"n": 0}
    def fake_post(*args, **kwargs):
        calls["n"] += 1
        raise TimeoutError("simulated")
    monkeypatch.setattr("src.webhooks.sender._post", fake_post)
    with pytest.raises(TimeoutError):
        send_with_retries("https://example.invalid", max_retries=3)
    assert calls["n"] == 3

example.invalid is a reserved domain, so the test cannot accidentally reach a real endpoint. The assertion is on the number of attempts, not on timing — timing assertions are the fastest way to build a flaky gate.

Leaf C — human-in-the-loop, draft only

The task is real work but fails the machine-check test, usually because "correct" is a design judgment, a visual result, or a policy call. Admit it, but change the output type: the agent produces a draft branch, and a person owns the merge.

A minimal rule set that is easy to enforce in review:

The agent's output goes to a branch prefixed draft/ and never to a protected branch.
The PR description must quote the acceptance command and its output, or state plainly that no command exists.
The reviewer's first question is "what would make this wrong and still compile?" — if there is no answer, the draft was not worth reviewing.

This leaf is where free-tier experimentation earns its keep, because the cost of a wrong answer is a closed PR, not an incident.

Leaf D — do not send

Reversibility fails, or the context boundary fails. Typical members: schema migrations, credential rotation, anything touching billing, infra/ directories, files under secrets/, and content you are not permitted to paste into a third-party service.

Keep the refusal executable rather than cultural:

# deny_paths.sh — illustrative; run before any agent invocation.
if git diff --name-only origin/main...HEAD | grep -qE '^(infra/|\.env|secrets/)'; then
  echo "BLOCKED: diff touches sensitive paths"
  exit 1
fi
echo "OK: no sensitive paths in diff"

A denylist is a backstop, not a policy. If your repository relies on this script as the only protection for sensitive data, fix the repository layout first — the script cannot see what the agent reads before it writes.

Where a free tier fits, and what it does not fix

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The admission tree above needs somewhere to run the guard and the acceptance command, and it needs an agent to produce the diff in the first place. MonkeyCode is an open-source project whose operator-supplied description includes free model access and a free server option; this account's operator also cites an allowance on the order of ten million tokens. Treat all three as product claims rather than measurements — quotas, model availability, and hosting terms change, so confirm the current numbers in the project's own documentation before you plan a weekend around them.

The mechanical fit is narrow and worth stating precisely: put the verification step — check_scope.py, the contract test, the denylist check — on the free server so that agent branches and human branches pass through the same gate. Identical checks in two places is how "it worked on my laptop" gets eliminated. What a free tier does not do is shrink Leaf D, fix flaky tests, or make an unverifiable task verifiable. A larger allowance mostly buys you more attempts at tasks you have not scoped.

Limits and who should skip this

Flaky suites break the gate. If pytest tests/webhooks -q fails one run in five on main, verification latency is undefined and Leaf B collapses into Leaf C.
The allowlist is only as good as the glob. src/webhooks/*.py does not match src/webhooks/v2/, and fnmatch treats * more loosely than you may expect. Test the guard with the escape case, not just the happy path.
Regulated or customer data. If you cannot lawfully paste the context, the task belongs in Leaf D regardless of how clean the diff would be.
Performance and migration work. Needs runtime evidence the harness does not produce; send it to a person with a profiler.
Teams without a trusted test command. Build that first. An agent on top of an untrusted suite just produces faster unverified diffs.

A verification pass you can repeat

Write three task cards: one that should land in Leaf A, one in Leaf B, one in Leaf D.
Run check_scope.py against a clean branch and confirm exit 0; add an out-of-scope file and confirm exit 1.
Run deny_paths.sh against a branch touching infra/ and confirm it blocks.
Run the acceptance command on a branch with no change and confirm it passes — a gate that fails on main is a gate you will disable.
Only then let an agent produce a diff, and record which leaf the task landed in.

If you try the project, keep the least glamorous piece: the task card and the scope guard. They are the parts that still work when the token budget runs out.

The Quota Wall: A Glossary and a Decision Tree for Agent Steps That Hit Free-Tier Limits

Taylor Lin — Sun, 08 Nov 2026 19:36:07 +0000

Consider this trace from a nightly agent job. It is a composite example, not a customer report, and the timings and counts are illustrative — but the shape is common.

03:40:02  step=summarize_prs    attempt=1  -> 200 OK   input_tokens=1180
03:40:07  step=write_changelog  attempt=1  -> 429      "rate limit: 20 rpm"
03:40:07  step=write_changelog  attempt=2  -> 429      (no sleep)
03:40:08  step=write_changelog  attempt=3  -> 429
03:40:08  job -> FAILED  "upstream error"

Three separate mistakes are packed into five seconds. Retries fired with no delay, the retry budget was spent on a condition that needed waiting rather than repeating, and the final error message threw away the only evidence that mattered — the response body.

Free model tiers make this failure mode routine, because "free" usually means "shared and rate-limited," not "unlimited." Below is the vocabulary I use to talk about it, a decision tree for choosing a response, and a dependency-free harness you can run in a few seconds against your own policy.

Glossary: six terms that decide everything

Admission — whether to send the request at all. Admission happens before the call: queue depth, tokens spent this hour, concurrency.
Soft throttle — a rejection that clears on its own within seconds or minutes. The classic 429 with a Retry-After header.
Quota wall — a limit that will not clear inside your deadline: a daily cap, exhausted credit, a disabled key. Retrying a quota wall is how a two-minute outage becomes an all-night loop.
Silent degradation — a 200 OK whose content is worse: truncated output, a dropped tool argument, a substituted smaller model. Status-code-only handling never sees it.
Replayability — whether the step can run again without duplicating a side effect (a comment posted twice, a ticket opened twice).
Deadline slack — remaining wall-clock budget minus the expected cost of one more attempt. When slack hits zero, every remaining branch is "fail."

Two derived terms get conflated and should not be: a tier switch moves the step to a different backend (another free model, a server you run yourself, a paid plan), while a retry means the same backend, later. Different costs, different failure modes.

The decision tree

Evaluate in order. The order is the policy; moving one branch changes behavior for every step.

[0] classify(response)  -------------------------- never decide on status code alone
 |
 +-- OK ...... commit
 |
 +-- SOFT_THROTTLE ... attempts left? slack > 5s? -- yes --> A: capped backoff + retry
 |                                                 \- no  --> D: fail loud (keep the body)
 |
 +-- TRUNCATED / SCHEMA_INVALID ... input over 60% of budget? -- yes --> B: degrade, retry once
 |                                                            \- no  --> D
 |
 +-- HARD_QUOTA ... has side effects? -- yes --> E: park for human
 |                  \- no ... replayable + tier left? -- yes --> C: switch tier, replay
 |                                                     \- no  --> D
 |
 \- UNKNOWN ... D: fail loud

Step 0 is classification, and it is the step people skip. Do it from three signals together: HTTP status, response body text, and finish_reason from the payload.

The artifact: `quota_triage.py`

Stdlib only, Python 3.10+, no network. Three parts: classify, decide, and a table-driven test with fake responses.

from __future__ import annotations
from dataclasses import dataclass, replace
from enum import Enum
import random

class Outcome(str, Enum):
    OK = "ok"
    SOFT_THROTTLE = "soft_throttle"
    HARD_QUOTA = "hard_quota"
    TRUNCATED = "truncated"
    SCHEMA_INVALID = "schema_invalid"
    UNKNOWN = "unknown"

def classify(status: int, body: str, finish_reason: str | None, parsed_ok: bool) -> Outcome:
    b = (body or "").lower()
    if status == 402:
        return Outcome.HARD_QUOTA
    if status == 429:
        # heuristic: some providers also use 429 for the daily cap
        return Outcome.HARD_QUOTA if ("daily" in b or "quota" in b) else Outcome.SOFT_THROTTLE
    if status in (401, 403):
        return Outcome.HARD_QUOTA if "quota" in b else Outcome.UNKNOWN
    if finish_reason == "length":
        return Outcome.TRUNCATED
    if 200 <= status < 300:
        return Outcome.OK if parsed_ok else Outcome.SCHEMA_INVALID
    return Outcome.UNKNOWN

The context object and the policy. Note that park_for_human is checked before the tier switch: a write with side effects never rides a replay branch, even when a second backend is idle.

@dataclass(frozen=True)
class Step:
    attempts_left: int
    input_tokens: int
    token_budget: int
    tiers_left: int            # e.g. 1 = one free-server leaf not tried yet
    replayable: bool
    has_side_effects: bool
    slack_s: float

def decide(o: Outcome, s: Step) -> str:
    if o is Outcome.OK:
        return "commit"
    if o is Outcome.SOFT_THROTTLE and s.attempts_left > 0 and s.slack_s > 5:
        return "backoff_retry"
    if o in (Outcome.TRUNCATED, Outcome.SCHEMA_INVALID) \
       and s.input_tokens > 0.6 * s.token_budget and s.attempts_left > 0:
        return "degrade_then_retry"
    if o is Outcome.HARD_QUOTA and s.has_side_effects:
        return "park_for_human"
    if o is Outcome.HARD_QUOTA and s.replayable and s.tiers_left > 0:
        return "switch_tier_replay"
    return "fail_loud"

def backoff_delay(attempt: int, base: float = 1.0, cap: float = 30.0) -> float:
    raw = min(cap, base * (2 ** attempt))
    return raw * (0.5 + random.random() / 2)   # jitter to 50-100% of raw

The test table is the part worth copying. It costs nothing to run and it fails loudly when your policy drifts.

BASE = Step(attempts_left=2, input_tokens=1200, token_budget=8000,
            tiers_left=1, replayable=True, has_side_effects=False, slack_s=90)

CASES = [
    # status, body,                  finish, parsed_ok, step,                                      want
    (200, "",                       "stop", True,  BASE,                                      "commit"),
    (429, "rate limit: 20 rpm",     None,   False, BASE,                                      "backoff_retry"),
    (429, "daily quota exceeded",   None,   False, BASE,                                      "switch_tier_replay"),
    (429, "daily quota exceeded",   None,   False, replace(BASE, has_side_effects=True),      "park_for_human"),
    (200, "",                       "length", True, replace(BASE, input_tokens=5200),         "degrade_then_retry"),
    (200, "",                       "stop", False, replace(BASE, input_tokens=5200),         "degrade_then_retry"),
    (429, "rate limit: 20 rpm",     None,   False, replace(BASE, slack_s=2),                  "fail_loud"),
    (500, "internal",               None,   False, BASE,                                      "fail_loud"),
]

if __name__ == "__main__":
    fails = 0
    for status, body, fin, ok, step, want in CASES:
        got = decide(classify(status, body, fin, ok), step)
        if got != want:
            fails += 1
            print(f"FAIL {status} {body!r:28} -> {got} (want {want})")
    print(f"{len(CASES) - fails}/{len(CASES)} policy cases pass")
    raise SystemExit(1 if fails else 0)

Run it with python quota_triage.py. Two caveats: the harness proves your policy, not your provider — it never touches the network — and the Step numbers are placeholders you should replace with your job's real budget.

Decision table: what each leaf costs when it is wrong

Signal	Leaf	Cost of a wrong call
`429` + `Retry-After`	A: backoff, retry	Burns deadline slack; harmless if bounded
`429` + "daily"/"quota"	C or E	Retrying a wall you cannot clear, all night
`401/403` + "quota"	C or E	An expired key sends you to a tier switch that also fails
`finish_reason=length`	B: degrade	Retrying unchanged truncates again — pure spend
`200` + missing field	B or D	Committing poisoned data downstream

Worked leaves

Leaf A — soft throttle: wait once, with a stated cap

backoff_delay(0) returns 0.5–1.0s; backoff_delay(4) returns 15–30s. Cap at three attempts and log the delay you actually slept, so a postmortem can tell "the provider was slow" apart from "our backoff was too short."

Leaf B — degraded payload: shrink before you repeat

Illustrative arithmetic: a step with a 10,000-token budget sent 8,600 tokens and came back with finish_reason=length. Since 8600 > 0.6 * 10000, degrade_then_retry fires. Degrade means drop optional tool definitions, cut retrieved context to top-k, and lower the output max_tokens. One retry, then fail_loud.

Leaf C — quota wall: switch tiers instead of sleeping

switch_tier_replay is the leaf where a second free backend earns its place: the same step runs against a different quota pool, so the replay is testable without a credit card. This is where MonkeyCode's free model access and free server option fit my workflow — two separate pools to alternate between, with the harness above deciding when to move. Treat availability as operator-supplied and re-read the project page for today's limits; any number quoted in a blog post ages badly.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Leaf D — side effects: park, do not replay

If a step writes — posts, files, charges — a quota wall returns park_for_human even when replayable=True, unless the write carries a dedupe key. The queue entry should hold the request body hash, the attempt count, and the raw error string.

Limitations and who should not use this

Body-text sniffing is a heuristic. Providers reword errors; keep classify as a patchable seam and log raw bodies for a week after any provider change.
429 is not standardized. Some backends use it for daily caps, some for per-minute limits, some for concurrency.
Latency, output-quality drift between tiers, and cost are out of scope for the harness.
"It worked on the second backend" is not a dedupe strategy. Never replay a non-idempotent write on a tier switch.

Skip this approach if your p99 requirement is tighter than your retry budget, if you need a contractual uptime guarantee, if your data cannot leave a region you can verify, or if writes dominate the workload. A decision tree does not fix a shared queue.

A five-minute check before trusting any policy

Paste three real error bodies into CASES. Real strings beat ten guesses.
Run the harness. Every failing case is a policy you were already running silently.
Set two hard caps: attempts per step, and steps per job.
Log outcome, attempt, delay_slept, and the raw body on every failure.

If you want to exercise Leaf C before spending anything, wire the harness to a second free pool first — the MonkeyCode project page is where I would start for that, and the disclosure above is the only reason it appears here.

Local, Ephemeral, or CI: A Placement Decision Tree for Agent Experiments

Taylor Lin — Sat, 07 Nov 2026 18:17:00 +0000

Local, Ephemeral, or CI: A Placement Decision Tree for Agent Experiments

Last Friday I had forty prompts to re-run against a changed prompt template. The suite took eleven minutes on my laptop, which meant the fan spun up, two browser tabs died, and I spent the whole time afraid to touch the keyboard. CI could have run it, but pushing forty throwaway iterations through a shared pipeline felt rude. A scratch box somewhere in the cloud was the obvious answer, except I had no idea which of the three places the job actually belonged in, and I had already wasted a morning on that same question the week before.

The DEV front page this week was full of community posts — Hacktoberfest wrap-ups, a goodbye post, a long thread about AI replacing us. Good reading, none of it answered my question. So I sat down and wrote the question out as a tree instead, once, so I would stop re-litigating it every Friday.

This article is that tree. It is about placement, not about models: given a job spec, which execution environment should run it, and why.

Glossary first, because the words do the work

Placement. The environment that executes a step or a batch of steps. Not the framework, not the model — the box.

Ephemeral runner. A machine you create for one job, run, and throw away. No state survives, which is either the point or the problem.

Reproducible from spec. Someone else can re-run the job from committed files alone — no manual steps, no uncommitted environment tweaks, no "oh, I had a .env locally".

Cold-start cost. Everything between "go" and the first useful output: image pull, dependency install, dataset download, credential exchange.

Environment drift. The gap between the machine where a job passed and the machine where it fails. Drift is invisible until you compare two runs.

Billable versus wall-clock time. CI bills queue-plus-run; a laptop bills attention; a scratch box bills neither but may disappear mid-job.

The tree

Q1. Does the job need to reach a host that only exists inside your private network?
    yes -> LEAF A (local, or CI with a network path)
    no  -> Q2

Q2. Does the job need a GPU, or more than a few GB of input data?
    yes -> LEAF D (dedicated machine; not a scratch box)
    no  -> Q3

Q3. Can the job be re-run from committed files in under five minutes?
    no  -> LEAF A (local, while you make it reproducible)
    yes -> Q4

Q4. Does the job need long-lived secrets it should not hold on a throwaway box?
    yes -> LEAF C (CI with scoped credentials)
    no  -> Q5

Q5. Will the run take more than ~45 minutes of wall clock?
    yes -> LEAF C (CI; survivable, resumable, logged)
    no  -> Q6

Q6. Is the output something a second person must open, link, or gate on?
    yes -> LEAF C (CI as the gating run)
    no  -> LEAF B (ephemeral runner)

The 45-minute number is a default, not a law. Set it to the point where you would stop watching the terminal. Everything above assumes the job is already spec-reproducible; if it is not, the tree routes you to fix that first, which is the single most useful outcome it can produce.

Worked leaves

Leaf A — local, deliberately

You land here when the job touches an internal host, or when it cannot be re-run from committed files. The second case deserves emphasis: placement is not the bug, reproducibility is. Run it locally, capture the manual steps as you go, and re-enter the tree tomorrow.

# Capture the environment you actually used, not the one you meant to use.
python -m pip freeze > runs/manual-$(date +%F)/frozen.txt

Leaf B — ephemeral runner

Short jobs, no private network, no long-lived secrets, output for your own eyes. This is the case an ephemeral box was designed for: you want a clean environment, you do not want to keep it, and the whole thing is over before cold-start matters.

# Pseudocode shape; substitute your own access details.
rsync -a ./evals ./jobs user@scratch-box:~/work/
ssh user@scratch-box 'cd ~/work && python -m evals.run --spec jobs/eval-small.json'
rsync -a user@scratch-box:~/work/runs ./runs/

This is also the leaf where a free tier earns its keep. MonkeyCode advertises free model access and a free server option, which makes leaf B the cheapest place to burn iterations — pull a scratch box, run the suite, pull the artifacts back, destroy it. Check the project's current terms yourself before you move anything you would mind losing; free tiers change, and I am not your source of truth for what is live today.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Leaf C — CI

The gating run. Anything a second person reads, anything that takes long enough that you would close the laptop, anything holding a scoped credential. Keep CI boring: one entry point, one spec file, artifacts uploaded.

# The shape that matters: spec in, artifacts out, no interactive step.
- run: python -m evals.run --spec jobs/eval-small.json --out runs/ci.jsonl
- uses: actions/upload-artifact@v4
  with:
    name: eval-run
    path: runs/ci.jsonl

Leaf D — dedicated machine

Large data, GPU, or a model too big for a scratch box. Expensive enough that placement should be a scheduled decision, not a reflex. Do not explore here; explore in leaf B and promote the winning spec.

The artifact: a classifier that ends the argument

One file, standard library only. It encodes the tree above and prints the reasons, so the next Friday debate takes four seconds instead of a morning.

#!/usr/bin/env python3
"""place.py - classify an experiment job spec into a runner placement.

Standard library only. Rules are ordered; the first match wins.
The thresholds are defaults; tune them to your own tolerance.
"""
from __future__ import annotations

import json
import sys
from dataclasses import dataclass

LONG_JOB_MIN = 45
BIG_DATA_GB = 5.0


@dataclass
class Job:
    name: str
    wall_clock_min: int
    needs_private_network: bool
    needs_gpu: bool
    data_gb: float
    reproducible_from_spec: bool
    holds_long_lived_secrets: bool
    shared_artifact: bool


def place(job: Job) -> tuple[str, list[str]]:
    why: list[str] = []
    if job.needs_private_network:
        why.append("needs reachability into a private network")
        return "A: local-or-network-path-ci", why
    if job.needs_gpu or job.data_gb > BIG_DATA_GB:
        why.append(f"hardware/data floor (gpu={job.needs_gpu}, data_gb={job.data_gb})")
        return "D: dedicated-machine", why
    if not job.reproducible_from_spec:
        why.append("cannot be re-run from committed files yet")
        return "A: local-until-reproducible", why
    if job.holds_long_lived_secrets:
        why.append("would need long-lived credentials on a throwaway box")
        return "C: ci-scoped-creds", why
    if job.wall_clock_min > LONG_JOB_MIN:
        why.append(f"wall clock {job.wall_clock_min}m exceeds {LONG_JOB_MIN}m")
        return "C: ci-long-run", why
    if job.shared_artifact:
        why.append("output must be opened or gated on by someone else")
        return "C: ci-gating-run", why
    why.append("short, reproducible, self-contained, private output")
    return "B: ephemeral-runner", why


def main() -> int:
    if len(sys.argv) != 2:
        print("usage: place.py jobs/eval-small.json", file=sys.stderr)
        return 2
    with open(sys.argv[1], encoding="utf-8") as fh:
        payload = json.load(fh)
    job = Job(**payload)
    placement, why = place(job)
    print(f"{job.name} -> {placement}")
    for line in why:
        print(f"  - {line}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

A spec to feed it:

{
  "name": "eval-small",
  "wall_clock_min": 11,
  "needs_private_network": false,
  "needs_gpu": false,
  "data_gb": 0.2,
  "reproducible_from_spec": true,
  "holds_long_lived_secrets": false,
  "shared_artifact": false
}

Applying the rules by hand, this spec lands on B: ephemeral-runner with the reason line "short, reproducible, self-contained, private output". Flip shared_artifact to true and it moves to C: ci-gating-run. Set reproducible_from_spec to false and it drops to A regardless of the other fields, which is the tree enforcing its own precondition. The outputs above are derived from the rules, not from a benchmark — the script is small enough that hand-checking is the honest verification.

A test plan for the placement itself

Placement decisions rot quietly. Three checks keep them honest:

Determinism check. Run the same spec twice on the same leaf and diff the output hashes. A diff means the job was never reproducible and the tree lied to you.
Drift check. Capture pip freeze (or your lockfile) on both runs and diff them. Empty diff is the expected result for leaf B.
Cost guard. Abort if a leaf-B run exceeds its wall-clock budget; that job belongs in leaf C next time, and the guard tells you so before the box disappears.

# Sketch: two runs, two hashes, one comparison.
for i in 1 2; do
  python -m evals.run --spec jobs/eval-small.json --out "runs/leaf-b/run-$i.jsonl"
done
sha256sum runs/leaf-b/run-*.jsonl

python -m evals.run is a placeholder for whatever your entry point is called. The two-run comparison is the part that matters.

Limitations, and who should skip this

This tree assumes jobs are batch-shaped and mostly idempotent. If your work is interactive debugging, you want leaf A and no tree at all. If you handle regulated data, scratch boxes are the wrong tool no matter how short the job is — go straight to leaf C with scoped credentials and an audit trail. If the job needs a GPU, the tree just says "not here" and defers to your capacity planning.

And if you only ever run one command a week, ignore all of this. The overhead of choosing is larger than the cost of choosing wrong.

If you do want to try leaf B without provisioning anything, the free model access and free server option in MonkeyCode are the lowest-friction way in — read the current terms on the project page first, then run the classifier against your own spec and see whether the tree agrees with your gut. When it disagrees, the reason line is usually the interesting part.

Two Writes, One Charge: An Idempotency Decision Tree for Agent Steps

Taylor Lin — Fri, 06 Nov 2026 16:54:50 +0000

Consider a nightly job that walks an invoice queue and calls a payments tool once per invoice. The run takes eleven minutes. In minute nine the worker is restarted — deploy, OOM kill, a dropped connection, the reason does not matter. The retry loop resumes from the last checkpoint and re-executes the step that was already in the air.

Two writes. One charge. One long morning.

None of that is fixed by a better prompt. It is fixed by deciding, per step, what "already done" means — and writing that decision somewhere a retry can read it. The DEV front page this week is full of debate about what agents will replace. The quieter corollary is that agents now touch payments, mailboxes, and shipping labels, where a duplicate is not a bad answer but a real event.

So the question here is narrow. For one given agent step, do you need an idempotency key, a local dedupe ledger, a compensating action, or nothing at all?

The tree first

Q1. Does the step change state outside the run?
    no  ................................ Leaf A: no key, cache if hot
    yes -> Q2

Q2. Does the downstream accept a caller-supplied dedupe key?
    yes ................................ Leaf B: pass the key
    no  -> Q3

Q3. Is the effect reversible, or safe to repeat verbatim?
    yes ................................ Leaf C: local ledger + tombstone
    no  ................................ Leaf D: record intent, compensate

Two rules sit above the tree. First, an unknown or missing effect declaration is treated as the most dangerous class, never the safest. Second, the decision is made from the step's schema and arguments, not from the model's prose about the step.

Glossary, because the words get overloaded

Effect boundary. The edge where a step leaves your process: an HTTP call to a payment API, an SMTP send, an object-store put. Inside the boundary you can retry freely; across it you cannot.
Idempotency key. A stable string that identifies this logical attempt, sent to the downstream so a repeat is recognized as a repeat.
Key scope. Run-scoped keys hash run_id + step_index + tool. Args-scoped keys also hash the canonicalized arguments. Use args-scoped for anything priced, so a corrected amount produces a new key instead of replaying the old charge.
In-flight marker. The row claiming a key before the effect happens. Without it, two workers both see "not done" and both proceed.
Dedupe window vs retry horizon. The window is how long the downstream remembers a key. The horizon is how long your retry loop may still fire. The window must cover the horizon, or you carry a local tombstone past the downstream's memory.
Compensation. A recorded, replayable inverse action for effects that cannot be deduped at all.

Note what is absent: exactly-once. Across a network boundary you get at-least-once delivery plus idempotent handling, which is effectively once only inside a declared window. Writing that window down is most of the work.

Four leaves, worked

Leaf A — read steps. get_invoice(id) changes nothing. Return execute_no_key. A key here only adds a row per call and a failure mode where a stale cached result outlives the data. If the read is expensive, cache on request content with a TTL, which is a different mechanism with different failure behavior.

Leaf B — keyed writes. charge_card against an API that accepts an Idempotency-Key header. Declare x-effect: keyed_write and key_scope: args, claim the key in the ledger, then pass the 32-hex digest downstream. On a duplicate claim with state done, replay the stored result instead of calling again. If you skip the claim and rely on the header alone, a retry that changes the amount silently becomes a second charge, because the header no longer matches.

Leaf C — blind writes that repeat safely. Sending the same receipt email twice is annoying, not catastrophic, and an object-store put to a deterministic key is naturally idempotent. Declare x-effect: blind_write. Claim locally, execute, and keep a tombstone row so a late retry returns replay_stored_result. The tombstone must outlive the retry horizon — this is the leaf where a too-short TTL quietly reopens the hole.

Leaf D — effects you cannot take back. A shipping label, an SMS to a customer, a physical dispatch. There is no key that undoes a truck. Record the intent with a durable row, execute, and on any ambiguity raise a compensation task plus a human-visible flag rather than retrying. Retrying here is the bug.

A reference implementation

Dependency-free, Python 3.9+, stdlib only. It is written for this article and has not been validated against a production store — swap the SQLite file for your own database before you point it at money.

#!/usr/bin/env python3
"""idempotency.py - decision tree + claim ledger for agent steps.

Reference implementation, not production-hardened.
Run: python3 idempotency.py
"""
from __future__ import annotations

import hashlib
import json
import sqlite3
import time

EFFECT_CLASSES = ("read", "keyed_write", "blind_write", "physical")


def canonical(args: dict) -> str:
    return json.dumps(args, sort_keys=True, separators=(",", ":"))


def effect_class(schema: dict) -> str:
    declared = schema.get("x-effect")
    # Unknown or missing declarations fall back to the most dangerous class.
    return declared if declared in EFFECT_CLASSES else "blind_write"


def step_key(run_id: str, index: int, tool: str, args: dict, schema: dict) -> str:
    parts = [run_id, str(index), tool]
    if schema.get("key_scope") == "args":
        parts.append(canonical(args))
    return hashlib.sha256("|".join(parts).encode("utf-8")).hexdigest()[:32]


class Ledger:
    def __init__(self, path: str = ":memory:") -> None:
        self.db = sqlite3.connect(path)
        self.db.execute(
            """CREATE TABLE IF NOT EXISTS ledger (
                   key TEXT PRIMARY KEY,
                   step TEXT NOT NULL,
                   state TEXT NOT NULL,
                   result TEXT,
                   created_at REAL NOT NULL)"""
        )

    def claim(self, key: str, step: str) -> bool:
        try:
            self.db.execute(
                "INSERT INTO ledger(key, step, state, created_at) VALUES (?,?,?,?)",
                (key, step, "in_flight", time.time()),
            )
            self.db.commit()
            return True
        except sqlite3.IntegrityError:
            return False

    def finish(self, key: str, result: dict) -> None:
        self.db.execute(
            "UPDATE ledger SET state=?, result=? WHERE key=?",
            ("done", canonical(result), key),
        )
        self.db.commit()

    def release(self, key: str) -> None:
        """Free a claim after a pre-effect failure so a retry can take it."""
        self.db.execute("DELETE FROM ledger WHERE key=? AND state='in_flight'", (key,))
        self.db.commit()

    def lookup(self, key: str):
        row = self.db.execute(
            "SELECT state, result FROM ledger WHERE key=?", (key,)
        ).fetchone()
        return None if row is None else {"state": row[0], "result": row[1]}


def plan(step: dict, schema: dict, ledger: Ledger) -> tuple:
    kind = effect_class(schema)
    if kind == "read":
        return ("execute_no_key", None)

    key = step_key(step["run_id"], step["index"], step["tool"], step["args"], schema)

    if not ledger.claim(key, step["tool"]):
        seen = ledger.lookup(key)
        if seen and seen["state"] == "done":
            return ("replay_stored_result", seen["result"])
        return ("wait_or_fail_fast", None)

    if kind == "keyed_write":
        return ("execute_passing_key", key)
    if kind == "blind_write":
        return ("execute_behind_local_dedupe", key)
    return ("execute_with_compensation", key)

The caller's contract is three lines: pass key downstream when the action is execute_passing_key, call ledger.finish(key, result) after a successful effect, and call ledger.release(key) when the call failed before the effect happened. If you cannot tell whether the effect happened, do not release — treat it as done and reconcile.

def demo() -> None:
    ledger = Ledger()
    schema = {"x-effect": "keyed_write", "key_scope": "args"}
    step = {"run_id": "r-1", "index": 3, "tool": "charge_card",
            "args": {"amount": 1200, "currency": "usd"}}

    action, key = plan(step, schema, ledger)
    assert action == "execute_passing_key"
    ledger.finish(key, {"id": "ch_1", "status": "captured"})

    action, replay = plan(step, schema, ledger)
    assert action == "replay_stored_result"

    # A corrected amount must not replay the original charge.
    edited = dict(step, args={"amount": 1100, "currency": "usd"})
    action, key2 = plan(edited, schema, ledger)
    assert action == "execute_passing_key" and key2 != key

    # Unknown effect class is treated as the dangerous one, not the safe one.
    assert effect_class({}) == "blind_write"

    # A read never touches the ledger.
    action, _ = plan(dict(step, tool="get_invoice"), {"x-effect": "read"}, ledger)
    assert action == "execute_no_key"

    print("all checks passed")


if __name__ == "__main__":
    demo()

Run python3 idempotency.py. The output is one line, all checks passed. The point of the demo is not coverage — it is that the three decisions a retry can make (execute, replay, wait) are testable without any network, which is what makes this reviewable in a pull request.

The two windows that actually bite

Window one: key scope drift. The same logical charge submitted with {"amount": 1200} and {"amount": 1200.0} hashes differently if you canonicalize carelessly. sort_keys=True plus separators handles ordering and whitespace; it does not handle numeric type coercion or currency casing. Normalize arguments before hashing, or you will accumulate near-duplicate keys that look fine in logs.

Window two: in-flight leases. A worker that crashes after claiming but before finishing leaves in_flight forever, and every retry gets wait_or_fail_fast. Add a lease: store an expiry, and let a claim on an expired row succeed. In the SQL above that is one more column and a conditional UPDATE; leaving it out is the single most common way this pattern deadlocks a queue.

Where a free tier genuinely fits

Two of the three moving parts here are unglamorous and cheap, which is exactly where a free tier earns its place.

Schema classification is a model task. Proposing x-effect for a few dozen tool schemas is a natural model pass: feed the tool description and parameters, ask for one of the four classes plus a one-line reason. Treat the output as a proposal that a human signs off, and keep the unknown-to-blind_write fallback. This is the MonkeyCode use I would actually defend — free model access (operator-supplied; check the current terms yourself) makes iterating on that classification prompt cost nothing, and the artifact it produces is a reviewed table in your repo, not a runtime dependency.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The ledger worker needs no GPU and no model call. At execution time plan() is deterministic string hashing plus one SQL statement. That makes it a plausible candidate for a free server option, with one hard condition: the storage must survive a restart. An in-memory ledger on a recycled instance is worse than no ledger, because it looks like protection. Verify persistence and uptime against your retry horizon before you host anything with a payment path on it.

Who should not use this

If your agent has no external side effects, or your retry horizon is a single process with no restart, this is scaffolding you do not need. If you already run a workflow engine with durable execution, use its semantics instead of hand-rolling a ledger — two dedupe layers with different windows is its own incident. And if a step is genuinely non-reversible and high-value, do not automate it at all; route it to an approval queue and stop pretending retries are the answer.

Pre-flight checklist

Every tool schema declares x-effect and key_scope, or inherits the blind_write fallback on purpose.
Canonicalization is tested with numbers, casing, and key ordering.
Dedupe window ≥ retry horizon + clock skew + longest in-flight duration.
In-flight rows expire; expired claims can be retaken.
Leaves C and D have a human-visible flag, not just a log line.
The ledger store survives a worker restart — tested, not assumed.

If you want to try the schema-classification pass before committing budget to it, a free model tier is a reasonable place to start; just make sure the ledger underneath it is durable before anything reaches a payment call.

Fail Loudly: A Tool-Schema Decision Tree for Agent Steps That Return Wrong Answers Without Failing

Taylor Lin — Thu, 05 Nov 2026 15:32:16 +0000

The bug report is always a variant of the same sentence: the agent ran, nothing threw, and the ticket got filed against the wrong customer.

There is no stack trace because nothing crashed. A field defaulted to null, the tool accepted it, and the model wrote a confident summary on top. Silent failures like this burn more debugging hours than exceptions do, because an exception tells you where to look.

What follows is a decision tree for that exact class of bug. Six glossary terms, five branches, one worked example per leaf, and a linter you can run against your own tool schemas in about a minute. The failure below is a constructed illustration, not a logged incident — the shape is what matters.

Glossary: the six terms this tree depends on

Coercion surface — the number of places where a wrong type is silently converted instead of rejected. '3' becoming 3 is a coercion. 'three' becoming 3 is a guess, and guesses are where silent bugs live.
Sentinel ambiguity — how many separate meanings null, '', 0, and [] carry inside one schema. Two meanings is a bug. Three is a support ticket.
Closed vocabulary — a field constrained to an enum or a regex versus left as a free string. Free strings are where model-invented status values appear.
Echo field — an input value the model must copy into the call verbatim: an ID, a hash, a request token. It is the cheapest correctness probe you will ever add, because a wrong echo means every other field is suspect too.
Error channel — how a tool reports failure back to the model. A tool that returns {'ok': False, 'reason': 'customer_id not found in input'} gives the model something to repair against. A tool that raises into the void gives it nothing.
Required-field density — required fields divided by total fields, per tool. Low density means the model is allowed to leave things out, which it will.

Step 1: Lint the schema before you blame the model

The tree's branch inputs come from a static scan, not from intuition. This script is stdlib-only and runs on Python 3.9+.

# schema_lint.py — no dependencies
import json
import sys


def walk(node, path='$'):
    yield path, node
    for name, sub in (node.get('properties') or {}).items():
        yield from walk(sub, f'{path}.{name}')
    items = node.get('items')
    if isinstance(items, dict):
        yield from walk(items, f'{path}[]')


def lint(schema):
    findings = []
    props = schema.get('properties') or {}
    required = set(schema.get('required') or [])
    total = len(props) or 1
    density = len(required) / total

    if total >= 3 and density < 0.75:
        optional = sorted(set(props) - required)
        findings.append(['R1', f'density {density:.0%} < 75%; omittable: {optional}'])

    for path, node in walk(schema):
        kind = node.get('type')
        if kind == 'string' and 'enum' not in node and 'pattern' not in node:
            findings.append(['R2', f'{path}: open string, no enum/pattern'])
        if kind == 'boolean':
            findings.append(['R3', f'{path}: boolean flag, prefer explicit enum'])
        if node.get('default') is None or kind in ('null', ['string', 'null']):
            findings.append(['R4', f'{path}: null default, sentinel ambiguity'])

    return findings


if __name__ == '__main__':
    with open(sys.argv[1]) as fh:
        schema = json.load(fh)
    for code, msg in lint(schema):
        print(f'{code}  {msg}')

Run it as python schema_lint.py ticket_tools.json. A realistic first pass on a four-tool schema usually prints eight to fifteen lines. That is not a bad schema; that is a schema written for a human caller.

Step 2: Read findings as branch inputs

The codes are not a score. Each one routes you to a branch.

Finding	Question it triggers	Branch
R1 (low density)	Can this call succeed with fields missing?	3 or 4
R2 (open string)	Does the model invent values here?	3
R3 (boolean flag)	Does `false` mean "no" or "unset"?	2
R4 (null default)	How many meanings does null carry?	2
No findings, still wrong	Is the contract right but the run unstable?	5
Output does not parse at all	Is the grammar even enforced?	1

Step 3: Walk to a leaf

Branch 1 — output will not parse → freeze the grammar

Worked leaf: the model returns prose wrapped around JSON, your json.loads throws, you retry three times and pay three times.

Action: enforce a structured-output mode if your endpoint has one. If it does not, wrap the call in an extractor that takes the first balanced {...} and validates it. The leaf ends when parse failures drop below 1 in 50 calls — not to zero, because zero is usually a sign you stopped measuring.

Branch 2 — parses, but the echo is wrong → echo-first contract

Worked leaf: input carries request_id: r-8841, the call comes back with request_id: r-8814 or none. Everything downstream is now attached to a possibly wrong record.

Action: make the echo field required, add it to the required list, and reject the call server-side on mismatch. This converts a silent failure into a loud one for the cost of one string comparison. It is the highest-return change in the whole tree.

Branch 3 — echo right, enum drifted → closed vocabulary

Worked leaf: your schema says status: string, the model sends "resolved-ish" or "Pending Review". Your writer stores it and the dashboard silently buckets it as unknown.

Action: replace the free string with an enum of the values your code actually branches on, and add an error channel entry that returns the allowed set on rejection. Keep the enum short — six values, not sixteen.

Branch 4 — shape right, required values empty → two-phase fill

Worked leaf: customer_id is present and syntactically valid but the model filled it with a plausible-looking value it never saw in the input. R1 flagged this as low density.

Action: split the call. Phase one asks the model to select from candidates you supply; phase two fills the remaining fields with the selection fixed. Two cheap calls beat one confident hallucination.

Branch 5 — everything correct, unstable across runs → snapshot and pin

Worked leaf: the same input yields correct output four times out of five, and you cannot tell which run will be the fifth.

Action: record input, schema hash, and output for twenty runs. If variance is in field selection rather than values, you have a decoding or sampling problem, not a schema problem. Pin what you can, measure what you cannot.

Step 4: Measure the echo rate before and after

One number tells you whether the tree worked:

def echo_match_rate(samples, field='request_id'):
    hits = sum(
        1 for s in samples
        if s.get('args', {}).get(field) == s.get('input', {}).get(field)
    )
    return hits / len(samples)

Record it at 20 samples before the change and 20 after. If the rate does not move, you fixed the wrong branch. Report the number alongside the schema hash, because a schema edit without a version tag makes every later comparison meaningless.

Step 5: Know when to stop

Stop tightening when three conditions hold: echo rate is at or near 1.0, every field your code branches on has a closed vocabulary, and rejections are rare enough that a human reading the error channel would not call it noise. A schema with zero rejections is not safer — it just stopped defending.

Limitations, and who should not use this

This tree assumes you own the tool schema. If you are calling someone else's function-calling API and cannot change field definitions, branches 3 and 4 are unavailable and you are limited to prompt-side fixes plus an echo check.

It also assumes failures are cheap to replay. In flows where each call has an external side effect — sending mail, charging a card — you cannot just re-run 20 samples. Snapshot the calls, replay against a stub, and treat the tree as a pre-production gate only.

Skip this entirely if your agent has fewer than three tools and a human reads every output. The lint pass takes a minute, but the versioned measurement loop costs real time and earns nothing at that scale.

Where a free tier actually fits

The measurement loop in Step 4 is the expensive part, and it is boring work: twenty calls per schema revision, repeated until the echo rate stabilises. That is a batch job, not an interactive session. MonkeyCode's operator-supplied free model access and free server option cover both halves — the calls come from the free model endpoint, and the batch harness runs on the free server instead of your laptop.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

The operator also states a free allowance of roughly 10 million tokens. Treat any quota figure as perishable: open the product page, read the current limit, and write the number into your own notes with today's date before you plan a harness around it. I have not verified model names, throughput, hardware, or whether the allowance is permanent, and this post makes no claims about any of them.

One honest ordering note: run the linter first. A schema fix costs nothing and removes most of the failures you were about to spend tokens measuring. If you still want the batch pass after that, the free model access and free server option are the two pieces worth opening first.

What to check before you copy this

Confirm your endpoint supports structured output before assuming branch 1 is solvable.
Check whether your tool errors reach the model at all. Many frameworks swallow them.
Version the schema hash in the same commit as the schema change.
Re-measure after every model or endpoint swap, not just after schema edits.

Retry, Repair, or Replay: A Failure-Triage Tree for Agent Steps

Taylor Lin — Tue, 27 Oct 2026 13:06:12 +0000

Step 6 of a 9-step agent job times out. The run is holding a checkpoint, an open HTTP call that may or may not have landed, and a retry counter that nobody wired up. The next decision is the expensive one — and most codebases make it with a bare except: retry().

That default is wrong in two directions at once. It hammers dead endpoints, and it re-fires steps that already wrote something into the world.

This post is a glossary, a decision tree, and four worked leaves, plus a dependency-free classifier you can run in under a second. The scenario above is illustrative, not a customer story; the code is the artifact you can actually check.

Terms worth pinning down first

Failure class. Whether the fault is in the transport (timeout, 429, 5xx, connection reset) or in the task (wrong output, wrong field, wrong decision). These need different budgets, not the same for loop.

Side-effect boundary. The first line in a step after which the outside world has changed: a row written, an email sent, a payment authorized. Everything before it is safe to throw away.

Checkpoint. A stable identifier for the last known-good state, granular enough that replay resumes at a step rather than at the job.

Retry. Same input, same prompt, same parameters. Cheap to reason about, only valid before the side-effect boundary.

Repair. Changed input: a narrowed instruction, an added schema, one failing example. Expensive in tokens, appropriate when the output parses but is wrong.

Replay. Re-run from a checkpoint behind an idempotency key, so a duplicate request returns the original result instead of creating a second one.

Poison step. A step whose failure fingerprint is unchanged after the repair budget is spent. Retrying it is a way to convert a bug into an outage.

Reconciliation. The out-of-band read that proves what actually happened downstream. Triage decides what to try; reconciliation decides what is true.

The three branches

Every recovery decision in an agent step reduces to three questions, asked in this order.

Is the fingerprint stable and non-transient after repair? If yes, stop. This is a poison step and no branch below improves it.
Are there side effects? If yes, the only safe action is replay from a checkpoint with an idempotency key — or abort if no checkpoint exists.
Is the transport healthy? Transient and under budget means retry. Transient and over budget means abort: a dead endpoint is not a prompt problem. Non-transient means repair, once or twice, then abort.

Order matters more than the rules themselves. Side effects must be checked before the retry branch, and poison must be checked before replay, or you will replay a step whose model output was garbage in the first place.

Decision table

transient	side effects	checkpoint	repair attempts	action
yes	no	—	0	retry (budget open)
yes	yes	`ckpt-17`	0	replay behind idempotency key
yes	yes	none	0	abort, reconcile by hand
no	no	—	1	repair once, record prompt hash
no	no	—	2	abort, quarantine
no	yes	`ckpt-17`	2	abort — poison beats replay

The classifier, runnable

Save as triage.py. Standard library only, CPython 3.10+ for the union syntax.

#!/usr/bin/env python3
"""triage.py - pick a recovery action for a failed agent step."""
from __future__ import annotations

from dataclasses import dataclass
from enum import Enum


class Action(str, Enum):
    RETRY = "retry"
    REPAIR = "repair"
    REPLAY = "replay"
    ABORT = "abort"


RETRY_BUDGET = 3
REPAIR_BUDGET = 2


@dataclass(frozen=True)
class Failure:
    step: str
    transient: bool = False        # timeout, 429, 5xx, reset
    schema_ok: bool = True         # did output match the contract?
    side_effects: bool = False     # did this attempt already write outside?
    checkpoint: str | None = None  # last known-good state id
    attempts: int = 1
    repair_attempts: int = 0
    signature: str = ""            # normalized error fingerprint


def triage(f: Failure) -> tuple[Action, str]:
    if not f.transient and f.repair_attempts >= REPAIR_BUDGET:
        return Action.ABORT, "poison step: fingerprint survived the repair budget"
    if f.side_effects:
        if f.checkpoint:
            return Action.REPLAY, f"re-run from {f.checkpoint!r} behind an idempotency key"
        return Action.ABORT, "side effects with no checkpoint: reconcile by hand"
    if f.transient and f.attempts < RETRY_BUDGET:
        return Action.RETRY, "transport fault, retry budget open"
    if f.transient:
        return Action.ABORT, "transport budget spent; a prompt edit will not fix a dead endpoint"
    if f.repair_attempts < REPAIR_BUDGET:
        return Action.REPAIR, "transport fine, output wrong: narrow the instruction once"
    return Action.ABORT, "no leaf matched: escalate with the trace"


CASES = [
    ("transient read", Failure("fetch", transient=True, attempts=1), Action.RETRY),
    ("transport exhausted", Failure("fetch", transient=True, attempts=3), Action.ABORT),
    ("write with checkpoint",
     Failure("charge", transient=True, side_effects=True, checkpoint="ckpt-17"), Action.REPLAY),
    ("write without checkpoint", Failure("charge", side_effects=True), Action.ABORT),
    ("contract violation", Failure("extract", schema_ok=False), Action.REPAIR),
    ("repair exhausted", Failure("extract", schema_ok=False, repair_attempts=2), Action.ABORT),
    ("poison beats replay",
     Failure("charge", side_effects=True, checkpoint="ckpt-17", repair_attempts=2), Action.ABORT),
]


def main() -> int:
    failures = 0
    for name, failure, want in CASES:
        got, why = triage(failure)
        ok = got is want
        failures += 0 if ok else 1
        print(f"{'ok  ' if ok else 'FAIL'} {name:<24} -> {got.value:<6} {why}")
    return 1 if failures else 0


if __name__ == "__main__":
    raise SystemExit(main())

Run python3 triage.py. Expected output, with the last line being the one that catches the classic mistake — replaying a step whose model never produced valid output:

ok   transient read           -> retry  transport fault, retry budget open
ok   transport exhausted      -> abort  transport budget spent; a prompt edit will not fix a dead endpoint
ok   write with checkpoint    -> replay re-run from 'ckpt-17' behind an idempotency key
ok   write without checkpoint -> abort  side effects with no checkpoint: reconcile by hand
ok   contract violation       -> repair transport fine, output wrong: narrow the instruction once
ok   repair exhausted         -> abort  poison step: fingerprint survived the repair budget
ok   poison beats replay      -> abort  poison step: fingerprint survived the repair budget

If a line differs, the test fails and the tree disagrees with the table above. Fix the tree, not the table.

Four leaves, one worked example each

Leaf 1 — Retry

A geocoding lookup times out on attempt 1 of 3. transient=True, no side effects, no writes. Action: retry with backoff (200ms, 800ms, 3200ms), log the attempt counter, and let the budget run out. The thing you must not do: reuse this leaf for the enrichment step that posts a normalized address to an internal service. That step has a side-effect boundary.

Leaf 2 — Replay

The charge step times out after the provider accepted the request. Whether the charge landed is unknown, which is exactly the state replay exists for. You have ckpt-17, so you re-submit the same request with the same idempotency key and the provider returns the original response instead of a second charge.

class ReplayLedger:
    """Append-only claim file: first writer wins, replayers get False."""

    def __init__(self, path: str) -> None:
        self.path = path

    @staticmethod
    def key(job_id: str, step: str, checkpoint: str) -> str:
        return f"{job_id}:{step}:{checkpoint}"

    def claim(self, k: str) -> bool:
        with open(self.path, "a+", encoding="utf-8") as fh:
            fh.seek(0)
            if k in {line.strip() for line in fh if line.strip()}:
                return False
            fh.write(k + "\n")
            return True

The checkpoint is not optional here. With side_effects=True and checkpoint=None, the classifier aborts — deliberately, because the honest action is a human reading the provider's list endpoint, not a script guessing.

Leaf 3 — Repair

The extract step returns valid JSON that is missing due_date. The transport is fine; the contract is not. Repair means the input changes: append the required schema, add the one failing example, bump repair_attempts, and record a hash of the new prompt so the next failure is comparable to the last. Two repairs maximum. After that the fingerprint is stable and the leaf flips to abort.

Leaf 4 — Abort

The charge step returns 402 three times, transient=False, repair_attempts=2, and it has side effects. Every branch below the poison check would do something plausible and wrong. Action: quarantine the job, keep the trace, page a human, and run reconciliation against the provider before any resubmission. The cost of this leaf is one interrupted job. The cost of skipping it is a duplicate-charge incident.

Where free capacity actually lands in this tree

The repair leaf is the speculative one. Two repair attempts per failing step, across a nine-step job, means inference on inputs that mostly fail — and every repair changes the prompt, which makes the cost hard to amortize with caching.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

That is the honest place to try MonkeyCode's free model access: point only the repair branch at it, keep retry and replay on your existing infrastructure, and compare the abort rate after a week. The other fit is the replay ledger — a tiny append-only process that benefits from the free server option because it needs uptime, not compute. Free-tier quotas, model lists, and server terms change; read the project's current documentation rather than any number quoted in a blog post, including this one, and check the limits before you put a real queue behind it.

Limitations, and who should skip this

The ledger above is not concurrency-safe. Single writer only; multi-worker setups need a real store with an atomic conditional write.
Triage classifies failures, not outcomes. A step that returns HTTP 200 with semantically wrong data lands in repair, and only reconciliation catches it.
The tree assumes idempotent downstream operations or a written compensation action. If neither exists, replay is a duplicate generator and this whole approach is unsafe.
If your steps cannot checkpoint — long streaming writes, non-resumable file uploads — the side-effects branch collapses to abort and you gain little beyond a vocabulary.
Do not put financial, medical, or deletion paths behind an automated replay without a reconciliation step that a human reads.

Test plan before production

Run python3 triage.py and require exit code 0. All seven assertions are regression tests, especially the last one.
Add a fixture for every step in your job, recording real observed values for transient, schema_ok, side_effects, and checkpoint.
Point the job at a sandbox downstream that records every write. Inject a timeout after the write and confirm exactly one write appears after replay.
Force checkpoint=None on a side-effecting step and confirm the job aborts instead of retrying. This is the single most valuable test in the list.
Run 50 jobs with a 5% injected failure rate and compare duplicate-write count against your current except: retry() baseline. Zero is the target; anything else means the ledger is leaking.

Start with the selftest, not the architecture diagram. If the repair leaf turns out to be where your tokens go, running those speculative attempts on MonkeyCode's free model access for a week is a cheap way to get evidence before you change anything that is currently working.

Which Agent Step Deserves a Cache? A Glossary, a Decision Tree, and Four Worked Leaves

Taylor Lin — Mon, 26 Oct 2026 11:36:18 +0000

A nightly agent job runs 40 steps. Last sprint someone added a response cache to save calls on a free model endpoint. Tonight the summary cites a documentation paragraph that was deleted six hours ago, and nobody notices until a human reads the output.

Caching is the cheapest optimization in an agent pipeline and the easiest way to ship stale facts. The problem is not "should we cache" — it is which step deserves a cache, and with what invalidation contract. This post gives you a glossary, a four-question decision tree, and a worked example at each leaf, plus a replay test you can run before trusting any of it.

Glossary: seven terms worth agreeing on first

Step — one model call plus its deterministic wrapper (prompt assembly, parsing, validation). Caching decisions belong to the step, not the pipeline.
Cache key — the exact bytes that identify a reusable result. If the key omits an input, you get a silent wrong answer, not a miss.
Normalization — the rewrite applied to a prompt before hashing (whitespace collapse, Unicode NFKC, lowercasing). It raises hit rate and raises collision risk in the same move.
Staleness class — a label for how bad a stale answer is: cosmetic, misleading, or unsafe. Only cosmetic is a safe default for TTL caching.
TTL — the wall-clock bound on reuse. A TTL is a promise you make to readers about how old your facts can be.
Input digest — a hash of the files or records the step reads, not just the prompt text. Prompt-only keys miss content changes.
Hit budget — the fraction of steps you expect to serve from cache. Write it down; an unmeasured cache is a guess with extra failure modes.

The decision tree

Ask four questions per step, in order. Stop at the first leaf.

Q1. Does the step read external mutable state (live DB, clock, web, file tree)?
    yes -> Q2
    no  -> LEAF A (exact memo, no TTL)

Q2. Is a stale answer ever unsafe (side effects, money, access, published text)?
    yes -> LEAF C (no cache)
    no  -> Q3

Q3. Do prompts vary only by whitespace/casing/Unicode form across runs?
    yes -> LEAF D (normalized key + namespace)
    no  -> LEAF B (TTL cache, TTL from staleness class)

Four leaves. Each one below is a runnable decision, not a vibe.

Leaf A — exact memo, no TTL

Pure steps: schema validation of a tool call, prompt linting, deterministic reformatting, unit conversion. The output cannot change unless the input changes, so a TTL is pointless — the input digest is the invalidation.

from dataclasses import dataclass
from hashlib import sha256
import json

@dataclass(frozen=True)
class Step:
    name: str
    prompt: str
    input_digest: str          # hash of files/records the step reads
    reads_mutable_state: bool
    stale_is_unsafe: bool
    normalizable: bool = False

def cache_key(step: Step, model_tag: str) -> str:
    payload = {
        "model": model_tag,
        "prompt": step.prompt,
        "input": step.input_digest,
    }
    blob = json.dumps(payload, sort_keys=True).encode("utf-8")
    return sha256(blob).hexdigest()[:16]

Note model_tag is in the key. Swapping the model behind the same prompt must not reuse the old answer — that is the single most common cache bug in free-tier pipelines.

Leaf B — TTL cache, TTL set by staleness class

Steps that read mutable state but tolerate staleness: doc-snippet retrieval, changelog summaries, format detection. Here the cache lives for a bounded time and the bound is chosen from the staleness class, not from "how much money would we save".

This is where placement matters. A cache needs to survive between runs but not between deploys, and it needs to be reachable from wherever the step executes. Running the cache on the same box as the worker removes a network hop and one credential.

MonkeyCode offers free model access and a free server option (both operator-supplied availability claims — confirm current limits on the project page), which makes Leaf B practical for a hobby-scale job: model calls on the free endpoint, cache and state on the free server.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

import time

TTL_BY_CLASS = {"cosmetic": 6 * 3600, "misleading": 15 * 60}

def should_reuse(step: Step, cached_at: float, staleness_class: str) -> bool:
    ttl = TTL_BY_CLASS.get(staleness_class)
    if ttl is None:
        return False           # 'unsafe' has no TTL — see Leaf C
    return (time.time() - cached_at) < ttl

One rule: store the staleness class next to the value, not in your head. When the class changes after a code review, you want the old entries refused automatically.

Leaf C — no cache at all

Anything with a side effect or an accuracy obligation: sending mail, writing to a repo, granting access, producing the final user-facing answer. Recompute. If a free-tier call budget forces the issue, shrink the step instead of caching it — cache the retrieval that feeds the step, not the conclusion.

def admit(step: Step) -> str:
    if not step.reads_mutable_state:
        return "leaf_a_exact_memo"
    if step.stale_is_unsafe:
        return "leaf_c_no_cache"
    if step.normalizable:
        return "leaf_d_normalized"
    return "leaf_b_ttl"

That function is the whole artifact. Pipe every step definition through it in a test and assert the mapping matches your written design; the test is what stops someone from adding a cache to the final-answer step during a Friday refactor.

Leaf D — normalized key with a namespace

Prompts that differ only in whitespace or Unicode form deserve a normalized key — but the namespace must record the normalization version, or an edit to your normalizer silently reuses entries built under the old rules.

import re, unicodedata

def normalize(text: str, version: str = "v1") -> str:
    t = unicodedata.normalize("NFKC", text).strip().lower()
    t = re.sub(r"\s+", " ", t)
    return f"{version}:{t}"

Test the collision behavior explicitly:

assert normalize("Summarize  this.") == normalize("summarize this.\n")
assert normalize("a") != normalize("a", version="v2")

If the second assertion ever fails, every cached entry is now ambiguous across normalizer versions.

The replay test: three checks before you enable anything

Hit-rate check. Replay recorded step inputs with the cache on. Record actual hits; compare to the hit budget you wrote down. A hit rate far above budget usually means your key is missing an input.
Invalidation check. Mutate the source document that feeds one Leaf-B step. The next run must miss. If it hits, your TTL is carrying all the safety and your digest is decorative.
Kill-switch check. Flip an environment flag that disables the cache and confirm the pipeline still produces identical output. A cache you cannot turn off in one move is an outage waiting for a bad deploy.

Decision table

Step	Mutable state	Stale unsafe	Leaf	Invalidation
Tool-call schema validation	no	n/a	A	input digest
Doc snippet retrieval	yes	no	B	TTL + digest
Final user answer	yes	yes	C	none (recompute)
Intent classification	yes	no	D	TTL + norm version

Limitations and who should not use this

Latency-sensitive paths. If a step runs once per request and the cache lives on a separate free server, the round trip can cost more than the recompute. Co-locate or skip.
Steps with hidden inputs. If your prompt assembly pulls in conversation history or a user profile, the digest must include it. Steps you cannot fully describe should stay at Leaf C.
Workloads with no repetition. Low-overlap traffic produces misses plus a cache to maintain. Measure first; if your replay shows a single-digit hit rate, delete the layer.
Anything with a compliance clock. If readers are entitled to fresh data, a TTL cache is a policy change, not a performance tweak. Get it reviewed as one.

The cache is not the interesting part. The interesting part is the sentence you can write afterwards: this step may serve a result up to N minutes old, and here is the test that proves it. Steps that cannot survive that sentence belong at Leaf C.

If you want to try the shape — free model calls plus a free server to host the cache and the replay harness — MonkeyCode's project page is the place to check current terms, then run the three checks above against your own recorded steps before enabling anything in production.

Token Forensics for Agent Runs: Five Waste Signatures and the Order to Fix Them

Taylor Lin — Tue, 20 Oct 2026 09:44:06 +0000

A team I worked with had a nightly agent job that was supposed to run for six hours. By 11 a.m. its daily allowance was gone, and the run had produced two usable summaries out of nine.

Nobody could say which step ate the tokens. The router logs showed requests; they did not show cost. So the first instinct was to blame the model.

It was not the model. It was step 7 resending the same 4,000-token tool output on every attempt, after a validation failure that fired four times.

That is the whole problem in miniature: agent budgets rarely die from one big call. They die from structural waste that no request log records.

Below is the accounting I now run before changing anything — a small meter, five waste signatures, and the order in which to fix them.

1. Start with the report, not the router

Routing decisions are the wrong first lever. If 40% of your billed tokens are retries of a call that always fails, no amount of re-routing helps; you will just fail more cheaply.

Measure per step first. Then the routing question becomes answerable with numbers instead of vibes.

Glossary (six terms, used consistently below)

Term	Definition	Signal it produces
Billed tokens	Sum of prompt and completion tokens the provider counts for a run	Absolute cost
Static tokens	System prompt plus tool schemas, paid on every single call	`overhead_ratio`
Reused tokens	Context carried into a step unchanged from an earlier step	`redundancy_ratio`
Retry amplification	Share of billed tokens spent on attempts that ended in failure	`retry_amplification`
Prompt growth	How much the last successful prompt outgrew the first	`prompt_growth`
Leaf	The single branch the decision tree lands on, i.e. one prescribed first action	`leaf`

The sixth term matters most. A tree that returns three suggestions per run is a tree nobody follows.

2. The meter (stdlib only, ~60 lines)

You need three numbers per step that providers already return in usage, plus two annotations you add yourself: how many tokens are static, and how many are reused from an earlier step.

The annotations are manual or heuristic. That is a feature — it forces you to look at the prompt.

#!/usr/bin/env python3
"""meter.py - per-step token accounting for agent runs. Python 3.10+, stdlib only."""
from __future__ import annotations

import json
import sys
from dataclasses import dataclass


@dataclass
class Step:
    name: str
    prompt_tokens: int
    completion_tokens: int
    static_tokens: int = 0   # system prompt + tool schemas, paid per call
    reused_tokens: int = 0   # context carried over unchanged from an earlier step
    attempt: int = 1
    ok: bool = True


def load(path: str) -> list[Step]:
    with open(path, encoding="utf-8") as fh:
        return [Step(**row) for row in json.load(fh)]


def summarize(steps: list[Step]) -> dict:
    prompt = sum(s.prompt_tokens for s in steps)
    completion = sum(s.completion_tokens for s in steps)
    if not steps or prompt == 0:
        return {"steps": len(steps), "billed_tokens": prompt + completion}

    billed = prompt + completion
    wasted_retry = sum(
        s.prompt_tokens + s.completion_tokens for s in steps if not s.ok
    )
    ok_steps = [s for s in steps if s.ok]
    first = ok_steps[0].prompt_tokens if ok_steps else 0
    last = ok_steps[-1].prompt_tokens if ok_steps else 0

    return {
        "steps": len(steps),
        "billed_tokens": billed,
        "overhead_ratio": round(sum(s.static_tokens for s in steps) / prompt, 3),
        "retry_amplification": round(wasted_retry / billed, 3),
        "redundancy_ratio": round(sum(s.reused_tokens for s in steps) / prompt, 3),
        "prompt_growth": round((last - first) / first, 3) if first else 0.0,
    }


ACTIONS = {
    "B_retry": "install a stop rule before touching anything else",
    "A_overhead": "shrink the system prompt and tool schemas",
    "C_redundancy": "collapse steps or externalize state",
    "D_drift": "cap the transcript, move history to a scratch file",
    "F_leave_it": "budget is healthy; change nothing",
}


def classify(m: dict) -> str:
    if not m.get("billed_tokens"):
        return "F_leave_it"
    if m["retry_amplification"] > 0.25:
        return "B_retry"
    if m["overhead_ratio"] > 0.60:
        return "A_overhead"
    if m["redundancy_ratio"] > 0.40 and m["steps"] >= 8:
        return "C_redundancy"
    if m["prompt_growth"] > 0.30:
        return "D_drift"
    return "F_leave_it"


def main(argv: list[str]) -> int:
    if len(argv) != 2:
        print("usage: meter.py steps.json", file=sys.stderr)
        return 2
    report = summarize(load(argv[1]))
    leaf = classify(report)
    report["leaf"] = leaf
    report["first_action"] = ACTIONS[leaf]
    print(json.dumps(report, indent=2))
    return 0


if __name__ == "__main__":
    raise SystemExit(main(sys.argv))

Input is a flat JSON array. Three rows are enough to show the shape:

[
  {"name": "plan",            "prompt_tokens": 3200, "completion_tokens": 410, "static_tokens": 2400, "reused_tokens": 0,    "attempt": 1, "ok": true},
  {"name": "fetch_schema",    "prompt_tokens": 5100, "completion_tokens": 120, "static_tokens": 2400, "reused_tokens": 2600, "attempt": 1, "ok": true},
  {"name": "validate_schema", "prompt_tokens": 5400, "completion_tokens": 90,  "static_tokens": 2400, "reused_tokens": 2600, "attempt": 3, "ok": false}
]

Run it:

python meter.py steps.json

Note: the script is dependency-free Python 3.10+. I have exercised it only against synthetic usage dicts, so treat its output as a structural description of your run, not as a benchmark of any model or provider.

3. The decision tree, in fix order

The branches are ordered by cost of ignorance, not by elegance. Retries are checked first because they are the cheapest thing to fix and the most expensive thing to leave alone.

retry_amplification > 0.25 → leaf B. A quarter of the budget is being spent on attempts that fail. Stop the run earlier.
overhead_ratio > 0.60 → leaf A. Most of each prompt is boilerplate you wrote once and now pay forever.
redundancy_ratio > 0.40 and at least 8 steps → leaf C. Long runs are resending the same material.
prompt_growth > 0.30 → leaf D. The transcript is growing faster than the work.
None of the above → leaf F. Do nothing. This leaf is real and should fire often.

The stop rule for leaf B

A ratio is a diagnosis; you need a hard stop in code.

DAILY_ALLOWANCE = 1_000_000          # your number, from your own account
MAX_WASTE_RATIO = 0.25
wasted = 0

for step in steps:
    result = run_step(step)
    if not result.ok:
        wasted += result.prompt_tokens + result.completion_tokens
    if wasted > DAILY_ALLOWANCE * MAX_WASTE_RATIO:
        raise SystemExit("stop: waste budget exhausted, fix the failing step first")

This turns an invisible leak into a visible crash. Crashes get fixed; leaks get re-billed.

4. Worked examples at each leaf

The numbers below are illustrative workloads, not measurements of any product.

Leaf B — retry amplification 0.41. Nine steps, 180k billed tokens, 74k of them on failed validation attempts. First action: make the validator return the exact field path, not "invalid input". Second: apply the stop rule above. Expected effect is bounded — you cap the loss, you do not recover the tokens.

Leaf A — overhead ratio 0.72. A 3,100-token system prompt plus four tool schemas against a 4,300-token average prompt. First action: move three rarely used tools behind a second call. If a schema is only needed after planning, it does not belong in the planner's prompt.

Leaf C — redundancy 0.55 with 12 steps. The same repository map is pasted into eight prompts. First action: externalize it to a file the agent reads by path, and pass a one-line index instead. Second: check whether steps 4 and 5 can merge — often they exist because of an earlier ordering constraint that no longer holds.

Leaf D — prompt growth 0.84. The first successful prompt was 2,900 tokens; the last was 5,300. First action: summarise every third turn into a scratch note and drop the raw turns. Second: measure again — if growth persists, the agent is accumulating state it never reads.

Leaf F — everything under threshold. The correct output of this workflow is often "no change". Record the report, set an alert on retry_amplification, and go do something else.

5. Where a free tier actually changes the math

One leaf benefits from cheap capacity, and it is not the one people expect.

Leaf C and leaf D often need a summariser: a step that compresses history before the expensive step reads it. That step is repetitive, low-stakes, and easy to validate — a good candidate for a free endpoint, because a bad summary fails loudly at the next step rather than silently.

That is where MonkeyCode fits in this workflow rather than in the headline. The project advertises free model access and a free server option, and an allowance of 10 million tokens at the time of writing — operator-stated figures that you should confirm against current terms before planning around them. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I would not move the planning step there first. Move the step whose failure is easiest to detect, run the meter again, and compare billed_tokens before and after. If the number does not move, the free tier was a distraction, not an optimisation.

6. Limitations, and who should skip this

Usage accounting is not portable. Cached tokens, reasoning tokens, and tool-call tokens are reported differently across providers. Normalise or your ratios lie.
The annotations are manual. static_tokens and reused_tokens come from your reading of the prompt, not from the API.
It measures spend, not quality. A run can hit leaf F and still be useless. Pair this with a task-level success check.
Skip it if you already have tracing. If your stack records per-span token counts and you trust them, use those numbers and borrow only the fix order.
Skip it if latency dominates. On a latency-bound pipeline, shrinking prompts for token reasons can cost you more than it saves.

Closing

The order matters more than the thresholds. Retries, then boilerplate, then redundancy, then growth — and then, usually, nothing.

If you want to test this today, save one run's usage fields to JSON and run the meter against it. It is small enough to finish before lunch, and the leaf it returns is a better starting point than another router rewrite. If you decide the summariser step is worth moving, the project's free tier is a reasonable place to try it — with the meter still running.

Context Triage for Agent Prompts: Terms, Five Branches, and a Dependency-Free Budget Checker

Taylor Lin — Mon, 19 Oct 2026 08:26:24 +0000

Consider a support-triage agent that reads a ticket, pulls four tool results, and asks a model to classify and draft a reply. It works locally. Then someone adds a fifth tool, the history buffer grows by one turn, and every call starts failing on length before the model ever sees the question. This is a composite scenario, not a logged incident — but it is the most common shape of the failure.

The tempting fix is "switch to a bigger window." That is also the fix that keeps you on a treadmill, because the window you get from a free model endpoint is the one constraint you do not control. The fix that survives is triage: decide, before the call, which spans of your prompt are load-bearing and which are recoverable.

This post is that decision procedure. Terms first, then a five-branch tree with a worked example at each leaf, then the code I use to make the decision mechanical instead of vibes-based.

1. Glossary: the six terms that do the work

Span. One named, independently removable block of text in the prompt: the system instruction, the ticket body, each tool result, each prior turn. Triage is impossible if your prompt is one concatenated string; it is trivial if it is a list of spans.

Hard window. The context limit the endpoint enforces. If you exceed it, the request fails or gets silently cut. Treat it as unknown until you read the documentation — do not infer it from a successful call.

Soft budget. The number you actually target, deliberately below the hard window, leaving room for the model's output and for tokenizer drift between your counter and the provider's.

Eviction class. The label that decides whether a span may be dropped. Three classes are enough: pinned (the task is meaningless without it), reconstructible (can be replaced by a stub or summary), recoverable (can be re-fetched on demand if the model asks).

Truncation debt. The information you removed that the model will later need. It is not free. A prompt that fits by dropping a tool result often produces a confident, wrong answer twenty seconds later.

Leaf. A terminal branch of the decision tree: one specific trimming action, plus the check that tells you the trim was safe.

2. The tree: five branches, five leaves

Ask the branches in order. The first one that matches is your leaf; stop there.

Branch 1 — Does one pinned span already exceed the soft budget?

If a single pinned span is too large, no amount of dropping history will save you. Chunk it.

Leaf 1A — Chunk and label. Split the span into numbered chunks that each fit alone, process chunk 1 first, and carry forward only the extracted fields, not the raw text.

Worked example: a 900-line log file attached to a bug ticket. Chunk by time window, ask for {error_signature, first_occurrence, affected_endpoint} from each chunk, then classify on the merged fields. The merge output is a few hundred tokens; the log never needs to re-enter the window.

Branch 2 — Is the overflow from accumulated conversation turns?

Leaf 2A — Windowed eviction with a carry-forward summary. Keep the last n turns verbatim, replace everything older with one summary span marked reconstructible, and cap the summary at a fixed token count so it cannot itself become the problem.

Worked example: twelve turns of a refactoring session. Keep turns 9–12 verbatim; compress 1–8 into a bullet list of decisions ("chose strategy B; rejected caching layer; rename pending"). The decisions are what future turns reference. The transcript is not.

Branch 3 — Is the overflow from tool output?

This is the branch people skip, and it is usually the right one.

Leaf 3A — Lossy projection at the tool boundary. Do not store what the tool returned; store what the next step needs. A grep result becomes a list of {path, line, match}, not the file. A JSON API response becomes the two fields the prompt actually names.

Worked example: a dependency-audit tool returns 4 KB of advisory JSON; the agent only needs (package, severity, patched_version). Project to that triple at the tool boundary and the span drops by an order of magnitude — with no loss, because the model was never going to read the rest.

Branch 4 — Is the overflow from the instruction block itself?

Leaf 4A — Deduplicate and demote. Long system prompts accumulate contradictory restatements. Split the instruction span into numbered rules, drop exact duplicates, and demote examples into a separate reconstructible span that is only included when the task is in the "hard case" set.

Worked example: an instruction block with three separate paragraphs explaining output JSON format. Collapse to one rule plus one schema block. If the schema is large, keep the schema and move the worked examples out.

Branch 5 — Nothing trims enough without cutting pinned content.

Leaf 5A — Reduce the unit of work. Do not shrink the prompt further; shrink the job. One call per file instead of one call per repository. This is where triage becomes planning, and it is a legitimate outcome, not a failure.

Worked example: "review this PR" against a 40-file diff. Convert to per-file review calls plus one aggregator call that receives only the per-file verdicts.

3. Make it mechanical: a budget checker with no dependencies

The tree is only useful if a machine applies it. This checker takes spans, applies eviction classes, and returns what survived and what was removed. The token counter is pluggable — the default character heuristic is a stand-in so the example runs without a tokenizer.

from dataclasses import dataclass

@dataclass
class Span:
    name: str
    text: str
    evict: str   # "pinned" | "reconstructible" | "recoverable"
    priority: int  # lower number = dropped earlier

def char_counter(text: str) -> int:
    return max(1, len(text) // 4)

def triage(spans, soft_budget, counter=char_counter):
    total = sum(counter(s.text) for s in spans)
    if total <= soft_budget:
        return list(spans), [], total

    droppable = sorted(
        (s for s in spans if s.evict != "pinned"),
        key=lambda s: (s.priority, -counter(s.text)),
    )
    kept, dropped = list(spans), []
    for span in droppable:
        if total <= soft_budget:
            break
        kept.remove(span)
        dropped.append(span)
        total -= counter(span.text)
    return kept, dropped, total

def render(kept, dropped):
    lines = [s.text for s in kept]
    for s in dropped:
        lines.append(f"[{s.name} omitted: {s.evict}]")
    return "\n\n".join(lines)

Run it on a synthetic fixture — counts below are fixture output from char_counter, not provider tokenizer numbers and not a benchmark:

spans = [
    Span("instructions", "..." * 200, "pinned", 0),
    Span("ticket",       "..." * 300, "pinned", 0),
    Span("tool_audit",   "..." * 900, "recoverable", 1),
    Span("history_1_8",  "..." * 700, "reconstructible", 2),
    Span("schema",       "..." * 250, "pinned", 0),
]
kept, dropped, total = triage(spans, soft_budget=1200)
print(total, [s.name for s in dropped])

The point of the render stub line is honesty: the model is told a span exists and was omitted, so it can ask for it back instead of assuming the absence means "nothing there." Silent removal is where truncation debt gets expensive.

4. Which leaf for which symptom

Symptom you observe	First check	Leaf	Failure if you pick wrong
Failure on first call, short history	Size of each pinned span	1A	Chunking a prompt that was fine wastes calls
Degrades after turn ~10	Turn count vs. budget	2A	Summarizing too aggressively loses earlier decisions
Degrades after adding a tool	Tool output size	3A	Dropping the tool result makes the model guess
Works until you edit the system prompt	Rule duplication	4A	Demoting examples breaks edge-case accuracy
Every leaf still over budget	Task granularity	5A	More trimming yields a confident wrong answer

The table is the practical artifact. In practice, most teams are in branch 3 and believe they are in branch 2.

5. Limitations, and who should skip this

Counters disagree. Your character heuristic and the provider's tokenizer will drift. That is why the soft budget exists; set it with margin, not to the last token.
Summaries are lossy in ways tests miss. If your task depends on exact strings (IDs, hashes, error codes), chunk summarization is the wrong leaf — keep those spans pinned and reduce the job instead.
This is not a cost optimization. Triage reduces failures; it does not necessarily reduce call count. Branch 5 in particular increases it.
Skip this if your prompts fit comfortably with headroom, if you have a single fixed prompt with no history or tools, or if silent truncation by the endpoint is already documented and you accept it.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

6. Where the free surfaces fit

The workflow above was written around two surfaces: a free server running the agent runtime (where span assembly and the budget check happen) and free model access at the endpoint (where the window is enforced). Keeping the triage step on the server side means the model endpoint only ever sees a prompt that already fits — the failure moves from "request rejected" to "span listed as omitted," which is a much cheaper failure to debug.

Two cautions. Free availability is not the same as unlimited capacity: read the current documentation for quotas, window sizes, and server limits rather than assuming last month's numbers still hold, and back off immediately on a 429 instead of retrying around it. And measure your own prompts — the fixture above tells you nothing about your traffic.

If you want to try the leaves against a live endpoint, the budget checker is the only thing you need to bring; MonkeyCode's free model access and free server option are the two surfaces this was written around, and the tree works the same without them.

Admission Control Before Routing: A Free-Tier Quota Ledger for Agent Jobs

Taylor Lin — Sat, 17 Oct 2026 06:09:28 +0000

Picture a nightly batch: 400 classification calls, one free model endpoint, one free server, one cron entry. At 03:12 the run exits with code 0. The logs are green. The next morning half the labels are wrong, because the run quietly started receiving truncated or fallback responses once the quota window closed — and nothing in the pipeline noticed.

That scenario is constructed, but the arithmetic behind it is not. Almost every free-tier agent job I have seen has routing logic and no admission logic. It knows where to send work. It does not know whether it can afford to send work at all.

This article builds the missing layer: a local quota ledger plus a four-leaf decision tree that runs before any request leaves the machine. MonkeyCode's free model access and free server option are the deployment targets I use for the worked examples, because that is the combination most likely to hit this failure. Treat the availability claims as operator-supplied and re-read the project's current terms before you depend on them.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Terms first, tree second

Five terms carry the whole design. If you only take one, take replay cost.

Quota window — the period after which the provider's allowance resets, if it resets. You do not control it, so you must assume it can close mid-batch.
Unit cost — tokens charged per work item, input plus output, measured rather than guessed.
Replay cost — the extra units burned by retries and re-runs. A 15% retry rate is a 15% tax on your budget, and it is invisible if you only count successful calls.
Headroom — budget - spend, where spend includes failed and retried attempts, not just the ones that returned 200.
Admission — the decision to not start a job, or to start a smaller version of it, before the first request is issued.

Routing asks where. Admission asks whether, and how much. Skipping the second question is how you get exit code 0 with wrong output.

The ledger artifact

The script below is standard library only and deliberately boring. It appends one JSON line per attempt, so retries are billable by construction.

# ledger.py - Python 3.11+, stdlib only
import json, os
from dataclasses import dataclass, asdict

LEDGER = os.environ.get("LEDGER_PATH", "usage.jsonl")
BUDGET = int(os.environ.get("QUOTA_BUDGET", "200000"))
SAFETY = 0.80          # never plan past 80% of headroom
REPLAY = 0.15          # measured retry tax, replace with your own

@dataclass
class Entry:
    ts: float
    job_id: str
    tokens_in: int
    tokens_out: int
    status: str        # ok | retry | failed
    idem_key: str

def append(entry: Entry) -> None:
    with open(LEDGER, "a", encoding="utf-8") as f:
        f.write(json.dumps(asdict(entry)) + "\n")

def load() -> list[Entry]:
    if not os.path.exists(LEDGER):
        return []
    with open(LEDGER, encoding="utf-8") as f:
        return [Entry(**json.loads(line)) for line in f if line.strip()]

def spent() -> int:
    return sum(e.tokens_in + e.tokens_out for e in load())

def headroom() -> int:
    return BUDGET - spent()

def plan(items: int, per_item: int, deadline_bound: bool) -> tuple[str, dict]:
    unit = per_item * (1 + REPLAY)
    want = items * unit
    left = headroom()
    if want <= left * SAFETY:
        return "A", {"items_now": items, "items_later": 0}
    if want <= left:
        now = int(left * SAFETY // unit)
        return "B", {"items_now": now, "items_later": items - now}
    if not deadline_bound:
        per_window = int(BUDGET * SAFETY // unit)
        return "C", {"windows": -(-items // per_window), "per_window": per_window}
    now = int(left * SAFETY // unit)
    return "D", {"items_now": now, "items_rerouted": items - now}

Run it with an explicit budget for the window you are about to use:

QUOTA_BUDGET=200000 LEDGER_PATH=usage.jsonl python -c "import ledger; print(ledger.plan(130, 900, False))"

Two design choices matter more than the code. First, status is recorded but never filtered out of spent() — a failed attempt still consumed quota. Second, SAFETY is a hard multiplier, not advice: it forces a plan to survive a 20% estimation error without touching the last fifth of the allowance.

The tree, with four worked leaves

Hold the fixture constant: BUDGET = 200,000, already spent 60,000, so headroom = 140,000; SAFETY = 0.80 gives a planning ceiling of 112,000; REPLAY = 0.15. All four numbers below are hand arithmetic against that fixture, not provider measurements.

Leaf A — admit unchanged. 40 items at 900 units, want = 40 × 900 × 1.15 = 41,400, under the 112,000 ceiling. Run it, record it, move on. Leaf A is where you want 90% of your batches to land; if it is not, your per_item estimate is wrong.

Leaf B — admit a shrunken batch. 130 items at 900 units, want = 134,550. That exceeds the ceiling but fits inside 140,000, so the right move is partial admission: items_now = 112,000 // 1035 = 108, with 22 deferred. Validate chunk one's output before submitting chunk two. If chunk one's labels look wrong, you just saved 22 items of quota and a full-window outage.

Leaf C — defer to the next window. 300 items at 900 units, want = 310,500, no deadline. per_window = 160,000 // 1035 = 154, so the job needs two windows. Note that even a fresh window cannot absorb it in one pass — deferral without re-chunking just moves the failure.

Leaf D — reroute the overflow. Same 300 items, but deadline-bound. Take 108 locally and push the remaining 192 to the free server or a local model path. Leaf D is the only branch that depends on a property of your job rather than of your budget: it is valid only if every item carries an idempotency key and the reroute target produces comparable output. If either is false, fall back to C and negotiate the deadline.

Test the tree, not the model

The decision logic is deterministic, so it is cheap to pin down:

Boundary fixtures. Assert that want == 0.8 × headroom returns A and want == 0.8 × headroom + 1 returns B. Off-by-one at the ceiling is the most common regression.
Injected clock and path. Set LEDGER_PATH to a temp file per test; never read the developer's real ledger.
Conservation property. For every fixture, items_now + items_later (or items_rerouted) equals the input count. A leaf that loses work items is a bug, not an optimisation.
Retry accounting. Append a status="retry" entry and assert that spent() increases. If it does not, your replay tax is not being billed.
Window rollover. Move the ledger file aside, re-run, and confirm the plan returns to leaf A rather than remembering stale headroom.

Limitations, and who should skip this

The ledger counts what your process recorded. It cannot see another process using the same account, and without a file lock two concurrent writers will interleave lines. It also has no pricing knowledge: per_item and REPLAY are yours to measure, and they drift when prompt length or retry behaviour changes. Most importantly, admission control does not override a provider's own rate limiter; if the endpoint returns 429, stop the batch instead of retrying around the limit.

Skip all of this if you run one interactive session at a time, if your work items are genuinely non-repeatable and cannot be idempotent, or if you already have real billing dashboards with per-key spend. For a nightly batch on a free tier with a closing quota window, the ledger is roughly forty lines and it converts a silent failure into a printed plan.

If you want a place to try the tree end to end, MonkeyCode's free model access and free server option give you a realistic budget constraint to run these leaves against — check the current quota and server terms on the project page first, then let the ledger, not the cron entry, decide what runs tonight.

Split Placement: Deciding Which Agent Steps Run Locally, on a Free Server, or on a Free Model Endpoint

Taylor Lin — Fri, 16 Oct 2026 05:08:38 +0000

02:10, nineteen steps done, thirty-one to go

Picture a nightly triage job. It has sixty steps, it starts at 02:00, and at 02:10 it has finished nineteen of them. Step twenty needs to pull text out of a PDF, and the function that calls the model endpoint has just started returning 429 on every third request.

The job does not fail. It stalls, half-done, with no record of which leaf of the pipeline was running when the wall appeared. (The scenario is illustrative; the numbers describe the shape of the problem, not a benchmark I ran.)

Most writing about free AI tiers answers one question: which provider should the whole job use. That question is too coarse. Inside a single job, different steps belong in different places, and the placement decision is the one you can actually test.

Why placement is a separate question from tier choice

A job card is a promise about a workload. A placement decision is a predicate about one step: does it need a filesystem, does it touch regulated data, is it retry-safe, how tight is its deadline. Two steps in the same job can have identical inputs and still belong on opposite ends of your infrastructure.

Two themes keep showing up in current AI coverage: features moving into the client, and fatigue with generated output. Both push the same direction. Compute that can stay local should stay local, and anything that leaves the machine should leave under an explicit rule rather than a default.

The free tier used as the concrete case below is MonkeyCode's, which its operator describes as including free model access and a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I have not independently verified the quota figure the operator quotes (described as 10M tokens) or the server specifications, so the harness below treats any quota as unknown and measures consumption instead of assuming headroom.

Glossary: ten terms this tree depends on

Placement leaf — the destination a step is assigned to. This article uses four: LOCAL, FREE_SERVER, FREE_MODEL, ESCALATE.

Data class — the egress sensitivity of a payload. I use three: PUBLIC, INTERNAL, REGULATED.

Deterministic step — a step whose output depends only on its input, so running it twice costs time and nothing else.

Retry-safe (idempotent) — a step that can be re-executed after a transport failure without duplicating a side effect.

p95_seconds — the measured 95th-percentile duration of a step on your hardware. Guessed values make the whole tree decorative.

Quota cliff — the moment a free allowance stops absorbing requests, after which every call is either throttled or billed.

Fallback ladder — the ordered list of endpoints a step may attempt before it escalates.

Outcome class — how a failed attempt should be interpreted: RETRYABLE, FATAL, or OK. Conflating the first two is the most common cause of hot retry loops.

Fixture hash — a short digest over the exact inputs a test used, recorded so a later run can prove it saw the same bytes.

Ledger row — one append-only record per attempt: step, endpoint, attempt index, outcome, elapsed seconds.

The tree: ordered predicates, first match wins

place(step)
|
+-- 1. data_class == REGULATED? -----------------> LEAF D: ESCALATE
+-- 2. deterministic AND p95 <= 2s AND no disk? --> LEAF A: LOCAL
+-- 3. needs_disk OR not text_only? --------------> LEAF B: FREE_SERVER
+-- 4. INTERNAL AND not retry_safe? --------------> LEAF D: ESCALATE
+-- 5. p95 > 0.5 * deadline? ---------------------> LEAF D: ESCALATE
+-- 6. otherwise (text, retry-safe) --------------> LEAF C: FREE_MODEL

The order is the contract, not a style choice. Move predicate 1 below predicate 6 and you have an egress incident with a passing test suite.

The four leaves, with a worked example each

Leaf A: `LOCAL`

Worked example: validate 12,000 issue bodies against a JSON Schema before anything else runs. The step is pure, takes under two seconds at p95, and never needs the network.

# placement.py - pure decision logic, no network, no API keys
from dataclasses import dataclass
from enum import Enum


class Leaf(str, Enum):
    LOCAL = "local"
    FREE_SERVER = "free_server"
    FREE_MODEL = "free_model"
    ESCALATE = "escalate"


class DataClass(str, Enum):
    PUBLIC = "public"        # safe to send to a third-party endpoint
    INTERNAL = "internal"    # company-internal, not customer data
    REGULATED = "regulated"  # PII, health, payment, or contract-bound


@dataclass(frozen=True)
class Step:
    name: str
    p95_seconds: float      # measured on your machine, not guessed
    deterministic: bool
    text_only: bool
    needs_disk: bool
    data_class: DataClass
    retry_safe: bool
    deadline_seconds: float | None = None


def place(step: Step) -> tuple[Leaf, str]:
    """Ordered predicates. First match wins, so order is the contract."""
    if step.data_class is DataClass.REGULATED:
        return Leaf.ESCALATE, "regulated payload: no third-party egress"

    if step.deterministic and not step.needs_disk and step.p95_seconds <= 2.0:
        return Leaf.LOCAL, "cheap pure function: egress would be waste"

    if step.needs_disk or not step.text_only:
        return Leaf.FREE_SERVER, "needs a filesystem or a non-text runtime"

    if step.data_class is DataClass.INTERNAL and not step.retry_safe:
        return Leaf.ESCALATE, "non-idempotent internal call on a shared endpoint"

    if step.deadline_seconds and step.p95_seconds > step.deadline_seconds * 0.5:
        return Leaf.ESCALATE, "free-tier latency variance can eat the deadline"

    return Leaf.FREE_MODEL, "stateless text transform, retry-safe"

Leaf B: `FREE_SERVER`

Worked example: extract text from 400 PDFs and keep an on-disk cache so a second run is nearly free. This step needs a filesystem and CPU time, not a model.

This is also the leaf where a free server option changes the cost shape. Compute-heavy, non-text steps have no reason to touch a model endpoint at all, and putting them there wastes the resource you are actually short on.

Leaf C: `FREE_MODEL`

Worked example: classify and summarize issue text, one call per item, no local state. Text in, text out, retry-safe, egress class PUBLIC.

Before this leaf is reachable you need a reason to believe the endpoint tolerates your request rate. That is what the ladder and the ledger below are for.

Leaf D: `ESCALATE`

Worked example: redact personal data before any summarization happens. The step is a queue write plus a human or in-boundary worker, never a third-party call.

Escalation is a leaf, not a failure. If your tree has no ESCALATE, it is not a decision tree; it is a preference list.

A reproducible harness in five steps

1. Pin the fixture hash. Record the exact bytes a run consumed, so a later result can be tied to a specific input set.

from hashlib import sha256

def fixture_hash(paths):
    h = sha256()
    for p in sorted(paths):
        h.update(p.encode())
        h.update(open(p, "rb").read())
    return h.hexdigest()[:12]

2. Pin the case table. Every leaf needs at least one case, including the ones you expect to be rare.

CASES = [
    (Step("parse_json", 0.3, True, True, False, DataClass.PUBLIC, True), Leaf.LOCAL),
    (Step("extract_pdf", 12.0, False, False, True, DataClass.INTERNAL, True), Leaf.FREE_SERVER),
    (Step("summarize_issue", 3.1, False, True, False, DataClass.PUBLIC, True), Leaf.FREE_MODEL),
    (Step("redact_pii", 4.0, False, True, False, DataClass.REGULATED, True), Leaf.ESCALATE),
]

3. Separate retryable from fatal. A 429 and a schema mismatch must not share a code path.

class Outcome(str, Enum):
    OK = "ok"
    RETRYABLE = "retryable"  # 429, 503, timeout
    FATAL = "fatal"          # 400, 401, schema mismatch


def run_ladder(step, endpoints, call, ledger):
    for attempt, endpoint in enumerate(endpoints, start=1):
        try:
            result = call(endpoint, step)
        except Retryable as exc:
            ledger(step.name, endpoint, attempt, Outcome.RETRYABLE, exc)
            continue
        except Fatal as exc:
            ledger(step.name, endpoint, attempt, Outcome.FATAL, exc)
            return Leaf.ESCALATE, exc
        ledger(step.name, endpoint, attempt, Outcome.OK, None)
        return endpoint, result
    return Leaf.ESCALATE, "ladder exhausted"

4. Prove the ladder with a stub, not with a live quota. A stub that fails on the first endpoint is deterministic, free, and runs in CI.

def test_ladder_falls_through():
    def call(endpoint, step):
        if endpoint == "model-a":
            raise Retryable("429")
        return "ok"

    endpoint, result = run_ladder(step, ["model-a", "free-server"], call, ledger=lambda *a: None)
    assert endpoint == "free-server"
    assert result == "ok"

5. Write a ledger row per attempt. Append-only JSONL is enough, and it is the only artifact that tells you where a quota went when it disappears.

import json, time

def ledger(step_name, endpoint, attempt, outcome, error, path="ledger.jsonl", t0=None):
    row = {
        "step": step_name,
        "endpoint": endpoint,
        "attempt": attempt,
        "outcome": outcome.value,
        "error": type(error).__name__ if error else None,
        "seconds": round(time.monotonic() - t0, 3) if t0 else None,
    }
    with open(path, "a") as fh:
        fh.write(json.dumps(row) + "\n")

Run the whole thing with pytest -q. If you want the leaf distribution for a real job, feed your step table through place() and count the leaves before you deploy anything.

Limitations and what I did not verify

Free tiers move. A tree that encodes this month's allowances will quietly lie next quarter, which is why the harness measures instead of assuming headroom. The p95_seconds values must come from your own runs; copy-pasting mine would produce confident nonsense.

The data-class verdicts are jurisdictional and contractual. REGULATED here means "handled under a reviewed boundary" and nothing more, so the compliance owner decides those rows, not the engineer writing the predicate.

The ladder costs latency. Each retryable failure adds a round trip, and any non-idempotent step placed on FREE_MODEL by mistake can duplicate a side effect instead of failing cleanly. Retry-safety is a property of your step, not of the endpoint.

The described MonkeyCode free tier and its reported quota are operator-supplied statements that I have not independently tested against live rate limits, model behavior, or uptime.

Who should not use this

Teams with hard contractual SLAs should not route deadline-critical steps through any free tier, whatever the tree says. Workloads dominated by large non-text payloads will spend more time uploading than computing and should keep those steps local or on a server you control. Anyone without an owner for the REGULATED branch should stop at the glossary and get one first.

If you have a nightly job with a mixed step table, run place() over it and post the leaf counts. The distribution usually says more about where your budget goes than the tree does, and it is a cheap thing to compare across two teams.

Routing Agent Jobs Across Free Tiers: A Glossary, a Decision Tree, and Four Worked Leaves

Taylor Lin — Thu, 15 Oct 2026 11:20:05 +0000

Two of this week's most-read posts on DEV were about AI fatigue and about tooling that lags behind model capability. Read together, they describe the same operational gap: models got cheaper and better, while the plumbing that decides where a job runs stayed informal, undocumented, and unreproducible.

The failure shape is familiar. A script that used to run on a laptop gets promoted to "the free tier" because the free tier is free. Nobody notices that the job is a webhook receiver — something has to connect to it. Nothing does. The retry loop runs for six hours, duplicate units pile up, and the only record of the routing decision is one sentence in a chat thread.

A week later, three questions have no answer: which lane did this job belong in, what evidence justified that lane, and what would move it out again.

So here is the artifact this post builds: a glossary, a decision tree with four leaves, a dependency-free router you can run (route_job.py), a test file that pins every branch, and a log query that turns routing records into a table you can argue with.

Glossary: the eight terms the tree depends on

Precision here saves arguments later. If two people mean different things by "free tier", the decision tree resolves nothing.

Job — one unit of work with a name, an input, an output, and an owner. Not a pipeline.
Lane — the placement class a job routes to. This post uses four: local_only, free_model, free_server, controlled.
Leaf — a terminal outcome of the tree, carrying one example, one expected routing record, and one exit condition.
Data class — what the input contains: public, internal, or sensitive. Assigned by the data owner, not by the engineer who is in a hurry.
Idempotent unit — work that replays from the start with no extra side effect. A duplicate is a wasted call, not a corrupted record.
Addressable — something must initiate a connection to the job: a webhook, a schedule, an SSH session. Addressability is a hosting requirement, not a model requirement.
Pinned model — a job whose output is reproducible only against a specific model revision. Reproducibility outranks cost.
Routing record — one JSON line written at dispatch time: job, lane, reason, data class, and the quota observed on that call.

The decision tree: five branches, in order

Branch order is the policy. Reorder it and you have a different system, so write the order down and test it.

Branch 1 — Can the answer be computed without a model? If yes, local_only.

Branch 2 — Is the input sensitive, or is there a contractual review? controlled. Boundaries beat convenience.

Branch 3 — Does the job need a pinned model? controlled. Free tiers move underneath you by design.

Branch 4 — Must something connect to the job, or does it run longer than a short request? free_server.

Branch 5 — Is a re-run cheap and safe? Not idempotent goes to controlled. Idempotent goes to free_model.

The two interesting leaves are free_model and free_server, because both assume you have a model endpoint and a host. Many free options exist for each; this post treats them as properties, not brands. MonkeyCode, for example, is an open-source project whose free model access and free server option map onto those two leaves.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Its current outreach states a free tier of 10,000,000 tokens plus a free server option. Treat both as configuration you read at run time, never as constants you hard-code: quotas and terms are set by the project and can change between the day you write the number down and the day you rely on it.

Four worked leaves

Leaf A — `local_only`

Example: a nightly disk-usage digest that reads /proc and prints a table. Why it stops here: the answer is computable without a model.

{"name":"disk-report","lane":"local_only","reason":"decidable without a model; keep it off the network"}

Exit condition: the day someone adds an LLM summarization step, this becomes two jobs. Keep the deterministic half exactly where it is.

Leaf B — `free_model`

Example: drafting a changelog entry from a public PR diff. Why it stops here: public input, single-shot, idempotent, no inbound connection.

{"name":"changelog-draft","lane":"free_model","reason":"disposable, non-sensitive, single-shot"}

Exit condition: the diff stops being public. Then Branch 2 fires and the job moves to controlled, no matter how cheap leaf B is.

Leaf C — `free_server`

Example: a watcher that receives a repo webhook, queues the event, and replies later. Why it stops here: it is addressable and long-running; the model call is incidental.

{"name":"repo-webhook","lane":"free_server","reason":"long-lived or addressable workload"}

Exit condition: if the watcher becomes part of an availability promise, a free server is the wrong instrument. Move it to infrastructure you own.

Leaf D — `controlled`

Example: redacting customer identifiers from support tickets before anything else touches them. Why it stops here: sensitive input plus contractual review.

{"name":"ticket-redaction","lane":"controlled","reason":"sensitive or contractual: do not route to a shared free tier"}

Exit condition: there is none that a cheaper endpoint can satisfy. This leaf exists to say no.

The artifact: a router you can run in one command

route_job.py has no dependencies and makes no network calls. It encodes the five branches as five early returns, in the order written above.

#!/usr/bin/env python3
"""route_job.py - classify an agent job into a placement lane. Python 3.9+, stdlib only."""
from __future__ import annotations
from dataclasses import dataclass, asdict
from enum import Enum
import json, sys


class Lane(str, Enum):
    LOCAL_ONLY = "local_only"
    FREE_MODEL = "free_model"
    FREE_SERVER = "free_server"
    CONTROLLED = "controlled"


class DataClass(str, Enum):
    PUBLIC = "public"
    INTERNAL = "internal"
    SENSITIVE = "sensitive"


@dataclass(frozen=True)
class JobSpec:
    name: str
    data_class: DataClass
    deterministic: bool
    idempotent: bool
    needs_inbound_endpoint: bool
    wall_clock_minutes: int
    requires_pinned_model: bool
    contractual_review: bool


def route(job: JobSpec):
    if job.deterministic:                                                  # Branch 1
        return Lane.LOCAL_ONLY, "decidable without a model; keep it off the network"
    if job.data_class is DataClass.SENSITIVE or job.contractual_review:    # Branch 2
        return Lane.CONTROLLED, "sensitive or contractual: do not route to a shared free tier"
    if job.requires_pinned_model:                                          # Branch 3
        return Lane.CONTROLLED, "pinned model required; free tiers move underneath you"
    if job.needs_inbound_endpoint or job.wall_clock_minutes > 15:           # Branch 4
        return Lane.FREE_SERVER, "long-lived or addressable workload"
    if not job.idempotent:                                                 # Branch 5
        return Lane.CONTROLLED, "non-idempotent: the cost of a duplicate is unbounded"
    return Lane.FREE_MODEL, "disposable, non-sensitive, single-shot"


def routing_record(job: JobSpec) -> dict:
    lane, reason = route(job)
    record = asdict(job)
    record["data_class"] = job.data_class.value
    record["lane"] = lane.value
    record["reason"] = reason
    return record


if __name__ == "__main__":
    with open(sys.argv[1], encoding="utf-8") as handle:
        jobs = [JobSpec(**{**raw, "data_class": DataClass(raw["data_class"])})
                for raw in json.load(handle)]
    for job in jobs:
        print(json.dumps(routing_record(job), sort_keys=True))

Steps to reproduce:

Save the file, and write a fixture jobs.json containing the four examples from the leaves above.
Run python route_job.py jobs.json | jq -c '{name, lane, reason}'.
Compare the four output lines against the expected records printed in each leaf. A mismatch is a branch-order disagreement, not a bug — decide which order you actually meant.

Expected output:

{"name":"disk-report","lane":"local_only","reason":"decidable without a model; keep it off the network"}
{"name":"changelog-draft","lane":"free_model","reason":"disposable, non-sensitive, single-shot"}
{"name":"repo-webhook","lane":"free_server","reason":"long-lived or addressable workload"}
{"name":"ticket-redaction","lane":"controlled","reason":"sensitive or contractual: do not route to a shared free tier"}

Pin every branch with a test, not a review

A decision tree that lives only in prose drifts. The file below pins the branch order. Run it with python test_route_job.py; the names also work under pytest.

# test_route_job.py
from route_job import JobSpec, DataClass, Lane, route

BASE = dict(deterministic=False, idempotent=True, needs_inbound_endpoint=False,
            wall_clock_minutes=3, requires_pinned_model=False, contractual_review=False)


def make(**overrides):
    return JobSpec(**{**BASE, **overrides})


def test_deterministic_never_leaves_the_machine():
    lane, _ = route(make(name="a", data_class=DataClass.PUBLIC, deterministic=True))
    assert lane is Lane.LOCAL_ONLY


def test_sensitive_beats_convenience():
    lane, reason = route(make(name="b", data_class=DataClass.SENSITIVE))
    assert lane is Lane.CONTROLLED and "sensitive" in reason


def test_pinned_model_beats_free_tier():
    lane, _ = route(make(name="c", data_class=DataClass.PUBLIC, requires_pinned_model=True))
    assert lane is Lane.CONTROLLED


def test_addressable_job_gets_a_server():
    lane, _ = route(make(name="d", data_class=DataClass.PUBLIC, needs_inbound_endpoint=True))
    assert lane is Lane.FREE_SERVER


def test_long_job_gets_a_server():
    lane, _ = route(make(name="e", data_class=DataClass.PUBLIC, wall_clock_minutes=45))
    assert lane is Lane.FREE_SERVER


def test_non_idempotent_never_lands_on_a_shared_model():
    lane, _ = route(make(name="f", data_class=DataClass.PUBLIC, idempotent=False))
    assert lane is Lane.CONTROLLED


def test_disposable_public_single_shot():
    lane, _ = route(make(name="g", data_class=DataClass.PUBLIC))
    assert lane is Lane.FREE_MODEL


if __name__ == "__main__":
    for name, fn in sorted(globals().items()):
        if name.startswith("test_") and callable(fn):
            fn()
            print("ok", name)

Two properties matter more than coverage. First, the assertions use is, so renaming a lane fails loudly instead of quietly changing meaning. Second, the precedence tests — test_sensitive_beats_convenience and test_pinned_model_beats_free_tier — fail the moment somebody reorders the branches. That is the point.

Turn routing records into a table you can argue with

Write one routing record per dispatch and append them to a JSONL file. Then:

jq -r '[.lane, .name] | @tsv' routing.jsonl | sort | uniq -c | sort -rn

Keep the counts next to a small ledger. Fill the cells from your own logs; do not paste numbers you have not observed.

Column	Why it is there
`job`	joins the routing record to the unit of work
`lane`	makes drift visible when a job silently changes placement
`reason`	the branch that fired — the only field that survives a rewrite
`retries`	idempotency failures show up here first
`quota_observed`	what the endpoint actually reported, not what the docs said
`review_flag`	who has to look before this job moves again

Limitations, and who should not use this

The tree is a policy encoding, not evidence about any provider. It has not been validated against your workload, and both thresholds — wall_clock_minutes > 15 and the public/internal split — are placeholders to replace with numbers from your own logs.

Do not adopt this approach if:

The workload is regulated, or the data class cannot be assigned by a named owner.
You need a contractual availability or residency guarantee. A free tier is not that, whatever today's terms say.
Your jobs need pinned model revisions. Branch 3 will keep sending them elsewhere, correctly.
You cannot write one routing record per dispatch. Without the ledger, the tree is just an opinion.

Also note that a free server is a placement, not an SLA. If the watcher in Leaf C sits on the critical path of a customer promise, label a free host as a prototype and plan the move before the promise is made.

What to do next

Before you move any job to a cheaper lane, run the fixture, pin the branches with the tests, and record the lane you chose. Then revisit the ledger monthly. The interesting column is not retries; it is reason, because that is where a changed assumption becomes visible — and a changed assumption is the only thing that should move a job between leaves.

If you want somewhere to run the free_model and free_server leaves while you build the ledger, MonkeyCode is an open-source project with free model access and a free server option. Start with the repository, run the fixture above against four of your own jobs, and compare your lanes to the expected records before you move anything real.

DEV Community: Taylor Lin

Admission Control for Free-Tier Coding Agents: A Glossary, a Decision Tree, and Four Worked Leaves

Glossary: six terms worth fixing before you argue

The tree: four questions, asked in order

Leaf A — admit as-is, with an allowlist guard

Leaf B — admit with a contract written first

Leaf C — human-in-the-loop, draft only

Leaf D — do not send

Where a free tier fits, and what it does not fix

Limits and who should skip this

A verification pass you can repeat

The Quota Wall: A Glossary and a Decision Tree for Agent Steps That Hit Free-Tier Limits

Glossary: six terms that decide everything

The decision tree

The artifact: quota_triage.py

Decision table: what each leaf costs when it is wrong

Worked leaves

Leaf A — soft throttle: wait once, with a stated cap

Leaf B — degraded payload: shrink before you repeat

Leaf C — quota wall: switch tiers instead of sleeping

Leaf D — side effects: park, do not replay

Limitations and who should not use this

A five-minute check before trusting any policy

Local, Ephemeral, or CI: A Placement Decision Tree for Agent Experiments

Local, Ephemeral, or CI: A Placement Decision Tree for Agent Experiments

Glossary first, because the words do the work

The tree

Worked leaves

Leaf A — local, deliberately

Leaf B — ephemeral runner

Leaf C — CI

Leaf D — dedicated machine

The artifact: a classifier that ends the argument

A test plan for the placement itself

Limitations, and who should skip this

Two Writes, One Charge: An Idempotency Decision Tree for Agent Steps

The tree first

Glossary, because the words get overloaded

Four leaves, worked

A reference implementation

The two windows that actually bite

Where a free tier genuinely fits

Who should not use this

Pre-flight checklist

Fail Loudly: A Tool-Schema Decision Tree for Agent Steps That Return Wrong Answers Without Failing

Glossary: the six terms this tree depends on

Step 1: Lint the schema before you blame the model

Step 2: Read findings as branch inputs

Step 3: Walk to a leaf

Branch 1 — output will not parse → freeze the grammar

Branch 2 — parses, but the echo is wrong → echo-first contract

Branch 3 — echo right, enum drifted → closed vocabulary

Branch 4 — shape right, required values empty → two-phase fill

Branch 5 — everything correct, unstable across runs → snapshot and pin

Step 4: Measure the echo rate before and after

Step 5: Know when to stop

Limitations, and who should not use this

Where a free tier actually fits

What to check before you copy this

Retry, Repair, or Replay: A Failure-Triage Tree for Agent Steps

Terms worth pinning down first

The three branches

Decision table

The classifier, runnable

Four leaves, one worked example each

Leaf 1 — Retry

Leaf 2 — Replay

Leaf 3 — Repair

Leaf 4 — Abort

Where free capacity actually lands in this tree

Limitations, and who should skip this

Test plan before production

Which Agent Step Deserves a Cache? A Glossary, a Decision Tree, and Four Worked Leaves

Glossary: seven terms worth agreeing on first

The decision tree

Leaf A — exact memo, no TTL

Leaf B — TTL cache, TTL set by staleness class

Leaf C — no cache at all

Leaf D — normalized key with a namespace

The replay test: three checks before you enable anything

The artifact: `quota_triage.py`

Leaf A: `LOCAL`

Leaf B: `FREE_SERVER`

Leaf C: `FREE_MODEL`

Leaf D: `ESCALATE`

Leaf A — `local_only`

Leaf B — `free_model`

Leaf C — `free_server`

Leaf D — `controlled`