Skip to main content
Back to reports Executive Deck
AI Engineering

Agent Loops with Codex and Claude Code

Agentic Coding

Agent Loops with Codex and Claude Code

A practical report on recurring AI coding work: what loops actually are, when they earn their cost, how to structure them in Codex and Claude Code, and why cron alone is the wrong mental model.

Engineering teams are moving from prompt-by-prompt assistance to recurring agents that inspect code, CI, documentation, research sources, and customer feedback without waiting for a human to ask again. That creates leverage only when the loop has a reason to wake up, a reliable finish line, and a budget cap.

An AI coding loop is worth running when it replaces repeated human checking with verified work. It is not worth running when it is only a scheduled prompt that checks quiet inputs and reports noise.

PublishedJune 24, 2026
AuthorChander Dhall Methodworks
AudienceEngineering and business leaders
Tools coveredCodex and Claude Code
Core testRepeated, verifiable, bounded

Executive Summary

The shift from one-off prompting to designing loops that prompt agents is a meaningful change in how engineering teams use AI coding tools. A loop is not a cron job. A loop is a bounded decision cycle that wakes up when there is a reason, inspects what changed, acts only when there is real work, verifies the result against a clear stop condition, records what happened, and exits.

The stakes are visible in simple arithmetic. A timer that checks a quiet source 120 times per month and finds six useful items performs twenty runs for each useful finding. An event trigger that wakes only when the source changes can reduce that to one run per useful finding. The difference is not clever prompting. It is the trigger design.

Most useful work starts below the loop level. A one-time prompt, an interactive session, or a saved skill often delivers the result at lower cost and lower risk. Loops earn their cost only when three conditions are present: the work repeats, the input changes in ways that matter, and the success test is verifiable from outside the agent.

Repeated runs consume token budgets, especially when the input does not change between runs or when the goal has no verifiable end. A loop that checks the same quiet source every fifteen minutes is usually a budget leak. A loop that pursues a subjective goal like "improve the product until it is good" has no stop rule and will keep creating runs until someone notices the bill or forces a halt.

Both OpenAI Codex and Anthropic Claude Code provide mechanisms for recurring and autonomous coding work, but the mechanisms differ. Codex supports project and standalone automations, worktrees for isolating changes, non-interactive execution, GitHub Actions, AGENTS.md for durable methods, and MCP connectors for external systems. Claude Code supports scheduled tasks through the /loop command and cron scheduling tools, desktop scheduled tasks, cloud routines, /goal for persistent objectives, headless and programmatic usage, GitHub Actions, and MCP connectors.

Strong loops share a common structure: a trigger tied to real change, a scope and tool list, a verifiable goal, a stop rule with caps on runs, turns, wall time, and spend, an output that lands somewhere reviewable, and memory that persists lessons learned. Non-technical loops are harder because verification has to turn subjective goals into reviewable evidence. When possible, separate verification from generation. The same agent writing and approving work is weak evidence.

1. What a Loop Is

A loop is not a recurring job. A loop is a bounded decision cycle: it wakes up when there is a reason, inspects what changed, acts only when there is real work, verifies the result against a clear stop condition, records what happened, and exits.

The distinction matters because the word "loop" often gets used loosely. Teams sometimes describe any scheduled task as a loop, or any agent that runs more than once. That framing leads to expensive mistakes. A timer that wakes an agent every hour is not a loop. It is a timer. The loop is what happens after the timer fires: the decision about whether to act, the work itself, the check that the work succeeded, and the record of what happened.

Cron is one way to wake a loop, but it answers only the question "when does this run?" It does not answer the more important questions: "What changed since the last run?", "Is action required?", "What counts as done?", and "When does this stop?" A cron job that fires every fifteen minutes and immediately calls an agent is not a loop. It is a timer with an agent attached. The agent may do useful work, or it may do nothing useful and still consume tokens. Without decision logic, verification, and a stop rule, there is no reliable way to tell the difference until the bill arrives.

The five parts of a loop

A well-designed loop has five parts, each of which can be implemented in different ways depending on the tools and the task.

Trigger. The event or justified cadence that starts a run. A trigger can be a human starting the loop manually, an event from an external system such as a pull request update or a CI failure, or a schedule. The best triggers are tied to real change in the world. A PR event is a better trigger than a fifteen-minute timer for a loop that processes pull requests, because the PR event fires only when there is a pull request to process.

Inspection. The first step of every run is a cheap check that can exit with "nothing changed." This is the decision point that separates a loop from a blind timer. If the loop inspects the state and finds nothing new, it exits without doing expensive work. If the loop skips this step, it will do expensive work on every run regardless of whether there is anything to do.

Action. The work the loop exists to perform. This is the part most teams focus on, but it is only one part of the loop. The action consumes the bulk of the token budget, so the inspection step exists to make sure the action happens only when it will produce value.

Verification. The check that the work succeeded. Verification can be as simple as an exit code from a build, as structured as a test suite, or as complex as a separate evaluator model reviewing the output. The key requirement is that the verification signal comes from outside the agent that did the work. The same agent writing and approving work is weak evidence.

Record. The output that lands somewhere reviewable, and the memory that persists lessons learned. A loop that produces output but does not record what it did is difficult to audit, difficult to debug, and difficult to improve. A loop that records what it did but does not persist lessons learned will make the same mistakes repeatedly.

Loops earn their cost when the work repeats, the success test is verifiable, and the trigger is tied to a real change in the world. They lose money quietly when any of those conditions is missing.

2. A Complete Loop in Practice

A CI failure loop is the simplest way to see the whole pattern. The trigger is not a clock. The trigger is a failed workflow. The loop reads the failure, decides whether action is needed, proposes a fix only when the evidence supports it, verifies the fix, records the result, and exits.

Start with the operating story. A pull request is opened. The normal CI workflow runs. If CI passes, nothing happens. If CI fails, a follow-up workflow gathers the failing logs and gives the agent a tightly scoped job: classify the failure, identify the smallest likely fix, run the relevant tests, and write either a patch or a triage note. The loop is valuable because it wakes on a real event and because it has a verifiable stop condition.

In Codex, the same pattern can run through the Codex GitHub Action or through codex exec in a controlled runner. The prompt includes the failing logs as context, the sandbox limits the allowed action, and the output is a patch artifact or a report. The job that calls Codex can be read-only, while a separate job with repository write permissions opens the pull request after the patch is generated. That split keeps model work away from direct write credentials.

gh run view "$RUN_ID" --log | codex exec --sandbox workspace-write "Classify the CI failure, make the smallest safe patch, run the failing test again, and write a summary of the evidence." npm test 2>&1 | codex exec "summarize the failing tests, identify the likely root cause, and propose the next three debugging steps" > ci-triage.md

In Claude Code, the same job can run as a scheduled task, a /loop command, a cloud routine, a headless command, or a GitHub Actions integration depending on where the repository and CI context live. The important part is the same: the loop starts from a real signal, not a blind timer, and it exits after a bounded attempt.

/loop every 30 minutes "Check open pull requests with failing CI. If no failing PR changed since the last run, record an empty run and stop. If a failure changed, classify it, propose the smallest fix, run the relevant tests, and ask for review before merge." claude -p "Read ci.log, classify the failure, identify the smallest likely fix, and write a reviewable triage note. Do not modify production configuration."

The loop contract now has a concrete shape. Trigger: CI failure or PR update. Scope: failing workflow logs and changed files. Tools: repository read, branch write, test runner, no production access. Goal: classify the failure and prepare a minimal fix or triage note. Verification: test result, diff scope, and human PR review. Stop rule: one bounded attempt, then escalation. Output: patch artifact, PR comment, or triage report. Memory: run log and a list of repeated failure patterns.

The loop is not valuable because it runs often. It is valuable because it turns a repeated interrupt into a bounded, reviewable workflow with evidence attached.

3. The Decision Ladder

Move up the ladder only when the lower rung has become predictable. Each rung adds autonomy, cost, and operational risk.

The ladder is a framework for deciding how much automation a task deserves. The rungs are ordered by increasing autonomy, increasing cost, and increasing risk. The right rung for a given task is the lowest rung that gets the job done. Moving up the ladder before the lower rung is stable multiplies every weakness in the prompt, tool access, verification, and budget model.

Rung 1: One prompt

A single prompt to an AI coding tool runs once, with the human reviewing and acting on the result. This is the right level for one-off questions, quick analyses, and small code changes that do not need to be repeated. The cost is low because the prompt runs once. The risk is low because the human reviews the output before anything happens. The promotion test for moving to the next rung is that the same prompt gets reused often enough that typing it repeatedly becomes friction.

Rung 2: Interactive agent session

A longer conversation with an AI coding tool lets the human steer work through multiple turns. This is the right level for exploration, debugging, and feature work that requires back-and-forth. The cost is moderate because the session can run for many turns. The risk is moderate because the human is present throughout and can correct course. The promotion test is that the steps stabilize into a repeatable method that no longer needs steering.

Rung 3: Skill or playbook

A saved, reusable method encodes the steps discovered during interactive sessions. In Codex, this might be an AGENTS.md file or a skill. In Claude Code, this might be a CLAUDE.md file, a skill, or project instructions. The skill defines what to do, but a human still triggers it and reviews the output. This is the right level for repeatable tasks like PR review, release notes, triage, and migration planning. The promotion test is that the skill works reliably without hand-holding and produces consistent results.

Rung 4: Scheduled automation

A skill or method can run on a schedule without human triggering. In Codex, this can be a standalone or project automation. In Claude Code, this can be a routine or a desktop scheduled task. This is the right level for regular maintenance where time is the real trigger, such as daily triage or weekly documentation drift checks. The cost is higher because run frequency multiplies cost. The promotion test is that the job frequently produces useful findings that justify the cost.

Rung 5: Event-driven loop

An event-driven loop wakes on an external event such as a CI failure, a PR update, a deploy event, or a changed source. This is the right level for work where the trigger is a real change in the world, not just the passage of time. The cost profile is better than scheduled automation because the loop runs only when there is something to do. The risk is highest because the loop acts autonomously in response to external events. The promotion test is that the loop exits cleanly when no action is required, and the verification signal is strong enough to catch failures before they reach production.

RungBest fitRisk levelPromotion test
One promptOne-off question, analysis, small code changeLowestSame prompt reused often
Interactive sessionExploration, debugging, feature work with steeringLowSteps stabilize into repeatable method
Skill or playbookRepeatable PR review, release notes, triageModerateSkill works reliably without hand-holding
Scheduled automationRegular maintenance where time is the triggerHigherJob frequently produces useful findings
Event-driven loopCI failures, PR updates, deploy checksHighestLoop exits cleanly when no action required

The ladder is not a maturity model where higher is better. Higher rungs fit only when lower rungs have become friction. A team that jumps directly to event-driven loops without stabilizing the underlying skill will multiply the cost of every prompt weakness, every missing guardrail, and every verification gap.

4. Fit Criteria

A loop fits when a fresh run has a reason to exist. The right question is not "Can an agent run this repeatedly?" The right question is "What change in the world makes a new run valuable?"

Most useful work starts below the loop level. A one-time prompt, an interactive session, or a saved skill often delivers the result at lower cost and lower risk. Before designing a loop, teams need to translate weak conditions into stronger operating patterns.

When the goal is subjective, make it measurable

A loop needs a stop rule. "Improve the product until it is good" has no stop rule because "good" is subjective and unbounded. Every run creates the conditions for another run. The loop will keep running until someone notices the bill or forces a halt. The fix is to replace the subjective goal with a measurable target: "Improve test coverage until it reaches 80%" or "Reduce linting errors until the count is zero" or "Generate documentation until every public method has a docstring." If the goal cannot be made measurable, the task is not ready for a loop.

When inputs rarely change, wake on change

A job that checks the same quiet source every fifteen minutes is usually a budget leak. If the source changes once a day, a fifteen-minute polling interval means 95 out of 96 daily runs find nothing to do. Each run still consumes tokens for the inspection step, and if the inspection step is not cheap, the waste compounds. Event triggers fit better for quiet sources. A webhook that fires when the source changes is more efficient than a timer that checks whether the source changed.

When production access is involved, stop at review

Payments, data migrations, security changes, and deploy controls need human gates. A loop that can write to production without human approval has a blast radius that scales with the frequency of runs. If a bad run can cause irreversible harm, the loop is not the right tool. The loop can prepare the change, generate the patch, and present evidence for review. The human still owns the final production decision.

When tool access is broad, narrow it first

A loop with broad connectors, network access, and repository write permissions has a larger blast radius than a narrowly scoped loop. If the loop only needs to read from one repository and write to a patch file, it does not need permissions to read from all repositories and write to production. The principle of least privilege applies to loops as it applies to any other automated system. Grant only the access needed for the job, and review that access regularly.

The best loop candidates have measurable goals, change-based triggers, narrow permissions, and a clear review gate before high-risk action.

5. Cost Model

Token budget is part of the design. A loop that runs without signal consumes the same monthly budget that could have funded higher-value coding, review, or research work.

The cost of a loop is not the cost of a single run. It is the cost of a single run multiplied by the number of runs per day multiplied by the number of days per month. A run that costs four cents seems cheap until it runs four times a day for thirty days and costs four dollars and eighty cents. If that run finds useful work to do on every run, four dollars and eighty cents may be a bargain. If that run finds nothing useful on most runs, four dollars and eighty cents is waste.

The formulas

Per-run cost = input tokens + output tokens + tool calls + human review time Monthly loop cost = per-run cost x runs per day x days per month Cost per useful finding = monthly loop cost / useful findings per month Break-even test = human time saved or risk reduced > monthly loop cost

The break-even test is the key decision point. A loop breaks even when the value it produces, measured in human time saved or risk reduced, exceeds the cost of running the loop. If a loop costs ten dollars per month and saves an engineer two hours of work, the loop breaks even if the engineer's fully loaded cost exceeds five dollars per hour. If the loop costs ten dollars per month and saves no measurable time, the loop does not break even regardless of how clever the automation is.

Polling versus event triggers

The most common source of waste in loop design is polling a quiet source on a frequent schedule. Consider a loop that checks for new pull requests four times per day. Over a month, that loop runs approximately 120 times. If each run costs four cents, the monthly cost is four dollars and eighty cents. If the loop finds actionable pull requests on six of those runs, the cost per useful finding is eighty cents. If the loop finds actionable pull requests on sixty of those runs, the cost per useful finding is eight cents.

The same loop triggered by a pull request event rather than a timer would run only when there is a pull request to process. If there are six pull requests per month, the loop runs six times instead of 120 times. The monthly cost drops from four dollars and eighty cents to twenty-four cents. The cost per useful finding drops from eighty cents to four cents.

The lesson is simple: prefer event triggers over scheduled triggers whenever the event is available. A PR event, a CI failure notification, a deploy webhook, or a changed-file notification is almost always more efficient than a timer that checks whether anything changed.

The early exit pattern

When an event trigger is not available, the next best option is an early exit pattern. The first step of every run is a cheap check that determines whether expensive work is needed. If the check finds nothing changed, the run exits immediately without consuming tokens for the main task. If the check finds something changed, the run proceeds to the main task.

The early exit pattern converts the cost of a full run into the cost of a check on most runs. If the check costs one cent and the full run costs ten cents, and the loop runs 100 times per month with only ten runs finding work to do, the cost drops from ten dollars to one dollar and ninety cents: ninety runs at one cent plus ten runs at ten cents.

The cheapest run is the run that does not happen. The second cheapest is the run that exits early. Useful loops reduce unnecessary runs and exit early when nothing changed.

Cost examples use simple arithmetic, not price quotes. Current provider rates, plan limits, model choice, context size, connector load, and run duration change the real number.

6. The Loop Contract

Every loop needs a written contract before it runs. The contract answers the questions that determine whether the loop will pay for itself or burn budget quietly.

A loop contract is a short document, typically one page or less, that specifies the essential properties of the loop. The contract exists for three reasons. First, writing the contract forces the designer to answer hard questions before the loop is built. Second, the contract provides a reference for anyone who needs to understand, modify, or retire the loop later. Third, the contract provides evidence for audits and reviews.

The eight fields

Trigger. What event or schedule starts a run? Is the trigger tied to real change, or is it a blind timer? If it is a timer, what is the justification for the frequency?

Scope. What files, systems, and problem boundaries does the loop operate on? What is explicitly out of scope?

Tools. What tools and permissions does the loop need? Is the tool set the smallest that can finish the job?

Goal. What is the loop trying to accomplish? Is the goal verifiable from outside the agent?

Verification. How does the loop know it succeeded? What signal proves progress or completion?

Stop rule. When does the loop stop? What are the caps on runs, turns, wall time, and spend? What triggers escalation to a human?

Output. Where does the output land? Is the output reviewable by a human?

Memory. How does the loop persist lessons learned? Where does the run history live?

Example contract

Loop: PR Review Follow-up Trigger: PR comment event, not a timer Scope: Open PRs in the main repository; excludes vendored code Tools: Read repository, write to PR branch, run tests; no production access Goal: Address accepted review comments and update the branch with passing tests Verification: Resolved comments, passing tests, and human review of the diff Stop rule: Max 3 attempts per comment batch; max 30 minutes wall time; escalate on second test failure Output: Updated PR branch with commit messages citing addressed comments Memory: Run log appended to .agent-loops/pr-review.log

The contract is not bureaucracy. It is a forcing function that prevents the most common loop failures: unclear triggers, unbounded goals, missing verification, and silent budget consumption. If any field in the contract is blank or vague, the loop is not ready to run.

7. Codex and Claude Code Patterns

Both products support recurring or autonomous coding work, but the mechanisms differ. The right choice depends on where the work runs, how long state must survive, what tools are needed, and how much permission the run receives.

OpenAI Codex patterns

Codex supports automations that can run in the background while the user does other work. Automations come in two forms: project automations, which are tied to a specific project context, and standalone automations, which run independently. In Git repositories, worktrees keep automation changes separate from the user's active local work, so the automation can make commits without disturbing uncommitted changes in the main working directory.

Codex automations use default sandbox settings that restrict what the automation can access. The sandbox provides isolation between the automation and the rest of the system. For recurring or scripted work, Codex provides a non-interactive mode called codex exec that can be invoked from command-line scripts or CI pipelines. Codex also provides a GitHub Action for running Codex tasks as part of a GitHub Actions workflow.

For encoding durable methods, Codex uses AGENTS.md files and skills. An AGENTS.md file describes how the agent is expected to behave in a given repository or project. Skills are reusable methods that can be invoked by name. For connecting to external systems, Codex supports MCP servers, plugins, and app connectors.

Claude Code patterns

Claude Code supports scheduled tasks through several mechanisms. The /loop command creates a persistent prompt that runs repeatedly until a goal is met or a stop condition is triggered. Cron scheduling tools allow Claude Code tasks to run on a schedule. Desktop scheduled tasks run on the user's machine on a schedule. Cloud routines run in Anthropic's infrastructure and can continue running when the user's machine is off.

The /goal command sets a persistent objective that the agent works toward across multiple sessions. Headless and programmatic usage allows Claude Code to be invoked from scripts, CI pipelines, and other automated systems. Claude Code also provides GitHub Actions integration for running tasks as part of a GitHub Actions workflow.

For connecting to external systems, Claude Code supports MCP connectors and routines connectors. Cost management features help track and limit token spend across sessions and tasks.

Choosing the right pattern

NeedCodex patternClaude Code patternFit
Return to the same conversationThread automationActive session with /goal or scheduled taskLong-running review, deployment watch, research thread
Run fresh recurring checksStandalone or project automationRoutine or desktop scheduled taskDaily triage, weekly docs drift, routine maintenance
Run inside CI or scriptscodex exec or Codex GitHub ActionHeadless mode or Claude Code GitHub ActionsCI repair, release notes, structured reports
Isolate repository changesDedicated worktree or patch artifactBranch or GitHub Action PR flowAutomation writes code without disturbing active work
Encode a durable methodAGENTS.md and skillsCLAUDE.md, skills, and project instructionsStable repeatable workflow before scheduling
Connect external systemsMCP servers, plugins, app connectorsMCP connectors, routines connectorsMinimum tool set for the job

Decision rule: Codex is strongest when the loop is repository-native, CI-native, or patch-artifact driven. Claude Code is strongest when the loop lives in an active coding session, a desktop or cloud routine, or a tool-connected workflow that benefits from MCP context. GitHub Actions is the cleanest home when the trigger is already a repository event and branch protections already exist.

Minimal setup patterns

A practical Codex pattern is to keep the agent in a sandboxed runner and pass logs or diffs as context. The final output can be a report, a structured JSON response, or a patch artifact that another job turns into a pull request.

gh run view "$RUN_ID" --log | codex exec --json "Classify the CI failure and return fields: category, likely_cause, suggested_fix, confidence." codex exec --sandbox workspace-write "Update docs for changed public APIs, run the docs check, and write a summary to docs-drift.md."

A practical Claude Code pattern is to keep local or cloud routines narrow: define the source, define the stop condition, and write the memory location into the prompt. A scheduled task without a memory file will rediscover the same facts repeatedly.

/loop every day at 8am "Check docs/api against changed public API files. If no API files changed, record an empty run and stop. If docs drift exists, draft a PR and write the run log to .agent-loops/docs-drift.log." /goal "Keep this PR branch aligned with review comments. Stop after three failed test attempts or when all requested changes are resolved."

Local loops fit short-lived polling and hands-on work where the user wants to watch the loop run. Cloud routines and automations fit durable work that must run when the laptop is closed or the user is offline. CI jobs fit repository events where logs, tests, and branch permissions already exist in the CI environment.

8. Worked Examples

Four examples show how the loop contract and product patterns apply to real tasks: a CI failure triage loop, a documentation drift loop, a research source-change loop, and a non-technical customer feedback clustering loop.

Example 1: CI failure triage loop

A common pain point in engineering teams is the time spent triaging CI failures. When a build breaks, someone has to look at the logs, identify the failure, decide whether it is a flaky test or a real problem, and either fix it or escalate it. This is repetitive work that follows a pattern, making it a candidate for a loop.

Trigger: CI failure webhook. The loop runs only when a build fails, not on a schedule.

Scope: The failed build's logs and the code changed in the commit that triggered the build.

Tools: Read CI logs, read repository, run tests locally, write to a triage report.

Goal: Classify the failure as flaky test, real failure in changed code, or infrastructure issue. For real failures, propose a minimal patch.

Verification: If a patch is proposed, the patch has to pass the same tests that failed. Classification accuracy is reviewed by a human in weekly triage review.

Stop rule: One attempt per failure. Escalate to human if classification confidence is below threshold or if patch does not pass tests.

Output: Triage report posted to the PR or build page. Patch committed to a branch for human review if applicable.

Memory: Run log appended to .agent-loops/ci-triage.log. Failure patterns added to a local knowledge base for future classification.

Useful result: The team gets a PR-ready patch, a clear triage note, or an escalation with evidence.

The loop pays for itself when the team spends significant time on CI triage and when the failure patterns are repetitive enough that an agent can learn to classify them. It does not pay for itself when failures are rare or when every failure requires deep investigation that the agent cannot perform.

gh run view "$RUN_ID" --log | codex exec --sandbox workspace-write "Reproduce the failure, classify it, make the smallest safe patch, rerun the failing test, and write evidence to ci-triage.md."

Example 2: Documentation drift loop

Documentation tends to drift out of sync with code over time. A function signature changes, but the docstring is not updated. An API endpoint is deprecated, but the documentation still refers to it. A new feature is added, but no documentation is written. Catching this drift manually is tedious, and most teams do it inconsistently.

Trigger: Weekly schedule on Monday morning, or on merge of any PR that touches API code.

Scope: Public API code and corresponding documentation files.

Tools: Read repository, compare code signatures to documentation, write to a drift report or open a PR with updates.

Goal: Identify documentation that is out of sync with code and propose updates.

Verification: Proposed updates match actual code behavior. Human review approves the PR before merge.

Stop rule: One pass per trigger. Escalate to human if drift affects more than ten items, because that suggests a larger documentation project.

Output: PR with documentation updates, or a drift report if updates cannot be automatically generated.

Memory: Run log appended to .agent-loops/docs-drift.log. Patterns of common drift added to AGENTS.md or CLAUDE.md to improve future detection.

Useful result: The team gets a reviewable documentation PR or a short drift report.

This loop pays for itself when documentation accuracy matters, such as for public APIs or internal developer platforms, and when drift is common. It does not pay for itself when the codebase is small enough that manual review is easy or when documentation is not a priority.

git diff --name-only origin/main...HEAD | codex exec "Identify public API changes that require docs updates. Draft the smallest docs patch and list any uncertain items."

Example 3: Research source-change loop

Teams that track external sources, such as competitor announcements, regulatory changes, or upstream library releases, often want to be notified when something changes. Polling these sources manually is tedious. A loop can automate the polling and summarization.

Trigger: Daily schedule, or event trigger from a source monitoring service if available.

Scope: A defined list of external sources, such as specific URLs, RSS feeds, or API endpoints.

Tools: Read from external sources with appropriate rate limiting and authentication, compare to previous snapshot, write to a digest report.

Goal: Identify changes in monitored sources since the last run and summarize them.

Verification: Digest cites sources for every claim. Human review checks accuracy and relevance.

Stop rule: One pass per trigger. Exit early if no sources have changed. Escalate to human if a source is unavailable or returns unexpected content.

Output: Digest posted to a Slack channel, email, or internal wiki page.

Memory: Snapshot of each source stored for comparison on next run. Run log appended to .agent-loops/research.log.

Useful result: The team gets a delta-only digest with citations, not a repeated summary of the same unchanged sources.

This loop pays for itself when the sources change frequently enough to justify monitoring and when the team acts on the information in the digest. It does not pay for itself when sources rarely change or when the digest is ignored.

/loop every 6 hours "Check the monitored sources listed in sources.json. If no source changed since the last snapshot, write an empty ledger entry and stop. If sources changed, summarize only the deltas with citations."

Example 4: Customer feedback clustering loop

Non-technical loops are harder because verification has to bridge subjective goals into reviewable evidence. A customer feedback clustering loop illustrates this challenge. The goal is to group similar feedback items so the product team can identify themes and prioritize work.

Trigger: Weekly schedule on Friday afternoon, or event trigger when feedback volume exceeds a threshold.

Scope: Customer feedback received in the past week from support tickets, surveys, and app store reviews.

Tools: Read from feedback sources, write to a clustering report.

Goal: Group feedback items into clusters by theme and rank clusters by volume and sentiment.

Verification: This is the hard part. The clustering is subjective, so verification has to be indirect. Options include a separate evaluator model reviewing the clustering and flagging low-confidence clusters, a human reviewing a sample of clusters each week, and the loop tracking whether product decisions were made based on the clusters as a lagging indicator of usefulness.

Stop rule: One pass per trigger. Escalate to human if clustering confidence is below threshold or if feedback volume is unusually high.

Output: Clustering report posted to a product team channel or wiki page, with links to representative feedback items in each cluster.

Memory: Previous clusters stored for comparison. Run log appended to .agent-loops/feedback-clustering.log. Human feedback on cluster quality used to improve future clustering.

Useful result: The product team gets a short list of themes with representative source items and a confidence note for each cluster.

This loop illustrates why non-technical loops are harder. The goal is inherently subjective, so the verification signal is weaker than a test suite. A separate evaluator model helps, but the evaluator is also subjective. Tracking downstream product decisions takes weeks. The loop designer compensates with smaller scope, more frequent human review, and clear evidence in each report.

claude -p "Cluster the new feedback rows in feedback.csv by theme. Include representative examples, confidence, and a reviewer checklist. Do not create product priorities without human review."

9. Verification Hierarchy

Verification turns recurring agent work from repeated guessing into repeated measurement. The hierarchy starts with the cheapest reliable signal and escalates only when the task demands it.

The purpose of verification is to answer the question "Did the loop succeed?" without relying on the agent's own assessment. The same agent writing and approving work is weak evidence because the agent has no external check on its own reasoning. A separate verification signal, even a simple one, provides evidence that an auditor or reviewer can evaluate independently.

Level 1: Exit code

The simplest verification is an exit code from a command. A build that exits with code zero passed. A lint check that exits with code zero found no errors. Exit codes are cheap to check and unambiguous. The limitation is that exit code zero means "the command succeeded" but not necessarily "the command did the right thing." A test suite can pass while missing important cases.

Level 2: Diff check

A diff check verifies that the changes the loop made are within expected bounds. Did the loop touch only the files it was supposed to touch? Did it avoid banned files like configuration or secrets? Did the changed paths match the expected scope? Diff checks are cheap and useful for scope control, but they do not prove that the changes are correct, only that they are in the right place.

Level 3: Deterministic tests

Unit tests, integration tests, smoke tests, and regression tests provide stronger verification because they check behavior, not just structure. If the loop's changes pass the test suite, there is evidence that the changes did not break existing functionality. The limitation is that tests only cover tested behavior. Changes that affect untested behavior will not be caught.

Level 4: Schema or structured output

For loops that produce reports, triage results, or API payloads, schema validation provides verification that the output has the expected structure. A JSON schema can verify that all required fields are present and have the right types. The limitation is that structure is not truth. A report can have the right structure and still contain incorrect information.

Level 5: LLM evaluator

For subjective outputs like writing quality, ambiguity detection, or summarization accuracy, an LLM evaluator can provide a verification signal. The evaluator is a separate model call that reviews the output and scores it against criteria. The limitation is that LLM evaluation is itself subjective and model-sensitive. Different models may give different scores, and the evaluator may have the same blind spots as the generator.

Level 6: Separate evaluator model

For higher-risk qualitative review, using a separate evaluator model, not the same model that generated the output, provides a stronger signal. The reasoning is that two different models are less likely to share the same blind spots than one model evaluating its own work. The limitation is that this is still not a production approval for high-blast-radius work. A human still owns the final decision.

LevelSignalBest useLimitation
1Exit codeBuild, lint, command successPasses can miss product intent
2Diff checkScope control, banned files, changed pathsDoes not prove behavior
3Deterministic testsUnit, integration, smoke, regressionOnly covers tested behavior
4Schema validationReports, triage, metadata, API payloadsStructure is not truth
5LLM evaluatorWriting quality, ambiguity, summarizationSubjective to model
6Separate evaluator modelHigher-risk qualitative reviewStill not production approval

The hierarchy is not a ranking where higher is better. It is a menu where each level has a cost and a benefit. The cheapest level that provides sufficient evidence for the risk level of the task is usually the right level. For low-risk tasks, exit codes and diff checks may be enough. For high-risk tasks, deterministic tests plus human review may be required regardless of what automated verification is available.

10. Governance by Blast Radius

Permission design starts with the damage a bad run could cause. The loop gets only the access needed for that tier.

The principle of least privilege applies to loops as it applies to any other automated system. A loop that can read, write, and delete across all repositories has a larger blast radius than a loop that can read from one repository and write to a patch file. The governance question is: if this loop fails in the worst possible way, what is the damage?

Tier 1: Read-only intelligence

Loops that only read data and produce reports or summaries have the lowest blast radius. A research loop that reads public sources and writes to a digest file cannot corrupt production data because it has no write access to production. A triage loop that reads CI logs and classifies failures cannot break the build because it cannot modify the build configuration. For Tier 1 loops, auto-run fits when the sources are safe and secrets are excluded from the read scope.

Tier 2: Repository writes on branches

Loops that can write to repositories have a larger blast radius, but the damage is limited if writes are confined to branches rather than the main branch. A documentation drift loop that opens PRs with updates cannot merge those PRs without human approval. A CI repair loop that commits patches to a feature branch cannot push to production without passing through the normal review process. For Tier 2 loops, the governance control is branch protection: the loop can write to branches, but merge requires human review.

Tier 3: Production actions

Loops that can take production actions, such as deploying code, modifying data, changing access controls, or initiating payments, have the highest blast radius. A bad run can cause immediate, possibly irreversible harm. For Tier 3 actions, human approval remains part of the runbook regardless of how reliable the loop has been in the past. The loop can prepare the action, generate the evidence, and present the recommendation. The human executes the action.

Tier 1

Read-only intelligence

Research, summaries, log analysis, issue clustering. Auto-run fits when sources are safe and secrets are excluded.

Tier 2

Repository writes on branches

Code patches, docs updates, test fixes. Run in a worktree or branch; require PR review before merge.

Tier 3

Production actions

Deploys, data changes, access changes, payments. Human approval remains in the runbook.

Governance controls are part of the loop, not afterthoughts. Trusted runners, sandbox settings, connector minimization, branch protections, secret handling, and a clear kill switch are design decisions that belong in the loop contract, not items to add later when something goes wrong.

11. Monthly Ledger

The loop ledger is the operational record that keeps automation honest. It tracks what each loop did, what it cost, and whether it was worth the cost.

A monthly ledger is a simple table that records key metrics for each active loop. The purpose of the ledger is to surface loops that are consuming budget without producing proportional value. Without a ledger, loops drift into background spend that no one monitors. With a ledger, the team can make informed decisions about which loops to keep, which to tune, and which to retire.

Ledger fields

Loop name: The identifier for the loop, matching the name in the loop contract.

Runs this month: The number of times the loop ran.

Useful findings: The number of runs that produced actionable output. A run that exits early with "nothing changed" is not a useful finding. A run that produces a report that no one reads is not a useful finding. A useful finding is output that led to a decision or action.

Total cost: The token cost of all runs this month, including runs that exited early.

Cost per useful finding: Total cost divided by useful findings. This is the efficiency metric.

Notes: Any observations about the loop's performance, including false positives, missed findings, or suggested improvements.

Example ledger

Loop nameRunsUseful findingsTotal costCost per findingNotes
CI failure triage4732$14.10$0.44High signal. Add memory for flaky test patterns.
Documentation drift53$2.40$0.80Acceptable. May reduce to monthly schedule.
Research digest304$9.00$2.25Low signal. Sources rarely change. Consider event trigger or retirement.
Feedback clustering44$6.80$1.70Product team uses clusters weekly. Keep running.

The ledger review is a monthly operating rhythm, not a one-time exercise. At the end of each month, the loop owner reviews the ledger and makes decisions. Loops with high cost per finding are candidates for tuning or retirement. Loops with low cost per finding are candidates for expansion. Loops that have not produced useful findings in multiple months are candidates for immediate retirement.

The research digest loop in the example ledger has a cost per finding of $2.25, much higher than the other loops. The note says the sources rarely change. This loop is a candidate for switching to an event trigger, if available, or retirement. The ledger surfaced the problem; without it, the loop would continue running indefinitely.

12. Launch Checklist

Every loop needs a short launch record. If any line is blank, the loop is not ready.

The checklist is a final verification that the loop contract is complete and that the loop is ready to run without immediate supervision. Each item corresponds to a failure mode that the checklist prevents.

  • The trigger is tied to real change or a justified cadence. This prevents blind timers that burn budget.
  • The loop has a written scope and a list of allowed tools. This prevents scope creep and excess permissions.
  • The first step can exit cheaply when nothing changed. This prevents expensive runs with no output.
  • The success test is deterministic where possible. This prevents reliance on subjective self-assessment.
  • The stop rule includes max runs, max turns, max wall time, and max spend. This prevents runaway loops.
  • The output lands somewhere reviewable: PR, patch, report, ticket, or digest. This prevents invisible work.
  • The run writes a short ledger entry with cost, outcome, and lesson learned. This prevents untracked spend.
  • The owner reviews the ledger monthly and retires loops with poor signal-to-cost. This prevents background automation from turning into background waste.

The checklist is a gate, not a form. If the team cannot check every box, the loop is not ready. It may be ready after more work on the contract, the verification, or the governance. But it is not ready now.

The goal is not more automation. The goal is more useful work per dollar, with less waiting and fewer repeated manual checks. A loop that passes the checklist has a reasonable chance of paying for itself. A loop that fails the checklist is likely to burn budget quietly.

13. Two Questions for Leaders

Two questions reveal whether a loop program is evidence-driven or enthusiasm-driven.

Question 1: Show the ledger for the most expensive loop last month. The answer reveals whether the team knows how often the loop ran, what it cost, how many useful findings it produced, and whether it was retired or tuned when the signal was weak.

Question 2: What stop rule terminates the longest-running automation? The answer reveals whether the loop has a real boundary. A healthy answer names max runs, max turns, max wall time, max spend, and the human escalation point.

Loops are not a marker of sophistication by themselves. The marker is evidence: a trigger tied to change, a result that can be verified, a cost that can be measured, and a stop rule that actually stops.

Sources

Sources cover product documentation, engineering practices, and technical context.

  1. OpenAI Codex documentation: Automations
  2. OpenAI Codex documentation: Non-interactive mode
  3. OpenAI Codex documentation: Best practices
  4. OpenAI Codex documentation: Worktrees
  5. OpenAI Codex documentation: Pricing and usage guidance
  6. Claude Code documentation: Run prompts on a schedule
  7. Claude Code documentation: Routines
  8. Claude Code documentation: Goals
  9. Claude Code documentation: How Claude Code works
  10. Claude Code documentation: Programmatic usage
  11. Claude Code documentation: GitHub Actions
  12. Claude Code documentation: Cost management
  13. Anthropic research: Building effective agents

Cost examples use illustrative arithmetic to demonstrate the formulas. Actual costs depend on provider pricing, plan limits, model choice, context size, and run duration.