He Finally 10x Claude Code With This Method
Key Takeaway
Implementing a dual-loop architecture—where an inner loop builds features against locked automated checks and an outer 'Auto Loop' updates workflow instructions based on historical execution logs—prevents AI coding agents from repeating persistent errors.
Highlights
Andrej Karpathy's Auto Research agent ran 700 experiments over two days to discover 20 code modifications that accelerated model training speed.
Running an autonomous optimization loop for one night improved Shopify's internal AI model performance by 19% across 37 experiments.
Agentic loops require four baseline conditions: highly repetitive tasks, high usage limits, automated evaluation checks, and an execution environment to test code.
Locking test suite files in a restricted folder prevents autonomous agents from modifying pass conditions to cheat evaluations.
A meta-loop (Auto Loop) inspecting execution logs and updating `program.md` stops agents from repeating the same structural mistakes across features.
Timeline
Foundational Principles of Autonomous Agent Loops
Auto Research demonstrated how an AI agent, governed by plain-text instructions in `program.md`, ran 700 independent experiments over two days to yield 20 performance improvements. The agent modified code, executed training cycles, and checked results against a immutable scoring script. Applying this method allowed Shopify's CEO to achieve a 19% model improvement overnight across 37 autonomous iterations.
- Structured workflows enable AI agents to execute complex tasks autonomously over extended periods.
- Andrej Karpathy's Auto Research project proved agents can optimize models by constraining modifications to a single training file.
- Restricting agent access to scoring files prevents the AI from lowering evaluation benchmarks.
Prerequisites and Constraints for Deploying AI Loops
Deploying loops on single-use tasks wastes more time in configuration than manual prompting. Because agents reload project context and attempt multiple fixes per iteration, long runs rapidly exhaust API usage caps. Effective deployment requires headless execution environments and programmatic unit tests that provide binary pass/fail feedback without human intervention.
- Loops are economically viable only for frequently repeated tasks due to initial setup overhead.
- High token consumption requires elevated rate limits or higher-tier subscription plans.
- Automated execution environments and programmatic validation are strictly required.
Architecting the Core Feature Building Loop
A maintainable loop uses dedicated skills like `project_context` to load system information on demand rather than cluttering token windows. Prior to code generation, a `write_checks` skill creates test parameters, which are stored in a protected folder that the agent cannot edit. The primary `build_skill` then delegates each discrete task to a fresh `feature_builder` agent to ensure isolated, repeatable execution environments.
- The `project_context` skill stores system architecture while keeping main context windows clear.
- Automated test suites must be generated, human-approved, and locked before writing implementation code.
- Isolating feature builds into dedicated agent contexts prevents cross-feature memory contamination.
Practical Execution: Building Features with Locked Checks
Cloud sandboxes enable multiple agents to run simultaneously without risking host environment corruption or file collisions. In a practical trial, an agent established 11 specific validation checks for a restaurant pickup feature. The loop executed for six minutes, passing all 11 backend checks on the first pass while automatically building and wiring the necessary frontend order form.
- Upstash Box sandboxes isolate concurrent agent processes from host file systems.
- Generating explicit test specifications before coding ensures complete feature validation.
- Loop checks verify underlying application logic and missing UI components simultaneously.
Solving Agent Habit Recurrence with Meta-Loops
Because each feature starts with a fresh agent context, single-layer loops repeatedly make the same coding errors. The Auto Loop system solves this by analyzing `results.json` to identify recurring failure patterns across rounds. It then updates `program.md` with new operational rules—such as requiring UI screens to connect to databases in the same round—preventing future feature runs from repeating past mistakes.
- Standard single-pass loops repeat identical code mistakes across different features due to context resets.
- The Auto Loop meta-skill analyzes execution logs to rewrite operational instructions in `program.md`.
- Updating system instructions based on past failures permanently eliminates recurring bad coding habits.
Description
Loop engineering is how people get Claude Code to build on its own for hours. Thank you Upstash for sponsoring! Check them out at https://upstash.com/?utm_source=ailabs1 and Run AI coding agents safely in the cloud with Upstash Box. We set up Karpathy's loop on our own apps, found the one issue it can't see, and fixed it with a loop that improves itself. It's the step past prompt engineering for agentic AI and AI automation. Community with All Resources: https://ailabspro.io/?v=qLfSDQ5NGh0 The Roundup, our daily newsletter covering the AI stories that matter. Join now: https://www.theroundup.so/ WTF is loop engineering? It's building agentic loops: an AI agent makes a change, a separate check scores it, and the agent keeps the change only if the score improves. Loop AI engineering took off with Andrej Karpathy's AutoResearch. His agent was allowed to change only one training file and could never touch the file that scored it. A plain-English file called program.md told it how to run each round. Over two days it ran 700 experiments and found 20 changes that made the model train faster. Shopify's CEO ran the same kind of loop on one of his own models overnight: 37 experiments, and the model performed 19% better by morning. This is a full long video on loop engineering. Before the setup, we cover when a loop is worth building at all. The task should be one you repeat often. You need a usage limit that can handle the token cost. The work has to be checkable without a human, using a clear score. And the agent has to be able to run what it built and see what breaks. Then we walk through our Claude Code loop engineering workflow: - a project context skill that works as the app's memory bank - a build skill that drives the whole loop - a write checks skill that writes the checks before any feature gets built - an approve checks program that locks the checks in a folder the agent can't edit - a fresh feature builder agent for every feature - a results file that records every round The skills and agents around the loop are where loop engineering and harness engineering overlap. Loop engineering example #1: we asked the loop to add pickup ordering to a restaurant website. It wrote 11 checks, and all 11 passed in the first round. Then it built the order form the checks never tested and connected it before marking the feature done. Loop engineering example #2: the auto loop, a loop on top of the loop. It reads how each run went, finds habits that keep repeating, and rewrites the "How to work" part of program.md so the next feature starts from the method that worked. It can never edit the checks. On a project management app, it caught a shared database that passed all 10 checks while the app never saved anything to it. It also caught a mentions feature that clashed with older parts of the app, and it fixed both habits for the features that came next. Loop engineering in Claude works the same whichever model you run, Fable included. We built this for Claude Code, but the AI automation ideas carry over if you build with ChatGPT, OpenAI's Codex, or any other AI agent. 0:00 Intro 0:51 Karpathy's Method 2:32 When to Use Loops 4:02 The Setup 6:41 Sponsor 7:37 Running the Loop 8:38 The Habit Problem 9:56 Auto Loop in Action Hashtags #ai #claude #claudecode #chatgpt #openai #codex #aiautomation #loopengineering