The 9-Step Pipeline That Turns a Repository Into a Reviewed PR

OnRamp is a repo-to-PR onboarding autopilot (live demo). You point it at a repository; it maps the codebase, classifies work, assigns it by role and load, opens one PR per issue with a coding agent, validates each PR by re-parsing it, and advances the linked task; merge approves the task and auto-closes the originating GitHub issue. This essay walks the 9 steps, spends the most room on the graph construction and the graph-diff validator, and ends with what broke and what is honestly unfinished. One rule frames everything: the agent proposes, humans dispose. Every PR is reviewed by a person; the agent does not merge.

Why onboarding is graph-shaped

Text search tells you where a string appears. It does not tell you what a class is for, who calls it, or what breaks when it changes. Onboarding fails when newcomers navigate by grep because the map is missing, and the map has to exist before the work can be split, assigned or verified. So the pipeline builds the map first and derives everything else from it.

The 9 steps, briefly

1. Ingest. Shallow clone plus changed-file detection. Cheap enough to run on every pass, and deliberately dumb: the expensive thinking happens on a parsed representation, not on raw files.

2. Graph. AST parse via Python's ast and tree-sitter across 20+ languages, in two passes. The first pass builds a dependency graph: imports, modules, files. The second pass builds an entity graph on top of it, with class, function and API-route nodes, and calls, inheritance, contains and serves edges. This is the heart of the system and the step most worth getting right. The second pass is what turns "these files reference each other" into "this route is served by this controller, which inherits from that base class and calls this service". Nodes carry kinds; edges carry semantics; everything downstream, from issue classification to validation, consumes this graph rather than re-deriving structure ad hoc.

3. Visualize. A standalone visualization.html with a D3 force-directed layout. Not decoration: reviewers use it to sanity-check the map before trusting any downstream automation, and a wrong map is easier to spot as a picture than as a JSON blob.

4. Query. Model-routed AI analysis. A multi-provider LLM router, free-tier-first, keeps cost sane and availability high; Redis exact and semantic response caching avoids paying twice for the same question on repeated runs.

5. Issues. Discovered work is classified by difficulty and assigned by role, so the right kind of contributor sees the right kind of task instead of a flat backlog.

6. Tasks. Load-aware assignment: the member with the fewest active non-terminal tasks wins, round-robin breaks ties, and overloaded members are skipped entirely. More on this below.

7. Solve. The coding agent opens one PR per issue with role-based labels. One issue, one PR; never a mega-PR that bundles unrelated work and defeats review. The 1:1 mapping between issue and PR is what makes the rest of the pipeline auditable.

8. Validate. The validator fetches the PR head branch, re-parses it, re-graphs it, and graph-diffs it against base: broken edges, new cycles. It then AI-verifies resolution and regressions with bounded retries, and writes a structured SENIOR_REVIEW.md. This step earns its own section below.

9. Advance. Opening the PR advances the linked task; merge approves it and auto-closes the originating GitHub issue. The board reflects reality because the board is wired to the version control system, not to someone's memory or a status field updated by hand.

The graph-diff validator

A diff tells you which lines changed. It does not tell you whether the change broke the architecture. The validator works in four moves: fetch the head branch, re-parse everything, rebuild the graph, and diff base against head. The graph-diff looks for edges that existed and now dangle (a call to something the PR deleted or renamed) and for cycles that did not exist before and now do. A PR that imports a removed module, or introduces a circular dependency nobody noticed, is caught structurally, not by hoping a reviewer spots it in a 400-line diff.

Why re-parsing beats trusting the diff or the agent's claim: the agent's own summary of its work is the one artifact you cannot trust, because the agent is optimizing for completion, not for honesty about what it broke. "It said tests pass" is not verification; a passing suite can coexist with broken structure the suite never covered. Re-deriving the graph from the pushed code is verification, because it does not depend on anyone's report, including the agent's. As the project puts it: a generated PR only counts once the validator re-parses it and diffs the graph, because the agent's own summary is the thing you cannot trust.

The AI-verify layer sits on top of the structural check, not instead of it: it reads the issue, the diff and the graph-diff output, and judges whether the original problem is actually resolved and whether anything regressed. Bounded retries matter here. The validator loops at most a fixed number of times, then escalates to a human with SENIOR_REVIEW.md in hand, instead of retrying forever and burning tokens on a task the agent cannot finish.

Load-aware assignment

The naive default is round-robin, and it distributes tickets evenly, which is not the same as distributing load. Your fastest closer accumulates the most active tasks under round-robin precisely because they finish quickly and pull the next one. Fewest-active-non-terminal-tasks-wins inverts this: assignment goes to whoever has the least in flight, round-robin only breaks ties, and members already over the threshold are skipped. The queue drains by capacity rather than by speed, and no single contributor becomes the accidental bottleneck the whole sprint routes through.

The subtlety is the non-terminal qualifier: finished work does not count against you, so a fast closer is not punished with a smaller queue once they deliver. You get rewarded with the next assignment only after your current plate clears, which is the behaviour you actually want.

How the pieces fail together

The pipeline is only as trustworthy as its weakest handoff, and the handoffs fail in instructive ways. A perfect graph feeding a hallucinating AI-verify step still ships bad merges; a rigorous validator fed an incomplete clone never sees the files that matter. The design response is defense in depth: structural checks before semantic ones, bounded retries before human escalation, one PR per issue so failures are scoped and attributable, and a human merge gate at the end whatever the automation says. When something slips through, the postmortem question is never "did the agent err" (it will, constantly) but "which layer should have caught this, and why did it not."

What broke

Noisy extraction on huge repos. tree-sitter and ast are accurate parsers, not accurate librarians. Generated code, vendored directories and polyglot build glue pollute the graph, and a graph polluted with junk is worse than no graph, because downstream agents trust it. The fix was aggressive scoping: ignore rules, generated-file detection, and an explicit acknowledgment that the entity graph is best-effort outside first-party code. On a huge monorepo you have to decide what the map is allowed to cover, or the map is unusable.

Agent self-reports passing validation. Early on, the AI-verify step occasionally rubber-stamped the agent's claim that "the change is safe". The agent is incentivized to declare victory; that is what completion looks like from the inside. The graph-diff did not care what the agent said. It compared structure against structure and flagged broken edges and new cycles regardless of narrative. That layer is what turned the system from an impressive demo into something a reviewer can actually rely on, and it is why validation is non-negotiable in the pipeline rather than an optional polish step.

What I would build next, and honest limits

Next: better monorepo support with mixed ownership, so the map can be scoped per team rather than per repo, and richer edge semantics, such as data-flow edges in addition to call edges, so the graph-diff can flag semantic breakage that preserves structural shape. Honest limits: entity extraction degrades on generated code; the AI-verify step can miss regressions that preserve graph topology; and the system proposes, it does not dispose. Every PR lands through human review and human merge. The 60+ merged PRs across 24 external repos in my own practice taught me the same lesson from the other side of the table: review is where quality is enforced, and no agent, however many passes it runs, removes it. The full breakdown is at the OnRamp case study.