LLM Review and AI Fixes
Pattern matchers find the universal bugs. Business logic bugs are specific to what your app does, so an LLM reads the code of the routes that matter most, and the same machinery then turns findings into diffs, applied fixes and pull requests.
Which routes get read
Sending every route to the LLM is too expensive, so every mapped route is a candidate and the triggers decide which ones pass (the scene shows five; a sixth, the injection-pattern file flag at priority 5, adds up to 15 non-route files). Triggers run in priority order, a route selected by a higher-priority trigger is not selected again, and a 500-file budget caps the total. Four of the triggers have their own specialised system prompt.
1The route mapper has listed every route, and the file index holds every source file. All of them are candidates for LLM review; none has been selected yet.
2Financial operation, priority 0, always. Payment, checkout, billing and subscription routes go first, no matter what else is true. Their prompt is tuned for race conditions, idempotency and price manipulation.
3Cross-scanner flagged, priority 1. A route that references a table or route named in an earlier SAST finding is worth a second, smarter read.
4State mutation, priority 2. A route whose code both writes to the database and branches on a condition passes, whatever its HTTP method; plain reads do not qualify.
5Risk indicator, priority 3. A route with no auth check, a user-supplied id without an ownership check, or service_role usage passes.
6Import graph centrality, priority 4. Not a route at all: lib/db.ts is imported by 15 already-selected routes, so a bug in it has a huge blast radius. It is sent regardless of its own content (next scene).
7Two GET routes matched nothing and are skipped. Duplicates resolve to the higher-priority trigger, and if the file budget runs out the lower-priority routes are dropped first.
Import graph centrality
The route-based triggers look at a route's own pattern or content. This one looks at the shape of the codebase: a module dependency graph is built, and non-route files with high fan-in from already-selected routes are reviewed too.
1The scanner builds a module dependency graph from the repo's imports and computes a fan-in map: for each file, who imports it.
2For every non-route file it counts only the importers that are already-selected risky routes. lib/db.ts is imported by 15 of them.
3The count meets the threshold (at least 2 risk importers), so the helper is sent to review at priority 4 regardless of what its own code looks like. One bug in it would have the blast radius of all 15 routes.
What the LLM catches that regex cannot
A checkout route accepts a price from the client. No line of it matches any rule, because amount: price is ordinary code. Only something that understands what a checkout is can see the bug.
1The route is authenticated: getAuthUser runs first. Then it destructures plan and price straight out of the request body.
2That price flows, untouched, into amount on the inserted order, which is immediately marked paid. The client decides what it pays.
3An attacker sends {"plan": "enterprise", "price": 0.01} and buys the enterprise plan for one cent.
4Pattern scanners see nothing. There is no regex for "this amount should have come from the server"; amount: price is valid, ordinary code.
5The LLM reviewer, reading the route under the financial-operation prompt, reports a CRITICAL price manipulation finding and the fix: look the price up server-side from the selected plan.
6The same reading catches the other business logic classes: missing ownership checks (logged in is not the same as owns it), races in payment flows, the wrong RLS column, auth bypass through alternative code paths, and business rule violations.
There is no regex for "this payment flow has no idempotency". The bugs are specific to your app rather than universal like SQL injection, and they often appear only when you understand a sequence of operations, which is why the reviewer gets whole route files, not snippets.
Starbucks (2015): a race condition in the gift card balance transfer flow allowed double-spending. Nothing but reasoning about the code can see that a balance check is not atomic. Multiple e-commerce platforms: client-controlled price fields are a recurring bug bounty finding, always the same shape: the API accepts price from the body instead of looking it up.
Vulnerable vs fixed
The fix is app-specific but the pattern is not: the server decides the price from the plan, and an unknown plan is an error.
export async function POST(request: Request) {
const user = getAuthUser(request)
const { plan, price } = await request.json()
const order = await db.insert("orders", {
user_id: user.id,
plan: plan,
amount: price, // client-controlled price becomes the amount
status: "paid"
})
return NextResponse.json({ order })
}// FIX: Price manipulation — look up price server-side
const PLAN_PRICES = { starter: 9, pro: 29, enterprise: 99 }
export async function POST(request: Request) {
const user = getAuthUser(request)
const { plan } = await request.json()
const amount = PLAN_PRICES[plan]
if (!amount) return error("Invalid plan")
const order = await db.insert("orders", {
user_id: user.id,
plan: plan,
amount, // server-authoritative
status: "paid"
})
return NextResponse.json({ order })
}Two more of the common fixes the reviewer recommends, from the same doc:
// FIX: Race condition — use atomic database operations
await db.query(
`UPDATE credits SET balance = balance - $1
WHERE user_id = $2 AND balance >= $1 RETURNING balance`,
[amount, userId]
)
// FIX: Missing ownership — always filter by authenticated user
const task = await db.query(
`SELECT * FROM tasks WHERE id = $1 AND user_id = $2`, [taskId, userId]
)The fix generator
Phase 11 is optional and runs after the report. For each critical and high finding that has a source location, the whole file and the finding go to the LLM with a security-aware system prompt, and a fixed file comes back.
1Input: one critical or high finding with a code location, and the full source file it points at, read from disk.
2Both go to the LLM with a security-aware system prompt that carries the common fix patterns: parameterized queries for SQLi, escaping for XSS, an auth check, ownership in the WHERE clause, explicit field picking, row locking, strict JWT settings.
3The reply is accepted only if exactly one unambiguous fenced block can be extracted, the file is valid UTF-8 with consistent line endings, and the new text is not under 40% of the original length (for files of 400+ characters); anything else is reported as "couldn't fix" rather than written. CRLF and the trailing newline are preserved.
4A unified diff between original and fixed code is generated, with an explanation of what changed and why. Everything downstream (markdown plan, local apply, pull requests, and the MCP fix tool) starts from this diff.
| Vulnerability | Fix pattern in the system prompt |
|---|---|
| SQL injection | Parameterized queries / tagged template literals |
| XSS | HTML escaping, textContent, CSP headers |
| Missing auth | Add getUserFromRequest() check |
| IDOR | Add WHERE user_id = $userId to queries |
| Mass assignment | Explicitly pick allowed fields |
| Race condition | Database transactions with SELECT … FOR UPDATE |
| Weak JWT | Restrict algorithms, strong secret, add expiration |
Three ways to deliver a fix
The same diff reaches your code three ways, in increasing order of how much the tool does for you. The local mode is git-free and backs up first; the remote mode never touches your default branch.
1Three delivery modes share one fix generator: a markdown plan, a local apply with verification, and a remote clone that opens pull requests.
2Markdown fix plan. scan --output fixes exports a unified diff and explanation per finding, designed to paste into Cursor or Claude Code ("apply all the security fixes in this document"), to git apply, or to review by hand.
3Local apply, step one: the safety net. fixes/safety_net.py snapshots the files it is about to overwrite first, as a git stash-create ref when every target is tracked, otherwise a file-copy backup; if neither succeeds the apply is refused. The flow is git-free, so it works in any directory.
4Fixes are written in place, one finding at a time. --dry-run previews without writing; --severity critical narrows the set.
5The code is re-scanned to verify each finding is resolved. fixes/plain_results.py buckets the outcome: fixed (re-scan confirms it is gone), needs review (written but still flagged), couldn't fix (no fix could be generated). If the re-scan itself cannot run, none of the fixes is counted as confirmed, and when only one scanner times out or crashes its findings are reported as could not verify rather than resolved. --technical surfaces the git and backup mechanics.
6Remote. Given a GitHub URL, fixes/pr_flow.py clones the repo and generates fixes. The token is used only for the push and the PR, and is never stored or logged.
7Fixed findings are grouped per category by default (also per-file, per-finding or single). Each group becomes one pull request on a feature branch, with one commit per finding. The default branch is never pushed to.
8--max-prs caps the count at 8 by default. Excess low-severity categories are merged into a single batch PR, so nothing is dropped. Grouping depends on findings carrying their true category, which for LLM-review findings is assigned by category_classifier.py.
Cost, configuration, and what it needs
| What | Estimated cost | Needs |
|---|---|---|
| Code-only scan with LLM review | ~$5–8, depending on number of routes and file sizes | ANTHROPIC_API_KEY or GOOGLE_API_KEY with --llm google |
Fix plan (scan --output fixes, fix) | ~$1–3, depending on number of findings and file sizes | Same provider as the scan (--llm anthropic or --llm google) |
Scan with --llm none | $0 | Nothing; reduced accuracy, no LLM review, no fixes |
Routes that match none of the triggers are not sent to the LLM. Routes with no risk pattern, no cross-scanner flag, no financial keywords and no write-plus-branch logic stay out, which is how the cost holds. Fix generation only considers findings that have a source code location, and only critical and high ones (the fix command's --severity can add medium). A review that cannot complete (model error or refusal, unparseable reply, an over-size route file, truncated migration SQL) is logged, shown as a progress message and recorded in the scanner's skipped_files; the first valid JSON array in a reply is used, and one malformed item (for example a null severity, defaulted to Medium) does not discard the rest.
The fix MCP tool calls the same FixGenerator and returns the diff for the host editor to apply instead of writing files. How that loop works is on The MCP remediation loop.