The Three-Step Loop for Reviewing AI-Generated Code

A dangerous property of AI-generated code is that it can look right. It may compile, follow the requested style, use sensible names, and handle the case described in the prompt. None of that establishes correctness. In the generated code I have reviewed, the failures that slip through are often plausible ones: a loop bound that fails when the input length is an exact multiple of the page size, a library call unavailable in the deployed version, or a quadratic algorithm where the requirement was linear.
Human-written code can fail in the same ways, so generated code does not require abandoning ordinary review. The useful posture is to treat it as work from a fast but untrustworthy teammate: worth reviewing rather than automatically accepting or rewriting, and entirely your responsibility once merged. That posture leads to a repeatable loop: read for intent in risk order, test against a deliberate case list, then choose an appropriate unit of repair.
Step one: read for intent, in risk order
Read the highest-risk layers first and style last.
- Algorithm shape and requirement match. Does the code solve the stated problem at the required complexity, or a nearby problem that happens to look similar?
- Boundaries and state. Check loop bounds, index arithmetic, empty input, invalid values, state transitions, cancellation, and concurrency where relevant.
- Integration and security fit. Confirm APIs against the deployed dependency versions. Check error propagation, authorization boundaries, parameterized query binding, safe process argument passing, and context-appropriate template encoding or sanitization.
- Style. Review it last because clean presentation does not compensate for defects in the earlier layers.
The risks depend on the domain. Payment code needs close attention to duplicate operations and partial completion. Authorization code needs explicit resource and identity boundaries. Concurrent code needs race, cancellation, and retry analysis. A generic list is a starting point, not a substitute for identifying the failure chains that matter in the system at hand.
Here is a generated pagination function to carry through the loop:
def paginate(items, page, page_size):
total_pages = len(items) // page_size + 1
start = page * page_size
return {
"items": items[start:start + page_size],
"page": page,
"total_pages": total_pages,
}
Before judging it, specify the contract. For this example, items is a sequence, page and page_size are integers, pages are 1-indexed, and both numeric arguments must be positive. Invalid values raise ValueError. Empty input has zero total pages, with a request for page 1 returning an empty item list. A positive page beyond the last page also returns an empty list while preserving the correct total.
Against that contract, the risk-ordered pass finds several defects. total_pages is wrong whenever the item count is an exact multiple of page_size: ten items with a page size of five yields three pages instead of two. start = page * page_size treats pages as 0-indexed, so page 1 skips the first five items. A zero page size raises ZeroDivisionError rather than the specified ValueError, and nonpositive page numbers are accepted despite the contract.
Step two: test against a deliberate case list
Reading and testing are complementary controls. Reading can identify the wrong algorithm or an unsafe integration. Tests can expose boundaries, state transitions, and combinations the review missed. Build the case list from several categories:
- Happy path. Exercise the ordinary behavior described by the contract. Passing it is useful but weak evidence on its own.
- Boundaries. Include empty input, a single element, the last page, a page beyond the last, invalid numeric values, and the exact-multiple case.
- Equivalence classes. Choose representatives for distinct contract behaviors, such as a full page, a partial final page, and an out-of-range page. Also vary relevant data and state boundaries even when inputs follow the same control-flow path.
- Adversarial input. Consider wrong types, extreme sizes, hostile strings, repeated operations, and invalid state transitions according to the domain.
For the pagination contract, derive these expectations independently of the implementation:
- Ten items, page 1, size 5: items 0 through 4 and two total pages.
- Ten items, page 2, size 5: items 5 through 9 and two total pages.
- Ten items, page 3, size 5: an empty item list and two total pages.
- Empty input, page 1, size 5: an empty item list and zero total pages.
- A page or page size of zero:
ValueError.
The corresponding repair is explicit:
def paginate(items, page, page_size):
if page <= 0 or page_size <= 0:
raise ValueError("page and page_size must be positive")
total_pages = (len(items) + page_size - 1) // page_size
start = (page - 1) * page_size
return {
"items": items[start:start + page_size],
"page": page,
"total_pages": total_pages,
}
Rerunning the case list against this version produces the specified item ranges, totals, empty results, and errors. Those cases do not prove the function correct, but they confirm that the identified defects were repaired without changing the chosen out-of-range behavior.
Execution is only one source of evidence. Documentation checks can confirm that APIs exist and behave as assumed. Compilation, type checking, and static analysis can catch some invalid calls and unsafe flows before runtime. Property-based tests are useful for invariants such as “returned items are always a slice of the input,” while differential checks can compare results with a trusted implementation when one exists.
Generating tests is not verifying correctness
Asking a model to generate test scaffolding can save work, but tests and implementation may encode the same misunderstanding. Verification requires an oracle independent of the implementation under test. That oracle does not have to be written by a person: it might come from a specification, a trusted reference implementation, a property, or a verified dataset.
Generated fixtures, parameterization, and expected values can all be useful if they are checked against that independent source. The key question is not who typed the assertion. It is whether the assertion was derived without relying on the behavior it is supposed to verify.
Step three: choose the unit of repair
When a case fails, determine whether the defect is local or structural, then account for the code’s risk and your confidence in the diagnosis.
A wrong index expression, missing validation guard, or isolated ceiling calculation can usually be fixed by hand and followed by the relevant regression checks. In the pagination example, the corrected start offset and ceiling division preserve the intended approach.
A mismatch such as offset pagination where cursor pagination is required, or an O(n²) scan where the requirement is O(n log n), calls for a broader decision. Re-prompting may be sensible when the approach is recoverable. Useful inputs include the exact failing case, observed and expected results, the intended algorithm, and the complexity requirement, plus any domain constraints that affect the design. These details improve the next attempt but do not guarantee it.
Other reasonable choices are to discard the output, redesign the interface, implement the unit manually, or escalate it for specialist review. Higher-risk code and unclear failures warrant a lower threshold for those options. The goal is not to preserve generated code; it is to reach an implementation you can explain and verify.
Decomposition keeps review tractable
The loop works best on bounded units. Large generated modules make it harder to isolate assumptions and decide whether a failure is local or structural. Ask for one coherent piece at a time: the pagination math, the request handler that uses it, then serialization.
Independent review of each piece is not enough. Once the units are assembled, check their shared contracts and run integration or end-to-end tests across boundaries such as validation, error mapping, retries, and serialization. Decomposition reduces the size of each review problem, but composition introduces risks that unit checks cannot see.
A compact baseline
Customize this sequence for the system and its risks:
- Restate the contract, including invalid and out-of-range behavior.
- Identify the domain-specific failures with the highest consequence.
- Review algorithm shape, complexity, boundaries, state, and integration.
- Confirm dependencies with documentation and suitable static tooling.
- Derive expectations from an independent oracle.
- Test contract behaviors, boundaries, adversarial inputs, and relevant properties.
- Fix locally only when the diagnosis and approach are sound; otherwise re-prompt, discard, redesign, implement, or escalate.
- After assembly, review contracts and run integration or end-to-end checks.
The same habit applies in interviews
Even when an interview does not permit AI assistance, this loop practices relevant engineering behavior: restating requirements, naming invalid cases, deriving expectations independently, tracing execution, and explaining why a repair is sufficient. Coding interview success also depends on correctness, complexity, testing, and clear explanation, so generated code can serve as review material without becoming part of the interview itself.
Treat generated code as untrusted until its contract, implementation, and integration have been checked with evidence appropriate to the risk. The output may be fast; responsibility for accepting it remains with the engineer.