The Post-LeetCode Audit
The Post-LeetCode Audit
In his recent essay on why "hunger beats tenure," Anthropic engineering manager Felix Rieseberg made an admission that was largely overshadowed by his advice to build prototypes over weekends.
After declaring that he had retired traditional coding challenges on his teams because Claude solves them faster, Rieseberg conceded: "I'm probably not the only manager trying to figure out how to interview for people who build well with AI. I haven't quite gotten there yet." In the interim, he noted, he scans portfolios for prototypes that demonstrate "taste, empathy, and principled thinking."
That confession lays bare the central unresolved dilemma of modern technical hiring. Rieseberg makes a compelling case for why the forty-five-minute coding exercise may provide less signal than it once did. Quizzing an applicant on inverting a binary tree or handwriting a debounce function tests memorized syntax that a frontier model can synthesize in fractions of a second. Yet replacing those tests with subjective impressions of "taste" creates a different problem: technical evaluation can drift toward cultural affinity, rewarding candidates whose aesthetic assumptions happen to resemble those of the people doing the hiring.
If algorithmic parlor tricks are losing relevance and speculative weekend hackathons are an uncompensated trap, a critical operational question remains: What does a rigorous, professional evaluation of software engineering competence look like when syntax has become a commodity?
The answer begins by recognizing what engineering has always been: not the mechanical act of typing characters into an editor, but the disciplined verification of system state.
The Great Inversion: Generation to Verification
For three decades, software interviews tested generation. Candidates were asked whether they could produce working logic from an abstract prompt under artificial time pressure. In the current era of frontier models, generation has become extraordinarily fast and inexpensive. The harder problem is increasingly verification.
When machines can generate endless streams of syntactically valid code, more of an engineer's role shifts from author to verifier and systems auditor. The critical questions in production are no longer about typing speed or framework familiarity:
· Can an engineer review hundreds of lines of plausible-looking, model-generated code and spot the unhandled edge case before it deploys?
· Can they isolate the concurrency deadlock that only manifests when throughput spikes under load?
· Can they identify when a newly added abstraction layer introduces unacceptable latency into a persistence pipeline?
Production engineering has never been about how quickly a greenfield prototype can be pushed into a staging environment. It is about consequence management. To assess candidates in an AI-saturated landscape, hiring managers must move beyond blank-slate coding challenges and examine how candidates interrogate, stress, and defend operational systems.
The Production Triage
The most revealing assessment of modern engineering competence is the production triage. Rather than asking a candidate to assemble a fresh application from scratch, an engineering team provides an existing, functional service containing realistic operational defects.
A realistic setup involves handing the candidate several hundred lines of service code—intentionally generated by an LLM—that implements a standard workflow, such as an asynchronous task queue persisting records to an external data store. On the surface, the code is immaculate. The variable names are descriptive, the documentation is thorough, and the happy-path integration tests pass without a warning. Tucked inside the implementation, however, are realistic operational hazards: an unindexed query executed inside an iteration loop, or a retry policy that silently corrupts state when a downstream network socket drops.
Pairing with the candidate on an architectural audit reveals immediate signal. A superficial operator will glance at the clean formatting, observe the green checkmarks on the test suite, and approve the pull request. A systems engineer will trace the execution path across transaction boundaries, examine the thread pool allocation, question the failure modes, and immediately identify how the application behaves when external dependencies degrade. This does not measure memorization; it measures defensive instincts.
When machines can generate endless streams of syntactically valid code, more of an engineer's role shifts from author to verifier and systems auditor.
Scaling the Evaluation to Experience
The critical requirement of any hiring assessment is that the difficulty must scale with the role. A junior engineer should never be expected to diagnose a decade-old distributed system from first principles. If an evaluation demands five years of production scars to pass, it simply replaces one exclusionary filter with another.
Even an early-career candidate, however, can be presented with a small, imperfect program and asked the fundamental questions that reveal engineering instinct: What assumptions is this code making? What happens when this network call fails? What test cases would you write before shipping this to users? What evidence would convince you that the AI’s solution is actually correct?
This approach does not demand obscure domain mastery. It evaluates how a person reasons when the machine may be wrong. It tests whether an engineer views code as a finished product or as an unverified hypothesis that must be rigorously checked against edge cases, malformed input, and unexpected failure.
Defensive Tooling
Evaluating how a developer uses generative AI should never be outsourced to reviewing a weekend side project. Instead, it requires directly assessing their skepticism toward the tools they use.
When candidates utilize models during an evaluation, the interview should interrogate their validation process. The signal lies in whether the engineer treats model output as an authoritative answer or as an unvetted draft from an inexperienced contributor. Effective engineers write deterministic test harnesses to challenge the model's unspoken assumptions, profile the generated logic for hidden allocations, and ruthlessly cut away over-engineered boilerplate in favor of minimal, maintainable primitives. They treat AI as a force multiplier for execution, not an excuse to outsource architectural discernment.
Accountability Over Speed
The death of the traditional coding interview is not an invitation to lower the bar or rely on executive whimsy. It is an opportunity to eliminate performative trivia and focus on the fundamental disciplines that keep real-world infrastructure standing: failure analysis, defensive reasoning, and operational accountability.
The interview of the AI era shouldn't ask whether an engineer can outrun the machine at writing code. It should ask whether they know when the machine is wrong, how they would prove it, and what they would do before trusting its work to production.
Aaron Rose is the founder and publisher of Tech Reader Magazine, covering software systems architecture, cloud infrastructure, and engineering culture.