Back to articles🏢Enterprise AI

AI Factories Still Need Engineers Where Judgment Matters Most

The AI factories posting huge merge counts still run on human judgment. Engineers just moved from typing code to guarding intent and approval.

Paul Lopez
··6 min read
The Dark Factory Has the Lights On

The Dark Factory Has the Lights On

Part 1 of 3. The firms shipping thousands of AI-written changes kept their engineers. They moved them to the few places where judgment is expensive.

Sunnyside Daycare looks like a paradise that runs itself. The kids are cheerful, the rooms are bright, and nobody seems to be in charge. When Woody and the toys arrive in Toy Story 3, it appears that the place simply works.

Then the film pulls the curtain back. A small inner circle under Lotso controls the gates, assigns the rooms and decides who gets punished. The quality of Sunnyside was never about automation. It depended on where those few sat.

Leaders are now hearing that AI labs run software factories with the lights off, and they are being asked why their own organizations are not doing the same. The factories with the biggest numbers look lights-out from a distance, but up close the engineers are still there, positioned at intent and at approval.

The Lights Are On Everywhere That Matters

The figures below come from Anthropic's October 2026 engineering account and from reporting on Stripe's agents in March 2026.

The numbers are real. Anthropic reports that merges into its monorepo went from roughly 2,500 a day in May to more than 30,000 a day by September, and that Claude now writes about 90 percent of Claude Code itself. Stripe's internal agents open more than 1,300 pull requests a week. If you stop reading there, the story writes itself: the machines build the software and the engineers watch.

Keep reading and the story changes. At Anthropic, a classifier sorts every change into a risk tier, low-risk changes merge on their own, medium-risk changes need a person, and an agent's approval never counts as a human's. At Stripe, a human reviews every one of those 1,300 weekly changes.

Risk-Tiered Merge Approval at Anthropic and Stripe

None of these organizations removed the engineers. They moved them, out of the typing and into the two places where a wrong call is expensive: deciding what should be built, and deciding what is allowed to ship.

PostHog sits in the middle of the spectrum. Its agents write most of its pull requests, and its humans still skim most of them. A dark factory where a person reviews the output is an expensive nightlight.

Cheap generation does not remove the need for judgment; it makes judgment the scarcest input in the building.

The pattern is consistent across all three. The factory automates the cheap, well-defined work and concentrates human attention on intent and on risk.

The One Dark Factory, and What It Took

There is one well-documented dark factory, and it is worth studying precisely because of what it took. StrongDM's AI team started with two rules: no human writes the code, and no human reviews it. To make that safe, the team built behavioral clones of the services its product talks to, so agents could test against Okta and Slack look-alikes at volumes no real service would allow. It keeps its test scenarios outside the codebase, hidden from the agents, so the builders cannot quietly tune the code to pass them.

That is not the absence of engineering. It is a different and very deliberate kind of engineering, done by a small team on a young product, with a stated benchmark of about a thousand dollars of model spend per engineer per day. The lights are off on the code. They are blazing on the test harness.

This is the exception that proves the pattern: the effort did not disappear, it moved into verification infrastructure.

Verification Is the New Bottleneck

When code gets cheap, everything that checks code becomes expensive. Anthropic's own account says the bottleneck moved to the developer infrastructure that was built for human scale, source control, builds, and continuous integration, and it ended up replacing git for its monorepo and retiring the merge queue to keep up. The same team describes a single pull request cycling through more than fifty automated review rounds overnight before a person stopped it, and says it does not yet have circuit breakers that tighten approval when quality slips.

That is the true shape of the frontier. Generating a change is solved well enough. Knowing whether the change is right, keeping a growing codebase understandable to the people accountable for it, and stopping a loop that has gone wrong are the unsolved parts, and they are exactly where the humans sit.

Addy Osmani calls the hidden cost here comprehension debt: the gap between how much code exists and how much of it any person actually understands. It accumulates silently, because the tests stay green while the team's grasp of the system erodes. Practitioners who tried lights-out and pulled back, Dex Horthy among them, make a related maintainability argument. Code nobody read is code nobody can confidently change, and the bill arrives the first time something breaks at 2 a.m.

So the leader's question is not whether to trust the model. It is this: for each loop in your delivery process, how cheap and how trustworthy is verification? Where it is cheap and hard to fake, the lights can dim. Where it is expensive or easy to fake, keep a person at the control point.

The Counterargument Deserves a Hearing

The strongest objection is that lights-on is a phase, not a destination. One company already runs a factory nobody reads, the models improve every quarter, and the labs themselves say end-to-end automation of software engineering is close. On that view, an enterprise that designs its process around human approval is building for a world that is about to end.

Those predictions of imminent full automation come from the companies selling the models, which is a reason to watch their operating practice more closely than their forecasts.

Part of that is right. Lights-out already works for narrow, well-specified work where verification is automated and cheap, and the share of work that qualifies will grow. But the setting is not one switch for the whole company. It is a dial on each loop, and it moves when verification gets cheaper, not when generation gets better.

Whoever can verify cheaply earns the right to automate, and everyone else is guessing.

The organizations furthest ahead are proving that point: they dim the lights on documentation and dependency bumps, and they keep them bright on authentication, billing, and anything an auditor will ask about.

Who Holds the Keys

Sunnyside never ran itself, and neither do the factories that look like they do; the real decision was always where to seat the few at the gates, not whether to keep them.

The question for an enterprise is not whether to turn the lights off but where its lights need to be. The answer comes from the cost of verifying each kind of change.

#ai-software-development#human-in-the-loop#enterprise-ai#code-review#ai-agents