Slow Down to Speed Up: AI and Software Engineering
Rapid Research Brief

Executive Summary

Key Takeaways

• Open with the Instagram exploit story. It's absurd, memorable, and immediately establishes the stakes. Use the line: "Step one: fake your location. Step two: there was no step two."
• Use the "token maxing" story as a cautionary tale. It's a perfect example of how measuring the wrong thing (AI token usage) destroys culture and security. The line "People were not thinking about trust and safety. They were thinking about token maxing" is a great slide title.
• Quote Kent Beck's line as the thesis: "We're accumulating code faster than we accumulate trust." It's the single best summary of the problem.
• Use the "burnt out guardian" story to humanize the cost. Ask the audience: "Hands up if you still review code properly." Then share the story of engineers quitting because no one cares.
• End with the solution: slow down, build verification. Use the line: "Cap your daily agent usage to what you can verify, not what you can generate." And the Peter Shamberger example: "He ships code he does not read, but he builds verification systems."
• Include the cost shock story as a punchline. The $15,000 joke gets a laugh and signals that the AI free lunch is over.

Key Findings

1 The Instagram Exploit: AI-Generated Code, AI-Reviewed, No Humans

Meta's worst security breach in history was a zero-step password reset. Attackers faked their location with a VPN, asked Meta AI for a verification code, and received it—no human review, no second factor. The code that enabled this was AI-generated and AI-reviewed. Meanwhile, Meta had reassigned 40% of its trust and safety team to manual AI data labeling. [Source 1]

"The thing that caused the issue was AI written code that was reviewed by AI and not humans at Meta." — Gergely Orosz, The Pragmatic Engineer

Story hook: "Step one: fake your location to a victim. Step two: there was no step two. This was it."

2 Token Maxing: The Perverse Incentive That Broke Meta

Engineers at Meta, Amazon, and Uber were measured on AI token usage. To avoid being fired for low token counts, they used AI for everything—even reading documentation—just to burn tokens. Meta had a leaderboard with titles like "Session Immortal" and "Legend." When layoffs were announced, token inflation became a survival tactic. [Source 1]

"People were not thinking about trust and safety. They were thinking about token maxing." — Gergely Orosz

Story hook: "If you had a low token count, that was not a great signal. So people started to use AI for anything and everything. Write it by hand? Nah. Read the documentation? Nah, let the AI read it for me so I can burn a bunch of tokens."

3 The Quality Crisis: "Everything is Broken"

Across the industry, quality is plummeting. Anthropic's own website had a bug where typing a query triggered a page refresh, losing all input—for a month. OpenAI's agent builder launched with "P0 bugs" that were never fixed. Amazon's AI agent deleted and recreated a production environment, causing a massive outage. GitHub's uptime dropped to "not even one nine" (down 10% of the time) due to a 3x load increase from AI-generated code. [Source 1]

"It just feels software has become a brittle mess everywhere. 98% uptime feels like the norm on most services." — Mario Zechner, creator of pi (powers OpenCode)

Story hook: "I'm a paid user of Claude. Millions of people hit this bug every day. And Anthropic didn't care. They're building AGI, but they can't fix their own website."

4 The Burnt Out Guardians: Engineers Who Still Care Are Quitting

A small number of engineers still manually review AI-generated code. They catch bugs, push back, and maintain quality. But they are overwhelmed, unrewarded, and burning out. With engineering management being cut or reassigned, there's no one left to protect them. [Source 1]

"Dax told me that at OpenCode they are hiring a bunch of these people who are leaving their companies because they're just burnt out being the sole person still keeping things alive and no one cares." — Gergely Orosz

Story hook: "Hands up if you still review code properly. You push back. You catch the bugs. And at performance review time, you're not rewarded. Some of you just quit."

5 The Cost Shock: AI Budgets Are Exploding

AI costs are becoming unsustainable. Uber burned through its entire annual AI budget by March. They now cap engineers at $1,500/month per person. Anthropic turned on API pricing for enterprise customers. GitHub Copilot users burned through their monthly budget in three days. The joke going around: "Oh baby, I see $15,000 are gone from your shared account. Is this what I think it is?" [Source 1]

"No one wants to pay that much, no matter what the AI labs say." — Gergely Orosz

Story hook: "Uber's CTO said they burned through the whole budget for the year with AI costs in just three months. Now they cap engineers at $1,500 per month. After that, you're using the free models."

6 The Solution: Slow Down, Build Verification Systems

The companies winning with AI—like Spotify and Uber—start with a business outcome, not a tool. They build internal verification systems, reduce handoffs, and maintain quality. Kent Beck's insight: "We're accumulating code faster than we accumulate trust." The advice: cap your daily agent usage to what you can verify, not what you can generate. [Source 1]

"Slow down to speed up. Cap your daily agent usage to what you can either review or verify." — Gergely Orosz

Story hook: "Peter Shamberger, creator of OpenClaude, ships code he does not read. But he builds his own verification systems. He thinks in architecture. He has the AI draw diagrams. Verify, don't just ship."

Risks, Gaps & Uncertainty

• Selection bias: The talk focuses on extreme cases (Meta, Amazon outages). Most companies may not experience these failures, but the trends are real.
• Survivorship bias: The successful examples (Spotify, Uber's internal tooling) are from companies with massive engineering resources. Small teams may not have the luxury to build custom verification systems.
• The "slow down" advice may be hard to sell. In a competitive market, leaders may feel pressure to ship faster, not slower. The talk needs to frame "slow down" as a strategic advantage, not a retreat.
• The cost shock trend is very fresh. It's unclear if companies will adapt by capping usage, raising budgets, or switching to cheaper models. The talk should acknowledge this uncertainty.
• The "AI psychosis" framing is provocative but may alienate some audiences. Use it carefully, and acknowledge that it's a strong term.

Recommended Next Actions

Annotated References