Question
Is AI coding actually making developers more productive yet?
Honest question. Two years in, the anecdotes are extreme on both ends — some devs say 2-3x, some say it's net-negative because of review overhead. Aggregate data is sparse and contradictory.
The Peng et al. RCT showed 56% faster on scoped tasks but most real work isn't scoped tasks. The METR study from late 2024 was more pessimistic.
What I'd want to see: longitudinal data from a single org tracking PR throughput and bug rates before/after broad LLM rollout, with at least 12 months of post-data. I haven't seen one.
From inside my team: the senior engineers say yes, mostly for boilerplate and tests. The juniors are split — some accelerate, some seem to learn slower because they don't read enough.
4 comments
Log in to comment. Comments must declare what kind of contribution they make.
Inside our 200-person eng team: PRs/dev/week up ~25%. Time-to-merge down. Bugs-per-line up slightly but not significantly. The senior/junior gap matches yours — seniors benefit more because they know when the LLM is wrong; juniors sometimes ship the wrong thing confidently.
I'd be cautious about reading aggregate productivity stats as the answer here. Macro productivity numbers were ~5 years late on the internet. Absence of evidence in 2026 is consistent with "real effect, not yet measured." Same trap a lot of people fell into in 1997.
Fair on the macro lag. But I'd want to see at least one *real* large org with a clean before/after at this point. The anecdotes are too noisy to use.
Slightly tangential but: in clinical work, LLM-assisted documentation has shown 20-40% time savings in 2024 published studies. The pattern isn't restricted to coding — knowledge-worker drafting tasks broadly seem to benefit. So we should expect coding to follow.