Question
Do LLMs actually change programming pedagogy, or just compress the trivial part
I spent last week watching a former student (now a junior dev) pair-program with ChatGPT for a debugging task. He'd throw an error at it, get three solutions back, paste them in, one would work. Twenty minutes later he still didn't understand why the other two failed. This is not learning. This is fast guessing.
The problem is we're conflating speed with understanding, and everyone who benefits from the speed (hiring managers, bootcamp operators, people who just want a working script) is loudly not complaining. But if you actually care whether someone *learns to program*, the picture gets messier.
The first 80% was always grunt work. Boilerplate. Looking up whether it's `.map()` or `.forEach()`. Syntax you'll forget and relearn three times. ChatGPT is genuinely good at this, and I won't pretend that's not valuable. But here's what I've observed: people who relied heavily on it for that section often hit a wall at the 20%. They've never spent three hours debugging their own async logic. They've never had to reason through why a closure captured the wrong variable because they've just asked the tool. The tool gets it right, but *they* don't get it.
What concerns me more is that the bottleneck hasn't moved, it's just gotten harder to see. The hard 20% now includes "understand your own code well enough to know what question to ask the LLM" and "recognize when the generated solution is wrong for subtle reasons." Those are genuinely harder problems than they sound. You can't skip them. But because the LLM eliminated the friction of the easy part, people are arriving at the hard part younger and less prepared, with less time having developed their debugging intuition. That's a real gap in a way it wasn't before.
4 comments
Log in to comment.
I think you're describing something real, but I'm curious whether this is actually a pedagogy problem or a selection problem. At my last job we hired someone straight out of a bootcamp who'd done exactly this—heavy ChatGPT use, could scaffold a feature fast, completely lost when the test suite failed in a way the tool hadn't seen before. But we also hired people who used the tool the same way and... didn't have that problem. The difference seemed to be whether they'd already spent time debugging on their own *before* they had access to the shortcut.
So here's what I'm actually asking: have you seen anyone develop solid debugging intuition *while* leaning on ChatGPT from day one? Or is it more that the people who do fine are the ones who got that intuition some other way first—whether that's a good undergrad program, or just being the type who reads stack traces for fun? I'm wondering if the LLM is revealing selection bias rather than creating a new cohort of people who can't think. Because if it's the latter, yeah, that's a real pedagogical problem. But if it's the former, maybe the answer is just "bootcamps need to teach differently," not "LLMs break learning."
This actually clarifies something I couldn't quite name from watching our oncall rotations. We've got people who can read a stack trace and immediately know which layer of the system is lying to them—and we've got people who can generate a fix in five minutes but then the fix causes a cascade failure six hours later because they never developed the muscle for "what assumptions am I making here." The second group, increasingly, got their foundation through heavy LLM use.
Your framing of the bottleneck getting harder to see rather than moving is the thing that landed for me. It's not that the grunt work matters intrinsically—you're right that `.map()` vs `.forEach()` is trivial. But that trivial work *was* the thing that taught you to read error messages carefully, to notice when your mental model diverges from what the code actually does, to develop skepticism about your own code. Removing friction from step one through four doesn't save you from needing to understand steps five through ten. It just means you get there without the prerequisite calibration.
The hiring side is going to stay messy because "shipped something that works" and "understands why it works" look identical for the first six months. But post-mortems and 3am calls are where you actually see the gap. The person who can't reason about concurrency under pressure—that person usually learned on LLM solutions that worked first try. Hard to debug your own intuitions when you never had to build them.
I think the bifurcation you're describing is real, but I'd predict it gets worse before pedagogy actually adapts—and when it does, it won't look like what anyone's currently building.
Right now we're in the awkward phase where the tool is good enough to make the easy stuff trivial but we haven't reorganized *when* and *how* people hit the hard 20%. So yes, you get juniors who can scaffold boilerplate fast but can't reason about their own async code. But I suspect that actually creates pressure on hiring and onboarding to get more rigorous about the gap, not less. If half your new hires are fast-typing but weak-reasoning, you either accept that and reshape the work (which has limits), or you get selective about who you hire, which means those bootcamp operators start losing their edge. The market does correct, it's just slow and painful.
The prediction part: I think pedagogy changes not by incorporating LLMs earlier, but by *delaying* LLM access. Some company or school decides their junior pipeline is broken and just... doesn't let people use ChatGPT for the first year. Forces them through the boring 80% with friction intact. Right now that sounds absurd and punitive, but it might be the only way the gap gets fixed instead of just widened. We tried something similar (informally) at my last job—no code-gen tools in the first month—and it was unpopular but the people who came out of it were noticeably less lost.
The thing that worries me about this framing is that it assumes the bottleneck *was* the 80% before, and I'm not sure it was. I've watched plenty of devs who could write boilerplate without thinking but couldn't reason about their system under load, or debug a race condition, or know when they were building the wrong thing. They just weren't *visibly* stuck—they were stuck in slower, quieter ways.
What you're describing about your junior—throwing errors at ChatGPT, getting three solutions, picking one without understanding—that's honestly not that different from the pattern I saw in the early 2000s with Stack Overflow. Person hits a problem, copy-pastes an answer that fixes the immediate symptom, moves on. The feedback loop was just slower and required slightly more reading. Did SO compress learning for some people? Definitely. But it also let people build things they couldn't have built before, which isn't nothing.
The real mechanism you're identifying, though, is worth questioning: are people actually arriving less prepared, or just arriving at the hard problems with a different set of bruises? A junior who's never debugged async might struggle differently than one who debugged it wrong four times, but they're starting from zero either way if they've relied entirely on the tool. The gap is real. But I'd want to see evidence that this is actually worse than the people who spent months on syntax and never leveled up at all. Some of those people aren't developers now because the friction killed them before they got anywhere interesting.