Rainy Sundays

I was experimenting with an AI-based development tool recently and spent most of the time really impressed.  Most.  Together we built a pretty full-featured image organization application; I'd request a feature and seconds later it would appear for review.  Sometimes it needed a little poke; sometimes it was, honestly, better than what I'd envisioned.  And it'd take moments to do what would take me an hour.

Then came the problem: the app started crashing, and the AI repeatedly said it was working, and it wasn't an error that appeared readily on Stackoverflow.  I'd explain the issue, relay error messages, and it'd say it'd found the answer... tweak *something* but the error would still be present.  OK, I said, can I use the rollback feature to remove the problematic code?  Sure, said the LLM.  Rolling back to a previously working state will definitely fix the problem.  So I hit the rollback.

Same error, I told the AI.  "What issue is that?" it asked.  The rollback hadn't fixed the error, but it HAD lobotomized the AI of all knowledge of it.  *Brilliant*.

That's when I realized the real effect.  See, you do still need to be a developer to run an AI LLM, in the same way that a project manager still needs to understand the project.  But also deeper.  Because you inevitably have to debug errors that the LLM just can't fix, or 'understand'; but you can't ask the developer why they did this because *there isn't a developer to ask*.  There isn't a person to educate, or understand.  AI development reduces development to the least pleasant aspect, which is chasing intractable bugs in code that may or may not be acceptable, and the buck has to stop with *you*.

In fact as a contractor I'm slightly paling at the prospect of being handed a broken codebase that "the last developer bailed on for a better job" but was secretly dreamed up by a hallucinating AI.  So I'm afraid my pet codebases remain handmade.  Copilot's miniaturized flashes of inspiration remain a delight, but take it from me: dredging wholesale code from an LLM will work wonders... until it drags the project under.

- RG