Back in April I made a post about vibe coding, namely a caveated defence of it, where I pointed out some of the positives it can have, pointed out the negatives, and how those negatives could be mitigated. I also ended it with a section about how a lot of the discourse at the time around vibe coding was elitist “real devs don’t use AI”, and how that didn’t sit right with me. While I still stand behind a lot of what I said in that post, I can’t in good conscience keep that post up, because I fundamentally no longer agree with the concept. I don’t believe vibe coding, and more specifically, the current scourge of large-language-model tools that seem to be deployed everywhere, often looking for a problem to “fix”, is a state of play that can be defended. I also want to make it clear that even though in this post I am going to be very negative about these tools, this is not an attack on (most of) those who use these tools, but more the tools themselves, and those who are trying to force them on us.

Where I stand on the points from the last post

Like I said I do agree with a lot of the points I made in the caveated defence post, namely the potential upsides, those being a potential way to lower the barrier of entering software engineering, or just doing some hobby programming, and potentially saving experienced developers some time here and there, at the very least it can give you the outline to fill the details in quite often. I also still very much agree with the caveats, the code quality ones, the erosion of expertise issue, and the inconsistent code style issue. I also very much still agree with the point I made about gate-keeping, that its generally unhelpful, and that people who do gate-keep generally need to gain some perspective.

What I have a harder time agreeing with is my points mitigating these negatives, especially the one about code quality. While it is true we do have tools to scan for code quality/security issues, as developers lean more on AI tooling to write code they likely wont fully understand, these tools will need to become smarter. For example there are already some tools that will use AI to generate test cases; so seemingly we’re entering a an arms race, where on both sides is effectively the same piece of software, just given a different brief, using enormous amounts of power, and a library of, at best, questionably acquired content to drive the responses. The code quality point was also, to some extent, made under the assumption that developers would perform a decent level of due diligence, checking over things for themselves. This has been counter to what I have witnessed online, even developers I have had respect for the attention to detail allowing very avoidable mistakes to slip through the gaps, if we get into a situation where AI is writing, the code, the tests, and the documentation (so developers are effectively entirely out of the loop), I shudder to imagine what will fall through the cracks.

The point on lack of due diligence feeds into the erosion of expertise issue, I feel the appeal to authority fallacy is often applicable to interactions with AI tools; people trusting its output because the computer is saying it, and the computer knows, only the computer doesn’t know, the computer is just stringing words together that sound right. This leads to issues like using deprecated/vulnerable API’s, and in the worst case, if allowing these tools to install dependencies (or installing them without first vetting them), falling prey to typosquatting, where malicious actors upload malware under package names that are similar to popular tools, to prey on LLM hallucinations. These tools, by their nature aim to eliminate the trial and error developers go through in writing software, this puts the onus back on the developers to understand the code generated, how it works, and crucially, why it works. If developers are having a chat in their IDE, and immediately sending that up, then the middle of the night on call issue will become much more of a problem, as ChatGPT can’t exactly be put on-call.

These issues then require the developers who don’t use the AI tooling to step up, whenever I’m reviewing code I know has had some LLM tool in the mix to generate the code, even from a developer I know and respect, I give it a lot more scrutiny. This is partly because of the reasons above, but also because I know some-to-all of the code I’m reviewing hasn’t been written based on experience and intentional decisions, but the roll of a dice in an LLM, and the next time it responded to an identical prompt, it could generate entirely different code. so when I’m reviewing this code, I’m not giving the good faith of “they will have used a normal for loop rather than a for each loop for a good reason, probably didn’t work for some reason”, I’m now either having to figure out if that’s the case, or raise many more comments on the PR in question.

A pattern is beginning to emerge here, using the LLM isn’t eliminating the work, its just moving it, or more commonly, requiring more work. The piper must always be paid.

A new age of grifting

What I also keep seeing in discussions around open source software pull requests, and bug bounty programs (especially cURL), is a common occurrence of AI generated slop being submitted, and wasting the time of these unpaid (or at best, underpaid) volunteers. This, while not just a gross thing to do, risks burning out these open source contributors, to which so much software relies on. This could result in more bugs slipping through, more vulnerabilities being missed in the pile of fake reports, or at the worst case, these contributors quitting, and libraries that so much modern software depends on going unmaintained.

Grifting seems to be the pattern for the companies peddling these tools, often, its new deep cognition, post cognition, pre cognition, but fundamentally they are still probability machines. The only real advancement recently I’ve seen is having them run tests/compilations to check that they’ve made something that works to some extent. What this practically means is you have an LLM model, basically running the tests on loop while making changes between each run, like some ocean-boiling catherine wheel, burning through credits which only makes it cost more, until it might reach something similar to the desired outcome. Or it gives up because the tokens ran out. There’s plenty of other ways that these companies are basically one big grift, but that’s its own blog post, or even, its own blog.

Then there’s the peer pressure from social media to use these tools, grandstanding that AI is going to revolutionise programming, while vilifying those who don’t want to use it as if they were a carpenter refusing to use a hammer. This is massively misrepresenting reality though, it isn’t going to revolutionise programming, at least while it still hallucinates, it’s going to hallucinate outsourcing the problem to someone else, and the use of LLM’s isn’t analogous to the hammer, as a hammer has a very well pre-defined outcome to an action, I hit a nail, and the nail becomes embedded in the material, by the statistical nature of an LLM, the outcome isn’t pre-defined, nor is it repeatable or internally consistent. It is difficult to find an existing tool that changed an industry in the same way LLM’s are, but that’s because we’ve never tried to invent the everything-and-nothing machine before, that can do any task you throw at it, but only to the level of a junior of that role. If I had to pick something it often reminds me of the more outlandish inventions from Wallace and Gromit, the next level of wrong trousers.

So where do I stand on LLMs in general now

My views on the technology behind LLM’s hasn’t changed, and has probably been abundantly clear for a while; I don’t like them, and without the correct supervision and responsible use, it’ll cause chaos.

Well, it’s not getting correct supervision and responsible use.

This has made me take a much more dim view on these tools from an ethical point of view, a lot of use of these tools end up being someone else’s problem, either the end user of some tool that was thrown together and may be pseudo-malware, to moving the responsibility of reviewing the code the AI has generated to the person who reviews the pull request, to just the global-warming speed-running machines these tools are turning out to be. That being said I try and not transfer my view of the tool onto the users, especially with the current pressures to use them, both the “peer pressure” aspect, but also companies like Microsoft forcing their employees to make use, to justify the massive investment. The way I have elected to view it is very similar to those who works at gambling companies, ethically, I could not work there, it just does not sit right with me, but I’m not gonna judge anyone else for where they make their income, not in this economy. By the same token, if your workplace is putting pressure on using these tools, or if you feel you’ll fall behind in your career from not making use, I’m not going to judge it.

The deeper we get into this “AI revolution”, the more it does seem like a .com bubble, that is just waiting to burst, every app and service under the sun is looking for a way to integrate AI, a good example from a post I saw a while ago pointing out that in the Google Weather app, there’s an AI summary poorly explaining the forecast, that the summary has pushed off screen, anytime I need to use any Microsoft tooling copilot is there like clippy 2.0, they are all seeming to be a solution, looking for a problem, that tends to cause new problems. In the mean time all I can do is try and avoid it, one good way I saw in that same blog post, is disabling the “AICore” app on newer Google Pixel’s more or less disables all local AI on the device, so at least I’m no longer having my phone battery wasted having it figure out how to tell me its going to be cloudy.

#AI#Ramblings