These days everybody is talking about vibe coding. Simply put, it works like this: you enter a prompt like „Make me a cool social media app like Facebook, but in a color theme with violet background“ into a coding-LLM and - voilà - it spits out thousands of lines of code (which sometimes even work quite decently).
This is great! The only problem with this kind of AI is that - according to information theory - they should not exist. At least not in the sense most people assume they work.
Let’s look at this a bit more in detail. In information theory, one way to measure the computational resources required to store an object is the Kolgomorov-complexity. It is defined as follows:
The (Kolgomorov-)complexity of a string is the length of the shortest possible description of the string in some fixed universal description language
Let’s use, for sake of simplicity, the python programming language as our description language.
Our Facebook-making prompt has a length of 90 characters, this makes 90 x 7 = 630 bits [1]. This means that the Kolgomorov-complexity of the prompt has to be some (very hard to calculate exactly) value below the length of the following description (program):
print("Make me a cool social media app like Facebook, but in a color theme with violet background")
Therefore the prompt obviously has a complexity of less than only 99 x 7 = 693 bits.
Now what about the output? Even a rough estimate is very difficult, we can only develop an intuition for the order of magnitude. The complexity of the produced code will be roughly the string length (in bits) of the code divided by some factor. This factor depends on how much the output code could be compressed (e.g. by creating an imaginary optimally efficient self extracting ZIP program)[2]. Unless the code produced by the LLM is really bad copy-pasta or contains long repetitive patterns, this factor should be rather small (like maximum 3 or so). We know from experience that program code is difficult to compress well.
Therefore, in our Facebook example the complexity of the output is definitely much larger than the complexity of the input. It seems that information is created out of nothing, in a way very similar to an impossible perpetual motion machine of the first kind which creates energy out of nothing.
If the shortest possible program describing the output is very long, how can it be described by a much shorter prompt? This is a contradiction!
The answer why this still works is simple and - of course - not new: the LLM is not using only the prompt and some general knowledge about programming. It is also „stealing“ an enormous amount of information from similar apps it has been trained on. The LLM knows what Facebook actually is and can, from the code of thousands of other apps and many descriptions and screenshots of Facebook on the internet, infer what kind of features and user interfaces are required to build a Facebook clone (or sort of).
Therefore, vibe coding is not really AI assisted programming, but rather AI assisted stealing of other people’s code (and ideas, concepts, designs etc.).
If we think a bit more about it, AI assisted programming - without the stealing capabilities - makes actually very little sense. At least not for a capable programmer. Let’s assume for a second Facebook was open source.
In this case our prompt could be manually (!) translated very quickly into the following code:
A shell one-liner:
$git clone https://github.com/Meta/Facebook
plus a very short (imagined [3]) python (let's keep it simple) snipplet:
import Facebook
fb = Facebook()
fb.theme.background_color = "#FA00FD"
fb.start()
Now this code has obviously a complexity which is comparable to the prompt (I’ll spare you and me the calculations). But, more importantly, it is also vastly superior to the prompt. This for two reasons:
- Because it's much more precise: instead of the vague „violet“ color, it specifies an exact RGB color value for a very specific part of the theme.
- Because it will, unlike the partly hallucinated, incomplete and untested output of an LLM, work perfectly well
Using a programming language is a very precise way to tell a computer what to do. A program is always superior to a description of functionality in natural language. When a developer is programming, a single requirement only rarely results in many lines of program code written. In cases where this is still required, it is because some basic functionality (actually another requirement!) is missing and needs to be written beforehand. As soon as this code (e.g. a new method or class) is finished, the original requirement can be written using only a few statements (like calling this new method in one line of code).
Writing thousands of lines of code based on a single short description of a requirement makes only sense in one case: if much of the basic functionality required to build the application is missing. Otherwise it is only a sign of poor programming style: „copy pasta“, failing to use suitable abstractions or failing to use existing open source code. And exactly the last point is the problem in our Facebook example: Facebook is - of course - not open source. For many other applications there is a plethora of open source frameworks and libraries which allow us to create complex applications with very little code. They can be usually coded in the compact style presented above.
If all software was available as open source code, a simple IDE with code completion would be completely sufficient for a halfway capable programmer to write most kinds of applications as fast as with LLM support. Any code required to build the application would be either already available (in this case it can be used with very few lines of code) or it would be completely novel (and in this case an AI could not have any idea how to write it because this use case did not yet appear in its training data).
In this case AI would simply not make sense (or would be only helpful to give hints and suggestions).
Therefore LLMs recreating code which already exists in open source form create more problems than they solve. Their code creation capabilities are only really interesting when they are asked to recreate closed source code.
Coding AI is mostly interesting in the context of stealing other people's work.
Is this a bad thing?
Not necessarily. It’s the beginning of the end of owning in general. I will write about this idea in one of my next blog posts.
[1] Assuming ASCII encoding
[2] To be precise, the length of this ZIP-style program would have to be included
[3] Admittedly a bit naive from an infrastructure perspective. Also Facebook is not coded in python.
Image: Shutterstock
Follow me on X to get informed about new content on this blog.
