These days, the internet is flooded with AI generated images and videos. The generative AI's which create them are trained on real fotos and video material. As this data is - by definition - perfect (nothing can be more real than reality), the material generated by AI must be of inferior quality. If such material is published in large quantities, the training data for future AI degrades. It's known that this leads to a performance degradation of AI models trained on such data.
This phenomenon is known unter the term „poisoning the well“.
But this might not be the case for other types of data. If the utility of the output for humans is not bounded, things might play out differently.
Let’s look at an example based on AI generated program code.
Let’s assume humankind continues to generate large amounts of computer code using AI („vibe coding“). Now two effects might limit the amount of this code which is actually used for the training of the next generation of AI systems:
- If the generated code does not meet minimal performance requirements, it might never get published (failed projects)
- The engineers who select training data for their AI might have a preference for the code of highly successful projects (like those with many Github stars)
Code quality is - at least to some minimal extent - a necessary condition for the success of a software project. But it is, of course, not sufficient: a project could become very popular rather based on a great idea than based on the quality of its code.
But successful projects should have - on average - also an above average technical implementation (from architecture down to implementation details).
Therefore, if we select the training data (generated by the current AI generation) for the next AI generation based on the utility for humans, we might - indirectly and unintentionally - train future AI generations using black box reinforcement learning with a genetic algorithm.
The training objective would be the utility of the code for humans, a quantity which can be easily measured by humans, even for superhuman program code no human can understand anymore.
As the utility for humans is not bounded, we could - possibly - create AI with superhuman performance on coding tasks in this way.
Let’s call this idea the „sweetening the well“ hypothesis.
Image: made by author with ChatGPT