AI is close to exhausting all the data available on the Internet for its training, as someone said who knows this industry well: Elon Musk, owner of X and CEO of xAI, among other companies.
Their solution is to cross the Rubicon of model training: use synthetic data. That is, the AI itself generates the data with which it will learn.
Why is it important. The scarcity of data to train models will mark the turning point in the development of AI. And it could slow down the pace of technological advancement as we know it in recent years.
The context. Large language models require enormous amounts of data and information to improve their performance. The exhaustion of this available real data, generated by humans through traditional means, forces the industry to look for valid alternatives with which to continue improving products such as chatbots or image generators.
- The idea is not new, it has already been applied with other AI projects. Gartner estimated that 60% of the data used in AI projects during 2024 was generated synthetically. Microsoft, OpenAI, Anthropic, Meta… are turning to it.
- Palmyra X 004a model designed to power existing AI applications, was trained like this. It cost $700,000.
- The estimated cost for a comparable sized OpenAI model It is 4.6 million dollars.
What is the difference with what Musk proposes? That the use of synthetic data has so far been a complement to real data, not a total replacement. And according to him, they will soon be the only possible source of training.
Between the lines. It is not the first similar signal sent by an industry heavyweight. Ilya Sutskever, former chief scientist at OpenAI, already made a very similar warning in December: “We have reached the peak of data and there will be no more.”
- The problem is that synthetic data can create a closed loop where biases and limitations are amplified.
- That would lead to the collapse of the model due to a progressive loss of creativity and precision.
Despite those risks, the industry is embracing them.
In Xataka | AI already understands real-world physics: NVIDIA Cosmos is the first step towards intelligent robots
Featured image | Xataka with Midjourney