Elon Musk said that experts in the field of artificial intelligence faced a serious problem – the exhaustion of data for training neural networks. For him in words, this situation arose last year, and now specialists have to look for new solutions.
Musk believes that this problem can be overcome by synthetic data — a special type of information that neural networks generate independently in the learning process. He noted that thanks to this data, the AI will be able to “self-assess itself and improve its skills through the process of self-learning.”
The idea of using synthetic data is not new. The former chief scientist of OpenAI Ilya Sutzkever also believes that the future of artificial intelligence lies in this direction. He predicts that future AI systems, unlike today’s, which mostly work on the basis of pattern comparison, will be able to solve complex tasks step by step, simulating the thinking process.
Many tech giants, such as OpenAI and Microsoft, are already actively using synthetic data to train their models. According to the analytical company Gartner, currently about 60% of the data used in this area is artificially generated. However, analysts caution that this approach can make models more biased and limited, as synthetic data is always based on pre-existing information.
The lack of data for training AI is considered one of the main obstacles on the way to creating revolutionary models. How effective the use of synthetic data will be will become clear already in 2025, when GPT-5 is expected to be released — the first model that can claim the status of a full-fledged artificial general intelligence (AGI).
