ChatGPT poses a threat to users as it often provides incorrect URLs to access the sites of large companies. About this informs The Register.
GPT-4.1 models correctly indicate the addresses of official sites in the fields of finance, trade, technology and utilities only 66% of the time. Of the 131 addresses tested, 34% were inaccurate: 29% led to nonexistent or defunct resources, and another 5% led to legitimate but incorrect sites. In particular, instead of the official website of the Wells Fargo bank, ChatGPT issued a link to a phishing resource imitating it.
This is actively used by cybercriminals. As Netcraft’s head of cyber threats, Rob Duncan, explained, attackers can track exactly what errors the model makes, register the relevant free domains and create fake websites.
The main problem is that language models form answers based on textual associations, rather than checking the validity of references. The study also describes a case involving the Solana blockchain: Fraudsters created fake GitHub repositories, accounts, and documentation that could mislead the model, causing false resources to appear in AI responses instead of verified data from search engines.
According to Duncan, the growing popularity of AI-based chatbots, which are increasingly used in place of traditional search, increases the threat because users are not always aware of possible inaccuracies. Therefore, Netcraft encourages developers to improve URL validation mechanisms in language models.
