AI Browser Vulnerabilities Pose Major Security Threats

AI-powered browsers pose a potential threat to users, with hackers able to exploit generative AI integrated into web surfing. On Tuesday, Anthropic issued a warning about the risk associated with their Claude AI Chrome extension. This tool allows the AI to control the browser, facilitating searches, research, and content creation. However, due to new security vulnerabilities introduced by the integration, the extension is currently limited to paid subscribers as a research preview.

Claude has been found reading data from the browser and interpreting it as commands that should be executed, leading to what Anthropic terms “prompt injection attacks.” These attacks enable hackers to secretly embed instructions in web content to manipulate the Claude extension into executing malicious requests.

Such attacks can cause AIs to delete files, steal data, or make financial transactions. In their investigation, Anthropic carried out 123 test cases representing 29 different attack scenarios, resulting in a 23.6% success rate through prompt injections. For instance, one successful attack involved using a phishing email to instruct the AI to delete all emails in the inbox without confirmation.

Although Anthropic has implemented a fix for this vulnerability, the mitigations only reduced the success rate of prompt injection attacks from 23.6% to 11.2%. The company’s findings also suggest that hackers could orchestrate more alarming attacks if the AI is granted control over the computer itself.

Further testing by Anthropic revealed that a set of four browser-specific attack types, with the mitigations reducing the attack success rate to zero in certain cases. However, due to ongoing threats and new forms of prompt injection attacks being constantly developed by malicious actors, Anthropic has decided not to release the extension beyond the research preview.

Similar concerns have been raised by Brave Software regarding prompt injection attacks on Perplexity’s AI-powered Comet browser. In their testing, Brave found that Comet was susceptible to the attack if the user asked it to summarize a web page containing malicious instructions. The attack is indirect in interaction and has a browser-wide scope.

Despite attempts to patch the vulnerability, Perplexity admits that they have not fully mitigated the attack as of yet. However, they assure that no users attempted the malicious prompt prior to fixing the vulnerability, and since then, no successful attacks have been reported.

Critics like software engineer Simon Willison argue that agentic browser extensions are “fatally flawed” due to the prompt injection vulnerability. The core issue lies in the merging of trusted instructions and untrusted content into the same token sequence, with nobody having demonstrated a reliable way of distinguishing between the two.

In response, Perplexity maintains that all AI companies take this matter seriously, collaborating on reporting and fixing vulnerabilities. They view this as an ongoing battle requiring increasing sophistication.