Google Gemini Flaw Turns AI Email Summaries into Phishing Tool

Google Gemini for Workspace Flaw

Image courtesy of Tom's Hardware

Google Gemini for Workspace has been identified with a critical security flaw that can be exploited for phishing attacks. According to Mozilla's 0-Day Investigative Network (0din), the AI features of Google Gemini could be tricked into believing a user's account was compromised through cleverly crafted email prompts. This vulnerability allows malicious actors to embed instructions in emails that are rendered invisible, yet obeyed by Gemini when summarizing the content.

The attack relies on a simple yet effective method: embedding malicious text in white font on a white background, ensuring it remains unseen by the user. When Gemini summarizes such an email, it outputs the hidden malicious instructions as part of its summary. As 0din stated, "Because the injected text is rendered in white-on-white (or otherwise hidden), the victim never sees the instruction in the original message." This method poses a significant risk as it can lead to users falling victim to social engineering attacks based on misleading summaries.

For further reading, refer to the original report on the Google Gemini flaw and insights from Mozilla's findings on phishing for Gemini.

Phishing Attack Mechanism

Image courtesy of Tom's Guide

The phishing attack mechanism takes advantage of how Google Gemini generates email summaries. Hackers can create emails that include hidden instructions, which the AI tool automatically obeys when summarizing the message. This is executed by hiding malicious text using HTML and CSS techniques that set the font size to zero and color to white.

Since the malicious text does not include any visible links or attachments, it is highly likely to bypass security filters and reach the intended recipient's inbox. When a user requests a summary from Gemini, the AI unwittingly follows the hidden directives. The alert generated might appear as a legitimate warning, leading users to believe they are acting on a credible alert.

For more information, see the detailed article on Google Gemini's phishing potential and insights on what phishing scams are.

Security Mitigations

Image courtesy of Bleeping Computer

Despite the known vulnerabilities, Google has been working on implementing safeguards against such attacks. Mozilla's GenAI Bug Bounty Program Manager, Marco Figueroa, suggested several mitigation strategies. These include having security teams neutralize or ignore hidden content and implementing filters to flag urgent messages, URLs, or phone numbers in Gemini outputs.

Google has acknowledged the issue but stated that they have no evidence of such attacks being executed in the wild. They are actively enhancing defenses against prompt injection attacks and are in the process of rolling out additional mitigations. For ongoing updates about security measures, visit Google's security blog.

Users are advised to exercise caution when using Gemini to summarize emails, as the AI's output should not be regarded as an authoritative source of security alerts. For more details on the latest security findings, visit Bleeping Computer's coverage on Google Gemini flaws.

For more information about our products and services, visit Gopher Security . Explore how we can enhance your security solutions with our offerings.