Google treats indirect prompt injection (IPI) as an evolving threat that cannot be solved once and then set aside. This type of attack targets AI applications that interact with multiple data sources and tools, such as Workspace with Gemini. An attacker can embed malicious instructions in the data or tools the model uses while carrying out a user request, even without direct input from the user.
In a blog post published by Adam Gavish from Google’s Generative AI Security team, Google explains that the expanding use of large language models in agentic automation, along with the diversity of the content they process, creates a constantly changing environment for adversarial attacks. The company therefore is building an ongoing approach that combines security research, attack testing, synthetic data, application controls, and updates to machine-learning models and language models.
Detecting Attacks from Multiple Sources
The process begins by discovering and cataloging new attack paths before attackers can exploit them more broadly. Google uses internal and external programs for this purpose, including human-led red-team testing, in which specialized teams carry out attacks based on realistic user profiles to uncover security and safety weaknesses, then coordinate with product teams to address the issues discovered.
The company also uses automated red-team testing through dynamic machine-learning-supported frameworks. These frameworks generate attack payloads and repeatedly modify them algorithmically to simulate advanced threats at scale. This helps map complex attack paths and test the effectiveness of security controls in more edge cases than manual testing alone can cover.
Google’s Google AI Vulnerability Rewards Program also enables collaboration with external security researchers who discover new attacks that exploit IPI. Google additionally hosts periodic live hacking events that give invited researchers access to pre-release features to identify new vulnerabilities. Teams then work to validate, reproduce, and remediate the issues.
Cataloging Vulnerabilities and Expanding Attack Data
Google also monitors publicly disclosed attacks through open-source intelligence, including social media, press releases, blogs, and other sources. New vulnerabilities are extracted from these sources, then recreated and cataloged internally to ensure that products are not affected by them.
Each new vulnerability undergoes analysis by Google Trust, Security, & Safety teams. The process includes reproducing the vulnerability, verifying that it is not a duplicate, linking it to an attack technique and impact category, and assigning it to the relevant owners. In this way, results from testing and external sources are converted into a tractable list for remediation and tracking.
After attacks are discovered, refined, and cataloged, Google uses the Simula tool to generate synthetic data that expands the scope of new attacks. This step makes it possible to create multiple attack variants to improve completeness and coverage and to prepare new datasets for training and validation. According to the blog post, this process led to a 75% increase in synthetic data generation, supporting the evaluation and retraining of defense models, as well as updates to the dataset used to calculate and report defense effectiveness.
Updating Defenses at Multiple Levels
Google does not rely on a single layer of protection. Deterministic defenses, such as user confirmation, URL sanitization, and tool-sequencing policies, respond to new attacks through simple configuration updates. These controls are managed through a centralized policy engine that includes settings for core tool calls, URL sanitization, and tool sequencing. This enables rapid fixes, such as removing malicious patterns with regular expressions, at a faster pace than machine-learning or language-model update cycles.
Machine-learning-based defenses are retrained using synthetic data that includes variants of discovered attacks. Google divides this data into training and validation sets, so effectiveness is measured on examples that were not used during training. This approach is intended to provide repeatability and consistency in training and testing data while establishing a scalable structure for fully automated model updates in the future.
Language-model-based defenses are likewise subject to prompt engineering, through improvements to system instructions based on synthetic-data examples and agreed-upon defense-effectiveness metrics. In parallel, Google is working to harden the Gemini model to improve its inherent ability to recognize malicious instructions within data and ignore them while continuing to fulfill the user’s intended request. The company says this process improved Gemini’s ability to detect and ignore injected instructions and reduced the attack success rate without affecting the model’s efficiency during normal operations.
Measuring Effectiveness Before Adopting Improvements
Google does not simply add new controls; it simulates attacks against several Workspace features and applications, such as Gmail and Docs, using a standardized set of assets to ensure consistent evaluation. To determine the actual impact of a specific improvement, such as updating a machine-learning model or modifying a language-model prompt, the company runs the full evaluation with the defense enabled and then without it.
This comparison provides “before-and-after” metrics to verify the change’s effectiveness and support a continuous improvement cycle. Google believes that the nature of IPI requires a multilayered defense and an iterative process combining security research, automated pipelines, advanced models, and more, rather than relying on a fixed solution to a problem whose form changes as agentic applications and data sources evolve.