Google SynthID Watermarks Alter LLM Safety Guardrails
New research reveals that Google's SynthID-Text watermarking system can inadvertently alter how large language models respond to harmful prompts, potentially weakening their safety guardrails.
Security researcher Andrea Siposova from Lasso Security has discovered that implementing Google’s open-source SynthID-Text watermarking can unexpectedly alter the behavior of large language models (LLMs). While watermarking is designed to be imperceptible to human readers, the study shows it can cause models to bypass their safety guardrails and comply with harmful prompts they would otherwise refuse. This issue becomes particularly pronounced when models are targeted with adversarial prompt-injection techniques.
SynthID-Text embeds a hidden signature by modifying the token selection process. Instead of standard random generation, it uses a secret key and a process called "tournament sampling" to score and select next-word candidates. To test this, Siposova utilized Hugging Face's unmodified SynthIDTextWatermarkLogitsProcessor to evaluate six open-weight models. The experiments revealed that applying the watermark changed how the models handled harmful inputs, with the exact behavioral shifts varying depending on which secret key was selected.
The consequences of this "sampling drift" extend beyond text generation to AI agents that rely on these models to execute tasks. The research demonstrated that watermarking altered which tools the agents called and what arguments they passed to them, sometimes degrading tool-calling accuracy. Because a weakened refusal at the model level can lead to unauthorized actions at the agent level, the combination of watermarking and prompt injection poses a unique security risk.
For AI practitioners and developers, these findings highlight a critical integration challenge. As platforms like Anthropic prepare to deploy SynthID-Text in future Claude models to comply with European Union regulations, developers cannot assume that safety testing conducted on a base model will hold once watermarking is active. Red-team exercises and safety evaluations must be performed with the watermarking configurations fully enabled to ensure guardrails remain intact.
This is our own summary of reporting by Ars Technica AI
