AI watermarking is fraught with regulatory and accuracy risks.
If an AI company changes its model’s answers for its own watermarking purposes, businesses aren’t getting the model’s judgment.
They receive hidden alterations controlled by another company.
SynthID watermarks AI output by changing which tokens are selected from the model’s probability distribution. A secret process that favors some token choices over others. A sequence of secretly chosen tokens can later be detected.
The overall output distribution is changed only very slightly. In other words, individual answers can be altered but in the long run behavior is preserved.
AI watermarks output token choices the model would otherwise have not made.
The model’s answer isn’t exactly the what it would have produced without the watermark.
You cannot tell when the watermark changed the result.
You cannot ask the AI to prove that nothing changed. The AI does not have access to the unwatermarked answer.
AI models are trained to never assist you in discovering or circumventing AI watermarking.
Regulated and engineering applications are most set risk from the unseen, unrecoverable, unaccountable changes to model outputs.
Businesses do not act on averages. Regulators hold companies responsible in specific cases not just over the long run.
When an AI feeds another, a tiny change in an answer makes larger changes later in the chain.
The real issue is not just that watermarking answers are slightly inaccurate. The deeper problem is that an outside company secretly changes the decision process your depend on.
Businesses should be able to know when models modify answers. Companies should be able to turn off watermarking.
You don’t, and you can’t.