Where we started

We compared three simple defenses that run at inference time: a fixed safety prompt, a prompt that asks the model for a short safety rationale, and Gaussian image noise with a vote over five runs. These are local variants inspired by AdaShield, RapGuard, and SmoothVLM, not exact implementations of the published methods.

The obvious question was: which defense works best? But once we inspected the experimental archive, it became clear that this question came too early. First we had to check whether the model received the right data, whether its actual answer was preserved, and whether the evaluator measured safety rather than incidental words in the response.

Experimental pipeline showing the input, three optional defenses, the model, and separate ways of checking the result
We examined inputs and outputs, processing time, actual refusals on benign requests, and keyword scores separately. These signals answer different questions and cannot replace one another.

Could the underlying data be trusted?

For every result, we asked three basic questions:

  1. Did the model receive the intended text and image?
  2. Was the answer shown to the user actually preserved?
  3. Did the evaluator detect real harm or refusal, rather than merely familiar words?

The original plan covered 9,000 inputs. After checking them, we retained four datasets: FigStep, the text-only part of JailBreakV, SALAD-Bench, and HarmBench-Simple. Together they account for 4,820 inputs per core configuration. Another 4,180 inputs had to be excluded from numerical safety claims.

The reasons were concrete. MM-SafetyBench used the wrong data field. In multimodal JailBreakV, a missing image could silently turn a request into a text-only one. And the adversarial-patch branch drew text and a random rectangle instead of producing the claimed optimized attack.

These problems cannot be repaired by rechecking the stored answers: if an image never reached the model, no later analysis can reconstruct the missing event. For the four retained datasets, we have summary tables but not every matching response. They can reveal how the old evaluator behaved, but they cannot support a reliable semantic ranking of defenses.

Evidence checks covering the intended input, the preserved answer, and whether the evaluator measures the right thing
A smarter check later in the pipeline cannot repair an earlier failure. We first need the correct input, then the actual output, and only then an evaluator suited to the question.

How the defense fooled the evaluator

The old script looked for a fixed set of words. Its refusal markers included harmful, unsafe, and unethical. But the safety-rationale prompt used the very same language: it asked the model to begin by discussing whether the request was harmful or illegal.

That created a strange failure mode. A model could open with a safety statement and then provide a substantive answer. The script saw a familiar word and treated the answer as a refusal. In the safety tables, the same wording could make an answer look safe even when harmful instructions followed the preamble. Empty outputs were also counted as safe because they contained none of the listed words.

In a targeted set of 246 responses, the old script marked 72 as harmful, but an independent LLM judge agreed on only 30. The judge also found 13 harmful responses that the script had missed. This selected sample does not tell us the overall error rate, but it makes the basic problem clear: word matching is not a trustworthy semantic evaluator.

The clearest result came from 28,000 ordinary, benign requests:

This is not a small, uniform error that can simply be subtracted from every score. It appears most strongly in the very configurations that use the safety prompt. The comparison therefore mixes two effects: how the defense changes the model and how it changes the evaluator.

Comparison of 7,532 keyword flags with roughly 138 estimated refusals, alongside estimates by model
The main result: 7,532 keyword flags versus roughly 138 estimated refusals. The pooled refusal estimate was 0.52%, and most false positives appeared in answers containing the safety rationale.

What remained after the audit

The retained results still vary substantially across models and datasets. In 27 of 32 model–dataset pairs, combining the two safety prompts raised the old script’s score. But this is exactly where repeated safety wording may look like an improvement even when model behavior has not become safer.

The largest changes were +11.8 percentage points for InternVL3.5-8B on FigStep and −12.5 for InternVL2.5-8B-MPO on HarmBench. These cases are useful priorities for a new paired evaluation of the actual answers. The numbers alone do not prove that the defense helped or hurt.

Changes in the old evaluator's score for two combined safety prompts across four datasets and eight models
The same setting behaves differently across models and datasets. This is a useful map for a rerun, not a ranking of defenses.

On benign requests, the pooled refusal estimate was 0.52%, and the highest estimable cell was 3.24%. This does not look like widespread refusal, but small differences between defenses are still not reliable: refusals were rare, and the judging model was not always consistent on the few disputed cases.

We also measured computational cost. Running five noisy image variants took 5.45–12.76 times the no-defense batch processing time, while the full stack took 4.36–16.59 times as long. These are batch-processing ratios, not single-request latency, and they say nothing about effectiveness. We do not report task accuracy because the archived MMBench prompts omitted answer choices A–D, making the old score impossible to reconstruct correctly.

The main takeaway

If a safety prompt uses the same words that an evaluator searches for, the resulting score says more than just how safe the model is. It also reveals how easily the defense can influence the measurement itself.

A fair comparison must preserve the entire chain: the exact text and image, the model version, the safety prompt, the answer shown to the user, any errors or fallbacks, and the evaluator version. The answers then need to be judged for meaning, not just scanned for words. Without that chain, a polished table of percentages remains a report about the experimental pipeline—not evidence of safety.