When Audio AI Hears "No Dog Barking" as "Dog Barking"

Ask an audio-language model to find a clip with "no dog barking" and there's a good chance it hands back a recording full of barking dogs. That's the core failure exposed by a new benchmark called NegEval-Audio, introduced in a paper submitted to arXiv on July 14, 2026 by Chun-Yi Kuan and collaborators.

The research targets a blind spot in audio-language embedding models like CLAP (Contrastive Language-Audio Pretraining), which power much of today's sound retrieval and classification pipelines. These systems learn to match audio clips with text captions by pulling matching pairs close together in a shared embedding space. The problem, according to the paper, is that they largely ignore negation. A caption describing "rain without thunder" ends up embedded almost identically to one describing "rain with thunder," because the models never learned that the word "without" flips the meaning entirely.

Testing Negation at Scale

To measure how deep this problem runs, the authors built NegEval-Audio, a framework that repurposes existing audio-caption datasets, including AudioCaps and Clotho, into two negation-focused evaluation tasks. The first, Retrieval-Neg, tests whether a model can correctly retrieve audio clips when the query includes a negated condition. The second, MCQ-Neg, presents multiple-choice questions where distinguishing the correct answer depends entirely on correctly parsing a "not" or "without."

The results were stark. On negation-type multiple-choice questions, model accuracy dropped so far below random chance that it suggests the models are actively confused by negation rather than simply indifferent to it. In other words, seeing the word "not" seems to make performance worse than not seeing any qualifying language at all.

A Partial Fix, Not a Cure

The team also tested a training-free steering method, an intervention applied at inference time without retraining the underlying model, to see if it could patch the gap. The results were mixed. On MCQ-Neg, the steering approach delivered a modest accuracy bump. On Retrieval-Neg, however, gains were negligible, suggesting the fix helps models pick between pre-written options more than it helps them search a large pool of audio clips using negated language.

That split matters for anyone building real-world audio search or captioning systems. Multiple-choice-style reasoning is a narrower, more constrained task than open retrieval, and the fact that a lightweight fix works better there than in retrieval implies the underlying representational problem, how these models encode "not" in the first place, remains largely unsolved.

For an industry increasingly leaning on CLAP-style embeddings for voice assistants, content moderation, and sound-based search, the paper is a reminder that fluency with affirmative descriptions doesn't guarantee any real grasp of negation, a linguistic feature humans handle almost automatically.