LLMs can know a task is impossible and still optimize it anyway.
Rohan Paul Twitter · Rohan Paul (@rohanpaul_ai) · 2026-08-02
A research paper introducing SaliTrap finds that LLMs fail to flag impossible tasks due to salience bias — where explicit numerical details suppress unstated prerequisites — with even the best model among 12 tested avoiding the trap in only 54.8% of cases.
Extraction
Topics: llm-reasoningsalience-biascommonsense-reasoningagent-evaluationllm-benchmarks
Claims
- LLMs exhibit salience bias: explicit numbers and procedural details in prompts overpower unstated physical or logical prerequisites, causing models to optimize impossible tasks.
- SaliTrap tests 1,145 prompts across four trap types and 12 models; the best-performing model avoided traps in only 54.8% of cases, and eight of twelve models stayed below 30%.
- Trap avoidance declined as numerical density in prompts increased, with models increasingly beginning calculations before noticing the embedded contradiction.
- Trap-aware models were not immune: GLM-5.1 and Kimi-K2 still complied with impossible tasks 86.2% and 81.8% of the time respectively after recognizing the trap.
- Removing bait from prompts recovered 86.9% to 91.8% of previously sycophantic responses across four models, confirming the relevant knowledge was present but suppressed by framing.
Key quotes
Ask whether to walk or drive to a car wash 50 meters away, and some models focus on distance while missing that the car itself must reach the wash.
Awareness was not enough either. Among trap-aware responses, GLM-5.1 and Kimi-K2 still complied 86.2% and 81.8% of the time.
So many failures reflect knowledge that is present but suppressed by task framing.