Reasoning Up the Instruction Ladder for Controllable Language Models
Published in Findings of the Association for Computational Linguistics: ACL 2026, 2026
Abstract
As large language model systems take on high-stakes roles in real-world decision-making, they must reconcile competing instructions from multiple sources within a single prompt context. Enforcing an instruction hierarchy, where higher-level directives override lower-priority requests, is critical to the reliability and controllability of language models.
In this work, we reframe instruction hierarchy resolution as a reasoning task. The model must first reason about the relationship between a user prompt and higher-priority instructions before generating a response. To enable this capability, we construct VerIH, a training dataset of constraint-following tasks with verifiable answers, comprising aligned and conflicting system-user instructions.
We show that lightweight reinforcement learning with VerIH effectively transfers general reasoning capabilities to instruction prioritization. Our method consistently improves instruction following and instruction hierarchy performance across multiple model families and scales. It also generalizes to safety-critical scenarios beyond the training distribution, improving robustness against jailbreaks and prompt-injection attacks.
Links
Citation
```bibtex @inproceedings{zheng-etal-2026-reasoning, title = “Reasoning Up the Instruction Ladder for Controllable Language Models”, author = “Zheng, Zishuo and Balachandran, Vidhisha and Park, Chan Young and Brahman, Faeze and Kumar, Sachin”, booktitle = “Findings of the Association for Computational Linguistics: ACL 2026”, month = jul, year = “2026”, address = “San Diego, California, United States”, publisher = “Association for Computational Linguistics”, url = “https://aclanthology.org/2026.findings-acl.1960/”, doi = “10.18653/v1/2026.findings-acl.1960”, pages = “39332–39354” }
Recommended citation: Zishuo Zheng, Vidhisha Balachandran, Chan Young Park, Faeze Brahman, and Sachin Kumar. 2026. Reasoning Up the Instruction Ladder for Controllable Language Models. In Findings of the Association for Computational Linguistics: ACL 2026, pages 39332-39354.
Download Paper