Cross-Model Agreement as an Escalation Signal for LLM Cascades: When It Works, When It Pays and How Much to Escalate

Cascaded large language model systems answer with a cheap model and escalate difficult queries to an expensive one. This study tested whether agreement between two cheap models from different families is a reliable, training-free signal for escalation, and when acting on it saves compute. Llama-3.1-8B and Qwen3-8B answered 1,100 numeric questions from GSM8K, MATH-500 and FinQA, and questions on which they disagreed were passed to Qwen3-32B. All models ran on the same two NVIDIA T4 graphics processors, and cost was measured in seconds of processor time. Agreement was a strong signal: Llama was correct on 93.3% of questions when the pair agreed and on 13.1% when it disagreed, with an area under the receiver operating characteristic curve of 0.90. Escalating the 34% of questions with disagreement captured 88% of the gain over random escalation that a perfect error detector could achieve, and the router matched the strong model's accuracy (86.3% versus 85.6%). Whether it saved compute depended on accounting: it used 17% less than always calling the strong model if the second cheap model was already running, and 8% more if that model had to be paid for. A pre-registered break-even rule wrongly predicted savings because the escalated questions were the expensive ones, on which the strong model wrote 2.0 times as many tokens. A corrected rule showed that routing pays once the strong model costs at least 3.7 times the cheap anchor, and a price list ranked which disagreements to escalate first. A single prompt instruction was also found to be able to fake or hide a routing result.