Does the “ask a frontier model for help” trick generalize?

Testing Hightouch’s Jev cascade on legal questions and three other datasets

November 9, 2026

I evaluated 200 legal multiple-choice questions using Jev with GPT‑6.1 Sol for questions where Jev was uncertain. This hybrid approach reached 87% accuracy — just two percentage points below Sol’s 89% but at 56% lower cost.

My exploration was inspired by Hightouch, who used a similar approach for an identity resolution task (Hacker News discussion). The idea is to use Jev, a decision modelDecision models, like Jev, do not generate text; they classify inputs and make decisions based on a given criterion. They are extremely low cost and low latency., for the bulk of the classifications at extremely low cost and latency; use Sol for only the questions Jev is least certain aboutIn my testing, $1 buys you roughly 30,000 Jev classifications, versus 500 Sol medium classifications..

Hightouch only tested identity resolution, and I wanted to know how far the technique generalizes. I tried several datasets from Hugging Face, including WildGuardTest, SciFact, and CLINC150. Escalation didn’t seem to help on these: Sol and Jev would agree on the same mistakes and have quite similar performance.

The CaseHOLD (Case Holdings on Legal Decisions) dataset proved more promising. The model is given a passage from a judicial opinion and asked to choose one out of five legal holdings. A simplified example:

Show the example questionExample question

A court explains that evidence discovered during a search cannot retroactively justify the arrest that supposedly authorized that search. It cites an earlier decision: holding.

Which holding best fills the blank?

  1. AAn arrest cannot justify a search when that search itself supplies the justification for the arrest. Answer
  2. BConsent searches do not require probable cause.
  3. CEvidence obtained through an unlawful search must always be excluded.
  4. DPolice may search whenever they reasonably suspect wrongdoing.
  5. EAn appellate court cannot make initial factual findings.

I sampled 200 questions and compared three scenarios: Jev alone, Sol alone, and a cascade. In the cascade strategy, Jev answers first. If the probability assigned to the answer is below the cutoff, it falls back to Sol.

Three line charts against the Jev confidence cutoff from 0.30 to 0.90. Accuracy rises from 79.5% to 87.5% (Sol alone 89.0%). Estimated cost per 1,000 questions rises from $0.06 to $0.99 (Sol alone $1.92). Estimated mean time per question rises from 0.41s to 3.73s (Sol alone 6.73s).
CaseHOLD, 200 questions, Jev 1.14 with Sol at medium reasoning. Questions go to Sol when Jev’s confidence is below the cutoff.

Jev alone scored 80% and Sol alone scored 89%. Raising the confidence cutoff routed more questions to Sol and generally improved accuracy, albeit with a higher cost and longer completion time. At a cutoff of 0.8, the cascade strategy scored 87% — two percentage points below Sol alone — at about 56% lower API cost. Thus, this can be a useful technique for organizations who need frontier-level intelligence without the associated costs.