Does the “ask a frontier model for help” trick generalize?

Testing Hightouch’s Jev cascade on legal questions and three other datasets

November 9, 2026

I evaluated 200 multiple-choice legal questions using Jev with GPT‑6.1 Sol for questions where Jev was uncertain. This cascade approach reached 87% accuracy — just two percentage points short of Sol’s 89%, but at 56% lower cost.

My exploration was inspired by Hightouch, who used a similar approach for an identity resolution task (Hacker News discussion). The idea is to use Jev, a decision modelDecision models, like Jev, do not generate text; they classify inputs and make decisions based on a given criterion. They are extremely low cost and low latency., for the bulk of the classifications at extremely low cost and latency; Sol is reserved for only the questions Jev is least certain aboutIn my testing, $1 buys you roughly 30,000 Jev classifications, versus 500 Sol medium classifications..

Hightouch only tested identity resolution. I wanted to know how far the technique generalizes. I tried several public datasets, including WildGuardTest, SciFact, and CLINC150. Escalation didn’t seem to help on these: Sol and Jev would agree on the same mistakes and have quite similar performance. So, YMMV.

The CaseHOLD (Case Holdings on Legal Decisions) dataset proved more promising. The model is given a passage from a judicial opinion and asked to choose one out of five legal holdings. A simplified example:

Show the example questionHide the example question

A court explains that evidence discovered during a search cannot retroactively justify the arrest that supposedly authorized that search. It cites an earlier decision: holding.

Which holding best fills the blank?

  1. AAn arrest cannot justify a search when that search itself supplies the justification for the arrest. Answer
  2. BConsent searches do not require probable cause.
  3. CEvidence obtained through an unlawful search must always be excluded.
  4. DPolice may search whenever they reasonably suspect wrongdoing.
  5. EAn appellate court cannot make initial factual findings.

I sampled 200 questions and compared three scenarios: Jev alone, Sol alone, and the confidence-based Jev-Sol cascade.

A 0.8 cutoff gets within two points of Sol’s accuracy at 56% lower cost.

Three line charts against the Jev confidence cutoff from 0.30 to 0.90, with 0.80 highlighted. Accuracy rises from 79.5% to 87.5%, reaching 87.0% at 0.80 (Sol alone 89.0%). Estimated cost per 1,000 questions rises from $0.06 to $0.99, $0.84 at 0.80 (Sol alone $1.92). Estimated mean time per question rises from 0.41s to 3.73s, 3.21s at 0.80 (Sol alone 6.73s).
CaseHOLD, 200 questions, Jev 1.14 with Sol at medium reasoning. Questions go to Sol when Jev’s confidence is below the cutoff.

As you can see in the graph, Jev alone scored 80% and Sol alone scored 89%. With the cascade, raising the confidence cutoff routed more questions to Sol and generally improved accuracy, albeit with a higher cost and longer completion time. At a confidence cutoff of 0.8, the cascade strategy scored 87% — two percentage points below Sol alone — at about 56% lower API cost. So at least on some tasks, this hybrid approach can be quite a useful technique for organizations who need frontier-level intelligence without the associated costs.