Quick Take
- The finding: JEV, a decision model that completes only 42.9% of ALFWorld tasks as an actor, raises a Qwen2.5-1.5B policy from 72.8% to 82.9% success on ALFWorld, and from 56.8% to 71.4% on WebShop, when it scores every action after the episode ends and is asked whether the action helped.
- Why it matters: Asking the wrong question undoes the gain. With the same hindsight, asking whether the task will succeed yields only 69.3% on ALFWorld, below outcome-reward training. The question and the context, not the model’s own acting score, decide how useful it is as a teacher.
- What it means: A poor actor can still teach well, because judging an action after the episode ends reveals consequences the actor had to anticipate. The 1.5B policy taught this way also outperforms a 7B policy trained with outcome rewards alone, on both environments.
In plain English
Some new AI models do not write long answers. They pick from a short list of choices, such as the next step an automated helper should take, or a score for a step already taken. Researchers at Zhejiang University and Baidu asked whether a model that is bad at taking those steps itself can still be a good teacher. They found that it can, if it looks at the finished attempt and is asked whether each step helped, rather than whether the whole task will succeed. Taught this way, a small student program did better on two standard test worlds than a larger program trained only on whether the attempt succeeded in the end.
What the study shows
Decision models answer a question about a context with a typed output, a choice among options with a probability for each, rather than generated text. The same model can choose an agent’s next action, evaluate actions already taken, or score every action so a policy can be trained with reinforcement learning. Most earlier evaluations treated them as actors and judged them by how well they act. Haochen Liu, Qianhao Zhu, Zixuan Wang and colleagues at Zhejiang University, with Peng Qiu and Yingqi Qu at Baidu, ask whether a decision model that acts poorly can still teach well.
- Actors skip the hard first step: All six models tested as actors, three dedicated decision models and three general language models given the same questions, succeed less often on ALFWorld tasks with ordered subgoals (Clean, Heat, Cool) than on Pick and Pick2. JEV completes 70.8% of Pick2 tasks but at most 18.8% of Heat, Cool or Clean in each state-change type. Decider v1.2 is the strongest actor overall at 63.6%, yet succeeds on only 18.8% of Heat tasks.
- Judges can credit the steps actors miss: When the same models see the whole trajectory and are asked whether each action helped (the paper’s Progress question), they credit the prerequisite steps that acting overlooked. Asking whether the task will succeed (Value) is less useful at separating helpful from useless actions.
- The useful judge is the useful teacher: JEV’s Hindsight plus Progress scores, added to GRPO outcome advantages, lift a Qwen2.5-1.5B policy from 72.8% to 82.9% on ALFWorld and from 56.8% to 71.4% on WebShop (Table 3, averaged over three evaluation seeds). The same pairing also wins on a 7B policy (77.6% to 83.8% on ALFWorld, 66.1% to 73.6% on WebShop). The 1.5B policy taught this way beats the 7B policy trained with GRPO alone on both environments.
- The wrong question can hurt: Hindsight plus Value reaches only 69.3% on ALFWorld, below the 72.8% GRPO baseline. The paper reports that, under hindsight, Value scores milestone actions only 0.05 higher than no-op actions, so the signal says little about which steps helped.
- Gains where order matters: Hindsight plus Progress improves over GRPO on every ALFWorld task type, with the largest gains on Cool (+23.0%) and Look (+20.7%). On tasks with dependency it reaches 86.8%. It also finishes sooner: 74.5% of ALFWorld tasks within 20 actions, versus 63.6% to 69.5% for the other 1.5B settings.
| 1.5B training method | ALFWorld success | WebShop success |
|---|---|---|
| GRPO (outcome rewards) | 72.8% | 56.8% |
| Prefix + Value | 79.0% | 67.7% |
| Prefix + Progress | 78.3% | 66.6% |
| Hindsight + Value | 69.3% | 68.3% |
| Hindsight + Progress | 82.9% | 71.4% |
Why it matters: one model, three jobs, different verdicts
The paper’s strength is a clean split of roles. The same decision models are tested as actors and as judges, then JEV is used as a teacher, with the question (Progress versus Value) and the context (prefix versus full trajectory) swapped one at a time. That design shows why acting scores are a poor summary of a model’s value for agent training: JEV completes 42.9% of the same ALFWorld evaluation tasks as an actor, including doing worst on the dependency tasks that its student later learns to handle. The teacher never has to solve those tasks itself to supply a useful training signal. Step-level scores also come without extra teacher calls at test time.
Method in brief
- Models as actors and judges: JEV, Decider v1.2 and GPT Decisions, plus three general language models given the same questions, including Qwen (prompted and constrained) and GPT Luna. Two of the general models are the base models of decision models in the comparison.
- Environments: ALFWorld household tasks and WebShop shopping tasks, both text-based, with some subgoals that must be completed in order. Episodes are capped at 30 actions.
- Teacher training: Qwen2.5-1.5B and 7B policies, 150 training steps, 16 tasks per batch. JEV’s step-level scores, normalized within each rollout group, are added to the outcome advantage of GRPO. The four question-and-context combinations are compared for the 1.5B policy in Table 3.
- Reporting: Actor rates are one evaluation per task. Policy success is averaged over three evaluation seeds. Code is at github.com/ZJU-REAL/decision-agents.
What remains open
- Short, text-only episodes: All experiments use two text environments with at most 30 actions. The authors flag software engineering, tool use and assistant tasks, with longer trajectories and more varied dependencies, as open ground.
- Offline judging only: Judges score actions after an episode ends. Online use, for filtering or reranking the next action, would see only the prefix. The paper leaves delayed or revised scores for later work.
- Closed teachers: JEV and GPT Decisions are closed models with undisclosed adaptation, so the study cannot say which training choices make a good judge. The authors point to open-weight models such as Decider v1.2 as a path to train directly for judging each action’s contribution.
- Baseline provenance: Table 3 notes that GRPO results come from GiGPO (Feng et al., 2025). Preprint, not peer reviewed.
- Conflicts: No funding or conflicts of interest statement appears in the paper.
Context: beyond acting benchmarks
Decision models such as JEV, GPT Decisions and Decider v1.2 have mostly been scored as actors, including work that documents surface shortcuts and missed prerequisites. Outcome-only reinforcement learning gives every action in a trajectory the same credit, while step-level labels are expensive to collect or require a language-model judge at every step. This paper treats judging, with the full trajectory visible, as the setting in which decision models may be most useful, and tests that claim in training, not only in analysis. It sits alongside our coverage of how decision models cut agent inference cost and of Google’s Gemini Decisions API.
What it means for agent builders and model labs
- Agent trainers: Do not retire a decision model just because it is a weak actor. Test it as a hindsight judge, and ask whether each action helped, before you decide it has no training value.
- Harness designers: The query is part of the system. A Value question with hindsight can be worse than outcome rewards; a Progress question with hindsight can beat a larger policy trained the usual way.
- Labs releasing decision models: Publish judge-quality numbers, not only acting success, and consider open-weight models that can be trained for step-level credit.
More on our Agent Research studies page.
About the analysis
Researched and written with AI assistance under the editorial responsibility of the i10X News team. The title image is an AI-generated image. Read our AI Use Policy. Spotted an error? Request a correction.
This article is based on the full text of Liu, Zhu, Wang, Li, Chen, Qiu, Qu, Lu, Xiao and Shen (2026), “Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents,” arXiv:2611.17167v1 (cs.AI, submitted November 12, 2026). All numbers come from the paper. The chart was redrawn by i10X from Table 3 of the paper, which is published under a CC BY 4.0 license. Code is released at https://github.com/ZJU-REAL/decision-agents. No conflicts of interest stated. AI tools assisted with research and drafting; selection, fact-checking and final editing were done by the i10X editorial team.
i10x Perspective
The useful sentence is not that JEV is a weak actor. It is that acting, judging and teaching are different jobs, and a score on the first one is a bad proxy for the third. Teams building agents should start measuring decision models by the questions they can answer after an episode ends, not only by the actions they pick while it is still underway.
