Qwen3.8-27B Adapter Doubles SWE-bench Pro Score for the Pi Coding Agent

Qwen3.8-27B-pi-adapted, a small LoRA adapter trained with reinforcement learning for the pi coding agent, raises hard SWE-bench Pro results from 29.4% to 62.7% and Terminal-Bench 2.1 from 39.5% to 55.8%, with far fewer tool calls.

Christopher Ort
By Christopher Ort · 2 min read
Share
A keyboard and glasses on a desk lit by a monitor at night.
Image: AI-generated, i10X
On this page
  1. Quick Take
  2. In plain English
  3. What happened
  4. Background
  5. How it works
  6. Why it matters
  7. What's still open
  8. Sources
  9. i10x Perspective

A small LoRA adapter for Alibaba’s Qwen3.8-27B has more than doubled the open model’s score on hard SWE-bench Pro tasks when it runs inside the pi coding agent. The adapter, Qwen3.8-27B-pi-adapted, was published on Hugging Face by the harness-hub organization and was trained with reinforcement learning for that one agent harness.

i10X Quick Take

Quick Take

  1. SWE-bench Pro (hard 51): 29.4% to 62.7% under pi, with 55% fewer tool calls per task.
  2. Terminal-Bench 2.1: 39.5% to 55.8% under pi, with 62% fewer tool calls.
  3. What it is: a rank-32 LoRA adapter on top of Qwen3.8-27B, not a new model.
  4. Runs on: vLLM, transformers serve and llama.cpp, with a GGUF version for local use.
  5. Training: AsyncGRPO in TRL, on two NVIDIA B200 GPUs per harness.

In plain English

Coding agents such as pi wrap a language model in a loop that reads files, runs commands and fixes errors. A model that is good at coding in general can still be clumsy inside a specific agent. This adapter is a small add-on file that teaches Qwen3.8-27B how to work inside pi. The result is far more solved tasks with fewer wasted steps, on a model you can run yourself.

What happened

The model card shows a real SWE-bench Pro task where the base model “searches and re-runs snippets until it hits the 100-step limit,” while the adapter finds the function, edits it, runs the tests and fixes the last case in 8 tool calls. The authors say the full recipe will follow in a release blog post so others can build adapters for their own harnesses.

Background

The gap between a model and the harness around it has become a theme in coding AI, as our report on harness versus model benchmarks showed. Qwen3.8-27B is already popular for local work, from offline game builds to agent experiments.

How it works

  • Method: reinforcement learning with AsyncGRPO, an experimental trainer in TRL, applied as a LoRA with rank 32 and alpha 64.
  • Tasks: 750 scale800 tasks interleaved with 300 TermGrade tasks, run in sandboxes with tool calls screened for safety.
  • Reward: task correctness from a verifier, plus a small bonus for using fewer tools.
  • Checkpoint: step 160 of a run that was stopped at step 191 of 1,000 planned.
  • Setup: thinking stays off, as in training. pi must use the OpenAI completions API, because the adapter was not trained on the Responses API.

Why it matters

For developers, a free adapter brings a local 27B model much closer to hosted coding agents on hard tasks.

For companies, harness-specific adapters are a cheap way to tune open models for internal agents without full fine-tuning.

For agent builders, the published recipe could make “one adapter per harness” a standard pattern.

What’s still open

  • Selection effects: the checkpoint was picked partly on SWE-bench Pro scores. On the 68 Terminal-Bench tasks not used for selection, the gain was still 16.2 points.
  • The GGUF version has not been re-evaluated, and lower-bit quantizations may drift further.
  • Whether the gains transfer to other agent harnesses.

Sources

  • harness-hub, “Qwen3.8-27B-pi-adapted” model card on Hugging Face
  • harness-hub, “Qwen3.8-27B-pi-adapted-GGUF” model card on Hugging Face

i10x Perspective

The lesson is less about Qwen than about fit: a well-matched adapter can matter as much as a bigger model. That is good news for anyone running open models on their own hardware. More in our Coding section.

Christopher Ort

Christopher Ort

Christopher Ort handles SEO, AI agent workflows and PR & partnerships at i10x.ai, the all-in-one AI workspace built by i10X Global Pte. Ltd. in Singapore. He is a graduate of Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU). He is the author of “The Style Penalty”, a study published on SSRN that analyses 1,576 evaluation decisions by four frontier AI models to show how AI-written CVs fare in automated hiring screens; the research has been covered by heise online and t3n.

Editor's picks

  1. CVE-2026-132121: Any website can hijack a local Supergateway and run your MCP tools
  2. CVE-2026-132122: agentic-radar scans can leak your OpenAI API key to a malicious repository
  3. CVE-2026-119220: Dokan MCP endpoint regex flaw lets attackers bypass WooCommerce permissions
  4. Reflection AI Gates Laser, Its Autonomous Penetration Testing Agent, Behind a Trusted Access Program
  5. BrightAI Raises $97 Million to Replace Scheduled Infrastructure Inspections With Physical AI