Z.ai GLM-Legal Beats Claude Fable 5 and Kimi K3 on Harvey LAB


Z.ai just released GLM-Legal. GLM-Legal is a legal AI model from Z.ai, built on GLM-5.3-Flash and trained for agentic legal work. It targets contract review, evidence analysis and compliance research. The key result is a 19.2% all-pass rate on Harvey’s Legal Agent Benchmark (LAB). That is nearly 3x its base model’s 6.7%. For developers, the catch is access: the API is application-only for now.

TL;DR

  • Size: Built on GLM-5.3-Flash, a 320B-total, 18B-active MoE with a 1M-token context and 128K max output. Z.ai lists no separate GLM-Legal parameter count.
  • Runs on: Z.ai’s hosted API only, by application. No public weights. The base model self-hosts on multi-GPU servers via vLLM or SGLang, with FP8 weights.
  • Performance: Top-10 on Harvey LAB at a fraction of frontier cost. Within 2.3 points of the LegalBench leader.
  • Best: 19.17% Harvey LAB task pass rate, #7 of 79 on Vals AI, at $0.15 per test.
  • Worst: 41.83% on Vals AI’s Legal Research Bench, below its own base model (45.19%).
  • Bottom line:
    • Best: frontier-class legal agent scores at about 1/128 the cost of Claude Fable 5.
    • Worst: gated beta with no price, no weights and slow runs (1h12m per task).

GLM-Legal is a domain-specialized version of GLM-5.3-Flash for law firms, in-house counsel and compliance teams. The base is Z.ai’s first natively multimodal GLM-5 model. It has 320B total parameters with 18B active. It mixes sparse and linear attention to cut long-context serving cost. It reads text, images, video and files, and outputs text.

Training ran in 2 stages:

The Z.ai team also describes the agent harness co-evolving with the model during post-training. Edge cases in training exposed harness bottlenecks. Harness fixes then exposed the model to more realistic constraints.

Harvey LAB is an open-source benchmark of 1,250 tasks across 24 practice areas. Over 75,000 expert-written rubric criteria grade the work. A task passes only if every criterion passes. Harvey calls this all-pass grading.

Vals AI runs the held-out set with 2 LLM judges and internet disabled. Agents get file tools plus docx, pptx and xlsx skills.

On that leaderboard, GLM-Legal scores 19.17% and ranks #7 of 79. Its base, GLM-5.3-Flash, scores 6.67%. The 12.5-point lift is the core claim, and an independent evaluator reproduced it.

Z.ai’s launch chart shows GLM-Legal ahead of Kimi K3 (12.92%), Claude Fable 5 (11.25%) and GPT-6 Astra (5.42%). Those figures match Vals AI. The chart omits 6 models ranked higher, led by Muse Spark 1.2 at 25.42%.

GLM-Legal also passes 92.5% of individual criteria. Top models sit near 90% to 95% on that measure.

Roughly. Z.ai claims parity on LegalBench and the Chinese-focused PLawBench without publishing numbers. Vals AI scores GLM-Legal at 86.23% on LegalBench. Claude Fable 5 leads at 88.56%. Kimi K3 scores 86.21%.

‘}).join(”);requestAnimationFrame(function(){requestAnimationFrame(function(){host.querySelectorAll(‘.fill’).forEach(function(f){f.style.width=f.dataset.w+’%’})})});post()}
var pct=function(d){return d.v.toFixed(2)+’%’};
var zai=[{n:’GLM-Legal’,v:19.17,c:’me’},{n:’Kimi K3′,v:12.92},{n:’Claude Fable 5′,v:11.25},{n:’GLM-5.3-Flash’,v:6.67,c:’base’},{n:’GPT-6 Astra’,v:5.42}];
var full=[{n:’#1 Muse Spark 1.2′,v:25.42},{n:’#2 Muse Spark 1.3 Max’,v:23.75},{n:’#3 Muse Spark 1.3′,v:22.92},{n:’#4 Wonder Full’,v:21.67},{n:’#5 Muse Spark 1.1′,v:20.00},{n:’#6 Gemini 4 Argon’,v:19.58},{n:’#7 GLM-Legal’,v:19.17,c:’me’},{n:’#8 Wonder’,v:17.08}];
var bh=document.getElementById(‘mtp-gl-bars’),bn=document.getElementById(‘mtp-gl-barnote’);
function view(v){R.querySelectorAll(‘[data-v]’).forEach(function(b){b.classList.toggle(‘on’,b.dataset.v===v)});
if(v===’zai’){bars(bh,zai,26,pct);bn.textContent=”The 5 models Z.ai chose for its launch chart. All figures match Vals AI.”}
else{bars(bh,full,26,pct);bn.textContent=”The full Vals AI ranking: 6 models from Meta and Google sit above GLM-Legal.”}}
R.querySelectorAll(‘[data-v]’).forEach(function(b){b.onclick=function(){view(b.dataset.v)}});
var cost=[{n:’GLM-Legal’,v:0.15,p:19.17,c:’me’},{n:’GLM-5.3-Flash’,v:0.57,p:6.67,c:’base’},{n:’Kimi K3′,v:3.83,p:12.92},{n:’Claude Fable 5′,v:19.23,p:11.25},{n:’GPT-6 Astra’,v:26.16,p:5.42}];
var lb=[{n:’Fable 5′,v:88.56},{n:’GLM-Legal’,v:86.23,c:’me’},{n:’Kimi K3′,v:86.21},{n:’Base’,v:83.93,c:’base’}];
var lr=[{n:’Fable 5′,v:49.52},{n:’Kimi K3′,v:46.15},{n:’Base’,v:45.19,c:’base’},{n:’GLM-Legal’,v:41.83,c:’me’}];
var drawn={};
function open(id){R.querySelectorAll(‘.tab’).forEach(function(t){t.classList.toggle(‘on’,t.dataset.t===id)});R.querySelectorAll(‘.panel’).forEach(function(p){p.classList.toggle(‘on’,p.id===id)});
if(id===’p1′)counters(document.getElementById(‘p1′));
if(id===’p2’)view(R.querySelector(‘[data-v].on’).dataset.v);
if(id===’p3′)bars(document.getElementById(‘mtp-gl-cost’),cost,27,function(d){return ‘$’+d.v.toFixed(2)});
if(id===’p4’){bars(R.querySelector(‘[data-set=lb]’),lb,100,pct);bars(R.querySelector(‘[data-set=lr]’),lr,100,pct)}
post()}
R.querySelectorAll(‘.tab’).forEach(function(t){t.onclick=function(){open(t.dataset.t)}});
post();
})();



Source link

  • Related Posts

    Alibaba Qwen Introduces QwenGyre: Elastic Reinforcement Learning RL for XLong-Horizon Agents

    Alibaba’s Qwen team has introduced QwenGyre, an elastic reinforcement learning RL framework for extreme-long-horizon agent training. QwenGyre is an online reinforcement learning system from Alibaba…

    Top 12 AI Live Translation Tools in 2026: Latency, Languages, Pricing

    AI live translation tools turn speech into translated captions or speech while people are still talking. The market now spans consumer apps, meeting add-ons, event platforms and developer APIs. We…

    Leave a Reply

    Your email address will not be published. Required fields are marked *