What we should do
Run one more focused Grok-versus-GLM-versus-Doctrine evaluation on memo reliability, numerical accuracy, constraint compliance, and timeout behavior before changing the production model.
Executive verdict
Keep Doctrine in place for now, and continue testing Grok as the leading challenger. Grok and GLM were the candidates that consistently organized the requested specialist work. Luna was fast and inexpensive, but repeatedly said it had created a memo when no memo existed. Terra showed promising judgment, but did not reliably finish the workflow. GLM 5.2 delivered every memo and specialist audit, but repeated the same budget-constraint error in all three runs. Muse Spark 1.1 is not currently available on OpenRouter, so it could not be tested under the same conditions.
What we should not do
Do not promote Luna or Terra based on speed or list price alone. In Tegy, finishing the work and delivering the promised artifact are part of the product.
The result in four numbers
Each scored model received the same information, tools, StrategyOS setup, and request to create a decision memo after using two specialist agents.
What each model was actually good at
“Completed” means the model returned an answer. “Memo delivered” means the requested file was genuinely created and captured—not merely mentioned in the answer.
Grok 4.5
Leading challenger- Completed all three strategy runs.
- Used the two requested specialists in all three runs.
- Delivered the real memo in two of three runs.
- Produced the most consistently structured strategy work.
- Trade-off: slowest candidate and highest observed workflow cost.
Tegy Doctrine
Current baseline- Completed and delivered a memo in two of three runs.
- One run failed after almost eleven minutes.
- Used both specialists on the two successful runs.
- One memo contained basic calculation errors.
- Still the safer production choice until a challenger is more reliable.
GPT-5.6 Terra
Promising judgment- Returned an answer in all three runs.
- Delivered the real memo only once.
- Did not produce a clean two-specialist execution trace.
- Its completed memo showed strong restraint around missing data.
- Worth watching, but not production-ready for this workflow.
GPT-5.6 Luna
Not ready- Fastest strategy responses by a wide margin.
- Lowest average reported strategy-run cost.
- Skipped the requested specialist work in all three runs.
- Created zero memo files while presenting links as if they existed.
- Suitable for lighter answers, not Tegy's current agent workflow.
GLM 5.2
Reliable but flawed- Returned an answer and created the real memo in all three runs.
- Completed both requested specialists in all three runs.
- Faster median than Grok and Doctrine; slower than Terra and Luna.
- Repeated the same material budget-constraint error in every memo.
- OpenRouter currently exposes text only, so no vision test was possible.
Strategy-work scorecard
Three isolated runs per model. Median time includes the failed Doctrine run; average cost includes successful turns with a reported terminal cost.
| Model | Answer returned | Real memo delivered | Clean specialist workflow | Median time | Average reported cost | Decision |
|---|---|---|---|---|---|---|
| GLM 5.2 | 3 / 3 | 3 / 3 | 3 / 3 | 4m 11s | $1.30 | Reliable, revise |
| Grok 4.5 | 3 / 3 | 2 / 3 | 3 / 3 | 6m 08s | $1.39 | Continue |
| Doctrine | 2 / 3 | 2 / 3 | 2 / 3 | 6m 51s | $1.09* | Keep baseline |
| GPT-5.6 Terra | 3 / 3 | 1 / 3 | 0 / 3 | 2m 51s | $1.13 | Watch |
| GPT-5.6 Luna | 3 / 3 | 0 / 3 | 0 / 3 | 1m 10s | $0.58 | Do not adopt |
Why Luna's low cost needs context: it did less of the
requested work. A cheaper run that skips specialist analysis and never
creates the deliverable is not a cheaper completed strategy project.
*Doctrine's failed run ended without a terminal cost value, so its average reflects the two successful runs and the suite total may understate that failed attempt.
Speed and observed workflow cost
Lower is better. These numbers reflect the complete Tegy workflow—not only the parent model's published token rate.
New candidate update: GLM measured, Muse pending
GLM 5.2 was run through the same strategy-artifact harness. Muse Spark 1.1 was checked against OpenRouter's live catalog but could not enter the benchmark because no OpenRouter route exists yet.
GLM 5.2 — measured result
3 / 3 delivered- All three runs created the memo and completed both specialists.
- Median time: 4m11s; average reported cost: $1.30.
- Every memo selected SMB and included calculations, risks, triggers, and a 30/60/90 plan.
- Every memo also proposed up to $500K of SMB spend plus a $150K SSO build, breaching the case's total $500K investment cap.
- One memo contained a malformed “Days 91–90+” plan heading.
Muse Spark 1.1 — availability result
Test pending- Meta describes it as a one-million-token multimodal reasoning model for agents, coding, tools, and computer use.
- It is designed to plan and delegate across parallel subagents.
- No Muse or Spark entry appeared in OpenRouter's live model catalog.
- Three plausible OpenRouter model IDs returned HTTP 400 “not a valid model ID.”
- Testing Meta's direct API would change the provider path and would not be the same experiment.
Interpretation: GLM 5.2 is the best artifact-delivery
performer in this small sample, but not the best decision performer.
Muse Spark 1.1 should be tested immediately if OpenRouter lists it;
until then it has no comparable Tegy score.
Naming note: the current Tegy production catalog maps
Doctrine 1.0 to Kimi K2.7 Code. GLM 5.2 was evaluated as a separate
candidate so the two results remain distinguishable.
Muse sources: Meta's Muse Spark 1.1 announcement and OpenRouter's live model catalog API, checked July 13, 2026.
Vision result
Yes for the four routes that expose image input. Each vision-tested model received the same chart image and had to filter segments, calculate two ratios, identify the winner, and report its retention.
| Model | Independent result | SMB | Mid-market | Winner and retention | Time |
|---|---|---|---|---|---|
| Luna | Correct | 3.50× | 6.35× | Mid-market · 84% | 12.1s |
| Terra | Correct | 3.50× | 6.35× | Mid-market · 84% | 13.2s |
| Grok 4.5 | Correct | 3.50× | 6.35× | Mid-market · 84% | 14.5s |
| Doctrine | Correct | 3.50× | 6.35× | Mid-market · 84% | 29.2s |
Only the first run for each model counts as independent vision
evidence. Later repetitions were exact gateway cache hits, so we did
not present them as additional model successes. OpenRouter exposes GLM
5.2 as text-only, and Muse Spark 1.1 has no OpenRouter route, so
neither received a vision score.
Published capability and list-price context
List prices help with planning, but they do not predict the cost of an agent workflow by themselves. Tool calls, repeated turns, specialist agents, caching, and whether the model actually finishes the task all matter.
| Model | Designed for | Context | Image input | Published input / output price |
|---|---|---|---|---|
| Muse Spark 1.1 | Agentic multimodal work | 1M | Officially yes; no OpenRouter route | Direct Meta API reported at $1.25 / $4.25 per 1M |
| GLM 5.2 | Long-context reasoning and agent work | 1.05M | No on OpenRouter | $0.85 / $2.50 per 1M |
| GPT-5.6 Luna | High-volume, cost-sensitive work | 1.05M | Yes | $1 / $6 per 1M |
| Grok 4.5 | General reasoning and agent work | 500K | Yes | $2 / $6 per 1M |
| GPT-5.6 Terra | Balance of intelligence and cost | 1.05M | Yes | $2.50 / $15 per 1M |
| Doctrine | Tegy's current deep strategy route | 262K | Yes | $0.72 / $3.49 per 1M |
Sources: Meta Muse Spark 1.1 announcement, Muse Spark direct-API price reference, OpenRouter GLM 5.2, OpenAI Luna model page, OpenAI Terra model page, OpenAI API pricing, and the July 11 Grok-vs-Doctrine report for the published Grok and Doctrine context. Prices are a July 13, 2026 snapshot and may change.
Why this is stricter than our first Grok test
The July 11 evaluation established that Grok could see images, use tools, and move quickly on short scenarios. This follow-up deliberately raised the bar.
Read the earlier Grok 4.5 vs Doctrine report.
How the test was kept fair
- Runtime
- Claude Agent SDK 2.1.165 with StrategyOS and 56 Tegy runtime skills.
- Conditions
- Same sandbox, tools, strategy prompt, reasoning effort, and output requirement.
- Specialists
- All five scored parent models were assigned Tegy Playbook as the subagent route.
- Execution
- Three strategy repetitions per model. The original four-model order rotated between runs; GLM was evaluated afterward under the same conditions.
- Vision
- One independent chart-reasoning request per model; cached repeats excluded from the evidence count.
- Cost source
- Terminal SDK/provider-reported USD cost. Failed Doctrine turn had no terminal cost value.
Recommended next decision
- Keep Doctrine as the production default today. None of the candidates demonstrated enough end-to-end reliability to justify an immediate switch.
- Advance Grok to a final head-to-head. Test ten artifact-heavy cases and score file delivery, numerical correctness, specialist synthesis, timeout rate, latency, and cost.
- Keep GLM 5.2 in that final round as the reliability benchmark. It completed the workflow perfectly, but the repeated investment-cap error means constraint accuracy must be a hard scoring gate.
- Do not advance Luna for Doctrine's role. Its speed is attractive for simpler response-only tasks, but its false artifact claims are a product-integrity risk.
- Retest Terra only if its sandbox artifact and delegation compatibility improves. Its strongest completed memo suggests useful analytical potential.
- Queue Muse Spark 1.1 for testing when it reaches OpenRouter. Its official agent and subagent design closely matches Tegy's workload, but it has no comparable evidence today.
Limitations and disclosure
- This is a focused operational evaluation, not a general intelligence benchmark.
- Three strategy runs are enough to expose workflow failures, but not enough to estimate production reliability precisely.
- GLM 5.2 was evaluated on the strategy-artifact case only because its OpenRouter route is text-only.
- Muse Spark 1.1 was checked for OpenRouter availability only. It was not benchmarked and is excluded from rankings and measured totals.
- Quality review was performed against visible outputs and required workflow events; no model was used as an automated judge.
- Absolute runway could not be calculated because the fictional case supplied a runway rule but not a company cash balance.
- Temporary candidate registrations and test-harness changes were removed after the run. No production model configuration changed.
- Internal chat IDs, session IDs, gateway log IDs, storage keys, credentials, and raw captures are intentionally excluded.
- No video is included because this report covers backend model behavior rather than an interactive UI change.