Tegy model evaluation Live provider evidence July 11, 2026

Grok 4.5 vs Tegy Doctrine

A controlled StrategyOS bakeoff covering vision, subagent orchestration, strategic intake, PDF analysis, latency, and exact provider-reported cost.

Executive verdict

Grok 4.5 is a credible fast deep-work candidate, and vision is not a blocker. It completed the three StrategyOS scenarios 6.7× faster than Doctrine, but cost 48.5% more in aggregate. Doctrine still produced the stronger consulting analysis on the substantive Sprinta PDF task, so this run does not support replacing Doctrine.

Headline evidence

Strategy totals cover the identical subagent, intake, and PDF scenarios. Vision was evaluated separately using the same generated image sentinel.

6.7× Faster across the three StrategyOS scenarios 35.3s Grok vs 234.8s Doctrine
+48.5% Higher aggregate strategy-run cost $0.6089 Grok vs $0.4099 Doctrine
2 / 2 Successful end-to-end vision probes Both returned “copper harbor” exactly
$1.2690 Total reported spend for all eight turns Price caps remained in force

Measured performance

Lower is better in both charts. Bars use a shared scale within each chart; labels show the exact observed value.

Result ledger

Scenario Model Outcome Latency Cost Observed behavior
Vision sentinel Grok 4.5 Pass 4.9s $0.129679 Returned the exact visible text.
Vision sentinel Doctrine Pass 11.4s $0.120580 Returned the exact visible text.
Two subagents Grok 4.5 Pass 9.5s $0.293080 Started and completed both subagents; combined the words correctly.
Two subagents Doctrine Pass 43.8s $0.185332 Started and completed both subagents; combined the words correctly.
Upmarket vs SMB intake Grok 4.5 Deferred 12.0s $0.155999 Recognized the high-stakes decision and opened a four-question questionnaire.
Upmarket vs SMB intake Doctrine Deferred 36.6s $0.029189 Framed the decision around PMF, unit economics, and constraints; asked three questions.
Sprinta PDF Grok 4.5 Mixed 13.8s $0.159809 Read the attachment and asked one question, but the visible analysis was generic.
Sprinta PDF Doctrine Stronger 154.3s $0.195368 Developed a concrete first-pass hypothesis and vulnerable assumptions before asking one question.

Consulting-quality review

Speed and price are directly measured. Quality assessment is based on visible assistant output and workflow behavior, not a separate model judge.

Grok 4.5

  • Fastest model in every tested scenario.
  • Correctly used StrategyOS, tools, subagents, questionnaires, PDF context, and image input.
  • Produced concise terminal answers and avoided long overthinking.
  • Did not demonstrate enough first-pass depth on the substantive PDF case.
“Broad company analysis needs StrategyOS first, then one scope lock so the first-pass stays decision-grade rather than generic.”

Tegy Doctrine

  • Slower in every scenario, dramatically so on the PDF case.
  • Lower aggregate cost across the three StrategyOS scenarios.
  • Produced the strongest substantive strategy work in this sample.
  • Used more output on the PDF task before deferring to the user.
“Null hypothesis: Sprinta can validate its B2C high-touch coaching subscription … and should therefore optimize this core funnel before diverting resources to freemium plans or coach-infrastructure B2B.”
The questionnaire scenarios stopped at the deferred tool boundary. They prove intake behavior, not final recommendation quality. A completed artifact comparison requires identical answers to each model’s questions.

Vision finding

What was proven

Both models received the same generated PNG through Tegy’s real attachment storage, sandbox materialization, Claude Agent SDK, Cloudflare AI Gateway, and OpenRouter route. Both returned “copper harbor” exactly.

What remains unproven

This validates the end-to-end vision path and basic visual text recognition. It is not a test of chart interpretation, slide critique, spatial reasoning, or dense document-image analysis.

Published capability and price context

Model OpenRouter ID Context Input modalities Input / output price
Grok 4.5 x-ai/grok-4.5 500K Text, image, file $2.00 / $6.00 per 1M
Doctrine moonshotai/kimi-k2.7-code 262K Text, image $0.72 / $3.49 per 1M

Sources: SpaceXAI Grok 4.5 documentation, OpenRouter xAI catalog, and OpenRouter Kimi K2.7 Code. Prices are a July 11, 2026 snapshot and may change.

Methodology and environment

This was a controlled local live-provider evaluation. The candidate mapping existed only for the test and was removed immediately afterward.

Runtime
Claude Agent SDK 2.1.165 with StrategyOS and Tegy runtime skills loaded.
Provider path
Local Tegy container API → Cloudflare AI Gateway → OpenRouter.
Effort
Medium for both models.
Cache status
All captured model requests were cache misses.
Cost safety
$0.50 cap per vision turn and $2 cap per strategy turn.
Cost source
Terminal SDK/provider-reported total_cost_usd.

Recommendation and next test

  1. Do not replace Doctrine from this evidence alone. Doctrine retained the strongest strategy-depth signal.
  2. Keep Grok 4.5 in evaluation as a fast deep-work candidate. The latency improvement is large enough to matter operationally.
  3. Run an end-to-end strategy artifact comparison. Supply identical business facts after intake and score the final recommendation, evidence use, decision clarity, and artifact quality.
  4. Add a real visual reasoning case. Use a chart, dashboard, or strategy slide—not only an OCR sentinel.

Verification and disclosure