Technical Guide

OpenRouter + Structured Output: Complete Guide

Production-ready setup for reliable JSON from 100+ models. Based on 672 real-world tests across 8 providers.

TL;DR: Raw OpenRouter JSON parsing varies wildly by model: 85-100%. Pick the right model OR add repair layer. 95% of failures are just markdown fences (trivial fix).

The Problem: OpenRouter Doesn't Guarantee Valid JSON

OpenRouter routes to 100+ models from different providers (OpenAI, Anthropic, Meta, Mistral, etc.). Each has different:

Real failure example:

```json
{
  "name": "Alice",
  "age": 30,
}
```

JSONDecodeError: Expecting property name enclosed in double quotes

This happened in 67% of tests without repair (64/192 failures).

When You DON'T Need JSON Repair

Be honest with yourself - you might not need this:

Model Reliability (Tested Feb 2026)

We tested 8 popular OpenRouter models with 7 different JSON tasks, 3 trials each (672 total tests):

Model Raw Success With StreamFix Notes
mistral-small-creative 100% 100% Best overall, free tier
gpt-4o-mini 100% 100% Fast, reliable, paid
llama-3.3-70b-instruct 95.8% 100% Good balance
ministral-8b-2512 95.8% 100% Fast, cheap
devstral-2512 91.7% 95.8% Code-focused
glm-4.7-flash 91.7% 95.8% Chinese model
seed-1.6-flash 87.5% 95.8% Budget option
kimi-k2.5 85.4% 95.8% Most failures

Key insight: Even "bad" models (85%) reach 96%+ with repair. Focus on cost/speed, not just raw JSON quality.

Production Setup (3 Options)

Option 1: Direct OpenRouter (No Repair)

Use when: Using mistral-small or gpt-4o-mini only (100% raw success)

from openai import OpenAI

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="YOUR_OPENROUTER_KEY"
)

response = client.chat.completions.create(
    model="mistralai/mistral-small-creative",  # 100% reliable
    messages=[
        {"role": "system", "content": "Output valid JSON only"},
        {"role": "user", "content": "Extract user data: Alice, 30 years old"}
    ],
    response_format={"type": "json_object"},  # Helps but not guaranteed
)

data = json.loads(response.choices[0].message.content)  # ✓ Works

Option 2: OpenRouter + StreamFix (Recommended)

Use when: Multi-model setup, need 98%+ reliability, using cheaper models

from openai import OpenAI

client = OpenAI(
    base_url="https://streamfix.up.railway.app/v1",  # Use StreamFix
    api_key="YOUR_STREAMFIX_KEY"  # Get free: /account/create
)

response = client.chat.completions.create(
    model="openrouter/mistralai/ministral-8b-2512",  # prefix: openrouter/
    messages=[
        {"role": "user", "content": "Extract: Alice, 30"}
    ],
    extra_body={
        "provider": {
            "openrouter": {
                "api_key": "YOUR_OPENROUTER_KEY"  # Pass OpenRouter key
            }
        }
    }
)

# JSON is guaranteed valid - no try/catch needed
data = json.loads(response.choices[0].message.content)  # ✓ Always works

StreamFix fixes: Trailing commas (15% of errors), markdown fences (95.5% of errors), unquoted keys, incomplete streams, single quotes, and more.

Option 3: Fallback Chain (Maximum Uptime)

Use when: Production apps needing 99.9% uptime

from openai import OpenAI

client = OpenAI(
    base_url="https://streamfix.up.railway.app/v1",
    api_key="YOUR_STREAMFIX_KEY"
)

response = client.chat.completions.create(
    model="openrouter/mistralai/mistral-small-creative",
    messages=[{"role": "user", "content": "..."}],
    extra_body={
        "provider": {
            "openrouter": {
                "api_key": "YOUR_OPENROUTER_KEY"
            }
        },
        "fallback_models": [  # Auto-retry with these if primary fails
            "openrouter/openai/gpt-4o-mini",
            "openrouter/meta-llama/llama-3.3-70b-instruct"
        ]
    }
)

# StreamFix automatically retries with fallbacks if primary fails
data = json.loads(response.choices[0].message.content)

Common Failure Patterns (from 672 tests)

1. Markdown Fences (95.5% of all failures)

```json
{"name": "Alice"}
```

Why: Models trained on GitHub code. Fix: StreamFix strips automatically.

2. Trailing Commas

{"name": "Alice", "age": 30,}

Models affected: kimi-k2.5, seed-1.6-flash. Fix: StreamFix removes.

3. Unquoted Keys

{name: "Alice", age: 30}

Models affected: Older/smaller models. Fix: StreamFix adds quotes.

4. Single Quotes

{'name': 'Alice'}

Why: Python-style output. Fix: StreamFix converts to double quotes.

Best Practices

✅ Do:

❌ Don't:

Quick Start

# 1. Get StreamFix API key (1000 free credits)
curl -X POST https://streamfix.up.railway.app/account/create?email=you@example.com

# 2. Update base URL in your existing code
client = OpenAI(
    base_url="https://streamfix.up.railway.app/v1",  # Change this
    api_key="YOUR_STREAMFIX_KEY"
)

# 3. Add openrouter/ prefix to model names
model="openrouter/mistralai/mistral-small-creative"

# 4. Pass OpenRouter key in extra_body
extra_body={
    "provider": {
        "openrouter": {"api_key": "YOUR_OPENROUTER_KEY"}
    }
}

# That's it! JSON parsing now: 33.3% → 98.4% ✓

Cost Comparison

Setup Success Rate Cost/1M tokens Wasted Cost (failures)
OpenRouter only (kimi-k2.5) 85.4% $0.03 $0.0044
OpenRouter only (mistral-small) 100% $0.10 $0
StreamFix + OpenRouter (kimi-k2.5) 95.8% $0.04 $0.0017

Bottom line: StreamFix adds ~$0.01/1M tokens but saves 67% of failures. ROI positive at any scale.

Related Resources