Auto model routing

The two Auto ids

Picking ih-auto-smart is the consent to that variable billing: most turns settle on the flat sheet, escalated turns settle at Sonnet 5's normal rates. If you never want frontier billing, use ih-auto.

Use it in your client

Both ids serve on the OpenAI Chat wire (/v1/chat/completions) and the Anthropic Messages wire (/v1/messages), so they work in Cursor, Claude Code, Kilo Code, and anything else that speaks either format. They are not available on the Responses wire (Codex).

curl https://app.inferencehub.tech/v1/chat/completions \
  -H "Authorization: Bearer sk-prov-live-YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ih-auto-smart",
    "messages": [{ "role": "user", "content": "Explain this stack trace: ..." }]
  }'

In Claude Code, set the model like any other id (or pick it from the /model list):

// ~/.claude/settings.json
{
  "env": {
    "ANTHROPIC_BASE_URL": "https://app.inferencehub.tech",
    "ANTHROPIC_AUTH_TOKEN": "sk-prov-live-YOUR_KEY"
  },
  "model": "ih-auto-smart"
}

In Cursor, add ih-auto or ih-auto-smart as a custom model on your InferenceHub endpoint — setup on the Cursor page.

Pricing: a flat sheet, honestly marked up

Auto bills a flat $1.00 per 1M input tokens and $5.00 per 1M output tokens, whatever model the router picks. Named models on InferenceHub pass through at native rates with no markup; Auto is the one deliberate exception — the spread between the flat sheet and the routed model's native price is the routing fee. In exchange, routine agent turns never cost frontier rates.

• No cache classes — the sheet has one input price; cached input tokens bill at the same $1.00 rate. If your workload is cache-heavy on a single model, naming that model directly (with its cache discount) can beat Auto — see Billing & plans.
• Escalated turns (Smart Auto) bill as a manual sonnet-5 pick — its normal per-token rates, not the sheet.
• On subscription plans, Auto turns debit the standard windows (the sheet is below the $4/1M premium threshold), and escalated Sonnet 5 turns do too — Auto never touches the premium budget. That makes it the all-day coding id on Plus and Max.

How a request routes

Routing is a deterministic rule table over the request itself — no LLM classifier sits in the hot path adding latency, and your request body never goes anywhere except to the model that serves it. The router reads the request's shape (estimated size, images, a narrow list of hard-reasoning cues, tool presence) and picks from the current pool:

The pool is curated for speed and tool reliability (every member passes a tool-calling gate and a decode-rate floor) and will change as models come and go — the disclosure header below always tells you what actually served the turn. Conversations are sticky: the same conversation keeps its routed model for ~15 minutes of idle, so multi-turn agent loops don't flap between models mid-task.

When Smart Auto escalates

Escalation is a capability floor, not an upgrade to the most expensive model. Smart Auto hands a turn to sonnet-5 — never Opus or Fable — when:

• the request needs something the pool can't do (images on the Anthropic wire, a prompt past the pool's limits, genuinely hard-reasoning cues), or
• a pool model just failed the turn — an empty completion or an unparseable tool call triggers one reactive retry on Sonnet 5 instead of handing you the failure.

Plain ih-auto never escalates: images get an honest 400, and hard turns route to the pool's strongest model. On the OpenAI Chat wire, images get a 400 from both ids.

You always see what served you

Auto is not a black box. Three places tell you the truth on every turn:

• the response's model field carries the routed model's id, not the sentinel;
• the x-ih-auto response header spells out the decision, e.g. sentinel=ih-auto-smart; routed=deepseek-3.2; class=default (escalated turns add escalated=1);
• the portal's usage view shows both the id you asked for and the model that served each request, with its cost.

Context compaction

Long agent sessions accumulate context until a request stops fitting the routed model's window — and until then, you pay to re-send the whole history every turn. On the Auto ids the gateway can compact the conversation instead. Compaction keeps the system prompt, the first user message, and the most recent exchanges, and drops the oldest middle turns in whole tool-call/tool-result groups (never splitting a pair). It applies only to the Auto ids — named models are always forwarded byte-faithful.

• Rescue (automatic) — a conversation that has outgrown roughly 80k estimated tokens is compacted just enough to keep serving, where the alternative was a context-window error.
• Opt-in (per request) — send x-ih-compact: 1 to trim to a ~40k-token working set, or a number (e.g. x-ih-compact: 20000) for an explicit target. Invalid values are ignored — the header can never fail a working request. In Claude Code: ANTHROPIC_CUSTOM_HEADERS="x-ih-compact: 1".

Billing follows what is actually forwarded: a compacted turn bills the compacted input, so opting in directly cuts what long conversations cost. Compacted turns disclose it on the same header, e.g. x-ih-auto: …; compacted=1; dropped=6. If a conversation can't be compacted safely, the gateway forwards it untouched rather than guess.