Models on Vertex AI (@gadhs/pi-vertex)

On this page

Exposes the models a project can call on Google Cloud Vertex AI to pi, with two properties the stock providers lack: per-model region routing and a static, editable JSON catalog. Auth is Application Default Credentials throughout — no API keys, no gcloud subprocess at request time, and no /login: every kind appears in /model on install.

Published as @gadhs/pi-vertex-anthropic through 0.6.0 and renamed when it stopped being only Anthropic (#77); the provider ids, one per publisher kind, did not change, nor did the config file.

One file, three publisher kinds

A catalog entry names its publisher kind, and the kind decides which provider id the model is registered under and which wire the extension delegates to. Nothing is reimplemented at the wire: each kind hands the request to code that already speaks that protocol, and owns only the routing — the region pin, the credential, the retry.

publisher Provider id Wire Models

anthropic (default)

vertex-anthropic

The official @anthropic-ai/vertex-sdk, a client per (project, region)

Claude — vertex-anthropic/claude-opus-5

google

vertex-gemini

pi’s own Vertex Gemini adapter, handed the project and the entry’s region per call

Gemini — vertex-gemini/gemini-3.8-flash

openai-compatible

vertex-maas

pi’s OpenAI-completions adapter against Vertex’s OpenAI-compatible endpoint, with a bearer minted from ADC

Every Model-as-a-Service publisher: vertex-maas/gpt-oss-120b, grok-4.20-reasoning, kimi-k2-thinking, qwen3-coder-480b, minimax-m2

A fourth publisher kind is a fourth row here, not a new way to configure.

Why per-model regions

On Vertex, “a region hosts a model” is not “your project can call it there”: availability and quota are granted per (project, model, region), whoever publishes the model. A real project measured on 2026-09-07. Read the SHAPE, not the cells: no single region serves everything, and grants change per project and over time. node probe.mjs prints this for yours, which is the only authoritative version.

Model global us us-east5 us-central1

claude-opus-5

200

200

quota-0

claude-haiku-4-5

200

quota-0

200

gemini-3.8-flash

200

200

not-found

not-found

gemini-3.1-pro-preview

200

not-found

not-found

not-found

gemini-2.5-pro

200

not-found

200

200

gpt-oss-120b

200

not-found

200

grok-4.20-reasoning

200

200

not-servable

kimi-k2-thinking

200

not-found

not-servable

Each entry pins its own region. For the anthropic kind the SDK owns the region→host mapping, including the us/eu multi-region .rep.googleapis.com hosts; the maas kind uses the same table (vertex-host.mjs, shared with the probe, because us-aiplatform… answers 400 for a multi-region); the google kind’s SDK derives its own host from the location, and agreed with the table on every cell above.

Status meanings: 200 usable (the only status to wire in) · quota-0 HTTP 429 that never clears (quota grant is zero) · data-sharing 403 until the project enables data sharing for that publisher · not-servable 400 · not-found 404.

Configuration

Config resolution order:

  1. $VERTEX_CONFIG (explicit path)

  2. <agentDir>/vertex-models.json (per-user override)

  3. models.json bundled with the package (the org default)

Pre-publisher installs used vertex-anthropic-models.json and $VERTEX_ANTHROPIC_CONFIG; those names are gone, not aliased — rename the file.

Top-level fields

Field Default Meaning

project

"$ANTHROPIC_VERTEX_PROJECT_ID"

GCP project. "$ENV_VAR" form reads the environment (falls back to GOOGLE_CLOUD_PROJECT / GCLOUD_PROJECT); a literal id also works for single-project setups.

models

required

The catalog — see below.

thinkingBudgets

minimal 1024 · low 4096 · medium 10240 · high 20480 · xhigh/max 32768

Token budgets per thinking level for budget-thinking anthropic models (Haiku). Partial overrides fine.

minOutputTokens

1024

Answer room reserved when a thinking budget would otherwise consume the whole output window (anthropic kind).

adcRetry

intervalMs 60000, no cap

Auth-expiry behaviour for every kind: how often to retry a turn that failed on expired Google auth. By default it retries for as long as the turn is alive - your escape is what ends it (#132); maxWaitMs is the opt-in cap, after which the turn ends on the original error and says it gave up. intervalMs: 0 disables.

defaults

sensible

Catalog-wide defaults for contextWindow (200 000), maxTokens (64 000), input, and cost, overridable per model.

Per-model fields

Field Default Meaning

id

required

The pi-facing model id (claude-opus-5, gemini-3.8-flash, gpt-oss-120b). Never contains a slash — pi refs are provider/id.

name

the id

Display name.

region

required

The region your project can call this model in.

publisher

anthropic

anthropic, google, or openai-compatible.

vertexModel

unset

openai-compatible only: the publisher-prefixed string Vertex’s endpoint wants (openai/gpt-oss-120b-maas).

reasoning

google: the catalogue’s, else true; maas: false

google and openai-compatible only. true when the model spends output tokens thinking: the probe sets it for a MaaS model where, having asked with reasoning_effort, it saw a reasoning block, and the battery (#81) set it where a model thinks inside its output budget without exposing one (MiniMax M2). Grok reports reasoning apart from output and ships false.

thinkingAllowance

4096 when the entry reasons, else none

google and openai-compatible only. Tokens the model spends thinking before it answers, added to whatever output budget a caller asks for and capped at maxTokens - so the judge’s 300-token verdict from a model that thinks 3 000 first gets both, the way the anthropic kind’s thinkingBudgets already grows max_tokens. Measured per model by the battery; the heavy thinkers carry their own number (Kimi K2 8192, MiniMax M2 8192, Qwen3-Next 16384, Gemini 2.5 Pro 2048).

costUntil

unset

ISO date, the last day cost is right, for an entry on an introductory rate. The unit tests fail from the next day until the rate is flipped, so a lapsed promotion becomes a deliberate edit rather than a silent under-count of session cost; the model-battery page marks such rows (†). Today: Gemini 3.7 and 3.8 Flash through 2026-12-31.

thinking

none

anthropic only: adaptive (Opus/Sonnet-class), budget (Haiku-class), or none. Refused on other kinds rather than ignored.

xhigh, offSupported

unset / true

anthropic only.

contextWindow, maxTokens, input, cost

see below

Standard pi model metadata; cost is per-million-token pricing for the cost display.

Where the numbers come from: an anthropic entry uses the file, then defaults. A google entry whose id the installed pi-ai’s Vertex catalogue lists inherits its contextWindow, maxTokens, cost and thinking-level map from there (the operator’s pi is the source, so a new Gemini appears when pi updates); the file overrides any of it; an unlisted id gets the file’s values or defaults, never a neighbour’s numbers. A maas entry uses the file, then defaults.

Every packaged entry carries a real rate. test/unit.test.ts refuses a zero-cost entry on any kind (pi would report its cost as zero forever) and checks Anthropic’s multipliers on the anthropic kind; Google’s and the MaaS publishers' rates are cited in the file’s //cost-* notes. An entry on an introductory rate carries costUntil, and the same tests fail the day after it (#90).

The shipped catalog

Model Kind Region Notes

claude-opus-5

anthropic

us

Daily driver in auto/manual

claude-opus-4-8

anthropic

us

Previous Opus

claude-fable-5-1

anthropic

us

Research model; the reviewer; thinking cannot be off

claude-sonnet-4-6

anthropic

global

Fast general model; the judge (auto and plan, at low) and the reviewer floor

claude-haiku-4-5

anthropic

global

Small/fast; the cheapest Claude

gemini-3.8-flash, gemini-3.7-flash, gemini-3.5-flash

google

global

Also answer from us; 3.8 Flash is the Explore and Verify helpers' model

gemini-3.1-pro-preview

google

global

The one Gemini Pro on Vertex today (preview)

gemini-2.5-pro

google

global

Also answers from the US regions

gpt-oss-120b

maas

global

Reasoning block observed; 131k window. Vertex drops the tail of roughly one streamed answer in six (measured; see the file’s //gpt-oss note) - a truncated tool call is a failed tool call

grok-4.20-reasoning, grok-4.20-non-reasoning

maas

global

Also us; no reasoning block exposed even when asked; 262k window on Vertex, whatever xAI’s sheet says (260k answers, 300k is refused)

grok-4.1-fast-reasoning

maas

global

131k window on Vertex (130k answers, 140k is refused)

kimi-k2-thinking

maas

global

Reasoning block observed; 262k

qwen3-coder-480b

maas

global

262k

qwen3-235b

maas

global

262k in, 16 384 out - a larger maxTokens fails every call

qwen3-next-80b-thinking

maas

global

Reasoning block observed; 262k

minimax-m2

maas

global

204k

contextWindow and maxTokens are what Vertex accepts, measured, not the vendor’s sheet: a window claimed larger than the real one deadlocks a session the way the modes page’s "Switching to a model the context does not fit" describes - pi cannot compact on a model the context does not fit, and it never sees the limit coming. tools/window-probe.mjs (repo tooling, not shipped) is how the numbers were taken and how to retake them: for each model, one headless pi run just under the claimed window and one just over, halving on refusal until a run answers, reporting the error text the adapter surfaced and whether pi-ai’s isContextOverflow recognises it. The refusals are not recognised for what they are: the OpenAI-compatible endpoint answers a streamed request’s error as a JSON array, which the adapter reports as 400 status code (no body) - a shape pi-ai’s Cerebras pattern happens to call an overflow, so the overflow is recovered from, but so would any other 400 from that endpoint be; Grok’s "Invalid arguments" and "Service temporarily unavailable" are plain errors with no recovery. The catalog being right is the only real defence.

Not shipped: DeepSeek V3.2 (429 quota-0 on the probe run), Llama and Mistral (request-access on the project). The probe finds them the moment a project can call them.

Editing the active config and running /model in pi reloads it — no restart.

Credential expiry (ADC rotation and retry)

Two behaviours cover an expired ADC session (agency reauth policy), together, for every kind:

  • Rotation pickup. Every attempt re-resolves its credential: the anthropic kind’s cached clients are fingerprinted against the ADC file, the google kind’s adapter reads ADC per request, and the maas kind mints its bearer inside the attempt. Re-running gcloud auth application-default login rewrites the file, and the next request uses it — no pi restart.

  • Turn retry. A request that fails on expired auth is re-issued on a timer instead of killing the turn: re-login in another terminal and the turn resumes by itself - for as long as the turn is alive, unless adcRetry.maxWaitMs caps it. Configure with adcRetry. Auth failures happen at connection, before any content streams, so a retry never duplicates output; non-auth errors propagate immediately. A turn that ends in an error is never announced as a resume: with a cap that closed, the notice says how long it waited and that it gave up; another error stops the retry and says so; after your escape nothing is said. The re-login itself is exercised live (test/adc-relogin.live.test.ts): a copy of the credential with a dead refresh token fails the attempt, is rewritten good while the pump waits, and the same turn completes on the next poll.

  • One immediate refetch (maas). The bearer is cached per process until two minutes before expiry and single-flight; a 401 on a cached token is refetched once and the request replayed before any event reaches pi, so the consumer sees one stream. A 401 on a fresh token is the timer’s business.

Discovering availability (probe.mjs)

Each team probes their own project and generates their catalog:

node probe.mjs                              # every kind, env project, default lists
node probe.mjs --publisher google           # one kind: anthropic | google | maas | all
node probe.mjs --discover                   # enumerate the publisher catalogs first
node probe.mjs --json > ~/.pi/agent/vertex-models.json

It prints one status matrix per kind and a ready-to-paste models block pinning each model to the first region that returned a real HTTP 200, with publisher and vertexModel filled in. For the maas kind it also asks for reasoning and reads the reply, marking reasoning: true only where a block came back. Every cost in the suggestion is a placeholder to fill from the publisher’s price sheet — the probe cannot infer pricing.

Thinking mapping

  • anthropic adaptive (Opus 4.7+/5, Fable 5, Sonnet 4.6): thinking.type: "adaptive" + output_config.effort; pi levels map minimal/low→low, medium→medium, high→high, xhigh/max→xhigh iff "xhigh": true (else clamped to high). The extension injects compat.forceAdaptiveThinking — without it pi-ai silently falls back to a 1024-token budget and drops the effort.

  • anthropic budget (Haiku, older Sonnet/Opus): thinking.type: "enabled"
    budget_tokens, growing max_tokens to keep budget_tokens < max_tokens.

  • "offSupported": false for anthropic models that reject disabled thinking (Fable 5).

  • google: pi’s level goes through pi-ai’s Gemini adapter (LOW / MEDIUM / HIGH; the catalogue’s map says what a model lacks — 3.x has no minimal). Thinking off is raised to low before the adapter sees it: pi-ai’s disabled config sends THINKING_LEVEL_MINIMAL for a Gemini 3 Flash and thinkingBudget: 0 for a 2.x, and Vertex refuses both (measured on 3.8 Flash, 3.7 Flash and 2.5 Pro), so every Gemini turn with thinking off was a 400 - including compaction, which runs thinking off. Off is not a thing these models offer; low is the floor pi-ai itself uses for the 3.x Pro line.

  • openai-compatible: pi’s level goes through as reasoning_effort; whether the vendor honours it is what the probe’s reasoning column measured. The system prompt always travels as role system. pi-ai would send it as developer for a reasoning model on a host its compat probe does not recognise, and the endpoint fronts vendors that drop a developer message unread - MiniMax M2 and Qwen3-Next answered a 113-character system prompt with 13-25 input tokens billed, meaning the mode prompt, the guidance and the agent body were silently absent on those two models (#113). system is the role every vendor behind the proxy honoured when probed. A vendor that opens its answer with a literal <think> tag in the content rather than a reasoning_content field - MiniMax M2 does - has the tag lifted into a thinking block as the stream passes (#112): text_* events become thinking_* up to </think>, the text after it is its own block, every partial and the final message agree, and a block that follows moves one index down. An unclosed tag is thinking to the end - the model ran out of budget mid-thought. Text that does not open with the tag passes untouched, byte for byte. The split is a view: on the way back in, this model’s own earlier turns get the tag put back - <think>…</think>, two newlines, the text, as one content string; the newline count after the tag is normalised, the rest is what the vendor wrote - because pi-ai replays an unsigned thinking block as nothing and MiniMax needs its reasoning in the history (its explore cell fell from 0.81 to 0.52 without it, 0.66 with it and the yield now the answer alone). Another model’s turns are left to pi-ai. Before this, pi’s transcript showed the reasoning as the answer and the cold reviewer’s verdict reader read it (think-tags.test.ts pins the split, the unclosed tag, the pass-through, the index shift and the rejoin; contract.test.ts pins the rejoin reaching the adapter).

What every message is stamped with

pi keys three things on the model stamped on an assistant message equalling the session model’s id: overflow recovery (skipped as "a different model’s error" otherwise), session restore (which looks the stamp up and falls back to the default model when it is not a registered id), and the cost lookup. pi-ai’s adapters stamp what they were handed or what the API answered with - the Vertex string for a MaaS model (openai/gpt-oss-120b-maas), the API’s dated name for Claude (claude-haiku-4-5-20251001, written over at message_start) - neither of which is the id here. Every kind’s stream is restamped with the pi-facing id before pi sees an event; the wire’s name is kept in responseModel, the field pi’s usage totals already prefer. Before this, no session on these providers ever got pi’s overflow recovery, and a resumed session could quietly come back on the default model.

Auth and login

ADC sources: gcloud auth application-default login, a service-account JSON via GOOGLE_APPLICATION_CREDENTIALS, or GCE/GKE workload identity. Each kind is registered with a literal adc placeholder key — pi’s documented way to mark a keyless provider configured — so every kind is in /model on install with no /login. The oauth sentinel from before publishers is kept beside it: a developer who logged in then has a stored record, and pi lets a stored record win, so without the sentinel that provider would vanish from /model. No kind forwards the placeholder: anthropic ignores it (own client), google strips it, maas replaces it with the bearer. If ADC is missing, requests fail with a clear google-auth-library error.

pi’s own built-in google-vertex provider is left alone. A developer who also configures it (env or /login) sees Gemini twice, under two ids; harmless.

Troubleshooting

Symptom Fix

A provider missing from /model

Another extension owns that provider id (only one may). Check pi --list-models. For vertex-anthropic on an install that logged in before publishers existed, see "Auth and login".

no GCP project resolvable

Set "project" in the config or export ANTHROPIC_VERTEX_PROJECT_ID.

Your override seems ignored — the models and regions are the packaged catalog’s

The pre-publisher names are not read at all: $VERTEX_ANTHROPIC_CONFIG and ~/.pi/agent/vertex-anthropic-models.json are silently passed over (no alias, no error). Rename to $VERTEX_CONFIG / vertex-models.json.

403 “data sharing … publisher 'anthropic'”

Enable data_sharing_enabled_provider=anthropic (setPublisherModelConfig), or pin the model to a region where it is already satisfied.

429 that never clears

Quota for that (model, region) is 0 — request quota or re-pin (see probe).

400 “not servable in region X” / 404 on a Gemini or MaaS model

Model not offered there; the newer Gemini and every MaaS model answer from global (and some from us) — pin there.

"thinking" … apply to anthropic entries only

You put an Anthropic thinking switch on a google or maas entry; use reasoning for those kinds.

Thinking seems ignored (anthropic)

"thinking" is probably budget on an adaptive model — fix the entry.

A 400 on every turn right after /model, and /compact is a 400 too

The context no longer fits the new model and pi’s recovery runs on that same model. /new starts fresh; or switch back and /compact there. The modes extension warns at the switch and offers to do the second for you (modes.adoc#_switching_to_a_model_the_context_does_not_fit).

400 on a Gemini entry only with thinking off

Your extension predates the floor: thinking off is raised to low for the google kind since 0.5.0; update.

Thinking seems ignored (maas)

That vendor does not expose a reasoning block; the probe’s column said so and the entry carries reasoning: false.

A reasoning model defers every judge call, or a small-budget caller only ever gets length

The model spends the caller’s output budget thinking. Set reasoning: true on the entry and, if the default 4096 is not enough, a thinkingAllowance sized to what it thinks (the battery’s records show it). Before #81 the judge’s 300 tokens ended 162-238 of 244 calls on length for four packaged models.

Edit this page · latest