Using local models from your machine (codex / Qwen Code / chat apps)

Run your local codex or Qwen Code CLI, or an OpenAI-compatible chat app, against the on-prem models through the ai-agents inference proxy — no OpenAI/Anthropic cloud, no GPU on your machine. Inference runs on the on-prem vLLM backends; the proxy handles auth, routing, and tool-call formatting.

your laptop (codex / qwen / chat app)
   │  OpenAI-compatible HTTPS + SSO token (lab ai)
   ▼
ai-agents proxy  (chat.ai-agents-dev.bor.labrats.work)
   │  resolves the model → backend, normalizes tool calls
   ▼
on-prem vLLM backends

Prerequisites

• codex CLI ≥ 0.13 (uses the Responses API) and/or Qwen Code CLI (qwen).
• Network access to https://chat.ai-agents-dev.bor.labrats.work (reachable off-cluster; the proxy rejects unauthenticated requests with 401).
• The lab CLI (infra.lab-cli 0.5.0 or later; standards scripts/setup-workstation.sh installs it) and membership of the aiagents-developer group.

1. Log in with SSO

People authenticate with a short-lived SSO token, never a static key or a cluster Secret (spec 0078):

lab ai login     # device code flow: open the URL in any browser, sign in with SSO
lab ai token     # prints a valid token (refreshes it when near expiry)
lab ai logout    # deletes the stored tokens

One login lasts a workday. Each CLI picks the token up its own way (below); don't export AI_AGENTS_API_KEY or OPENAI_API_KEY in your shell profile. Static API keys are for machines only (GitHub Actions, in-cluster services).

2. See available models

curl -s -H "Authorization: Bearer $(lab ai token)" \
  https://chat.ai-agents-dev.bor.labrats.work/v1/models | jq '.data[].id'

That list is the source of truth for what you can use — the served models change, so this guide names none. Wherever an example below says <model-id>, put an id from it. An id can be listed while its backend is down; requesting it then fails (see Troubleshooting).

3. codex CLI (Responses API)

codex ≥ 0.13 only speaks the Responses API (wire_api = "responses"), which the proxy now supports.

Add to ~/.codex/config.toml:

model          = "<model-id>"
model_provider = "labrats"

[model_providers.labrats]
name     = "labrats ai-agents proxy"      # required — codex errors if empty
base_url = "https://chat.ai-agents-dev.bor.labrats.work/v1"
wire_api = "responses"
auth     = { command = "lab", args = ["ai", "token"] }

codex runs lab ai token for the bearer, re-running it every 5 minutes and after a 401, so a session outlives one token (auth replaces env_key; the two are exclusive).

Run it:

codex                       # interactive (prompts before running tools)
codex exec "summarize README.md"   # non-interactive

Per-invocation (no config edit):

codex -m "<model-id>" \
  -c model_provider=labrats \
  -c 'model_providers.labrats.name="labrats"' \
  -c 'model_providers.labrats.base_url="https://chat.ai-agents-dev.bor.labrats.work/v1"' \
  -c 'model_providers.labrats.wire_api="responses"' \
  -c 'model_providers.labrats.auth.command="lab"' \
  -c 'model_providers.labrats.auth.args=["ai","token"]'

4. Qwen Code CLI (chat-completions)

qwen uses chat-completions. Add an openai provider in ~/.qwen/settings.json:

{
  "security": { "auth": { "selectedType": "openai" } },
  "model": { "name": "<model-id>" },
  "modelProviders": {
    "openai": [{
      "id": "<model-id>",
      "name": "labrats ai-agents proxy",
      "envKey": "AI_AGENTS_API_KEY",
      "baseUrl": "https://chat.ai-agents-dev.bor.labrats.work/v1"
    }]
  }
}

qwen-code reads its key once at startup, so launch it through lab ai exec, which puts a fresh token in AI_AGENTS_API_KEY (and OPENAI_API_KEY) for that process only:

lab ai exec -- qwen -p "list the files here and summarize"

A session lasts one token lifetime; restart it after that.

dsh reads AI_AGENTS_API_KEY from $DSH_HOME/.credentials.yaml, which lab ai keeps current — see the dsh section of standards ai-tools.md.

5. Verify (one-shot tool call)

codex exec --dangerously-bypass-approvals-and-sandbox \
  "run 'echo it-works' with your shell tool and report the output"

You should see the command execute and it-works reported back — that confirms the model is reaching its backend and tool calling works.

6. Chat apps (SillyTavern, Chatbox, Open WebUI)

Any app with an OpenAI-compatible (chat completions) provider works. Set:

These apps store the key once and can't run lab ai token themselves, so paste a fresh token when calls start failing with 401 — a token lasts 8 hours (spec 0078 d2). Never paste a static API key instead.

Troubleshooting

Notes

• This targets the dev proxy (ai-agents-dev). A prod endpoint requires a separate promotion.
• Tool calling is normalized by the proxy (each backend's call format is surfaced as standard OpenAI/Responses tool_calls), so standard OpenAI-compatible clients work.