FreeLLMAPI

One OpenAI-compatible endpoint. Every free LLM provider. Real-time dashboard.

Aggregate the free tiers from Google, Groq, Cerebras, NVIDIA, Mistral, OpenRouter, GitHub Models, Cohere, Cloudflare, HuggingFace, Z.ai, Ollama, Kilo, Pollinations, LLM7, OVH AI Endpoints, plus any custom OpenAI-compatible endpoint — behind a single /v1/chat/completions drop-in. A smart router picks the best available key for each request, fails over transparently when a provider rate-limits you, queues requests instead of dropping them, routes outbound traffic through an HTTP proxy pool of your choice, and tracks everything in a live WebSocket dashboard.

---

What's new in v1.3

• 🛰️ HTTP proxy pool — pool many proxies, choose a strategy (round-robin, least-latency, random, or single), per-key assignment that rotates hourly, dashboard with stats, geolocation, health-checker, bulk select / enable / disable / test / delete, and a France-only find-proxies scraper. A single proxy URL set in Settings is also tracked in the dashboard.
• 📊 Live dashboard v2 — in-flight requests now show the proxy in use, input tokens and a growing output token counter that ticks up as the stream comes back. New session totals (Tokens IN / Tokens OUT).
• 🔑 Keys page v2 — model discovery now syncs the catalog: new models from the provider are added, stale ones are removed. Bulk auto-discover for custom endpoints, search bar, inline label/RPM/TPM editing.
• 🌐 26 providers — Groq, Cerebras, Mistral, OpenRouter, NVIDIA NIM, GitHub Models, Cohere, Cloudflare, Google, Z.ai, HuggingFace, Ollama Cloud, Kilo Gateway, Pollinations, LLM7, OVH AI Endpoints, Amazon Bedrock (EU, paid — your own AWS capacity), Anthropic (native format), Claude accounts (sign in with your Claude account), DeepSeek (paid — your own account), ModelScope (free — your own account), Alibaba Cloud Model Studio (free Token Plan — your own account), Google Antigravity (sign in with your Google account), OpenAI Codex (legacy ChatGPT sign-in), OpenAI ChatGPT Plan (official “Sign in with ChatGPT”), plus any custom OpenAI-compatible URL.
• 🛠 Operational fixes — fixed fd-leak crash on test-all, dispatcher cache, SOCKS test endpoint, model-stats scoping per user, pool-strategy persistence across restarts.
• 🧪 728 server tests passing in ~27s. Forgejo Actions workflow on every push and every v* tag.

---

Contents

• What's new in v1.3
• Features
• Supported providers
• Quick start
• Install script
• Docker
• Node.js + PM2
• Migrating from an older install
• Using the API
• HTTP proxy pool
• Dashboard
• Configuration
• How routing works
• Project structure
• Continuous integration
• Contributing
• Disclaimer

---

Features

Gateway

• Single OpenAI-compatible endpoint — /v1/chat/completions, /v1/models. Any OpenAI client library works unchanged.
• Smart failover router — tries provider keys in scored order (success rate, latency, rate-limit headroom). Falls over to the next key/provider silently.
• Request queueing — when all keys for a model are rate-limited, requests are held and retried instead of immediately returning 429. Configurable timeout.
• Per-user isolation — each user has their own provider keys, gateway keys, proxies, and request history. Zero cross-tenant data leakage.
• Gateway API keys — mint scoped keys (freellmapi-…) for your apps with per-key RPM/TPD limits. Enable/disable without deleting.
• Outbound rate limits — set per-provider-key RPM/TPM caps enforced between the router and the upstream API.
• Context handoff — injects a compact system message when a session switches model mid-stream so the new model knows where things left off.
• HTTP proxy pool — distribute outbound calls across many HTTP proxies (round-robin, least-latency, random), or pin to a single proxy. Per-key hourly assignment.
• AES-256-GCM encryption — all provider API keys are encrypted at rest. The plaintext never leaves your server.

Dashboard

• Live WebSocket dashboard — in-flight request counter, per-request routing events (which provider/model/proxy is being tried), per-key RPM gauges, overview counters — all pushed in real time without polling.
• In-flight proxy + tokens — active rows show the proxy name, input tokens, and a growing output token counter that ticks up live as the model streams.
• Session token totals — Tokens IN and Tokens OUT aggregated over the dashboard session.
• Per-user live counts — counts and events are scoped to the logged-in user.
• Keys page — masked key display, inline label/RPM/TPM editing, health status, one-click model discovery, custom endpoint with bulk auto-discover.
• Per-key deep stats — click any key to expand: latency percentiles (p50/p95/p99), 24-hour hourly bar chart, per-model breakdown, error breakdown, recent 25 requests, active cooldowns.
• Live status dots — each key's indicator updates in real time when the server detects a 401/403/429. No manual refresh needed.
• Playground — interactive chat with model selector, system prompt editor, and a live routing indicator showing which provider/model/proxy is handling the current request.
• Proxy pool page — bulk select, enable / disable / delete / test, per-proxy stats panel, geolocation, health checker.
• Analytics — request volume, error rates, cost estimates, per-provider breakdown, exportable history.
• One-click update with live progress — when the server is behind origin, an admin gets an Update now button. Clicking it runs the whole pipeline on the server (git pull → npm install → build → restart) and the banner shows it as a progress bar: each step is ticked off as it finishes, the running one spins, and a failed step is marked in red with the compiler output beside it. The install and build stages can take minutes, so the bar says which one you are in instead of showing a frozen "Updating…".

Operations

• Structured JSON logger — every request logs timestamp, level, req_id, user, key prefix, proxy, provider, and model. Secrets masked. Daily rotation with configurable retention.
• Universal installer — install.sh auto-detects Docker vs Node.js+PM2, generates keys, writes .env, builds, and starts the service. Prompts to migrate from an older install.
• Migration script — migrate-from-old.sh finds an older FreeLLMAPI install, verifies the encryption key, and imports provider keys and request history non-destructively.
• find-proxies script — scans a list of candidate free-proxy URLs, geo-IP-verifies the country, tests connectivity, and writes the working set to working-proxies.txt for one-click pool import.
• Docker — multi-stage Dockerfile, docker-compose.yml, named volume for the database.
• PM2 — ecosystem.config.cjs with autorestart, memory cap, and structured log rotation.
• Kubernetes / Podman — k8s/ manifests for podman play kube or kubectl apply.
• Forgejo Actions — .gitea/workflows/ci.yml and release.yml run install + build + test on every push and every v* tag.

---

Supported providers

Provider	Models	Auth
Google Gemini	Gemini 2.5 Flash, 2.0 Flash, 2.5 Pro preview	API key
Groq	Llama 3.3/4, Qwen3, Gemma, compound-beta	API key
Cerebras	Qwen3 235B, Llama 3.3 70B	API key
Mistral	Large 3, Medium 3.5, Codestral, Devstral	API key
OpenRouter	20+ free-tier models via :free routes	API key
GitHub Models	GPT-4.1, GPT-4o, Phi-4, Llama 3.3	GitHub PAT
Cloudflare Workers AI	Kimi K2, GLM-4.7, Llama, Granite	Account ID + token
Cohere	Command R+, Command-A (trial key)	API key
NVIDIA NIM	40 RPM free tier (eval-only ToS)	API key
HuggingFace	Inference router → DeepSeek, Kimi, Qwen3	API key
Z.ai (Zhipu)	GLM-4.5, GLM-4.7 Flash	API key
Ollama Cloud	GLM-4.7, Kimi K2, Qwen3	API key
Kilo Gateway	:free routes, no key required	Keyless
Pollinations	GPT-OSS 20B, no key required	Keyless
LLM7	GPT-OSS, Llama 3.1, GLM, no key required	Keyless
OVH AI Endpoints	Qwen3.5 397B, Llama 3.3, no key required	Keyless
ChatJimmy (Taalas HC1)	Llama 3.1 8B hard-wired into silicon — ~15–19k tokens/sec, no key required, 6,144-token context, ~19 req/min per IP	Keyless
Amazon Bedrock (EU)	Your AWS account's Bedrock models, EU regions only — paid, not a free tier	Bedrock API key (bearer)
Anthropic	Claude Sonnet 4, Opus 4, Haiku 3.5, 3.5 Sonnet, 3 Opus	API key
Claude (account)	The Claude models your Pro/Max/Ultra subscription gets — discovered from your account, metered against plan limits	Sign in with Claude
DeepSeek	DeepSeek V4.1 Flash, V4 Pro (from your account's /models) — paid, prepaid balance	API key
ModelScope	Qwen, DeepSeek, GLM and more from your ModelScope account's API-Inference — ~2,000 requests/day free quota	API token
Alibaba Cloud Model Studio	Qwen family plus third-party models from your Model Studio Token Plan account (Singapore, ap-southeast-1) — discovered from your account	API key
Google Antigravity	The Gemini (and, per your plan, Claude) models your Google account gets in Antigravity — discovered from your account	Sign in with Google
OpenAI Codex	The models your ChatGPT plan gets in Codex — discovered from your account (legacy Codex-client flow)	Sign in with ChatGPT
OpenAI ChatGPT Plan	The models your ChatGPT plan gets through OpenAI's official “Sign in with ChatGPT” route — discovered from your account	Continue with ChatGPT (official)
Custom endpoint	Any OpenAI-compatible URL — llama.cpp, LM Studio, vLLM, Ollama, Novita, etc.	API key (or keyless)

Keyless providers don't require an API key. The Keys page stores a sentinel row for them so routing treats the platform as configured.

ChatJimmy is Taalas' public HC1 demo: Llama 3.1 8B etched into custom silicon, answering at roughly 15–19k tokens/sec. It needs no account, no key and no cookie — every call is stateless — but it has two measured limits: 6,144 tokens of context per request (a longer prompt returns an empty 200, which the gateway reports as a model error and fails over) and 19 requests per minute per egress IP (the 20th returns 429 + Retry-After: 60). No daily token quota was observed: over 100k tokens went through one IP in a single session.

It also has no tool-calling surface: the adapter flattens tool results into text, but the model cannot emit structured tool_calls. Agentic clients (omp, Claude Code, Cline…) send a system prompt plus tool schemas that alone exceed 6,144 tokens — pinning ChatJimmy for them returns 422 with context too small, and even a small enough prompt would fail their tool loop. Use ChatJimmy for short, non-agentic prompts; pin a large-context model for coding agents. Routing by auto is safe: the context gate skips ChatJimmy for oversized prompts and falls through to a model that fits.

Since the only identity ChatJimmy can meter is the outbound IP, the provider is registered multiKeyless: add one ChatJimmy row per proxy on the Keys page. Each row keeps its own proxy binding, the router rotates keys least-recently-used on every request, and a chatjimmy: 19 provider RPM cap moves to the next row before the upstream limiter trips — so N rows on N proxies give roughly 19N requests/minute. Rows that share a proxy also share that IP's budget, so never add more rows than you have distinct proxies. Without a proxy pool, every row leaves from the server's own IP and the 20/min limit is not multiplied. Discovery lists its single model as chatjimmy/llama3.1-8B.

Cloudflare Workers AI imports exactly what the Workers AI model-search API returns: the Cloudflare-hosted @cf/... models (39 text-generation + 8 audio when this was written). Cloudflare's dashboard also shows a unified catalog of third-party models — minimax/m3.1-flash, anthropic/claude-…, openai/gpt-… — but those are AI Gateway models, not Workers AI: the search endpoint never returns them, and they are billed per token from your AI Gateway balance (or require your own provider key, BYOK). Without credits, the very same /ai/v1/chat/completions endpoint answers 402 Insufficient balance; add money to your gateway or use BYOK. They are therefore not imported on purpose: listing them would only add paid, unusable entries to a free-tier catalogue. Fetch all models is not stale — every model the Workers AI API lists is already in the catalogue.

Amazon Bedrock (EU) is a paid provider — it exists so you can route your own AWS capacity through the same endpoint, and it is off unless you add a key. It uses the AWS-recommended bedrock-runtime OpenAI-compatible endpoint with an Amazon Bedrock API key as a plain bearer token — no SigV4 signing and no AWS SDK. Pick the region with BEDROCK_REGION (EU allowlist, default eu-central-1); a non-EU value is refused so inference never silently leaves the EU. Because Bedrock model access is granted per AWS account, no models ship in the catalog: add the key, then Keys → Fetch all models imports exactly what your account is entitled to.

DeepSeek is also paid (prepaid balance — an empty balance answers 402 Insufficient Balance, which fails the request over to your other providers and benches that key for a day). Create a key at platform.deepseek.com and add it on the Keys page; discovery runs automatically and imports the models your account lists (deepseek-flash, deepseek-v4-pro) with their context window, output ceiling, vision and tool-calling support.

ModelScope is free (registered-user API-Inference: ~2,000 requests/day shared across all models, ≤500/model/day, no credit card — requires an Alibaba Cloud account binding and real-name verification). Create a token at modelscope.cn/my/myaccesstoken and add it on the Keys page; discovery imports the API-Inference-enabled models your account can call (Qwen3.5 family, DeepSeek, GLM…) with their context window and output ceiling.

Alibaba Cloud Model Studio is Alibaba's first-party Model Studio (DashScope) API — distinct from ModelScope above. This integration targets the Singapore (ap-southeast-1) Token Plan endpoint (https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1, the only region the Token Plan exists in, billed in plan Credits), which is OpenAI-compatible and accepts a Bearer API key. Create a key in the Model Studio console and add it on the Keys page. No models ship in the catalog: adding the key runs discovery against /models, which imports the ~20 models the Token Plan covers (prefixed alibaba/), with their context window and tool/vision metadata.

Google Antigravity uses no API key: pick it on the Keys page and click Sign in with Google. This runs the same OAuth flow as the Antigravity IDE/CLI, and Google always returns to http://localhost:51121/oauth-callback (the only address it allows for Antigravity's client). When the browser runs on the same machine as FreeLLMAPI, that page is served by FreeLLMAPI and sign-in finishes by itself. When FreeLLMAPI runs elsewhere (e.g. you open http://172.17.1.2:3001), that page fails to load — copy its full address from the browser's address bar and paste it into the box under the Sign in button; the server finishes the sign-in. No public HTTPS is needed either way. The account is stored encrypted like any key, gets a proxy from your pool, is health-checked, rate-limited and routed exactly like other providers, and its models are discovered automatically (addressed as antigravity/<model>). Each model family is listed once — antigravity/claude-opus-4-6, not a -thinking or (Low)/(High) variant — and the client's thinking level (reasoning_effort: none, low, medium, high) picks the right Antigravity model and thinking setting: a thinking level for Gemini, a thinking budget for Claude and GPT-OSS. The model list refreshes itself shortly after startup and every 6 hours, so models Google adds, renames or retires show up without pressing Fetch all models. Signing the same account in again repairs it if Google revoked the grant; add more Google accounts to rotate between them.

OpenAI ChatGPT Plan (the chatgpt platform) is OpenAI's official, sanctioned way to use a ChatGPT subscription from a third-party client — the “Sign in with ChatGPT” flow documented for open-source and locally hosted apps. There is no separate “Codex Pro” product: Codex is included in ChatGPT Free, Go, Plus, Pro, Business, Edu and Enterprise, and the plan is what decides which models and how much usage your account gets. Pick ChatGPT plan (official sign-in) on the Keys page and click Continue with ChatGPT; OpenAI shows a consent screen, then redirects to http://127.0.0.1:1456/auth/callback. When the browser runs on the same machine as FreeLLMAPI that page is served by FreeLLMAPI and sign-in finishes by itself; when FreeLLMAPI runs elsewhere, copy the full address of the page OpenAI opened (it starts with http://127.0.0.1:1456/auth/callback?state=…) and paste it into the box under the button. The request needs exactly two things from you: signing in with the account that owns the plan, and leaving the port free during sign-in.

• It uses your existing plan, not a separate allowance. Requests are billed against the limits you already have in ChatGPT, so a plan limit hit comes back as the same rate-limit error and routing fails over to your other providers. There is no separate API bill and no pay-as-you-go key. You keep control in ChatGPT: Settings → Usage shows usage and lets you set an app limit, and Settings → Connected apps disconnects this app at any time.
• Official route, no impersonation. FreeLLMAPI calls the public Responses API (https://api.openai.com/v1/responses) with store:false and stream:true, exactly as the SIWC documentation requires — never ChatGPT's internal backend-api. No Originator, client-version or account-id headers are sent; the OAuth bearer token is the identity. The issued client id (oaiapp_…) is stored and reused, and the ID token is verified against OpenAI's published JWKS (issuer, audience, expiry and nonce).
• Models come from your account. Discovery lists what your plan can actually use (GET https://api.openai.com/v1/models, filtered to entries marked visibility: "list") and imports them as chatgpt/<model>, with the account's context window and image support where published.
• Some request fields do not exist on this route. temperature, top_p, max_output_tokens, metadata, previous_response_id, truncation and user are dropped instead of being sent and refused. reasoning_effort works (minimal/low/medium/high; xhigh maps to high, none means no reasoning), as do function tools and tool results. Image generation, file search, Code Interpreter, hosted MCP and tool_search are not available.
• Signed-in accounts behave like other keys. The credential is encrypted at rest, bound to a proxy from your pool like Antigravity and Codex (it never moves egress IP on its own), and health-checked by listing models. Quota is not polled — the plan has no documented quota endpoint, so a limit hit is learned from the upstream 429.
• Backups carry the account. The CSV export contains the credential, so re-importing the file restores the account without another sign-in. One ChatGPT account is kept as one row; signing the same account in again repairs a revoked or expired grant instead of duplicating it.
• Legacy Codex rows still work. The older codex platform (the Codex-CLI–compatible sign-in on port 1455) is kept untouched for setups that already use it. Prefer ChatGPT plan (official sign-in): it is the route OpenAI sanctions for applications like this one.

Claude (account) (the claude platform) signs a Claude subscription (Pro/Max/Ultra) in with the Claude Code OAuth flow and routes requests against that account's plan limits — distinct from the API-key anthropic platform, which bills per token. Pick Claude (Claude account) on the Keys page and click Sign in with Claude; Anthropic redirects to http://localhost:54545/callback. When the browser runs on the same machine as FreeLLMAPI that page is served by FreeLLMAPI and sign-in finishes by itself; when FreeLLMAPI runs elsewhere, copy the address and paste it into the box under the Sign in button.

• One account, one identity, one IP. Each account keeps the single egress proxy it signed in through and is exempt from the pool's rotation/rebinding — Anthropic meters OAuth traffic per account, and a wandering IP is the classic account-farming signal. Add several accounts to add capacity: routing drains one account (keeps using it while it works) and only moves to the next when the current one rate-limits or errors, then loops back. This also preserves Anthropic's prompt cache per account instead of re-billing the whole context on every switch.
• Per-account priority. Each Claude row carries a routing priority (edit it on the row, Priority (lowest first)): the lowest value is served first, and a higher-priority account is used whenever it can serve — lower ones only take over while it is rate-limited or spent, then it comes back automatically when its window resets. Set your own account to 1, a lent account to 2, another to 3. The default (100) keeps the plain drain order, and every other provider is unaffected.
• The wire looks like Claude Code. Anthropic classifies OAuth requests by how closely they match genuine Claude Code traffic; requests that don't match are metered against the account's small "extra usage" bucket instead of the plan limits. This provider sends the Claude Code header set, billing header with body attestation, _-prefixed custom tools and the JSON metadata identity, and adopts the minimum version Anthropic names when it rejects a stale one.
• Spent accounts rotate automatically. An exhausted account answers 400 "out of extra usage"; the gateway reads that as the account being done, benches it for the window and moves to the next account.
• Quota reads like Codex and Antigravity. The Keys page shows the account's plan meters — Session (5h), Weekly (7d) and the Fable weekly meter — as mini bars, each hidden until Anthropic actually reports it (a Pro account without a Fable bucket shows no Fable row; a freshly signed-in account shows nothing until observed, never an error). Two sources feed them at zero extra inference cost: every real /v1/messages answer carries the account's unified-limit headers (harvested on the turn), and when nothing fresh was observed recently the gateway reads GET /api/oauth/usage — an undocumented endpoint rate-limited per account and shared with Claude Code's own polling, so a 429 there just serves the last known numbers. A spent-account answer (400 "out of extra usage") flips the weekly pool to drained at once — same recording pattern as the Codex rememberQuotaExhausted wall — and keeps the refill time already known. Snapshots are per-account in memory (keyed by refresh token, like Codex's quota-header cache) on top of the route's persisted key_quota_snapshots refresh throttle (one provider read per key per minute, shared by all tabs).
• Honest caveat. Anthropic documents subscription OAuth as intended for its own apps; routing it through a gateway is a personal-use choice Anthropic could enforce against. Grants also expire ~30 days after the interactive sign-in — the dashboard then asks for a new sign-in rather than silently failing.

---

Quick start

git clone https://git.pandem.fr/outage.sh/FreeLLMapi freellmapi
cd freellmapi

# Recommended — universal installer
bash install.sh

# Or manually
cp .env.example .env
# Edit .env — set ENCRYPTION_KEY, ADMIN_USERNAME, ADMIN_PASSWORD
npm install && npm run build
node server/dist/index.js

Open http://localhost:3001 and log in with your admin credentials.

---

Install script

bash install.sh             # auto-detects Docker vs Node.js+PM2
bash install.sh --docker    # force Docker Compose
bash install.sh --node      # force Node.js + PM2
bash install.sh --port 8080 # custom port
bash install.sh --lan       # bind to 0.0.0.0 (LAN access)
bash install.sh --yes       # non-interactive / CI

The script detects Docker or Node.js, generates ENCRYPTION_KEY, prompts for admin credentials, writes .env, builds, starts the service, waits for the health check, and prints the access URL and management commands. After .env is written it asks if you want to migrate keys from an older install.

---

Docker

cp .env.example .env
# Set ENCRYPTION_KEY, ADMIN_USERNAME, ADMIN_PASSWORD in .env

docker compose up -d

# Verify
curl http://localhost:3001/api/ping

# Logs
docker compose logs -f

# Update
git pull && docker compose build && docker compose up -d

The database is stored in a named Docker volume (freellmapi-data) and survives container restarts and image rebuilds.

LAN access: set HOST_BIND=0.0.0.0 in .env to expose the port to your local network.

---

Node.js + PM2

Requires Node.js 20+.

cp .env.example .env   # edit before proceeding

npm install
npm run build

pm2 start ecosystem.config.cjs
pm2 save
pm2 startup            # enable autostart on reboot

# Logs
pm2 logs api-gateway

# Update
git pull && npm run build && pm2 restart api-gateway

The database lives at server/data/freeapi.db. Back it up before updates.

---

Migrating from an older install

If you ran a previous version of FreeLLMAPI and want to carry your provider keys forward:

# Auto-search the machine for an old install
bash migrate-from-old.sh

# Or point directly at the old repo directory
bash migrate-from-old.sh /path/to/old/freellmapi

The script:

1. Locates the old freeapi.db and reads its ENCRYPTION_KEY
2. Verifies decryption works before touching anything
3. Handles ENCRYPTION_KEY mismatches — adopt the old key (recommended if the new DB is empty) or re-encrypt everything on the fly with the new key
4. Imports all api_keys rows, skipping duplicates. Optionally imports request history.
5. Never deletes from either database — fully non-destructive

After running, restart the server and go to Keys → Check all to confirm the imported keys are still valid with each provider.

---

Using the API

FreeLLMAPI speaks standard OpenAI API. Point any client at your server:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-gateway-key-here",
)

response = client.chat.completions.create(
    model="auto",   # router picks best available, or name a specific model
    messages=[{"role": "user", "content": "Hello!"}],
)
print(response.choices[0].message.content)

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-gateway-key-here" \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","messages":[{"role":"user","content":"Hello!"}]}'

Listing models

curl http://localhost:3001/v1/models \
  -H "Authorization: Bearer freellmapi-your-gateway-key-here"

Returns only models the gateway can actually route. A model is listed when it is switched on on the Models tab and you hold an enabled key for its provider. A catalogue entry is a promise: discovery clients list whatever they are handed, so advertising a model you turned off — or one with no key behind it — just puts a choice in your model picker that can only ever fail.

Add ?available=0 to see the full catalogue instead, each entry tagged available with an unavailable_reason of disabled (switched off on the Models tab) or no_key (no enabled key for that provider) — useful for checking what adding a given provider key would unlock.

Use auto to let the router choose, or request a specific model by name. Ids are provider-qualified (groq/openai-gpt-oss-120b), so the same model served by several providers stays distinguishable; the bare name also resolves.

Model prices

Every model carries a price — free routes included — and it is the paid equivalent: what those tokens would cost on a paid API. A free route's own list price is $0, which is true and tells you nothing; the number worth seeing on a free-tier gateway is what you didn't pay.

{
  "id": "openrouter/inclusionai/ling-3.0-flash-vl:free",
  "pricing": {
    "prompt": "0.00000006",     // per token, OpenRouter's wire convention
    "completion": "0.00000018",
    "input_per_million": 0.06,
    "output_per_million": 0.18,
    "source": "openrouter",     // openrouter | models.dev | litellm | fallback
    "estimated": false          // true = no feed knew it; flat fallback rate
  },
  "list_price": { "input_per_million": 0, "output_per_million": 0 }  // what this route charges
}

Prices are resolved per model from OpenRouter, models.dev and LiteLLM, so a provider that publishes no prices of its own (NVIDIA NIM, Groq, Cerebras…) still gets one from whichever feed knows the model. Ids are matched across naming conventions — @cf/ prefixes, model:120b size tags, _ separators, :free / -free suffixes, qwen-3 vs qwen3 — because the benchmark only exists under the paid spelling. A model no feed has heard of (stealth releases, first-party routes like groq/compound) gets the flat fallback rate with estimated: true, so a client can say "about $x" rather than quote a number it cannot stand behind.

Per-request cost

Every completion reports what it cost — chat (streaming and non-streaming) — in the usage.cost field OpenAI-compatible clients read as the authoritative charge:

"usage": { "prompt_tokens": 120000, "completion_tokens": 30000, "cost": 0.066 }

This is what lets a harness show a running total for traffic routed through the gateway. Coding agents (omp, Cline, …) park models discovered from a gateway at zero catalog cost, precisely because a gateway's prices are its own — so usage.cost is not just the preferred source, it is the only one.

It is the same number the Analytics tab's savings figure is built from: one resolver prices the request, the response reports it, and the ledger stores it. The API and the dashboard cannot quote different figures.

Showing prices in a coding harness

usage.cost drives a harness's session cost total. Its model picker is a separate matter: harnesses refuse to price models discovered from a gateway using that gateway's plain /v1/models. OMP says so outright — "discovered proxy and gateway models … discovery keeps them at a local-unknown zero cost" — and that is a fair default, because an arbitrary OpenAI-compatible endpoint has no agreed way to state a price, so trusting one would be guesswork. The effect is that every model reads free in the picker no matter what /v1/models says.

LiteLLM's management endpoints are that agreed way, and the gateway serves them. You do not need a second provider entry or a second copy of your key — add discovery.type: litellm to the provider you already have:

providers:
  my-freellmapi:              # whatever you already called it
    baseUrl: http://172.17.1.2:3001/v1
    apiKey: FREELLMAPI_KEY    # the key you already configured
    api: openai-completions
    discovery:
      type: litellm           # ← the only line you add

Discovery strips the trailing /v1 and probes /model_group/info, /v2/model/info, /model/info and /v1/model/info at the host root; runtime calls keep using your /v1 base URL unchanged. All four are served, at the root and under /v1, so either convention works. The client parses input_cost_per_token / output_cost_per_token, max_input_tokens, max_output_tokens, supports_vision, supports_reasoning and supported_openai_params:

{
  "model_group": "openrouter/inclusionai/ling-3.0-flash-vl:free",
  "input_cost_per_token": 6e-8,      // → $0.06/M in the picker
  "output_cost_per_token": 1.8e-7,   // → $0.18/M
  "max_input_tokens": 262144,
  "max_output_tokens": 128000,
  "supports_reasoning": true,
  "mode": "chat"
}

Nothing is pinned locally, so a refresh picks up new models, rotated rosters and re-priced routes on its own. Model ids are the same provider-qualified ones /v1/chat/completions accepts, so a model chosen from the picker routes without translation.

If your harness has no LiteLLM discovery path, there is a static fallback — a generated models.yml fragment. It needs re-running whenever the catalogue changes, which is why it is the second choice:

curl -H "Authorization: Bearer freellmapi-your-gateway-key-here" \
  "http://localhost:3001/v1/models/omp-overrides.yaml?provider=freellmapi" \
  > ~/.config/omp/models.yml

---

OpenAI models

The default setup works out of the box — just add an OpenAI-compatible API key on the Keys page and you're ready:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-gateway-key-here" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4o","messages":[{"role":"user","content":"Hello!"}]}'

---

Anthropic models

Anthropic models use the same gateway key — the router transparently translates your OpenAI-format request to Anthropic's native API format. No extra setup needed beyond adding an Anthropic API key on the Keys page:

response = client.chat.completions.create(
    model="claude-sonnet-4",
    messages=[{"role": "user", "content": "Hello!"}],
)

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-gateway-key-here" \
  -H "Content-Type: application/json" \
  -d '{"model":"claude-sonnet-4","messages":[{"role":"user","content":"Hello!"}]}'

Anthropic also supports native tool calling, thinking/reasoning, and streaming through the same gateway key — the router handles all protocol translation automatically.

DeepSeek models

Add a DeepSeek API key on the Keys page — its models are discovered automatically and addressed as deepseek/<model>:

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-gateway-key-here" \
  -H "Content-Type: application/json" \
  -d '{"model":"deepseek/deepseek-flash","reasoning_effort":"high","messages":[{"role":"user","content":"Hello!"}]}'

DeepSeek models think by default. reasoning_effort: "none" switches thinking off (thinking.type: "disabled"); low/high/xhigh map onto DeepSeek's low/high/max. Tool calling and streaming work as usual — the gateway replays each turn's reasoning_content, which DeepSeek requires whenever tools are present.

Thinking / reasoning effort

Pass reasoning_effort to control how much a reasoning model thinks. Accepted values: none | minimal | low | medium | high | xhigh.

curl http://localhost:3001/v1/chat/completions \
  -H "Authorization: Bearer freellmapi-your-gateway-key-here" \
  -H "Content-Type: application/json" \
  -d '{"model":"nvidia/z-ai/glm-5.2","reasoning_effort":"none","messages":[{"role":"user","content":"2+2"}]}'

The gateway translates the level into whatever shape each upstream actually honors, so you never send provider-specific fields yourself: Gemini gets thinkingConfig.thinkingBudget, Nemotron 3.5 gets chat_template_kwargs.enable_thinking + thinking_token_budget, Zhipu gets thinking.type, DeepSeek gets thinking.type + its low|high|max effort, Mistral gets its 2-value none|high enum. /v1/models advertises compat.supportsReasoningEffort per model so a client can show the control only where it works.

GLM-5.x caveat (genuine model limitation, not a gateway one). GLM's chat template resolves the effort to max unless the value is literally high — the template logic is 'high' if reasoning_effort == 'high' else 'max'. So the gateway clamps low/medium/high all onto high, and xhigh onto max. low and medium therefore think the same amount as high — the graduation between them cannot be expressed on GLM. If you want GLM to think less, use none or minimal, which turn thinking off outright via enable_thinking: false. Anything else reasons at high or above.

Image generation

POST /v1/images/generations is OpenAI-compatible. The first run after upgrading seeds a Pollinations FLUX (keyless) model on the imagegen platform that uses Pollinations.ai's anonymous legacy image endpoint — no card, no key, no rate limit to speak of. The row appears in Models / Fallback Chain with a rose Image badge and an auto-seeded sentinel key, so image generation works out of the box with zero configuration.

curl -X POST http://localhost:3001/v1/images/generations \
  -H "Authorization: Bearer freellmapi-your-gateway-key-here" \
  -H "Content-Type: application/json" \
  -d '{"prompt": "a red apple on a white background, studio lighting", "n": 1, "size": "1024x1024"}'

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-gateway-key-here",
)

img = client.images.generate(
    model="imagegen/pollinations-flux",
    prompt="a red apple on a white background, studio lighting",
    n=1,
    size="1024x1024",
)
# img.data[0].b64_json is a base64-encoded image

Request fields:

Field	Type	Default	Notes
prompt	string (1-4000)	required	The text prompt.
model	string	auto	The image-gen model id, e.g. imagegen/pollinations-flux.
n	integer (1-4)	1	Number of images. The Pollinations endpoint serves one per request, so n>1 issues sequential calls.
size	"WxH"	1024x1024	Width x height in pixels, both clamped to 64-2048.
response_format	url | b64_json	b64_json	b64_json is the only format actually served (the Pollinations endpoint returns raw image bytes); url is accepted for OpenAI client compatibility.

Response shape (200 OK):

{
  "created": 1753819200,
  "data": [
    { "b64_json": "iVBORw0KGgo...", "revised_prompt": "a red apple on a white background, studio lighting" }
  ]
}

Errors:

• 401 invalid_api_key - the bearer token is missing or wrong.
• 422 no_image_gen_model - no image-gen model is enabled in the Fallback Chain. Enable imagegen/pollinations-flux (or another image-gen row) and retry.
• 422 invalid_request - the body is malformed or prompt is missing.
• 502 image_generation_failed - the upstream returned a 5xx; the message includes the upstream status text.

Add another image-gen backend: the imagegen platform is a normal provider slot. To add e.g. a Hugging Face-backed image-gen model, register a second provider on the same /v1/images/generations route, seed rows with supports_image_generation = 1 via a follow-up migration, and the route will pick whichever row is at the top of the fallback chain.

Audio: text-to-speech, speech-to-text, speech-to-speech

Audio models live in the same catalogue as chat models. Fetch all models on a provider key catalogues its audio models next to its chat models, adds them to the Fallback Chain and tags them, and the router picks a model and key for each request exactly as it does for chat (key rotation, cooldowns, proxies, failover, queueing).

Tag	Meaning	Endpoint
TTS (emerald)	text → speech	POST /v1/audio/speech
STT (amber)	speech → text	POST /v1/audio/transcriptions
STS (rose)	speech in → spoken reply out	POST /v1/audio/speech-to-speech

The tags appear as chips in Models / Fallback Chain; type tts, stt or sts in its search box to list only those models. GET /v1/models reports them too: supports_tts / supports_stt / supports_sts flags, type (chat, image, tts, stt or sts — a chat model that also speaks or listens stays chat), and endpoints, the exact gateway endpoints that serve the model. Dedicated audio models (Whisper, Orpheus, …) carry supports_chat: false and are never routed for /v1/chat/completions; multimodal chat models with audio input/output (Gemini, gpt-4o-audio, Voxtral Small, …) keep serving chat and are reached through the provider's chat surface for audio.

model takes platform/model_id, a bare model_id, or auto. An id that no enabled model of that surface answers to — e.g. the OpenAI SDK defaults tts-1 / whisper-1 — routes as auto. The X-FreeLLMAPI-Model response header names the model that served the request.

# Text → speech
curl http://localhost:3001/v1/audio/speech \
  -H "Authorization: Bearer freellmapi-your-gateway-key-here" \
  -H "Content-Type: application/json" \
  -d '{"model":"auto","input":"Hello from FreeLLMAPI","voice":"alloy","response_format":"mp3"}' \
  -o hello.mp3

# Speech → text
curl http://localhost:3001/v1/audio/transcriptions \
  -H "Authorization: Bearer freellmapi-your-gateway-key-here" \
  -F file=@hello.mp3 -F model=groq/whisper-large-v3-turbo

# Speech → speech (the model hears the clip and answers aloud)
curl http://localhost:3001/v1/audio/speech-to-speech \
  -H "Authorization: Bearer freellmapi-your-gateway-key-here" \
  -F file=@question.wav -F voice=alloy -o answer.wav

from openai import OpenAI

client = OpenAI(base_url="http://localhost:3001/v1", api_key="freellmapi-your-gateway-key-here")
client.audio.speech.create(model="auto", voice="alloy", input="Hello").write_to_file("hello.mp3")
print(client.audio.transcriptions.create(model="auto", file=open("hello.mp3", "rb")).text)

Request fields

Endpoint	Field	Notes
/v1/audio/speech (JSON)	input	required, 1–4096 chars
	voice	provider voice name; omitted → the provider's default voice
	response_format	mp3 (default), opus, aac, flac, wav, pcm — providers return the closest format they can produce, and Content-Type always describes the real bytes (Gemini, for example, answers WAV)
	speed, instructions	passed through where the provider supports them
/v1/audio/transcriptions (multipart, ≤200 MB, AUDIO_MAX_UPLOAD_MB)	file	required; any length — see Long recordings below
	language, prompt, temperature	hints, passed through where supported
	response_format	json (default, {"text": …}), text, verbose_json
/v1/audio/speech-to-speech (multipart, ≤25 MB)	file	required
	voice, instructions, response_format (wav default)	the reply's transcript, when the provider reports one, is in the percent-encoded X-Audio-Transcript header

Errors: 401 invalid_api_key; 400 invalid_request (malformed body, missing file); 422 no_tts_model / no_stt_model / no_sts_model (nothing of that surface is enabled — run Fetch all models on a key whose provider offers it); 429 when every candidate is rate-limited past the queue timeout; otherwise the last upstream status with code tts_failed / stt_failed / sts_failed.

Long recordings. A transcription upload that is over 24 MB or longer than ~10 minutes is split on the gateway: ffmpeg re-encodes it to 16 kHz mono FLAC (lossless, and the rate Whisper listens at anyway) and cuts it every ~10 minutes (LONG_AUDIO_CHUNK_SECONDS), moving each cut into the nearest pause so no word is cut in half. The pieces are transcribed in order, each given the end of the previous text as context, and returned as one text; the X-Audio-Pieces header says how many there were. If a piece is rate-limited or refused, the request continues from that piece on the next key/model instead of starting over. It is still one request on Live and in Analytics, priced on the full duration. This needs ffmpeg and ffprobe on the gateway host (on PATH, or FFMPEG_PATH / FFPROBE_PATH); without them, short recordings work as before and uploads over 24 MB are refused with 413 audio_too_large. Free-tier quotas still apply — Groq's free Whisper allows about 2 hours of audio per hour per key.

Provider coverage. An unknown OpenAI voice name (alloy, nova, …) falls back to the provider's default voice rather than failing.

Provider	TTS	STT	STS	Notes
Google AI Studio	*-tts Gemini models (voice Kore)	every Gemini chat model + gemini-3.5-transcribe	Live / native-audio models	TTS answers WAV. STS uses the Live API WebSocket and needs a WAV (16-bit PCM) upload. "Fetch all models" now reads Google's own model listing.
Groq	Orpheus (troy / fahad)	Whisper large v3 / v3-turbo	—	Orpheus returns WAV only, max 200 chars per request.
Mistral	voxtral-mini-tts	Voxtral transcribe + Voxtral Small (chat)	—
OpenRouter	dedicated TTS models, gpt-audio	dedicated STT models, audio-input chat models	gpt-audio	Discovery requests output_modalities=all so audio models are listed.
Kilo Gateway	audio-output chat models	audio-input chat models	audio in/out chat models
Zhipu	GLM-TTS (tongtong)	GLM-ASR	GLM-4-Voice
Cloudflare Workers AI	Deepgram Aura 1/2, MeloTTS	Whisper ×3, Deepgram Nova-3	—	"Fetch all models" now reads the Workers AI model search.
Hugging Face router	Kokoro, Sesame CSM, Zonos, Chatterbox (DeepInfra)	Whisper (hf-inference), Parakeet, Nemotron ASR, Qwen3-ASR, Cohere Transcribe	PersonaPlex (fal)	Billed against HF credits; a free token without credits gets 402.
NVIDIA NIM	Magpie multilingual, Chatterbox multilingual	Parakeet family, Conformer, Whisper large v3, Nemotron Omni (chat)	—	Some Riva models are gRPC-only; those refuse (and fail over) when the key is bound to an outbound proxy.
Cohere	—	Cohere Transcribe (+ Arabic)	—	Discovery now reads Cohere's native chat-model listing.
OVH AI Endpoints	NVIDIA Riva TTS (en/es/de/it)	Whisper large v3 / v3-turbo	—	Keyless.
LLM7	—	Whisper, Voxtral	—
Antigravity	—	the transcription models the account is offered	—
Custom (OpenAI-compatible)	/audio/speech	/audio/transcriptions	audio chat models	Works with vLLM, speaches, LocalAI, etc.; models are tagged from listing metadata or well-known names (Whisper, Kokoro, …).
Cerebras, DeepSeek, Anthropic, Ollama Cloud, OpenCode Zen, GitHub Models, ModelScope, Bedrock	—	—	—	No audio API offered. Audio models any of them later list are tagged by the same discovery rules.
Pollinations	—	—	—	Audio exists only on the paid gen.pollinations.ai (pollen) API, not the keyless endpoint this gateway uses.

Gateway API keys

Create keys in the dashboard under Keys → Gateway keys. The raw key (freellmapi-…) is shown once at creation — only the hash is stored server-side, so you should copy it right away.

Each gateway key is provider-agnostic — the same key works for OpenAI models, Anthropic models, Google models, or any other configured provider. The router picks the best provider based on your fallback chain. You never need a different gateway key per provider.

How to add provider credentials (e.g., Anthropic, OpenAI, etc.):

1. Go to Keys page in the dashboard
2. Select the provider platform (e.g., Anthropic)
3. Paste your provider API key (e.g., Anthropic's sk-ant-… key, OpenAI's sk-… key)
4. Click Fetch all models to import that provider's catalog
5. Start making requests — the gateway key (freellmapi-…) routes to the right provider automatically

Example: Anthropic setup

1. Get an API key from console.anthropic.com
2. On the Keys page, select Anthropic and paste the key
3. Click Fetch all models
4. Use your gateway key with model="claude-sonnet-4" — the router handles the translation to Anthropic's native API format transparently

---

HTTP proxy pool

v1.3 ships a full HTTP proxy pool. Configure it from the Proxies page or via env / .env:

Env var	Default	Description
PROXY_URL	—	Single proxy URL. Wins over DB. e.g. http://user:pass@host:port
PROXY_POOL_STRATEGY	none	none (off), random, round-robin, least-latency
PROXY_BYPASS	—	Comma-separated platform names that skip the proxy (e.g. groq,google)

Single proxy — set PROXY_URL once. The dashboard tracks request count, success rate, latency, and bytes.

Pool — add rows in the Proxies page (label, URL, optional country), pick a strategy, and the router distributes each request across the pool. Per-key assignment rotates every hour to spread load and avoid burning any single egress. Google Antigravity accounts are exempt from the hourly rotation: an account is bound to the proxy it signed in through, and Google must keep seeing it on that one IP — rebinding it would make one account appear from many egress IPs. Rebind an account by hand from the Proxies page.

Bypass list — platforms you don't want proxied (e.g. low-latency providers like Groq that you want to reach directly).

A bound key never falls back to a direct connection. When a key is bound to a proxy, its outbound calls leave through that proxy — or the request is refused. It is never silently downgraded to a direct connection: a dead, disabled or deleted proxy makes the key unroutable until a proxy recovers, so the server's own public IP is never handed to a provider carrying your key. The one sanctioned direct case is the per-key force direct toggle (and a key that was never bound because you run no proxy infrastructure at all). This is what keeps several provider accounts behind one gateway from looking like one account farming from a single IP.

SOCKS proxies — supported via socks5:// and socks4:// URLs.

find-proxies script — scans a candidate URL list, geo-IP-verifies the country (France by default), probes connectivity, and writes the working set to working-proxies.txt for bulk import:

# 1. Find a batch of working proxies
node scripts/find-proxies.mjs --out working-proxies.txt

# 2. Import in the dashboard: Proxies → Bulk import → paste file

---

Dashboard

Every tab shares one full-width operator look: a compact title row, the same panels (icon + title header), KPI tiles, dense tables and meaning-only colour as the Live and Analytics dashboards. Live, Analytics and the Playground chat fit the window on a desktop screen; the other tabs scroll.

Models page

• Bounded model list: the routing table initially renders 50 models to keep the browser responsive. Load more adds the next 50 rows from the current filtered and sorted list; searching starts again at the first 50.

Live dashboard (v1.3 highlights)

• In-flight proxy name — every active row shows which proxy the request is using
• Growing token counts — output_tokens ticks up live as the model streams
• Session totals — Tokens IN and Tokens OUT aggregated over the dashboard session
• Per-user counts — events scoped to the logged-in user
• Real-time routing events — which provider/model was tried, latency, outcome, all pushed over WebSocket
• Full-screen, no-scroll layout — Live and Analytics use the whole screen width and, on any desktop-sized viewport (≥1024×640), fit the window exactly: KPI rail on top, then charts on the left and In-flight over Recent on the right, with long lists scrolling inside their own panel. 21:9 / 4K screens (≥2200 px) give In-flight and Recent their own columns. Phones and short windows keep the normal scrolling layout.
• In-flight and Recent as one aligned table — both lists share the same columns (time, model, gateway key, provider key, proxy, in, out, tok/s, latency or state, cost) under a sticky header, so every row has the same shape however long the key and proxy names are. Wide panels get one 26 px line per request; narrower ones keep the figures aligned with the route on a second line; phones stack them. Truncated names show in full on hover.
• Ranked lists instead of label-heavy bar charts — Top models (and the per-provider panels on Analytics) print rank, full name and exact value with a thin bar, so long model ids never get clipped.
• Recent-sample KPIs — Avg latency and Avg speed (tok/s) sit next to Tokens IN/OUT; all four describe the Recent list and say so (last N). The Recent header counts ok / failed / cancelled.

Keys page

Add provider credentials for any supported service. Features per key:

• Live status dot — updates in real time: green (healthy), amber (rate-limited), red (error/disabled/401)
• Cached quota snapshots — the Keys list renders the last persisted Antigravity/Codex quota immediately, even after a browser reload or server restart, then refreshes stale provider data in the background. Refreshes are centralized and limited to one account read per minute, regardless of tabs or browsers. Codex uses response headers from real traffic whenever available; otherwise, the central scheduler sends at most one minimal, non-stored quota probe per account per minute so every connected account has a visible value.
• Inline editing — pencil icon to edit label, RPM cap, TPM cap without re-entering the key
• Stats panel — click the dot or ⌄ chevron to expand deep stats:
  • Total requests, success rate, error count, rate-limit hits
  • Latency: min / p50 / avg / p95 / p99 / max
  • 24-hour bar chart, colour-coded by error rate
  • Per-model breakdown with success rate and token counts
  • Error breakdown with occurrence counts
  • Last 25 requests with model, latency, tokens, and timestamp
  • Active cooldowns with reset times
• Model discovery — one click calls the provider's /v1/models endpoint, adds new models to the catalog, and removes stale ones. Removal is decided per provider, not per key: free tiers hand out different rosters per account, so a model is retired only once no key on that provider lists it any more. The summary line counts what actually changed, so a refresh that changed nothing reads all zeros. Custom endpoint mode imports all advertised models in one shot.
• Click the summary line to see what changed, not just how much: every model added and retired, and every thinking level, context window, and price that moved, each with its value on both sides of the change (33K → 131K, in $1/M / out $2/M → in $4/M / out $9/M). One entry per counted change, so the number and the list can never disagree.
• Your keys outrank the catalog. The signed catalog also retires models it used to ship and no longer lists — but never one your own key still serves. The catalog is a snapshot of what is free somewhere; your key listing is what your account is actually given, and when they disagree your account wins. Without that rule the two fought: discovery added a model your key serves, then the next catalog apply (every boot, and twice a day) deleted it again, so it never appeared no matter how often you refreshed.

Custom endpoints: use the "Custom" provider to connect any OpenAI-compatible URL. Toggle "Auto-discover" to bulk-register all models the endpoint advertises.

Export & re-import (the backup round trip): the Export button on the Keys page writes every key of the account to a CSV with the columns Name,Platform,Key,Created At,Base URL,Row ID. It includes the real, unmasked secrets, so the file is a backup and must be kept safe. Dropping the file back into Batch import keys restores the rows, including their original identity and provider metadata.

• Each row goes back to the provider it came from. The Platform column is honoured per row, so a backup holding keys for several providers restores them all — the provider picked in the form is only the fallback for rows that name none. Created At, Base URL, and Row ID preserve the exported row's metadata when available.
• OAuth accounts come with it. OpenAI Codex, OpenAI ChatGPT Plan and Google Antigravity rows contain the account credentials needed to restore them without another sign-in. A row that does not carry a valid account credential is refused rather than stored as a row that would fail on its first request.
• Legacy CSVs remain importable. The older four-column format (Name,Platform,Key,Created At) is still accepted. A custom row in that legacy format has no Base URL, so it is skipped and reported rather than imported without the endpoint it needs. New custom rows must include Base URL.
• Skipped rows are reported. Batch import shows the row number, optional label, platform, and reason for each skipped row; credential values are not included in the report.
• The import binds every new key to a proxy exactly like adding one by hand, and the hourly rotation leaves Google Antigravity accounts alone (see HTTP proxy pool).

Proxies

The proxy pool page lets you:

• Add HTTP / HTTPS / SOCKS proxies with labels and (optional) country
• Pick a pool strategy: none, random, round-robin, least-latency
• Bulk select with checkboxes and apply enable / disable / delete / test across many rows
• Click a row to expand its stats panel (request count, success rate, latency, bytes)
• Health-checker pings every enabled proxy on a schedule and marks failing ones

Playground

Interactive chat with:

• Full model selector (providers, auto)
• System prompt editor
• Live routing indicator — shows the provider, model, and proxy the router is currently trying while a request is in flight

Analytics

Request history with per-provider breakdown, token usage, latency trends, and cost estimates — all on one screen, in the same tiles and panels as the Live page: twelve KPI tiles, the trend chart and per-provider rankings on top, per-model and recent-error tables below (each scrolling inside its own panel with a sticky header; from 1280 px wide no column is pushed out of view). Each provider row prints its value plus success rate or request count; hover it for the full breakdown: requests, success rate, errors, cancellations, latency, and tokens.

Three things the numbers mean, because each is easy to read wrong:

• Avg latency is time to first token — how fast the gateway starts answering. Avg duration is the whole request: queue wait, any failover attempts, and the full response stream. A long duration is normal for an agentic turn with a big context and says nothing bad about the provider; latency is the figure that makes providers comparable. Latency shows — when nothing in the window recorded a first-token time (it is measured on the streaming path).
• Est. savings is what the tokens you actually sent would have cost on paid APIs, priced per request at the moment it was served and stored on the row (requests.paid_cost_usd). It is a measurement of the selected range, not a projection, so it only changes when traffic does — a pricing refresh or a retired model never rewrites what last week cost. Errors and cancellations cost nothing and are excluded.
• Requests counts everything, including requests cancelled or abandoned before routing resolved. Those belong to no provider and no model, so the per-provider and per-model panels leave them out — which is why those panels can total less than the Requests card. The card's tooltip reports the gap so the two reconcile.

Range windows are aligned to whole UTC hours (24h) or whole UTC days (7d, 30d), and the charts cover exactly the window the cards count.

---

Configuration

Variable	Default	Description
ENCRYPTION_KEY	(required)	64-char hex string for AES-256-GCM. Generate: node -e "console.log(require('crypto').randomBytes(32).toString('hex'))"
PORT	3001	HTTP listen port
HOST_BIND	127.0.0.1	Bind interface. Set 0.0.0.0 for LAN access.
ADMIN_USERNAME	—	Bootstrap admin username (first run only)
ADMIN_PASSWORD	—	Bootstrap admin password, min 8 chars (first run only)
PROXY_RATE_LIMIT_RPM	120	Max /v1 requests per minute per client IP. 0 = disabled.
REQUEST_ANALYTICS_RETENTION_DAYS	90	Days to keep request history
REQUEST_ANALYTICS_MAX_ROWS	100000	Maximum request history rows
REQUEST_QUEUE_TIMEOUT_SECONDS	60	Seconds to hold a queued request before returning 429
MODEL_REFRESH_INTERVAL_MINUTES	720	Scheduled model catalogue refresh: every enabled key re-runs discovery through its own proxy. 0 = disabled.
FREELLMAPI_CONTEXT_HANDOFF	—	Set on_model_switch to inject context messages on model switch
BEDROCK_REGION	eu-central-1	Amazon Bedrock EU region. EU-only allowlist; non-EU values are refused.
SIWC_REDIRECT_PORT	1456	Loopback port of the official “Sign in with ChatGPT” callback (http://127.0.0.1:<port>/auth/callback). Change it if 1456 is taken.
SIWC_VERIFY_ID_TOKEN	(on)	Set off to skip only the ID-token signature check (issuer, audience, expiry and nonce are still enforced) — an escape hatch for an OpenAI JWKS outage.
PROXY_URL	—	Single proxy URL. e.g. http://user:pass@host:port
PROXY_POOL_STRATEGY	none	none / random / round-robin / least-latency
PROXY_BYPASS	—	Comma-separated platform names to skip the proxy
DASHBOARD_ORIGINS	—	Extra CORS origins for the dashboard (comma-separated)
LOG_LEVEL	INFO	DEBUG | INFO | WARN | ERROR | FATAL
GATEWAY_INSTANCES	1	PM2: number of gateway worker processes

---

How routing works

When a request arrives at /v1/chat/completions:

1. Resolve the model — auto uses the whole fallback chain; anything else looks up provider keys for that model ID.
2. Follow your order — candidates are tried in the fallback-chain order you set on the Models page (drag to reorder), with rate-limited models demoted by a temporary penalty.
3. Pick a proxy — for the resolved key, look up its current pool assignment (or fall back to the single PROXY_URL); the assignment rotates every hour so each key gets a fair share.
4. Rotate keys fairly — for the chosen provider, keys are tried least-recently-used first. Every key gets served once before any key repeats, and rotation state lives in the database (api_keys.use_seq), so it survives restarts. A key you just added has never been used, so it goes to the front of the queue and is picked on the very next request for that provider. Within that order the router takes the first key that isn't on cooldown, hasn't hit its outbound RPM/TPM cap, and has a healthy proxy.
5. On failure — a rate-limit response (429) puts the key on a cooldown; an auth failure (401/403) marks it invalid and pushes a real-time status update to the dashboard.
6. Queue if exhausted — if all candidates are on cooldown the request parks in an in-memory queue and retries when the earliest cooldown expires, up to REQUEST_QUEUE_TIMEOUT_SECONDS.
7. Context handoff — if the winning model differs from the previous turn in the same session, a compact system message is prepended so the new model has context.

Every routing decision is logged (provider, model, key prefix, proxy, latency) and streamed to the live dashboard via WebSocket.

When a model gets auto-disabled

If a provider answers 404/410 for a model, the gateway can disable its catalog row so the router stops paying a round-trip to a dead endpoint on every request. That verdict requires proof the model is gone, not that one key couldn't reach it:

• the provider says so in words — retired, end of life, no longer available, no endpoints found — which is authoritative on its own, or
• a bare 404/410 reproduces on 2+ different keys within 15 minutes.

A bare 404 on a single key is not enough. On gateways that grant model access per account (NVIDIA NIM, Bedrock, OpenRouter tiers) a 404 usually just means that key isn't entitled, and it also comes from a dead proxy's error page or a transient routing fault. Such a request fails over normally and the model stays enabled.

Auto-disabled rows are marked auto_disabled and stay visible on the Models page so you can re-enable them. Keys → Fetch all models also restores them automatically when the provider lists the model again. Requesting one returns a 400 that says the disable was automatic and how to undo it — rather than implying you turned it off.

---

Project structure

freellmapi/
├── server/src/
│   ├── routes/        REST endpoints — /api/*, /v1/*
│   │                    (proxy, keys, gateway-keys, admin, fallback, responses, settings…)
│   ├── services/      Router, queue, health checker, scoring, WebSocket push, auth, context handoff, proxy-health
│   ├── providers/     Per-provider OpenAI-compatible adapters (google, openai-compat, cohere, cloudflare)
│   ├── db/            SQLite (better-sqlite3), migration runner, model catalog migrations
│   └── lib/           Crypto, proxy pool, logger, error handling
├── client/src/
│   ├── pages/         Keys, Proxies, Gateway keys, Playground, Analytics, Live dashboard, Fallback, Premium, Admin…
│   └── components/    Shared UI components (shadcn/ui)
├── shared/            TypeScript types shared between server and client
├── desktop/           Electron desktop wrapper
├── scripts/           find-proxies.mjs and other ops helpers
├── k8s/               Kubernetes / Podman play kube manifests
├── .gitea/workflows/  Forgejo Actions (ci.yml, release.yml)
├── .github/workflows/ GitHub Actions (ci.yml, docker.yml)
├── ecosystem.config.cjs  PM2 process definition
├── Dockerfile         Multi-stage Docker build
├── docker-compose.yml
├── install.sh         Universal installer
├── migrate-from-old.sh  Import keys from an older FreeLLMAPI install
└── find-proxies.mjs   Free-proxy scraper / verifier

---

Continuous integration

Two Forgejo Actions workflows ship with the repo and are picked up automatically by your Forgejo runner:

Workflow	Trigger	What it does
.gitea/workflows/ci.yml	push to main, pull request	npm ci → build server → build client → run 728 server tests
.gitea/workflows/release.yml	push of any v* tag (e.g. v1.3)	same as CI, then posts a green-build summary to the run page

A push of the v1.3 tag will start a release build on the runner and report success or failure on the release page. Run the same suite locally with:

npm install
npm run build
npm test -w server

---

Contributing

git clone https://git.pandem.fr/outage.sh/FreeLLMapi freellmapi
cd freellmapi
npm install
npm run dev        # server on :3001, Vite dev server on :5173

• npm run build — full production build (server + client)
• npm test -w server — server test suite (728 tests)
• npm run build:server — server only
• npm run desktop:dev — Electron desktop wrapper in dev mode
• npm run desktop:dist — build a distributable Electron binary

PRs welcome. Keep changes focused; include tests for new server behaviour.

---

Disclaimer

FreeLLMAPI routes requests to third-party AI provider APIs under their respective free-tier terms. It does not circumvent rate limits — it distributes load across multiple API keys you legitimately own. Review each provider's Terms of Service before use. NVIDIA's free NIM tier is for evaluation only. The authors are not responsible for ToS violations or API key misuse.

---

MIT License · Based on FreeLLMAPI