Add a Custom OpenAI-Compatible Provider

The compatible provider reads ~/.config/kli/providers.json and registers one entry per OpenAI-compatible endpoint you define there — a local server (Ollama, llama.cpp, vLLM), a self-hosted gateway, or any third-party service that speaks the OpenAI Chat Completions or Responses wire format. For the rest of kli's providers, see Models & Providers.

This guide writes that file and connects the result.

Create providers.json

Create ~/.config/kli/providers.json. The file is a single JSON object keyed by provider id. Each key becomes a provider name you select against; each value describes one endpoint.

A minimal entry for a local Chat Completions server:

{
  "local-llama": {
    "base-url": "http://localhost:11434/v1",
    "key-env": "LOCAL_LLAMA_KEY",
    "models": [
      { "id": "llama3.1:70b", "context-window": 131072 }
    ]
  }
}

The key local-llama is the provider id. kli sends requests to the endpoint built from base-url, and reads the API key from the environment variable named in key-env.

Set the request URL

kli builds the request URL from base-url plus a fixed path determined by api. Set base-url to the root your server exposes, with no trailing path component:

• For the Chat Completions API (the default), kli appends /chat/completions. A base-url of http://localhost:11434/v1 yields http://localhost:11434/v1/chat/completions.
• For the Responses API, kli appends /responses. Set url-path to append a different segment instead.

Any trailing slash on base-url is trimmed before the path is joined.

Choose the wire format

Set api to match the format your endpoint speaks. The default is openai-completions.

{
  "my-gateway": {
    "base-url": "https://gateway.example.net",
    "api": "openai-responses",
    "url-path": "/v1/responses",
    "key-env": "GATEWAY_KEY",
    "models": [
      { "id": "gpt-oss-120b", "context-window": 131072 }
    ]
  }
}

The entry above sends requests to https://gateway.example.net/v1/responses: kli trims any trailing slash from base-url and joins url-path onto it. url-path only takes effect when api is openai-responses. With openai-completions it is ignored and the path is always /chat/completions.

A value of api other than openai-completions or openai-responses is an error and the entry fails to load.

Provide the API key

A compatible entry never holds a secret. The key-env field names an environment variable; kli reads the key from that variable's value at request time. Set it in the shell or service unit that launches kli:

export LOCAL_LLAMA_KEY="sk-..."

For a local server that ignores authentication, name an env var and set it to any non-empty placeholder.

If you omit key-env, the entry registers with no environment credential. You can then persist a static key for that provider id from inside a session:

/auth key local-llama sk-...

The static key is stored and reused; it is not echoed back.

List the models

Each entry needs a models array. An entry with an empty array registers the provider but offers nothing to select. Each model is an object:

{ "id": "llama3.1:70b", "name": "Llama 3.1 70B", "context-window": 131072 }

• id — the model id sent to the endpoint, and the id you select against. Required.
• name — the display label. Defaults to id when omitted.
• context-window — the context size in tokens, used for budgeting.
• options — optional semantic model option schemas for this model. See Add semantic options.

For where this file sits among kli's config paths, see Files and paths.

Add semantic options

Semantic options describe model capabilities in kli terms. They are not raw request fields. The selected transport lowers them to the wire shape it supports: for example, reasoning-effort becomes OpenAI reasoning_effort or Responses reasoning.effort, while Anthropic maps it to its thinking request object.

Add an options object at the provider level to give every model the same option schema:

{
  "my-gateway": {
    "base-url": "https://gateway.example.net/v1",
    "api": "openai-responses",
    "key-env": "GATEWAY_KEY",
    "options": {
      "reasoning-effort": {
        "values": ["off", "low", "medium", "high"],
        "default": "off"
      },
      "text-verbosity": {
        "values": ["low", "medium", "high"]
      }
    },
    "models": [
      { "id": "fast-model", "context-window": 65536 },
      {
        "id": "plain-model",
        "context-window": 32768,
        "options": { "reasoning-effort": null }
      }
    ]
  }
}

A model-level options object merges over the provider-level object. Re-declaring an option replaces that option's schema for just that model. Setting an inherited option to null removes it from that model.

Each option schema may carry:

The schema type can be enum, boolean, integer, number, or string. Built-in semantic option ids are reasoning-effort, text-verbosity, service-tier, and prompt-cache-retention; their enum universes and transport lowering are documented in Providers and Transports.

Add extra headers

To send headers on every request to an endpoint — a routing tag, an org id, a gateway token — add a headers object. Each name and value is appended to the outgoing request, alongside the Authorization: Bearer header kli sets from the resolved key.

{
  "my-gateway": {
    "base-url": "https://gateway.example.net/v1",
    "key-env": "GATEWAY_KEY",
    "headers": {
      "x-org-id": "acme",
      "x-route": "fast"
    },
    "models": [{ "id": "fast-model", "context-window": 65536 }]
  }
}

Use the provider

kli reads providers.json when the compatible provider installs, so start a fresh session after editing the file. Each entry appears under its provider id, with its models available to select. See /kli/guides/connect-a-provider for switching providers and models inside a session.