# Add a Custom OpenAI-Compatible Provider


The `compatible` provider reads `~/.config/kli/providers.json` and registers one entry per OpenAI-compatible endpoint you define there — a local server (Ollama, llama.cpp, vLLM), a self-hosted gateway, or any third-party service that speaks the OpenAI Chat Completions or Responses wire format. For the rest of kli's providers, see [Models & Providers](/kli/models/providers-and-transports).

This guide writes that file and connects the result.

## Create providers.json

Create `~/.config/kli/providers.json`. The file is a single JSON object keyed by provider id. Each key becomes a provider name you select against; each value describes one endpoint.

A minimal entry for a local Chat Completions server:

```json
{
  "local-llama": {
    "base-url": "http://localhost:11434/v1",
    "key-env": "LOCAL_LLAMA_KEY",
    "models": [
      { "id": "llama3.1:70b", "context-window": 131072 }
    ]
  }
}
```

The key `local-llama` is the provider id. kli sends requests to the endpoint built from `base-url`, and reads the API key from the environment variable named in `key-env`.

## Set the request URL

kli builds the request URL from `base-url` plus a fixed path determined by `api`. Set `base-url` to the root your server exposes, with no trailing path component:

- For the Chat Completions API (the default), kli appends `/chat/completions`. A `base-url` of `http://localhost:11434/v1` yields `http://localhost:11434/v1/chat/completions`.
- For the Responses API, kli appends `/responses`. Set `url-path` to append a different segment instead.

Any trailing slash on `base-url` is trimmed before the path is joined.

## Choose the wire format

Set `api` to match the format your endpoint speaks. The default is `openai-completions`.

```json
{
  "my-gateway": {
    "base-url": "https://gateway.example.net",
    "api": "openai-responses",
    "url-path": "/v1/responses",
    "key-env": "GATEWAY_KEY",
    "models": [
      { "id": "gpt-oss-120b", "context-window": 131072 }
    ]
  }
}
```

The entry above sends requests to `https://gateway.example.net/v1/responses`: kli trims any trailing slash from `base-url` and joins `url-path` onto it. `url-path` only takes effect when `api` is `openai-responses`. With `openai-completions` it is ignored and the path is always `/chat/completions`.

A value of `api` other than `openai-completions` or `openai-responses` is an error and the entry fails to load.

## Provide the API key

A `compatible` entry never holds a secret. The `key-env` field names an environment variable; kli reads the key from that variable's value at request time. Set it in the shell or service unit that launches kli:

```sh
export LOCAL_LLAMA_KEY="sk-..."
```

For a local server that ignores authentication, name an env var and set it to any non-empty placeholder.

If you omit `key-env`, the entry registers with no environment credential. You can then persist a static key for that provider id from inside a session:

```
/auth key local-llama sk-...
```

The static key is stored and reused; it is not echoed back.

## List the models

Each entry needs a `models` array. An entry with an empty array registers the provider but offers nothing to select. Each model is an object:

```json
{ "id": "llama3.1:70b", "name": "Llama 3.1 70B", "context-window": 131072 }
```

- `id` — the model id sent to the endpoint, and the id you select against. Required.
- `name` — the display label. Defaults to `id` when omitted.
- `context-window` — the context size in tokens, used for budgeting.
- `options` — optional semantic model option schemas for this model. See [Add semantic options](#add-semantic-options).

For where this file sits among kli's config paths, see [Files and paths](/kli/config/files-and-paths).

## Add semantic options

Semantic options describe model capabilities in kli terms. They are not raw request fields. The selected transport lowers them to the wire shape it supports: for example, `reasoning-effort` becomes OpenAI `reasoning_effort` or Responses `reasoning.effort`, while Anthropic maps it to its `thinking` request object.

Add an `options` object at the provider level to give every model the same option schema:

```json
{
  "my-gateway": {
    "base-url": "https://gateway.example.net/v1",
    "api": "openai-responses",
    "key-env": "GATEWAY_KEY",
    "options": {
      "reasoning-effort": {
        "values": ["off", "low", "medium", "high"],
        "default": "off"
      },
      "text-verbosity": {
        "values": ["low", "medium", "high"]
      }
    },
    "models": [
      { "id": "fast-model", "context-window": 65536 },
      {
        "id": "plain-model",
        "context-window": 32768,
        "options": { "reasoning-effort": null }
      }
    ]
  }
}
```

A model-level `options` object merges over the provider-level object. Re-declaring an option replaces that option's schema for just that model. Setting an inherited option to `null` removes it from that model.

Each option schema may carry:

| Field | Type | Effect |
| --- | --- | --- |
| `type` | string | Option type. Usually omitted for built-in semantic options, whose type is known globally. |
| `values` | array | Admitted enum values. Required for enum options. |
| `default` | scalar | Default value when the model is selected without an explicit value. |
| `min` | number | Minimum for numeric option types. |
| `max` | number | Maximum for numeric option types. |

The schema type can be `enum`, `boolean`, `integer`, `number`, or `string`. Built-in semantic option ids are `reasoning-effort`, `text-verbosity`, `service-tier`, and `prompt-cache-retention`; their enum universes and transport lowering are documented in [Providers and Transports](/kli/models/providers-and-transports#semantic-options).

## Add extra headers

To send headers on every request to an endpoint — a routing tag, an org id, a gateway token — add a `headers` object. Each name and value is appended to the outgoing request, alongside the `Authorization: Bearer` header kli sets from the resolved key.

```json
{
  "my-gateway": {
    "base-url": "https://gateway.example.net/v1",
    "key-env": "GATEWAY_KEY",
    "headers": {
      "x-org-id": "acme",
      "x-route": "fast"
    },
    "models": [{ "id": "fast-model", "context-window": 65536 }]
  }
}
```

## Use the provider

kli reads `providers.json` when the `compatible` provider installs, so start a fresh session after editing the file. Each entry appears under its provider id, with its models available to select. See [/kli/guides/connect-a-provider](/kli/guides/connect-a-provider) for switching providers and models inside a session.