LLM providers and models

Kumoss configures two model roles in config.yaml. Both are applied by the core’s LiteLLM Router; there is no provider enum in Kumoss, so any model string LiteLLM understands is accepted by the configuration. Whether a model works is a different matter; see Model requirements below.

Three extra settings apply to both roles:

The core passes drop_params=True to LiteLLM, so a provider that does not support one of these parameters ignores it instead of failing.

Kumoss’s agents are tool-calling loops, so the configuration accepting a model string does not mean the model can run a session. Both roles must provide:

The shipped models (Claude and Gemini on Vertex AI, vertex_ai/, which the core’s test suite exercises) satisfy all of these, as do the built-in defaults (Anthropic Claude direct, anthropic/) and the same Claude models on Azure AI Foundry (azure_ai/). Kumoss has not tested every provider LiteLLM lists.

A LiteLLM model string is <provider>/<model-id>:

Routing is purely prefix based. Kumoss does not validate that the model id exists; a typo surfaces as an error from the provider on the first call. The reference list of model ids is models.litellm.ai; the provider list with credential keys is the LiteLLM provider docs.

Model selection lives in config.yaml; credentials live in core/.env. They must belong to the same provider:

Changing a model identifier to a different provider means changing the credential variables too. config.yaml is baked into the core image, so a model change requires docker compose build core; a credential change in core/.env only requires restarting the core container.

At boot the core calls litellm.validate_environment(model=…​) for each configured model (or for each llm.model_list entry when that key is set). If LiteLLM reports missing environment variables, the core refuses to start with:

Treat this check as a convenience, not a guarantee:

The exact behaviour depends on the installed LiteLLM version (the core pins litellm>=1.93.0).

The tables below list the LiteLLM model string format, the environment variables LiteLLM reads by default, and the litellm_params keys you would use instead when writing an llm.model_list entry. The lists come from LiteLLM’s documentation. Kumoss has not tested every provider listed here; the checked-in config.yaml and the code defaults both target Anthropic directly (anthropic/), and the core’s test suite (core/tests/shared/config/test_llm_config.py) exercises the OpenAI-style and Vertex AI validation paths.

A single direct provider, both roles on the same vendor, no router customisation. This is the checked-in setup, and the simplest working one.

An equivalent setup on Azure AI Foundry, whose credentials LiteLLM cannot validate at boot (a missing value fails on the first call):

Equivalent example on Google Vertex AI:

For how to paste the key, and what the file-path alternative needs, see the Vertex AI example in Environment variables.

llm.model_list follows the LiteLLM Router format and is passed to the Router verbatim. Use it only when you need something the two plain model strings cannot express:

You do not need it to keep credentials in core/.env: plain model strings already read the provider’s default variables.

Rules when model_list is non-empty:

If all you want is the default provider credentials from core/.env, you do not need model_list at all; the two plain model strings are the right configuration.

Core exits with LLM credentials missing from environment: …​. The provider chosen by llm.model or llm.small_model needs the listed variables in core/.env. Either you changed the model prefix without changing the credentials, or you rely on custom variable names through model_list without also setting the default-named ones.

Core boots but the first session fails with an authentication or "model not found" error. The provider has no boot-time validation mapping (for example azure_ai), the key is present but invalid, or the <model-id> part of the string is wrong for that provider. Check the model id against models.litellm.ai and the provider’s own console.

Boot validation complains about a default variable although you use custom names. Expected. Set the default-named variable to any value, even empty (validate_environment treats an empty string as present); only the os.environ/…​ reference in model_list is used at call time.

model or small_model does not match any model_name. When model_list is set, the two role strings are aliases. LiteLLM raises an error on the first call for an alias with no matching entry. Make both values equal to a model_name from the list.

Requests time out or the endpoint is unreachable. Each chat completion call has an llm.timeout budget (600 seconds by default) and num_retries=3 inside LiteLLM. The web-search Responses API fallback path uses the same timeout but is otherwise different: it retries up to 4 attempts with 30-second sleeps between them and sets no num_retries, so a slow or flaky OpenAI-compatible endpoint can stall that path for a long time. For self-hosted endpoints, remember that the core runs inside the compose network: localhost refers to the core container, not to the host. Use the service name or the host’s routable address in api_base, and make sure any TLS certificate is trusted by the core image.

Provider rejects temperature or max_tokens. The core sends drop_params=True, so unsupported parameters are dropped. If the provider still rejects a value (for example a completion cap below 32000), lower llm.max_output_tokens.

litellm calls load_dotenv() at import time unless LITELLM_MODE is set to PRODUCTION. Running the core, or pytest, on your workstation (rather than inside the container) therefore silently loads core/.env into the process environment — including any real secrets it holds — even though nothing in Kumoss’s own code does this. The working directory does not matter: python-dotenv searches upwards from the directory of the calling package, so with the virtual environment at core/.venv the file core/.env is found from any directory. Variables already present in the environment are not overwritten by the file. To reproduce a clean-environment boot, set LITELLM_MODE=PRODUCTION, move core/.env aside, or use a virtual environment outside core/; otherwise a variable you think is unset may actually be coming from core/.env.