Kumoss-managed state

With the state bucket set to a name — the shipped configuration — Kumoss owns state:

storage:
  provider: "RUSTFS"
  bucket: "kumoss-artifacts"
  terraform_state_bucket: "kumoss-terraform-state"
  endpoint_url: "http://object-storage:9000"
  public_endpoint_url: "http://localhost:9000"
  region: "us-east-1"

This is why it ships on: it is the quickest way to get a working remote backend when there is none yet, on RUSTFS the bucket name is the only value involved, and the repositories need no backend block at all.

Under this model, a backend block the repository declares is replaced, not combined: none of its values survive. If your repositories already keep state in their own backend, use Repository-declared backend instead.

State goes to the same object-storage provider as artifacts, in a separate bucket (a blob container on STORAGE_ACCOUNT). The separation is deliberate: state must not inherit the lifecycle, expiry, or presign policy you apply to generated reports and plans.

See Credentials required below for how the IaC sidecar authenticates against the state store when the rendered backend does not embed static keys.

What Kumoss manages automatically

When What happens

When	What happens
Core start-up	The state bucket (or container) is created if missing, with the same credentials used for artifacts. If the store is unusable the core fails to boot, alongside the database, Redis, and artifact-bucket checks.
Before every `init`	The core writes a `backend_override.tf` into the workspace, addressing the state bucket with a key derived from the project.
Per project	The state key is assigned from a digest of the repository, cloud scope, and root-module path — no configuration, no per-repository setup.
On apply	The pinned workspace is moved, not copied, so the override and the initialized `.terraform/` travel with it: the apply writes to the same state the plan was made against, without re-running `init`.
On every run	`backend_override.tf` is excluded by the seeded `.gitignore`, so it never reaches a commit or a pull request.

Core start-up

The state bucket (or container) is created if missing, with the same credentials used for artifacts. If the store is unusable the core fails to boot, alongside the database, Redis, and artifact-bucket checks.

Before every init

The core writes a backend_override.tf into the workspace, addressing the state bucket with a key derived from the project.

Per project

The state key is assigned from a digest of the repository, cloud scope, and root-module path — no configuration, no per-repository setup.

On apply

The pinned workspace is moved, not copied, so the override and the initialized .terraform/ travel with it: the apply writes to the same state the plan was made against, without re-running init.

On every run

backend_override.tf is excluded by the seeded .gitignore, so it never reaches a commit or a pull request.

The override wins over whatever the repository declares: Terraform and OpenTofu merge override.tf *over the rest of the configuration, so one file both _introduces a backend where the repository declares none and replaces one it does declare. Kumoss never edits the repository’s committed HCL. The core logs the target on every init:

Terraform state: rustfs bucket=kumoss-terraform-state key=<project_id>/terraform.tfstate

Per-provider configuration

The backend type follows storage.provider. There is no separate setting for it, and no combination in which state and artifacts live in different stores.

The bundled store, or any S3-compatible server — the default storage.provider, and the least work: in the Compose stack the bucket name is the only value to add.

storage:
  provider: "RUSTFS"
  terraform_state_bucket: "kumoss-terraform-state"
  endpoint_url: "http://object-storage:9000"
  region: "us-east-1"

Rendered into each workspace as backend_override.tf:

terraform {
  backend "s3" {
    bucket = "kumoss-terraform-state"
    key    = "<project_id>/terraform.tfstate"
    region = "us-east-1"

    access_key = "rustfsadmin"
    secret_key = "rustfsadmin"

    endpoints = {
      s3 = "http://object-storage:9000"
    }

    use_path_style              = true
    skip_credentials_validation = true
    skip_region_validation      = true
    skip_requesting_account_id  = true
    skip_metadata_api_check     = true
    skip_s3_checksum            = true
    use_lockfile                = true
  }
}

The skip_* flags exist because a non-AWS S3 implementation has no account-id endpoint, no EC2 instance-metadata service, and no region validation; the checksum skip accommodates servers without S3 trailing checksums. use_path_style addresses buckets as <endpoint>/<bucket> rather than as a virtual host.

AWS S3.

storage:
  provider: "S3"
  bucket: "acme-kumoss-artifacts"
  terraform_state_bucket: "acme-kumoss-tfstate"
  region: "eu-west-1"

endpoint_url and public_endpoint_url are ignored: Terraform and the AWS SDK both build the regional endpoint from region. With static keys configured:

terraform {
  backend "s3" {
    bucket = "acme-kumoss-tfstate"
    key    = "<project_id>/terraform.tfstate"
    region = "eu-west-1"

    access_key = "AKIAEXAMPLE"
    secret_key = "examplesecret"

    use_lockfile = true
  }
}

With no static keys — the recommended production shape — the credential lines are omitted entirely and the engine resolves credentials as any AWS SDK client does, from an instance profile, IRSA, or a workload identity on the IaC sidecar’s container:

terraform {
  backend "s3" {
    bucket = "acme-kumoss-tfstate"
    key    = "<project_id>/terraform.tfstate"
    region = "eu-west-1"

    use_lockfile = true
  }
}

Blank the RustFS keys when you switch to S3. storage.access_key_env and storage.secret_key_env still name RUSTFS_ACCESS_KEY and RUSTFS_SECRET_KEY by default, and core/env.sample ships them set to rustfsadmin. A non-empty value always wins, so leaving them set embeds rustfsadmin into the backend block and every init fails against AWS.

Azure Blob Storage.

storage:
  provider: "STORAGE_ACCOUNT"
  bucket: "kumoss-artifacts"
  terraform_state_bucket: "kumoss-terraform-state"
  endpoint_url: "https://acmekumoss.blob.core.windows.net"
  public_endpoint_url: "https://acmekumoss.blob.core.windows.net"

terraform_state_bucket names a blob container, not a bucket. The account name is derived from endpoint_url; region is ignored.

terraform {
  backend "azurerm" {
    storage_account_name = "acmekumoss"
    container_name       = "kumoss-terraform-state"
    key                  = "<project_id>/terraform.tfstate"
    access_key           = "azure-shared-key"
  }
}

The shared key is mandatory and is always embedded. The core refuses to boot with provider: STORAGE_ACCOUNT and an empty STORAGE_ACCOUNT_KEY (storage.provider STORAGE_ACCOUNT requires env var STORAGE_ACCOUNT_KEY.), because the artifact adapter needs that key to sign SAS download URLs. The renderer contains an Entra ID branch (use_azuread_auth = true) for a keyless account, but no supported configuration reaches it in this release. Treat managed identity and Entra ID for Azure state as unavailable under this model. If you need keyless state, turn managed state off and declare an azurerm block with use_azuread_auth = true, as shown in Repository-declared backend; the artifacts keep using the shared key.

Credentials required

Two different components touch the state store, and they authenticate separately.

Component What it does Credentials it uses

Component	What it does	Credentials it uses
Core	Creates/checks the state bucket at boot	`storage.*_env` variables in `core/.env` — `RUSTFS_ACCESS_KEY`/`RUSTFS_SECRET_KEY`, the AWS default chain, or `STORAGE_ACCOUNT_KEY`
IaC sidecar	Reads and writes state on every `init`, `plan`, and `apply`	The static keys embedded in `backend_override.tf`: always on `RUSTFS` and `STORAGE_ACCOUNT` (the Azure shared key), and on `S3` when keys are configured. Otherwise, on `S3` only, the sidecar container’s own ambient credentials (`AWS_*`, IRSA, instance profile)

Core

Creates/checks the state bucket at boot

storage.*_env variables in core/.env — RUSTFS_ACCESS_KEY/RUSTFS_SECRET_KEY, the AWS default chain, or STORAGE_ACCOUNT_KEY

IaC sidecar

Reads and writes state on every init, plan, and apply

The static keys embedded in backend_override.tf: always on RUSTFS and STORAGE_ACCOUNT (the Azure shared key), and on S3 when keys are configured. Otherwise, on S3 only, the sidecar container’s own ambient credentials (AWS_*, IRSA, instance profile)

This split is easy to get wrong: with provider: S3 and no static keys, giving the core an instance role is not enough — the engine runs in the IaC sidecar, so that container needs the role too. The sidecar must also be able to reach storage.endpoint_url; in the Compose stack both core and iac are on bridge-network, so http://object-storage:9000 resolves, but on another platform allow that egress explicitly.

Minimum permissions on the state bucket, in practice:

Provider Permissions

Provider	Permissions
AWS S3	`s3:ListBucket` on the bucket; `s3:GetObject`, `s3:PutObject`, `s3:DeleteObject` on `<bucket>/*` (this covers the `.tflock` object). Add `s3:CreateBucket` only if you want the core to create the bucket at boot rather than pre-creating it.
Azure Blob Storage	The account shared key, which conveys full container access.
RustFS / other S3-compatible	The access/secret key pair configured for the store.

AWS S3

s3:ListBucket on the bucket; s3:GetObject, s3:PutObject, s3:DeleteObject on <bucket>/* (this covers the .tflock object). Add s3:CreateBucket only if you want the core to create the bucket at boot rather than pre-creating it.

Azure Blob Storage

The account shared key, which conveys full container access.

RustFS / other S3-compatible

The access/secret key pair configured for the store.

Backend minimums lists the per-cloud backend minimums in more detail, including the variables to set when state lives in a different account or project from the resources.

State keys

State is keyed by project, not by session or directory:

<state_bucket>/<project_id>/terraform.tfstate

project_id is the SHA-256 hex digest of three newline-joined values:

the normalized repository URI, as host/path
the cloud scope (scope_id — subscription, project, or account), trimmed and lowercased
the normalized root-module path within the repository (iac_path)

Normalization means two spellings of the same project never split into two states: the repository URI is lowercased, loses a .git suffix and a userinfo username; the module path is trimmed of surrounding and duplicate slashes. Only https:// URIs reach this point — the API rejects every other scheme.

These all resolve to one project

These all resolve to one project
`https://github.com/acme/infra.git`
`https://github.com/acme/infra`
`https://acme@github.com/acme/infra.git`
`HTTPS://GitHub.com/Acme/Infra.git`

https://github.com/acme/infra.git

https://github.com/acme/infra

https://acme@github.com/acme/infra.git

HTTPS://GitHub.com/Acme/Infra.git

An empty iac_path is a valid project — the root module is the repository root. The path comes from the root detection described in Repository layout; note that the web application sends . for a repository whose Terraform files are at the top level, while an API call that omits iac_path sends an empty value, and the two are hashed as different projects. Because all three inputs matter, a change to any of them is a different project and therefore a different state file.

Locking

State locking is enabled and is native to the store — Kumoss provisions no extra infrastructure for it.

Provider Mechanism Notes

Provider	Mechanism	Notes
`RUSTFS`, `S3`	`use_lockfile = true`	The engine’s S3-native lock: a `<key>.tflock` object written with a conditional put next to the state. No DynamoDB table is used or needed. Requires OpenTofu ≥ 1.10 or Terraform ≥ 1.10 — per those engines' release notes — and an S3 implementation that supports conditional writes (the bundled engines are 1.12.6 and 1.16.0).
`STORAGE_ACCOUNT`	Blob lease	Native to the `azurerm` backend and always on; no flag is rendered.

RUSTFS, S3

use_lockfile = true

The engine’s S3-native lock: a <key>.tflock object written with a conditional put next to the state. No DynamoDB table is used or needed. Requires OpenTofu ≥ 1.10 or Terraform ≥ 1.10 — per those engines' release notes — and an S3 implementation that supports conditional writes (the bundled engines are 1.12.6 and 1.16.0).

STORAGE_ACCOUNT

Blob lease

Native to the azurerm backend and always on; no flag is rendered.

Whether RustFS honours the conditional-write semantics use_lockfile relies on has not been verified in this repository.

Verify

To check all three layers in a deployed environment, see Find the active state.

Run a session, then confirm that the core wrote the override and which key it chose:

docker compose logs core | grep "Terraform state:"

Terraform state: rustfs bucket=kumoss-terraform-state key=<project_id>/terraform.tfstate

List the state bucket and check that the logged key exists after the first init. For the bundled RustFS store, see Operations below.

Operations

Inspect the bundled store. RustFS speaks the S3 API, so any S3 client works. From the host, through the nginx server on port 9000 (it forwards the Host header unchanged, so SigV4 signatures verify):

# One-off: the backend uses path-style addressing, so the client must too.
aws configure set default.s3.addressing_style path

AWS_ACCESS_KEY_ID=rustfsadmin AWS_SECRET_ACCESS_KEY=rustfsadmin \
AWS_DEFAULT_REGION=us-east-1 \
  aws --endpoint-url http://localhost:9000 \
  s3 ls s3://kumoss-terraform-state/ --recursive

Reset local state. docker compose down -v removes the object_storage_data volume and with it all state and all artifacts. That is usually what you want on a workstation and never what you want elsewhere.

In production, on the state bucket: versioning and soft delete on; encryption at rest with your own key if policy requires it; access restricted to the core and the IaC sidecar identities; excluded from any lifecycle expiry you apply to the artifacts bucket; and included in your backup and restore drills. This bucket is the record of what Kumoss has built, and Kumoss never deletes an object from it. See also Operations — Data services and persistence.

Next steps

Troubleshooting: boot, init, and empty-state failures.
Configuration reference — the storage section: every field used on this page.