Quay on Kubernetes

Plain Kubernetes deployment for https://registry.brunner.ninja in the default namespace. Quay updates are deliberately simple: change both matching quay.io/projectquay/quay image references in quay.yaml, commit, and push to main. Gitea Actions performs the Kubernetes update.

The deployment uses plain YAML. There is no Helm chart, Operator, Kustomize, Argo CD, or custom container build.

Persistent data

Quay itself has no persistent volume. Durable state remains in existing external services:

  • metadata, users, permissions, and tags: PostgreSQL database quay through postgresql.default.svc.cluster.local;
  • image layers and artifacts: Rook Ceph RGW bucket quay;
  • events and build logs: the existing Valkey service in default.

Applying quay.yaml does not recreate or delete the PostgreSQL database or RGW bucket. The CephObjectStoreUser declaration is kept separately in ceph-object-user.yaml and is not managed by CI.

Configuration Secret

The historical manifest stored the complete private Quay configuration in a ConfigMap. The Deployment now mounts default/secret/quay-config instead.

On its first run, install.sh copies the existing ConfigMap directly into that Secret without printing the configuration. After the new pods are healthy, the script prints the command that can be used to remove the legacy plaintext ConfigMap. It does not delete it automatically.

For future configuration changes, copy config.yaml.example to the ignored config.yaml, fill every placeholder, and run ./install.sh. Preserve the existing SECRET_KEY and DATABASE_SECRET_KEY; they are persistent Quay cryptographic keys and should not be regenerated during normal updates.

DEFAULT_UI: angular keeps the legacy logged-out landing page, including its public repository browsing experience. Anonymous access remains controlled by FEATURE_ANONYMOUS_ACCESS: true. The Angular UI is deprecated upstream, so a future Quay release may require returning to the React UI.

An exported Kubernetes Secret YAML is not a safe public backup because its values are merely base64 encoded. Keep an encrypted off-cluster copy of the actual configuration in addition to PostgreSQL and RGW backups.

Initial installation

With an administrator kubeconfig active, run:

./install.sh

This creates the configuration Secret when necessary, applies the existing Rook object user and all Quay resources, and waits for both Deployments.

The registry Deployment has startup, readiness, and liveness probes. Both Deployments use Recreate, accepting downtime and ensuring old and new Quay versions do not access PostgreSQL concurrently.

Automatic tag updates

To update Quay, change both identical image: fields in quay.yaml, for example from a 3.8 image to the selected 3.9 image, then commit and push:

git add quay.yaml
git commit -m "Update Quay to 3.9.x"
git push origin main

The workflow triggered by a quay.yaml change validates the manifest, performs a server-side dry run, applies the complete file, waits for both Deployments, and checks the public /health/endtoend endpoint.

Kubernetes handles pod replacement through each Deployment's native Recreate strategy. If the pod template did not change, applying the file does not restart the pods. When an image reference or another pod-template field changes, Kubernetes stops that Deployment's old pod before starting its replacement.

If the new registry cannot migrate or become healthy, CI fails and leaves the registry stopped or unhealthy for investigation. It does not automatically start the old image because a completed database migration might not be backwards compatible.

Quay requires supported intermediate database migrations. Reach a distant target using successive supported-version commits rather than skipping from 3.8 directly to 3.18. Kubernetes actions require no manual intervention, but a Git commit cannot make an unsupported database-version jump safe.

The workflow runs only when quay.yaml or the workflow itself changes, so a README-only commit does not restart the registry.

Gitea Actions bootstrap

CI requires one repository Actions secret:

  • KUBE_CONFIG_BASE64: restricted Quay deployment kubeconfig.

After ./install.sh has created all application resources, run once:

./create-ci-kubeconfig.sh

Store its final base64 line as the Gitea Actions secret KUBE_CONFIG_BASE64. After that bootstrap, tag updates are Git-only.

The deployment identity can update only the existing Deployments quay and quay-repomirror, Service quay, and Ingress quay in default. It cannot read or modify Secrets, PostgreSQL, Valkey, Ceph resources, or the RGW bucket.

The workflow matches the existing brunner.ninja runner: Kubernetes API URL https://host.containers.internal:6443, with TLS server name 192.168.0.2.

Local validation

Run:

./test.sh

The test checks shell syntax, requires both Deployments to use one identical non-latest image reference, and validates the Kubernetes manifests with kubeconform.

S
Description
Config files to run quay in kubernetes
Readme GPL-3.0
72 KiB
Languages
Shell 100%