137 lines
5.1 KiB
Markdown
137 lines
5.1 KiB
Markdown
# Quay on Kubernetes
|
|
|
|
Plain Kubernetes deployment for `https://registry.brunner.ninja` in the
|
|
`default` namespace. Quay updates are deliberately simple: change both matching
|
|
`quay.io/projectquay/quay` image references in `quay.yaml`, commit, and push to
|
|
`main`. Gitea Actions performs the Kubernetes update.
|
|
|
|
The deployment uses plain YAML. There is no Helm chart, Operator, Kustomize,
|
|
Argo CD, or custom container build.
|
|
|
|
## Persistent data
|
|
|
|
Quay itself has no persistent volume. Durable state remains in existing
|
|
external services:
|
|
|
|
- metadata, users, permissions, and tags: PostgreSQL database `quay` through
|
|
`postgresql.default.svc.cluster.local`;
|
|
- image layers and artifacts: Rook Ceph RGW bucket `quay`;
|
|
- events and build logs: the existing Valkey service in `default`.
|
|
|
|
Applying `quay.yaml` does not recreate or delete the PostgreSQL database or RGW
|
|
bucket. The `CephObjectStoreUser` declaration is kept separately in
|
|
`ceph-object-user.yaml` and is not managed by CI.
|
|
|
|
## Configuration Secret
|
|
|
|
The historical manifest stored the complete private Quay configuration in a
|
|
ConfigMap. The Deployment now mounts `default/secret/quay-config` instead.
|
|
|
|
On its first run, `install.sh` copies the existing ConfigMap directly into that
|
|
Secret without printing the configuration. After the new pods are healthy, the
|
|
script prints the command that can be used to remove the legacy plaintext
|
|
ConfigMap. It does not delete it automatically.
|
|
|
|
For future configuration changes, copy `config.yaml.example` to the ignored
|
|
`config.yaml`, fill every placeholder, and run `./install.sh`. Preserve the
|
|
existing `SECRET_KEY` and `DATABASE_SECRET_KEY`; they are persistent Quay
|
|
cryptographic keys and should not be regenerated during normal updates.
|
|
|
|
`DEFAULT_UI: angular` keeps the legacy logged-out landing page, including its
|
|
public repository browsing experience. Anonymous access remains controlled by
|
|
`FEATURE_ANONYMOUS_ACCESS: true`. The Angular UI is deprecated upstream, so a
|
|
future Quay release may require returning to the React UI.
|
|
|
|
An exported Kubernetes Secret YAML is not a safe public backup because its
|
|
values are merely base64 encoded. Keep an encrypted off-cluster copy of the
|
|
actual configuration in addition to PostgreSQL and RGW backups.
|
|
|
|
## Initial installation
|
|
|
|
With an administrator kubeconfig active, run:
|
|
|
|
```sh
|
|
./install.sh
|
|
```
|
|
|
|
This creates the configuration Secret when necessary, applies the existing
|
|
Rook object user and all Quay resources, and waits for both Deployments.
|
|
|
|
The registry Deployment has startup, readiness, and liveness probes. Both
|
|
Deployments use `Recreate`, accepting downtime and ensuring old and new Quay
|
|
versions do not access PostgreSQL concurrently.
|
|
|
|
## Automatic tag updates
|
|
|
|
To update Quay, change both identical `image:` fields in `quay.yaml`, for
|
|
example from a 3.8 image to the selected 3.9 image, then commit and push:
|
|
|
|
```sh
|
|
git add quay.yaml
|
|
git commit -m "Update Quay to 3.9.x"
|
|
git push origin main
|
|
```
|
|
|
|
The workflow validates every push and pull request. A successful push to
|
|
`main` performs a server-side dry run, applies the complete manifest, waits for
|
|
both Deployments, and checks the public `/health/endtoend` endpoint.
|
|
|
|
Kubernetes handles pod replacement through each Deployment's native `Recreate`
|
|
strategy. If the pod template did not change, applying the file does not restart
|
|
the pods. When an image reference or another pod-template field changes,
|
|
Kubernetes stops that Deployment's old pod before starting its replacement.
|
|
|
|
If the new registry cannot migrate or become healthy, CI fails and leaves the
|
|
registry stopped or unhealthy for investigation. It does not automatically
|
|
start the old image because a completed database migration might not be
|
|
backwards compatible.
|
|
|
|
Quay requires supported intermediate database migrations. Reach a distant
|
|
target using successive supported-version commits rather than skipping from
|
|
3.8 directly to 3.18. Kubernetes actions require no manual intervention, but a
|
|
Git commit cannot make an unsupported database-version jump safe.
|
|
|
|
Applying an unchanged pod template does not restart the registry, so a
|
|
README-only commit still verifies the live deployment without causing Quay's
|
|
`Recreate` downtime.
|
|
|
|
## Gitea Actions bootstrap
|
|
|
|
CI requires one repository Actions secret:
|
|
|
|
- `KUBE_CONFIG_BASE64`: restricted Quay deployment kubeconfig.
|
|
|
|
After `./install.sh` has created all application resources, run once:
|
|
|
|
```sh
|
|
./create-ci-kubeconfig.sh
|
|
```
|
|
|
|
For remote cluster access:
|
|
|
|
```sh
|
|
KUBECTL_SSH_HOST=arschrock ./create-ci-kubeconfig.sh
|
|
```
|
|
|
|
Store its final base64 line as the Gitea Actions secret
|
|
`KUBE_CONFIG_BASE64`. After that bootstrap, tag updates are Git-only.
|
|
|
|
The deployment identity can update only the existing Deployments `quay` and
|
|
`quay-repomirror`, Service `quay`, and Ingress `quay` in `default`. It cannot
|
|
read or modify Secrets, PostgreSQL, Valkey, Ceph resources, or the RGW bucket.
|
|
|
|
The workflow matches the existing brunner.ninja runner: Kubernetes API URL
|
|
`https://host.containers.internal:6443`, with TLS server name `192.168.0.2`.
|
|
|
|
## Local validation
|
|
|
|
Run:
|
|
|
|
```sh
|
|
./test.sh
|
|
```
|
|
|
|
The test checks shell syntax, requires both Deployments to use one identical
|
|
non-`latest` image reference, and validates the Kubernetes manifests with
|
|
kubeconform.
|