10. Quality Requirements

Table of Contents

1. Quality Tree
2. Quality Scenarios

Table 1. High level quality requirements
Requirement	Description
Reliability	The cluster and the applications hosted on it have to be available 24/7; there is no accepted maintenance window ([CP-001 No Maintenance Window ]). Stateful data has to be recoverable after node or volume loss.
Maintainability	Every change to the cluster has to be possible as a Git commit, reviewable and revertible, without direct cluster access ([CT-004 GitOps single source of truth ]). Operating the cluster has to remain feasible for a very small team ([CO-001 Small Team ]).
Security	There must not be a locally-managed user database; every login has to be delegated to the organization’s OpenID Connect provider ([CT-005 OpenID Connect only ]). Runtime secrets must not be stored in Git ([CT-007 Secrets via External Secrets ]).

ID	Scenario
RE01	Applications hosted on the cluster are used around the world and around the clock. The cluster must have no accepted downtime window for planned maintenance: node upgrades, certificate renewal ([RE02 Recoverability / Backup] \-adjacent, see Certificate Renewal) and Argo CD reconciliation must all happen without taking the ingress or the Kubernetes API offline. Multiple control-plane replicas, `MachineHealthCheck`-driven remediation ([CT-002 Kubernetes via Cluster API ]), and 2+ replicas for user-facing components (Traefik, Keycloak) support this.
RE02	Given a lost `PersistentVolume` or a lost PostgreSQL cluster (e.g. Keycloak’s database), a new cluster can be recovered from the last base backup plus archived WAL segments in S3-compatible object storage, with a recovery point objective of minutes rather than the interval between scheduled backups. See CloudNativePG Backup and Restore and Data Protection.
MT01	Given a merged commit to this repository, Argo CD applies the change to the cluster automatically within its polling/reconciliation interval, without any manual `kubectl apply`. Given an out-of-band manual change on the cluster, `selfHeal` reverts it on the next reconciliation loop. See GitOps Reconciliation.
MT02	Maintaining the cluster and its add-ons must be possible for a very small team (basically the full army of myself, me, and I — [CO-001 Small Team ]). This is achieved by keeping every operational task expressible as "edit a file in Git", by pinning chart versions explicitly ([CT-004 GitOps single source of truth ]), and by keeping infrastructure bootstrap (rarely touched) cleanly separated from day-to-day GitOps (frequently touched).
SE01	A user with a valid Keycloak account in realm `k8s` can authenticate to the Kubernetes API (`kubectl oidc-login`), Argo CD, Grafana and Headlamp using the same credentials, without any of these tools maintaining its own user database. Group membership in Keycloak alone determines authorization (e.g. Kubernetes RBAC via `oidc:cluster-admin`). See OIDC Login and Single Sign-On.
SE02	No runtime secret (passwords, API tokens, private keys) is ever committed to this repository. Every runtime secret is fetched at reconciliation/runtime by the External Secrets Operator or the Secrets Store CSI Driver from the Bitwarden Secrets Manager `ClusterSecretStore` ([CT-007 Secrets via External Secrets ]).

Applications hosted on the cluster are used around the world and around the clock. The cluster must have no accepted downtime window for planned maintenance: node upgrades, certificate renewal ([RE02 Recoverability / Backup] \-adjacent, see Certificate Renewal) and Argo CD reconciliation must all happen without taking the ingress or the Kubernetes API offline. Multiple control-plane replicas, MachineHealthCheck-driven remediation ([CT-002 Kubernetes via Cluster API ]), and 2+ replicas for user-facing components (Traefik, Keycloak) support this.

RE02

Given a lost PersistentVolume or a lost PostgreSQL cluster (e.g. Keycloak’s database), a new cluster can be recovered from the last base backup plus archived WAL segments in S3-compatible object storage, with a recovery point objective of minutes rather than the interval between scheduled backups. See CloudNativePG Backup and Restore and Data Protection.

MT01

Given a merged commit to this repository, Argo CD applies the change to the cluster automatically within its polling/reconciliation interval, without any manual kubectl apply. Given an out-of-band manual change on the cluster, selfHeal reverts it on the next reconciliation loop. See GitOps Reconciliation.

MT02

Maintaining the cluster and its add-ons must be possible for a very small team (basically the full army of myself, me, and I — [CO-001 Small Team ]). This is achieved by keeping every operational task expressible as "edit a file in Git", by pinning chart versions explicitly ([CT-004 GitOps single source of truth ]), and by keeping infrastructure bootstrap (rarely touched) cleanly separated from day-to-day GitOps (frequently touched).

SE01

A user with a valid Keycloak account in realm k8s can authenticate to the Kubernetes API (kubectl oidc-login), Argo CD, Grafana and Headlamp using the same credentials, without any of these tools maintaining its own user database. Group membership in Keycloak alone determines authorization (e.g. Kubernetes RBAC via oidc:cluster-admin). See OIDC Login and Single Sign-On.

SE02

No runtime secret (passwords, API tokens, private keys) is ever committed to this repository. Every runtime secret is fetched at reconciliation/runtime by the External Secrets Operator or the Secrets Store CSI Driver from the Bitwarden Secrets Manager ClusterSecretStore ([CT-007 Secrets via External Secrets ]).

10. Quality Requirements

1. Quality Tree

2. Quality Scenarios