Minor Postgresql Upgrade¶

there are 2 type of postgresql version upgrade 1. supervised upgrade, where it will automatically upgrade all replicas, but will not continue with the current master. either switchover to replica or restart the current master will finish the upgrade process 2. unsupervised upgrade, where controller with automatically upgrade all replicas and master. there are 2 primary upgrade type (primaryUpdateMethod) here 1. restart, if possible, perform an automated restart of the pod where the primary instance is running 2. switchover, a switchover operation is automatically performed, setting the most aligned replica as the new target primary, and shutting down the former primary pod.

Supervised Upgrade¶

create your postgresql cluster

kubectl apply -f cluster.yaml

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: echo-postgresql-upgrade-test
  namespace: cnpg-system
spec:
  imageName: ghcr.io/cloudnative-pg/postgresql:17.2
  instances: 2
  storage:
    size: 1Gi
  primaryUpdateStrategy: supervised

upgrade the cluster

kubectl apply -f cluster.yaml

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: echo-postgresql-upgrade-test
  namespace: cnpg-system
spec:
  imageName: ghcr.io/cloudnative-pg/postgresql:17.4
  instances: 2
  storage:
    size: 1Gi
  primaryUpdateStrategy: supervised

it will rollout restart replica but not primary, and waiting for us to switchover/delete the master pod.

echo-postgresql-upgrade-test-1             1/1     Running   0          4h22m
echo-postgresql-upgrade-test-2             1/1     Running   0          86s

zufar.dhiyaullhaq@MacBookPro 4-minor-postgresql-upgrade % k get cluster
NAME                           AGE     INSTANCES   READY   STATUS                     PRIMARY
echo-postgresql-upgrade-test   4h32m   2           2       Waiting for user action    echo-postgresql-upgrade-test-1

switchover the master, there will be downtime for 5-10s

kubectl cnpg promote echo-postgresql-upgrade-test echo-postgresql-upgrade-test-2

zufar.dhiyaullhaq@MacBookPro 4-minor-postgresql-upgrade % k get cluster                                                                   
NAME                           AGE     INSTANCES   READY   STATUS                                       PRIMARY
echo-postgresql                8d      5           5       Cluster in healthy state                     echo-postgresql-8
echo-postgresql-upgrade-test   4h37m   2           1       Waiting for the instances to become active   echo-postgresql-upgrade-test-2

after replica became master. the old master (which is replica right now) will be automatically restarted

echo-postgresql-upgrade-test-1             1/1     Running   0          66s
echo-postgresql-upgrade-test-2             1/1     Running   0          7m28s

Unsupervised Upgrade¶

I really recommed to use switchover method istead of restarting master since it's take more time for downtime

upgrade the cluster

kubectl apply -f cluster.yaml

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: echo-postgresql-upgrade-test
  namespace: cnpg-system
spec:
  imageName: ghcr.io/cloudnative-pg/postgresql:17.5
  instances: 2
  storage:
    size: 1Gi
  primaryUpdateStrategy: unsupervised
  primaryUpdateMethod: switchover

operator will upgrade the replica, switchover the master to the newly upgraded replica, and upgrade the old master automatically. there will be downtime for 5-10s during the switchover.

https://cloudnative-pg.io/documentation/1.26/rolling_update/#automated-updates-unsupervised https://cloudnative-pg.io/documentation/1.26/postgres_upgrades/

Learning¶

unsupervised upgrade with restart primaryUpdateMethod take more than 1 minutes downtime of primary
switchover is prefered instead since it only take 5-10s downtime

Issues¶

Upgrade to unknown version¶

when upgrading to unknown version, replica pod will start crashing due to image not found/etc. to fix this, just update the version to the correct version and delete the replica pod which is in error state

Manifests¶

cluster.yaml

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: echo-postgresql-upgrade-test
  namespace: cnpg-system
spec:
  imageName: ghcr.io/cloudnative-pg/postgresql:17.5
  instances: 2
  storage:
    size: 1Gi
    storageClass: gtf-ack-essd-pl0-wait
  ## supervised upgrade strategy ##
  # primaryUpdateStrategy: supervised
  ## unsupervised upgrade strategy ##
  primaryUpdateStrategy: unsupervised
  primaryUpdateMethod: switchover

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: echo-postgresql-upgrade
  namespace: cnpg-system
  labels:
    app: echo-postgresql-upgrade
spec:
  replicas: 1
  selector:
    matchLabels:
      app: echo-postgresql-upgrade
  template:
    metadata:
      labels:
        app: echo-postgresql-upgrade
    spec:
      containers:
      - name: echo-postgresql-upgrade
        image: ghcr.io/zufardhiyaulhaq/echo-postgresql:v1.0.0
        ports:
        - containerPort: 8080
          name: http
        env:
        - name: HTTP_PORT
          value: "8080"
        - name: POSTGRESQL_HOST
          value: "echo-postgresql-upgrade-test-rw.cnpg-system.svc.cluster.local"
        - name: POSTGRESQL_PORT
          value: "5432"
        - name: POSTGRESQL_DATABASE
          valueFrom:
            secretKeyRef:
              name: echo-postgresql-upgrade-test-app
              key: dbname
        - name: POSTGRESQL_USER
          valueFrom:
            secretKeyRef:
              name: echo-postgresql-upgrade-test-app
              key: user
        - name: POSTGRESQL_PASSWORD
          valueFrom:
            secretKeyRef:
              name: echo-postgresql-upgrade-test-app
              key: password
        resources:
          requests:
            cpu: "100m"
            memory: "128Mi"
          limits:
            cpu: "500m"
            memory: "512Mi"
---
apiVersion: v1
kind: Service
metadata:
  name: echo-postgresql-upgrade
  namespace: cnpg-system
spec:
  selector:
    app: echo-postgresql-upgrade
  ports:
    - protocol: TCP
      port: 8080
      targetPort: http
  type: ClusterIP