Argo CD

Overview

Point an Argo CD Application at helm/garrison in this repository. Argo syncs chart releases when Git (or image tags via Helm parameters) changes.

Chart files in the repo:

File	Purpose
helm/garrison/values.yaml	Default values
helm/garrison/values-production.example.yaml	Production overrides — copy and customize
helm/garrison/argocd-application.example.yaml	Example Application manifest

First deploy checklist

1. Cluster — Kubernetes with a Gateway API controller (for example NGINX Gateway Fabric, Cilium Gateway, or Istio) and storage class for Postgres PVCs.
2. Registry — CI-built images at registry.gitlab.com/churchtechhelp/garrison/{api,crdt-merge,ui,www}. Pin tags to $CI_COMMIT_SHORT_SHA rather than latest in production.
3. Registry auth — If the registry is private, create an imagePullSecret in the garrison namespace and set global.imagePullSecrets in your values.
4. Secrets — Create garrison-secrets from secrets.env.example before the first sync (see Secrets).
5. TLS — Configure TLS on your cluster Gateway listener (for example cert-manager Certificate referenced by the Gateway). The chart attaches HTTPRoute resources to that Gateway; it does not create the Gateway itself.
6. UI image — Build the UI with VITE_API_URL set to your public API hostname; the chart does not inject API URL at runtime.
7. Values — Copy values-production.example.yaml, customize it, and wire it into your Argo CD Application (see Production values).
8. Apply — Create the Argo CD Application (example below or helm/garrison/argocd-application.example.yaml).
9. Verify — Run the Production smoke checklist against your public API URL.

Production values

helm/garrison/values-production.example.yaml is a template, not something you edit in place on the public repo. Copy it, customize for your cluster, and reference the copy from Argo CD.

1. Copy and customize

On your workstation (or in a private Git repo):

cp helm/garrison/values-production.example.yaml values-production.yaml

Edit values-production.yaml. Typical changes:

Field	What to set
global.imageRegistry	Your registry (for example registry.gitlab.com/churchtechhelp/garrison)
api.image.tag, ui.image.tag, crdtMerge.image.tag	CI image tags — pin to $CI_COMMIT_SHORT_SHA, not latest
httpRoute.parentRefs	Name, namespace, and sectionName of your cluster Gateway HTTPS listener
httpRoute.hosts	Public hostnames for API, app, and optional www
api.existingSecret	Name of the Kubernetes Secret you created (default garrison-secrets)
global.imagePullSecrets	Uncomment if the registry is private

Do not put Laravel env vars (APP_URL, MAIL_*, SESSION_*, etc.) in this file when using api.existingSecret — keep them in the Kubernetes Secret (see Secrets). Only APP_KEY and DB_PASSWORD at minimum; see helm/garrison/secrets.env.example for the full list.

2. Store the file for Argo CD

Argo CD reads Helm values from Git. Pick one approach:

Option A — Private values repo (recommended)

Push values-production.yaml to a private repository (for example garrison-deploy-config). Use a multi-source Application: the chart comes from this repo; values come from the private repo.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: garrison
  namespace: argocd
spec:
  project: default
  sources:
    - repoURL: https://gitlab.com/ChurchTechHelp/garrison.git
      targetRevision: main
      path: helm/garrison
      helm:
        releaseName: garrison
        valueFiles:
          - values.yaml
          - $values/values-production.yaml
    - repoURL: https://gitlab.com/your-org/garrison-deploy-config.git
      targetRevision: main
      ref: values
  destination:
    server: https://kubernetes.default.svc
    namespace: garrison
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Register both repos in Argo CD. After you change values-production.yaml in the private repo, sync the Application (or rely on automated sync).

Option B — Helm parameters on the Application

Keep only values.yaml from the chart repo and override production fields with spec.source.helm.parameters (as in the example Application below). Works for a few keys; awkward for long env blocks or httpRoute.parentRefs.

Option C — Plain Helm (no Argo CD)

helm upgrade --install garrison ./helm/garrison \
  --namespace garrison --create-namespace \
  -f values-production.yaml

See Kubernetes for the full install flow.

3. Upgrades

Migrations run automatically in API, queue, and scheduler init containers on every rollout (php artisan migrate --force --isolated). Bump image tags via values or Argo CD parameters; no separate migration toggle is required.

Commit or push value changes to the repo Argo CD watches, then sync. Image tag bumps alone can be done via parameters without touching the values file.

Example Application

Replace repoURL and paths with your Git remote. Store secrets outside Git (e.g. Argo CD sealed secrets, external secret operator, or a private values file).

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: garrison
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://gitlab.com/ChurchTechHelp/garrison.git
    targetRevision: main
    path: helm/garrison
    helm:
      releaseName: garrison
      valueFiles:
        - values.yaml
        - values-production.example.yaml
      parameters:
        - name: global.imageRegistry
          value: registry.gitlab.com/churchtechhelp/garrison
        - name: api.image.tag
          value: latest
        - name: crdtMerge.image.tag
          value: latest
        - name: ui.image.tag
          value: latest
        - name: api.existingSecret
          value: garrison-secrets
        - name: httpRoute.enabled
          value: "true"
        - name: httpRoute.parentRefs[0].name
          value: external-gateway
        - name: httpRoute.parentRefs[0].namespace
          value: gateway-system
        - name: httpRoute.parentRefs[0].sectionName
          value: https
        # Marketing site is optional (default www.enabled: false):
        # - name: www.enabled
        #   value: "true"
        # - name: www.image.tag
        #   value: latest
  destination:
    server: https://kubernetes.default.svc
    namespace: garrison
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

The example below uses the in-repo values-production.example.yaml for illustration. In production, replace that entry with your customized file from a private values repo ($values/values-production.yaml) or drop it and use parameters only.

Gateway API (HTTPRoute)

Production examples use Gateway API HTTPRoute resources instead of Ingress. The chart creates one route per hostname (API, UI, optional www) and attaches them to an existing cluster Gateway via httpRoute.parentRefs.

The chart does not create the Gateway — only HTTPRoute resources that reference it. TLS termination and certificates are configured on the Gateway listener, not in this chart.

Legacy Ingress is still available (ingress.enabled: true); enable one exposure method only.

Finding httpRoute.parentRefs values

parentRefs must point at a Gateway that already exists in your cluster. The example values (external-gateway, gateway-system, https) are placeholders.

Field	Source on the cluster Gateway
parentRefs[0].name	metadata.name
parentRefs[0].namespace	metadata.namespace
parentRefs[0].sectionName	spec.listeners[].name for the HTTPS listener (port 443)

List Gateways — pick the one that should receive public traffic for Garrison:

kubectl get gateway -A

List listener names on that Gateway (name, protocol, port):

kubectl get gateway <gateway-name> -n <gateway-namespace> \
  -o jsonpath='{range .spec.listeners[*]}{.name}{" ("}{.protocol}{" :"}{.port}{")\n"}{end}'

Use the HTTPS listener’s name as sectionName (often https, but confirm from the output).

In values (values-production.yaml):

httpRoute:
  enabled: true
  parentRefs:
    - name: my-actual-gateway-name
      namespace: my-gateway-namespace
      sectionName: https

As Argo CD parameters (same three values):

- name: httpRoute.parentRefs[0].name
  value: my-actual-gateway-name
- name: httpRoute.parentRefs[0].namespace
  value: my-gateway-namespace
- name: httpRoute.parentRefs[0].sectionName
  value: https

If no Gateways appear, install a Gateway API controller and create a Gateway first (NGINX Gateway Fabric, Cilium Gateway, Istio, Envoy Gateway, etc.). Check that CRDs exist:

kubectl api-resources | grep gateway.networking.k8s.io

After sync, confirm routes attached to the parent Gateway:

kubectl get httproute -n garrison
kubectl describe httproute -n garrison

If a route is not accepted, describe usually reports a parentRef mismatch (wrong sectionName, namespace, or hostname rules on the Gateway listener).

Secrets

Do not commit production APP_KEY, database passwords, or other Laravel env to Git. The chart loads all keys from the Kubernetes Secret named in api.existingSecret (default in values-production.example.yaml).

Create garrison-secrets from a file (recommended)

Keep secrets in a local file that is never committed (chmod 600). Argo CD does not need this file — create the Secret in the cluster once with kubectl.

Start from helm/garrison/secrets.env.example in the repository. Customize domains, mail settings, and credentials.

Env file — secrets.env:

APP_KEY=base64:your-generated-key-here
APP_URL=https://api.garrisonbible.com
SESSION_DOMAIN=.garrisonbible.com
SANCTUM_STATEFUL_DOMAINS=app.garrisonbible.com
FRONTEND_URL=https://app.garrisonbible.com
CORS_ALLOWED_ORIGINS=https://app.garrisonbible.com
MAIL_MAILER=smtp
MAIL_HOST=smtp.example.com
MAIL_PORT=587
MAIL_SCHEME=tls
MAIL_USERNAME=your-smtp-user
MAIL_PASSWORD=your-smtp-password
MAIL_FROM_ADDRESS=noreply@garrisonbible.com
DB_PASSWORD=your-strong-db-password

kubectl create namespace garrison
kubectl create secret generic garrison-secrets -n garrison \
  --from-env-file=secrets.env

Separate key files — useful if you store each value in its own file:

kubectl create secret generic garrison-secrets -n garrison \
  --from-file=APP_KEY=./app-key.txt \
  --from-file=DB_PASSWORD=./db-password.txt

Generate APP_KEY with php artisan key:generate --show from a local Laravel install. Use the same DB_PASSWORD in the Secret that Postgres will use (postgres.auth.password is empty in the production example, so Postgres reads DB_PASSWORD from this Secret).

The chart ConfigMap still sets in-cluster service wiring (DB_HOST, REDIS_HOST, NOTE_CRDT_MERGE_URL, etc.) and overrides the Secret when both define the same key.

Verify:

kubectl get secret garrison-secrets -n garrison
kubectl describe secret garrison-secrets -n garrison   # shows keys, not values

To rotate credentials, delete and recreate (or use your cluster’s sealed-secrets / external-secrets workflow):

kubectl delete secret garrison-secrets -n garrison
kubectl create secret generic garrison-secrets -n garrison --from-env-file=secrets.env

Wire the Secret into the chart

In your production values file:

api:
  existingSecret: garrison-secrets

Argo CD does not create this Secret; it must exist in the garrison namespace before the first sync. The chart loads all Secret keys into the API, queue, scheduler, and bundled Postgres (for DB_PASSWORD).

Other secret options

• Argo CD secret management (Sealed Secrets, External Secrets Operator)
• api.secret.appKey / postgres.auth.password in Helm values (avoid in production Git)

Registry pull credentials (GitLab images)

CI pushes app images to registry.gitlab.com/churchtechhelp/garrison/{api,crdt-merge,ui,www}. GitLab Container Registry is private for private projects — the cluster needs pull credentials even when garrison-secrets (Laravel/DB) is configured.

Postgres (postgres:16-alpine) and Redis (redis:7-alpine) pull from Docker Hub and do not use global.imagePullSecrets. If only app pods fail, fix registry auth below; if Postgres/Redis fail, check cluster access to docker.io (firewall, rate limits, or image mirror).

1. See which image failed

kubectl get pods -n garrison
kubectl describe pod -n garrison <pod-name> | tail -20

Look for Failed to pull image and the full image reference (for example registry.gitlab.com/churchtechhelp/garrison/api:latest).

2. Confirm the tag exists in GitLab → Deploy → Container Registry. If latest is missing, CI may not have run on main yet — pin api.image.tag, ui.image.tag, and crdtMerge.image.tag to a SHA tag that exists.

3. Create a pull Secret in the garrison namespace.

GitLab deploy token with read_registry (or your username + personal access token with read_registry):

kubectl create secret docker-registry garrison-registry -n garrison \
  --docker-server=registry.gitlab.com \
  --docker-username='<token-username>' \
  --docker-password='<token-password>'

Or from a local Docker config file (chmod 600, never commit):

kubectl create secret docker-registry garrison-registry -n garrison \
  --from-file=.dockerconfigjson=./gitlab-registry-dockerconfig.json

4. Wire it into Helm so API, queue, scheduler, UI, and CRDT merge pods use it.

In values:

global:
  imagePullSecrets:
    - name: garrison-registry

Or as Argo CD parameters (when using parameters instead of a values file):

- name: global.imagePullSecrets[0].name
  value: garrison-registry

Sync the Application after creating the Secret and adding the parameter.

Image promotion

After CI pushes $CI_COMMIT_SHORT_SHA tags:

1. Update Helm parameters or a small production-values.yaml in a secure repo with the new tag, or
2. Use Argo CD Image Updater (optional).

Argo CD Image Updater

Image Updater polls the registry and patches your Application’s Helm parameters (it does not watch Git or registry pushes by itself). With syncPolicy.automated, Argo CD rolls out after parameters change.

Install (controller in the Argo CD namespace):

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj-labs/argocd-image-updater/stable/config/install.yaml
kubectl rollout status deployment/argocd-image-updater-controller -n argocd

Registry credentials — reuse your cluster pull secret so Image Updater can list tags:

kubectl patch configmap argocd-image-updater-config -n argocd --type merge -p '
data:
  registries.conf: |
    registries:
      - name: GitLab Garrison
        prefix: registry.gitlab.com
        api_url: https://registry.gitlab.com
        credentials: pullsecret:garrison/garrison-registry
  interval: 2m
'
kubectl rollout restart deployment/argocd-image-updater-controller -n argocd

ImageUpdater CR — keep global.imageRegistry and short repository names (api, ui, …) on the Application. Map Helm name + tag parameters per image. Use newest-build and ignore latest so CI’s $CI_COMMIT_SHORT_SHA tags are picked up:

kubectl apply -n argocd -f - <<'EOF'
apiVersion: argocd-image-updater.argoproj.io/v1alpha1
kind: ImageUpdater
metadata:
  name: garrison
  namespace: argocd
spec:
  namespace: argocd
  writeBackConfig:
    method: argocd
  commonUpdateSettings:
    updateStrategy: newest-build
    ignoreTags:
      - latest
    pullSecret: pullsecret:garrison/garrison-registry
  applicationRefs:
    - namePattern: garrison
      images:
        - alias: api
          imageName: registry.gitlab.com/churchtechhelp/garrison/api:latest
          manifestTargets:
            helm:
              name: api.image.repository
              tag: api.image.tag
        - alias: ui
          imageName: registry.gitlab.com/churchtechhelp/garrison/ui:latest
          manifestTargets:
            helm:
              name: ui.image.repository
              tag: ui.image.tag
        - alias: crdtmerge
          imageName: registry.gitlab.com/churchtechhelp/garrison/crdt-merge:latest
          manifestTargets:
            helm:
              name: crdtMerge.image.repository
              tag: crdtMerge.image.tag
        - alias: www
          imageName: registry.gitlab.com/churchtechhelp/garrison/www:latest
          manifestTargets:
            helm:
              name: www.image.repository
              tag: www.image.tag
EOF

The chart’s image helper treats repository values that contain / as already registry-qualified (for example after Image Updater writes registry.gitlab.com/churchtechhelp/garrison/api). It will not prepend global.imageRegistry again, so doubled paths like registry.../garrison/registry.../garrison/api cannot occur.

Reverb, queue, and scheduler all use the same api.image.repository and api.image.tag Helm values. Image Updater only needs to track the api image — when api.image.tag changes, Argo CD rolls out every workload that shares that image, including Reverb.

Application parameters should still use short repository names:

- name: global.imageRegistry
  value: registry.gitlab.com/churchtechhelp/garrison
- name: api.image.repository
  value: api
- name: api.image.tag
  value: <sha>

Enable www on the Application if you track that image: www.enabled: "true".

Verify:

kubectl get imageupdater garrison -n argocd
kubectl logs -n argocd deployment/argocd-image-updater-controller --tail=50

Recover from a bad image path — reset repositories to short names and a known-good tag, then sync:

argocd app set garrison -n argocd \
  --parameter api.image.repository=api \
  --parameter ui.image.repository=ui \
  --parameter crdtMerge.image.repository=crdt-merge \
  --parameter www.image.repository=www \
  --parameter api.image.tag=YOUR_SHA
argocd app sync garrison -n argocd

Sync and health

• API readiness and liveness use HTTP /up on port 80.
• Argo CD sync waves order the rollout (lower numbers sync first):
  1. Postgres PVC (-20)
  2. ConfigMap, chart-managed Secret, and Postgres/Redis Services (-15)
  3. Postgres and Redis Deployments (-10)
  4. CRDT merge worker (5)
  5. API, queue, scheduler, UI, and their Services (10) — init containers wait for Postgres/Redis, then run migrate --isolated
  6. HTTPRoutes (20)
• API, queue, and scheduler pods wait for in-cluster Postgres (and Redis where needed) via init containers, then apply pending migrations before the main container starts.

Optional components

Component	Values key	Default
Scheduler	scheduler.enabled	false in values.yaml; true in values-production.example.yaml
Marketing site	www.enabled	false
External Postgres	postgres.enabled: false, externalDatabase.enabled: true	bundled Postgres
External Redis	redis.enabled: false, externalRedis.enabled: true	bundled Redis

See also

• Kubernetes — Helm install, external DB/Redis, upgrades
• Production smoke checklist — post-deploy verification