16. Cheat sheet

Everything worth having to hand, in one place.

The five facts

1. An APPLICATION says: this git path, rendered this
   way, belongs in this cluster and namespace.

2. SYNC STATUS = "does live match git?"
   HEALTH       = "is it working?"
   THEY ARE INDEPENDENT. Synced ≠ working.

3. Argo CD runs `helm template`, NEVER `helm install`.
   There are no releases and no `helm rollback`.

4. WAVES wait for HEALTH. A resource that never becomes
   healthy blocks the sync forever.

5. WEBHOOKS. Without one you poll every 3 minutes, and
   that is why people think Argo CD is slow.

CLI

argocd login argocd.acme.com --sso
argocd app list
argocd app get checkout
argocd app get checkout --show-operation    # what the last sync did
argocd app diff checkout
argocd app diff checkout --local ./apps/checkout
argocd app history checkout
argocd app resources checkout
argocd app logs checkout --follow

argocd app sync checkout
argocd app sync checkout --prune
argocd app sync checkout --dry-run
argocd app sync checkout --resource apps:Deployment:checkout
argocd app sync checkout --force            # kubectl replace — DOWNTIME
argocd app rollback checkout 12
argocd app terminate-op checkout            # cancel a stuck sync
argocd app wait checkout --health --sync --timeout 600

argocd app set checkout --sync-policy none  # ← THE INCIDENT COMMAND
argocd app set checkout --sync-policy automated --self-heal --auto-prune
argocd app delete checkout --cascade=false  # keep the workloads

argocd repo add https://github.com/acme/x.git --username git --password $T
argocd cluster add prod-eu --name prod-eu   # grants CLUSTER-ADMIN
argocd proj list
argocd admin export > backup.yaml
argocd admin settings resource-overrides health ./cr.yaml --argocd-cm-path ./cm.yaml

A production Application

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: checkout
  namespace: argocd
  finalizers: [resources-finalizer.argocd.argoproj.io]
spec:
  project: shop
  source:
    repoURL: https://github.com/acme/manifests.git
    targetRevision: main
    path: apps/checkout
  destination:
    name: prod-eu
    namespace: shop
  syncPolicy:
    automated: { prune: true, selfHeal: true, allowEmpty: false }
    syncOptions:
      - CreateNamespace=true
      - PruneLast=true
      - ApplyOutOfSyncOnly=true
      - ServerSideApply=true
    retry:
      limit: 5
      backoff: { duration: 5s, factor: 2, maxDuration: 3m }
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jsonPointers: [/spec/replicas]      # the HPA owns this

Annotations

argocd.argoproj.io/sync-wave: "-1"        # order; default 0

argocd.argoproj.io/hook: PreSync          # PreSync|Sync|PostSync|SyncFail|Skip
argocd.argoproj.io/hook-delete-policy: HookSucceeded
                                          # |HookFailed|BeforeHookCreation(default)

argocd.argoproj.io/sync-options: Prune=false        # never auto-delete
argocd.argoproj.io/sync-options: Delete=false       # survive app deletion
argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
argocd.argoproj.io/sync-options: Validate=false

argocd.argoproj.io/compare-options: IgnoreExtraneous

notifications.argoproj.io/subscribe.on-health-degraded.slack: alerts

Sync options

Option	Effect
`CreateNamespace=true`	create the destination namespace
`PruneLast=true`	delete after everything else is healthy
`ApplyOutOfSyncOnly=true`	send only changed resources — big win
`ServerSideApply=true`	fixes the 256 KB annotation limit; tracks ownership
`RespectIgnoreDifferences=true`	do not sync ignored fields
`Replace=true`	`kubectl replace` — last resort
`FailOnSharedResource=true`	refuse if another app owns a resource
`Validate=false`	skip schema validation

Status combinations

Sync	Health	Meaning
Synced	Healthy	correct
Synced	Progressing	rolling out — wait
Synced	Degraded	git describes something broken. Re-sync will not help
OutOfSync	Healthy	drift, or not yet synced
OutOfSync	Degraded	broken and wrong
Synced	Missing	something deleted it
any	Unknown	a Lua health check errored

AppProject — the security boundary

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata: { name: shop, namespace: argocd }
spec:
  sourceRepos:
    - https://github.com/acme/shop-manifests.git
  destinations:
    - { name: prod-eu, namespace: 'shop-*' }
  clusterResourceWhitelist:              # EMPTY BY DEFAULT = none allowed
    - { group: '', kind: Namespace }
  namespaceResourceBlacklist:
    - { group: rbac.authorization.k8s.io, kind: ClusterRoleBinding }  # ESSENTIAL
  syncWindows:
    - kind: deny
      schedule: '0 0 * * *'
      duration: 24h
      applications: ['*-prod']
      manualSync: true                   # or you cannot deploy in an incident

RBAC

# argocd-rbac-cm
data:
  policy.default: role:readonly          # NEVER role:admin
  policy.csv: |
    p, role:shop-admin, applications, *, shop/*, allow
    p, role:shop-admin, applications, delete, shop/*, deny
    p, role:shop-admin, exec, create, shop/*, deny
    g, acme:shop-leads, role:shop-admin

Deny always wins. exec and logs are separate, dangerous permissions.

Repository Secret

apiVersion: v1
kind: Secret
metadata:
  name: repo-manifests
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository    # ← WITHOUT THIS, NOTHING
stringData:
  type: git
  url: https://github.com/acme/manifests.git
  githubAppID: "123456"
  githubAppInstallationID: "7891011"
  githubAppPrivateKey: |
    ...

secret-type: repo-creds + a URL prefix = a credential template (longest prefix wins). OCI: oci:// in the Application, no prefix in the Secret.

Settings that matter

# argocd-cm
data:
  application.resourceTrackingMethod: annotation    # NOT the default label
  timeout.reconciliation: 300s                      # after webhooks work
  resource.exclusions: |
    - apiGroups: ["*"]
      kinds: [Event, EndpointSlice, Endpoints]
      clusters: ["*"]
  admin.enabled: "false"                            # once SSO works
  exec.enabled: "false"                             # leave it off

# argocd-cmd-params-cm
data:
  controller.replicas: "3"
  controller.sharding.algorithm: round-robin
  reposerver.parallelism.limit: "10"
  application.namespaces: "team-*"                  # + project sourceNamespaces

# and the StatefulSet — FORGETTING THIS SILENTLY STRANDS CLUSTERS
env:
  - name: ARGOCD_CONTROLLER_REPLICAS
    value: "3"

Metrics

Metric	Alert on
`argocd_app_info{health_status="Degraded"}`	`== 1 for: 15m`
`argocd_app_info{sync_status="OutOfSync"}`	`== 1 for: 1h`
`argocd_app_reconcile` p99	`> 10s` — the scaling signal
`argocd_app_sync_total{phase="Failed"}`	any rate
`argocd_cluster_connection_status`	`== 0`
`argocd_git_request_total`	rising → rate limiting

Alert on duration, never instantaneous state.

ApplicationSet safety

spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]   # a typo must FAIL, not render ""
  syncPolicy:
    applicationsSync: create-update         # never auto-delete in production
    preserveResourcesOnDeletion: true

Generators: list, clusters, git directories (adding a directory deploys), git files (usually best), matrix, merge, pullRequest, scmProvider.

Diagnosis order

1. argocd app get <app>                    status, health, last error
2. argocd app get <app> --show-operation   which phase, which resource
3. argocd app diff <app>                   what differs
4. argocd app history <app>                when did it last work
5. kubectl describe <failing resource>     the REAL Kubernetes error
6. kubectl logs -n argocd deploy/argocd-repo-server        render failures
   kubectl logs -n argocd sts/argocd-application-controller  reconciliation

Incident procedures

STOP SELF-HEAL REVERTING MY FIX
  argocd app set <app> --sync-policy none
  ... fix ... then commit and re-enable
  (PUT THIS IN THE RUNBOOK BEFORE YOU NEED IT)

STUCK SYNC
  argocd app get <app> --show-operation
  argocd app terminate-op <app>
  kubectl describe <the blocked resource>

ARGO CD BROKE ITSELF
  git revert <bad commit> && git push
  helm template argocd argo/argo-cd -f values.yaml | kubectl apply -f -

ACCIDENTAL DELETION
  re-sync from git — stateless comes back
  PVCs may not: check reclaimPolicy

Do not

✗ leave Applications in the `default` project
✗ ship an AppProject without clusterResourceWhitelist
  and a ClusterRoleBinding blacklist
✗ use a Helm semver range in production
  (desired state changes with no commit)
✗ use :latest, or any mutable tag
✗ enable exec, or expose the UI publicly
✗ keep the local admin account after SSO works
✗ put Secrets in git
✗ use Image Updater's write-back-method: argocd in prod
✗ delete an ApplicationSet casually
✗ ignoreDifferences on broad paths to silence noise
✗ run without webhooks at scale (git rate limits)
✗ forget ARGOCD_CONTROLLER_REPLICAS when sharding
✗ set a CPU limit on the application controller
✗ manage Kubernetes resources with Terraform's k8s provider

The pre-production checklist

☐ webhooks configured
☐ resourceTrackingMethod: annotation
☐ ServerSideApply=true, ApplyOutOfSyncOnly=true
☐ AppProjects with destinations + cluster whitelist
   + ClusterRoleBinding blacklist
☐ SSO working, groups claim present, local admin off
☐ exec.enabled: false
☐ Events excluded from the resource cache
☐ a secrets answer (ESO recommended)
☐ PVCs annotated Prune=false,Delete=false
☐ ApplicationSets on applicationsSync: create-update
☐ notifications on failure only
☐ alerts on Degraded 15m / OutOfSync 1h / reconcile p99
☐ HA manifests, not the default install
☐ the "stop self-heal" command in the runbook

That is Argo CD. It sits directly on top of Kubernetes and reads its object model constantly; it is the natural counterpart to Terraform, which owns everything outside the cluster; and the Vault guide answers the one problem it deliberately leaves open.

Written by Sepuri Sai Krishna