The ACME protocol is the reason most sysadmins no longer set calendar reminders for certificate renewal. If you run anything from a single VPS to a few hundred Kubernetes ingresses, knowing how automated issuance works under the hood makes the difference between a renewal job that heals itself and one that fails quietly for 60 days. This deep dive covers the full ACME message flow, the three challenge types, rate limits, renewal timing and where automation breaks in real deployments.
Where ACME Came From and What It Standardizes
ACME (Automatic Certificate Management Environment) was built by the Internet Security Research Group for Let’s Encrypt and published as RFC 8555 in March 2019. Before ACME, getting a certificate meant generating a CSR, pasting it into a web form, clicking a link in a validation email and copying files onto a server by hand.
ACME replaces all of that with a JSON-over-HTTPS API. Every request is signed with an account key using JWS (JSON Web Signature), so the CA always knows which account is asking. Let’s Encrypt, ZeroSSL, Google Trust Services, Sectigo and DigiCert all offer ACME endpoints today. That makes the protocol vendor-neutral, not a Let’s Encrypt feature.
The ACME Issuance Flow Step by Step
A client such as certbot, acme.sh, lego, Caddy or cert-manager goes through the same sequence every time:
1. Fetch the directory and a nonce. The client reads the CA’s directory URL to find endpoints, then requests a fresh anti-replay nonce from newNonce.
2. Register or reuse an account. A POST to newAccount binds a public key to the account. Lose that key and you lose the ability to revoke through that account.
3. Create an order. The client lists the identifiers it wants (example.com, www.example.com). The CA returns one authorization per identifier, each with a set of challenges.
4. Complete challenges and poll. The client provisions a token, tells the CA it is ready, and polls until each authorization becomes “valid” or “invalid”.
5. Finalize and download. The client submits a CSR to the finalize URL, waits for the order to go “valid” and downloads the certificate chain in PEM format.
Authorizations are cached for a while (Let’s Encrypt reuses them for up to 30 days), so renewals often skip straight from step 3 to step 5. This explains a confusing pattern: a broken challenge setup can go unnoticed through one renewal and fail on the next.
HTTP-01, DNS-01 and TLS-ALPN-01 Compared
HTTP-01 asks you to serve a token at http://yourdomain/.well-known/acme-challenge/TOKEN over port 80. It is the simplest option and what certbot’s –webroot and –nginx plugins use. It cannot issue wildcard certificates, and it needs port 80 reachable from the internet. Redirects to HTTPS are fine, since the CA follows them.
DNS-01 requires a TXT record at _acme-challenge.yourdomain. It is the only challenge that supports wildcards, and it works for hosts that are never exposed publicly, such as internal APIs or staging clusters. The cost is a DNS provider API token sitting somewhere in your infrastructure, plus propagation delays that can run from 10 seconds on Cloudflare or Route 53 to several minutes on slower providers.
TLS-ALPN-01 (RFC 8737) validates over port 443 using a special self-signed certificate negotiated with the acme-tls/1 ALPN protocol. Caddy and Traefik support it natively. It suits environments where port 80 is closed by policy, but most CDNs and TLS-terminating load balancers do not pass it through.
Since 2025, public CAs must use multi-perspective issuance corroboration, meaning validation requests come from several network locations. A firewall that only allows one CA IP range, or geo-blocking that drops traffic from outside your country, now breaks HTTP-01 in ways that used to work.
Myth: “ACME Is Set-and-Forget”
The most persistent misconception is that installing certbot equals solved renewals. ACME automates issuance. It does not guarantee that the new certificate reaches the process actually serving traffic, and it does not tell you when the renewal job stopped running.
A typical scenario: a team moves a site behind a CDN proxy or a new load balancer. The certbot systemd timer keeps firing twice a day, but HTTP-01 requests now land on the CDN, not the origin, and every attempt fails. Certbot logs the error to /var/log/letsencrypt/ and nobody reads it. Let’s Encrypt shut down its expiration reminder emails in June 2025, so that safety net is gone too. The first signal is a browser warning about 30 days later.
This is why external checks on the live endpoint matter more than the client’s own exit codes. Advance warnings at 30, 14, 7 and 1 days before expiry catch exactly this kind of silent drift. For broader fleet coverage, see monitoring Let’s Encrypt certificates at scale.
Renewal Timing, ARI and Shrinking Lifetimes
Certbot renews when 30 days remain on a 90-day certificate, which leaves 30 days of retries. That margin is about to shrink. CA/Browser Forum ballot SC-081 cuts maximum validity to 200 days from March 2026, 100 days from March 2027 and 47 days from March 2029. Let’s Encrypt has announced its own plan to move to 45-day certificates by 2028, and already offers 6-day short-lived certificates through ACME profiles. The background is covered in the 47-day standard.
The ACME Renewal Information (ARI) extension, published as RFC 9773 in 2025, lets the CA suggest a renewal window to the client. If a CA has to mass-revoke certificates after a compliance incident, ARI-aware clients like recent certbot, lego and cert-manager renew early on their own. Hardcoded “renew at day 60” cron jobs do not.
Rate limits also matter. Let’s Encrypt allows 50 certificates per registered domain per week, 5 duplicate certificates per exact set of names per week, and 5 failed validations per account, per hostname, per hour. A CI pipeline that requests a fresh certificate on every deploy can hit the duplicate limit by Wednesday. Always test against the staging endpoint first.
Common ACME Mistakes in Production
No deploy hook. Certbot writes new files to /etc/letsencrypt/live/, but Nginx and HAProxy keep the old certificate in memory until they reload. Without –deploy-hook “systemctl reload nginx”, the server keeps serving the expiring certificate even though renewal “succeeded”. Java keystores, Postfix and appliances that need a PKCS#12 import have the same issue.
Forgotten CAA records. A CAA record that only lists digicert.com will block Let’s Encrypt issuance outright. Someone adds the record during a security review, and renewals fail 60 days later. An experienced SRE checks dig CAA example.com before blaming the ACME client.
Over-privileged DNS tokens. Storing a Cloudflare global API key on every web server for DNS-01 turns one compromised host into full control of your zone. Scope tokens to a single zone and to DNS edit only, or delegate _acme-challenge via CNAME to a dedicated validation zone using a tool like acme-dns.
Practical automation patterns for each of these are in the guide on automating certificate deployment with ACME.
When ACME Is Not the Right Fit
Air-gapped networks cannot reach a public CA at all. Running a private ACME server such as Smallstep step-ca or HashiCorp Vault’s PKI engine is the usual answer there. EV and most OV certificates still require human organization validation, so ACME only automates the final issuance after vetting. Legacy appliances, older F5 BIG-IP firmware and many hardware HSM workflows have no native ACME client, which means a script that pushes certificates through the vendor API.
FAQ
Does ACME work only with Let’s Encrypt?
No. ACME is an IETF standard (RFC 8555), and ZeroSSL, Google Trust Services, Sectigo, DigiCert and private CAs like step-ca all support it.
Which challenge type should I use for wildcard certificates?
DNS-01 is the only option. HTTP-01 and TLS-ALPN-01 cannot validate wildcard identifiers.
How often should an ACME client attempt renewal?
Run the check at least twice daily and let the client decide when to renew, ideally using ARI. Renewing when one third of the lifetime remains leaves enough time for retries.
Final Practical Takeaway
ACME turns certificate issuance into a few signed API calls, but the parts around it – challenge reachability, CAA records, deploy hooks, DNS credentials and rate limits – are where production outages happen. Before lifetimes drop to 47 days, run a dry run with certbot renew –dry-run or your client’s equivalent, confirm the reload hook fires, and check the certificate your endpoint actually serves rather than trusting the renewal log.