Automating letsencrypt and haproxy

Letsencrypt is a great project for getting every website on https. It provides a great client that will create your certificates and can even automatically set up your running apache to use the new ssl certificate for https. There is also some experimental support for automatically configuring nginx but the problem is that I'm using neither to terminate my https connections. For my websites I run the Haproxy loadbalancer to route various requests to all the different webservers I have running, some requests just go to apache, some go through Varnish for more caching. To have everything nicely working with https (especially to have Varnish working) I terminate the SSL part of the connection in Haproxy and forward everything over plain http to the other services on my VPSes.

To use Letsencrypt I run the letsencrypt-auto tool in standalone mode so it will spawn its own temporary webserver for the domain verification and then I concat the generated certificate with the private key and put it in the ssl directory of haproxy. Haproxy is configured to load all certificates in its own ssl directory and automatically uses the correct one for the domain when using the https frontend. 

The problem is that the letsencrypt certificates are only valid for a relative short amount of time. most ssl providers generate certificates that are valid for one or two years, letsencrypt certificates are currently valid for three months and that might even be shorter in the future. With this strategy letsencrypt is trying to push sysadmins to automate their certificate renewal so even shorter validity length can be used. This is a great thing if a certificate gets comprimised, the comprimised certificate will only be valid for a short amount of time.

Configuring haproxy for letsencrypt

To have haproxy load the certificates I use the following in my https frontend:

frontend www-https
        bind 0.0.0.0:443 ssl crt /etc/haproxy/ssl/
        reqadd X-Forwarded-Proto:\ https

        acl letsencrypt-request path_beg -i /.well-known/acme-challenge/
        use_backend letsencrypt if letsencrypt-request

        ...backend routing config here...

with this frontend Haproxy will load all certificates from the /etc/haproxy/ssl/ directory and use the correct one based on the SNI field of the http requests. It also has a reference to the letsencrypt backend and uses it if the path starts with "/.well-known/acme-challenge/". This makes sure that all http requests for letsencrypt validation are routed to the letsencrypt standalone webserver independently of the domain and used backend server technology. The backend definition for letsencrypt is as follows:

backend letsencrypt
        mode http
        server letsencrypt 127.0.0.1:9999

This config just tells Haproxy that the letsencrypt webserver is reachable at port 9999

Automating certificate creation and renewal

The only thing left to do is automating the process of requesting the certificate by calling the letsencrypt client, moving some files around and then reloading haproxy. This is done by the following bash script:

#!/bin/bash

# Path to the letsencrypt-auto tool
LE_TOOL=/opt/letsencrypt/letsencrypt-auto

# Directory where the acme client puts the generated certs
LE_OUTPUT=/etc/letsencrypt/live

# Concat the requested domains
DOMAINS=""
for DOM in "$@"
do
    DOMAINS+=" -d $DOM"
done

# Create or renew certificate for the domain(s) supplied for this tool
$LE_TOOL --agree-tos --renew-by-default --standalone --standalone-supported-challenges http-01 --http-01-port 9999 certonly $DOMAINS

# Cat the certificate chain and the private key together for haproxy
cat $LE_OUTPUT/$1/{fullchain.pem,privkey.pem} > /etc/haproxy/ssl/${1}.pem

# Reload the haproxy daemon to activate the cert
systemctl reload haproxy

This will run the letsencrypt client in unattended standalone mode on port 9999 and requests the certificates supplied on the command line. The rest is explained in the inline comments.

This script is executed with cron before the previous certificate expires. For example

# Renew certificates on the first day of every month on a random time
42 0  1 * * /opt/letsencryt-haproxy domain1.nl
37 13 1 * * /opt/letsencrypt-haproxy otherdomain.com