Introduction

Traefik is a reverse proxy that allows you to have all of your web services behind a single front end. It can be used for easy management of Docker based services (with automatic SSL certificate generation), and also to handle external services (located on another device/IP or hosted in Docker).

The main benefits (and the reasons that I am using Traefik) are:

It is a container aware proxy. It will automatically pick up any changes to running Docker containers (configuration is made using labels on the target container)
You can setup a wildcard DNS record for your subdomain (if you have one) to point to Traefik so that any new entries do not need to be added manually for each service (See my DNS server post for more information)
Other non-Docker services can be manually added using a configuration file (rules.yaml/toml)
It can used in Kubernetes, allowing for high availability hosting

Setup Overview

These are the requirements to establish the setup that I currently have for Traefik:

DNS entries for sites (minimal can be editing the local Hosts file if you don't have a DNS server, but a provider is required if you want SSL certificates)
Docker host to run Traefik and other containers

Traefik Dashboard

The dashboard for Traefik is the best way to check how all of the traffic and routing is working and for troubleshooting services or access The documentation for the dashboard can be found here

The main page of the dashboard will show useful information about the services and ports that it is listening on (example below listens on Port 80/443 for web and also 8888 for Metric information):

Adding Containers

When setting up a service to be proxied by Traefik, there are a few things to consider that can be illustrated by the below flow diagram:

Entrypoint: the port/protocol you want the service to listen on (usually 80/443)
Router: in charge of connecting incoming requests to the services
Middleware: any changes that need to made to the request/authentication
Service: the container or service that this will be routed to (Docker container or manual entry)

Non-standard port mappings

A note to make in the example above is that is a docker container listens on multiple ports (running a docker ps will show the ports that the container is listening on) or a non-standard web port then you will need to specify it in the traefik.http.services.NAME.loadbalancer.server.port label to allow Traefik to correctly point to it

Example here is checking the Service for Plex to ensure that it uses the correct port (32400)

Certificate Management

Once we have the services setup correctly, we would like a valid certificate to appear in the title bar to ensure extra security is applied to the site. For each of the services that you want to have a certificate generated for, you will specify in the labels which certificate resolver to use For Traefik I use the "myresolver" as the name, and pass the parameters to the Traefik docker-compose.yml (full config file can be seen at the bottom of this page)

services:
  traefik:
    labels:
      - "--certificatesresolvers.myresolver.acme.dnschallenge=true"
      - "--certificatesresolvers.myresolver.acme.dnschallenge.provider=dreamhost"
      - "--certificatesresolvers.myresolver.acme.email=email@domain.com"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
    environment:
      - "DREAMHOST_API_KEY=XXXXXXXXXXXX"

Once Traefik is aware of the certificate resolver to use, you can simply add the traefik.http.routers.plex.tls.certresolver=myresolverlabel to the container as in the Plex example above and it should place the certificate in the letsencrypt/acme.json file automatically

Extra Security (ssllab test)

Once you have the certificates applied correctly to the connected containers, you can look into adding extra security to the TLS connections used. A good site to test with (if your domains are publicly resolvable) is SSL Labs

With the following configuration I am currently receiving an "A" score for SSL (as of 29th March 2022) Traefik docker-compose.yml:

labels:
      - "traefik.http.middlewares.traefik-headers.headers.accesscontrolallowmethods=GET, OPTIONS, PUT"
      - "traefik.http.middlewares.traefik-headers.headers.accesscontrolalloworiginlist=https://abowden.net"
      - "traefik.http.middlewares.traefik-headers.headers.accesscontrolmaxage=100"
      - "traefik.http.middlewares.traefik-headers.headers.addvaryheader=true" 
      - "traefik.http.middlewares.traefik-headers.headers.allowedhosts=traefik.domain" 
      - "traefik.http.middlewares.traefik-headers.headers.hostsproxyheaders=X-Forwarded-Host"
      - "traefik.http.middlewares.traefik-headers.headers.sslredirect=true"
      - "traefik.http.middlewares.traefik-headers.headers.sslhost=traefik.domain" 
      - "traefik.http.middlewares.traefik-headers.headers.sslforcehost=true"
      - "traefik.http.middlewares.traefik-headers.headers.sslproxyheaders.X-Forwarded-Proto=https"
      - "traefik.http.middlewares.traefik-headers.headers.stsseconds=63072000"
      - "traefik.http.middlewares.traefik-headers.headers.stsincludesubdomains=true"
      - "traefik.http.middlewares.traefik-headers.headers.stspreload=true"
      - "traefik.http.middlewares.traefik-headers.headers.forcestsheader=true"
      - "traefik.http.middlewares.traefik-headers.headers.framedeny=true"
      - "traefik.http.middlewares.traefik-headers.headers.contenttypenosniff=true"
      - "traefik.http.middlewares.traefik-headers.headers.browserxssfilter=true"
      - "traefik.http.middlewares.traefik-headers.headers.referrerpolicy=same-origin"
      - "traefik.http.middlewares.traefik-headers.headers.featurepolicy=camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';"
      - "traefik.http.middlewares.traefik-headers.headers.customresponseheaders.X-Robots-Tag=none,noarchive,nosnippet,notranslate,noimageindex,"

Traefik rules.yaml:

tls:
  options:
    default:
      minVersion: VersionTLS12
      cipherSuites:
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384   # TLS 1.2
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305    # TLS 1.2
        - TLS_AES_256_GCM_SHA384                  # TLS 1.3
        - TLS_CHACHA20_POLY1305_SHA256            # TLS 1.3
      curvePreferences:
        - CurveP521
        - CurveP384
      sniStrict: true

Full copies of the config files are available at the bottom of this page

Middleware

Middleware is used in the middle of the request process, to make a change or add extra controls to the request as it is passing through

In my case I am using the following middlewares to add extra security:

redirect-web-to-websecure (redirect HTTP to HTTPS)
middlewares-basic-auth (Adds a basic authentication prompt for username/password)
middlewares-rate-limit (adds request rate limiting to slow down brute force attempts)
middlewares-ipwhitelist (whitelist to only allow internal IP addresses to access internal services)
traefik-headers (default headers to add for extra SSL security)

Docker-compose.yml file

version: "3.3"

networks:
  web:
    external: true

services:
  traefik:
    image: "traefik:v2.5.3"
    container_name: "traefik"
    restart: unless-stopped
    command:
      - "--api=true"
      - "--providers.docker=true"
      # Do not expose containers unless explicitly told so
      - "--providers.docker.exposedbydefault=false"
      - "--providers.file=true"
      - "--providers.file.filename=/etc/traefik/rules.yaml"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.web.http.redirections.entryPoint.to=websecure"
      - "--entrypoints.web.http.redirections.entryPoint.scheme=https"
      - "--entrypoints.web.http.redirections.entrypoint.permanent=true"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.websecure.http.middlewares=middlewares-basic-auth@file"
      - "--entrypoints.metrics.address=:8888"
      - "--metrics.prometheus=true"
      - "--metrics.prometheus.entryPoint=metrics"
      - "--certificatesresolvers.myresolver.acme.dnschallenge=true"
      - "--certificatesresolvers.myresolver.acme.dnschallenge.provider=dreamhost"
      - "--certificatesresolvers.myresolver.acme.email=email@domain.com"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
      - "--serverstransport.insecureskipverify=true"
      - "--accessLog.filePath=/logs/access.log"
    security_opt:
      - "no-new-privileges:true"
    labels:
      - "traefik=name"
      - "traefik.enable=true"
      # Middleware
      - "traefik.http.routers.traefik-rtr.middlewares=traefik-headers,middlewares-rate-limit@file,middlewares-basic-auth@file" 
      - "traefik.http.middlewares.traefik-headers.headers.accesscontrolallowmethods=GET, OPTIONS, PUT"
      - "traefik.http.middlewares.traefik-headers.headers.accesscontrolalloworiginlist=https://abowden.net"
      - "traefik.http.middlewares.traefik-headers.headers.accesscontrolmaxage=100"
      - "traefik.http.middlewares.traefik-headers.headers.addvaryheader=true" 
      - "traefik.http.middlewares.traefik-headers.headers.allowedhosts=traefik.domain" 
      - "traefik.http.middlewares.traefik-headers.headers.hostsproxyheaders=X-Forwarded-Host"
      - "traefik.http.middlewares.traefik-headers.headers.sslredirect=true"
      - "traefik.http.middlewares.traefik-headers.headers.sslhost=traefik.domain" 
      - "traefik.http.middlewares.traefik-headers.headers.sslforcehost=true"
      - "traefik.http.middlewares.traefik-headers.headers.sslproxyheaders.X-Forwarded-Proto=https"
      - "traefik.http.middlewares.traefik-headers.headers.stsseconds=63072000"
      - "traefik.http.middlewares.traefik-headers.headers.stsincludesubdomains=true"
      - "traefik.http.middlewares.traefik-headers.headers.stspreload=true"
      - "traefik.http.middlewares.traefik-headers.headers.forcestsheader=true"
      - "traefik.http.middlewares.traefik-headers.headers.framedeny=true"
      - "traefik.http.middlewares.traefik-headers.headers.contenttypenosniff=true"
      - "traefik.http.middlewares.traefik-headers.headers.browserxssfilter=true"
      - "traefik.http.middlewares.traefik-headers.headers.referrerpolicy=same-origin"
      - "traefik.http.middlewares.traefik-headers.headers.featurepolicy=camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';"
      - "traefik.http.middlewares.traefik-headers.headers.customresponseheaders.X-Robots-Tag=none,noarchive,nosnippet,notranslate,noimageindex,"
    ports:
      - "80:80"
      - "443:443"
      # Metrics
      - "8888:8888"
    networks:
      - web
    environment:
      - "DREAMHOST_API_KEY=XXXXXXXXXXXX"
    volumes:
      - "./letsencrypt:/letsencrypt"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./rules.yaml:/etc/traefik/rules.yaml:ro"
      - "./logs:/logs"

Rules.yaml file

http:
  routers:
    router-traefik:
      entryPoints:
        - websecure
      rule: Host(`dashboard.domain`)
      service: api@internal
      middlewares:
        - "middlewares-rate-limit@file"
        - "middlewares-basic-auth@file"
        - "middlewares-ipwhilelist@file"
      tls:
        certResolver: myresolver
  middlewares:
    middlewares-basic-auth:
      basicAuth:
        users:
          - "username:(encrypted password)"
        headerField: X-WebAuth-User
    middlewares-rate-limit:
      rateLimit:
        average: 100
        period: 1m
        burst: 100
    middlewares-ipwhilelist:
      ipWhiteList:
        sourceRange:
          - "127.0.0.1/32"
          - "192.168.0.0/24"
tls:
  options:
    default:
      minVersion: VersionTLS12
      cipherSuites:
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384   # TLS 1.2
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305    # TLS 1.2
        - TLS_AES_256_GCM_SHA384                  # TLS 1.3
        - TLS_CHACHA20_POLY1305_SHA256            # TLS 1.3
      curvePreferences:
        - CurveP521
        - CurveP384
      sniStrict: true

Troubleshooting Steps

It's taken me quite a while to get to this setup with Traefik, and I've tried to boil down the troubleshooting steps to following the items in the "Setup Overview" section. This is the order that I will troubleshoot a container if it's not resolving the way that I planned:

Check the Traefik Dashboard to make sure that the Router appears in the HTTP Routers section
Click into the router to see the data flow at the top of the page
Ensure that it is on the correct Entrypoint
Ensure that the Router is container@docker (unless manually added service)
Go to the linked HTTP service, double check the URL and port number resolve correctly (can do a wget from the Docker host to check the HTTP result returned from the internal Docker IP)
Check the labels that are set on the container against other images (example is Plex)
If all else fails, check the logs of the Traefik Docker container

Final Thoughts

This is still a work in progress, and extra labels and configuration are being added as I discover them or through extra security requirements

There are still some improvements that I'd like to make to the setup:

Use Docker secrets to store the DREAMHOST_API_KEY
Test out a High Availability setup (either 2 standalone instances, or with Kubernetes)
Adding better error pages and monitoring (currently a generic 404 page will be shown if a host name is not available)

Setting up Traefik with SSL certificates for a Homelab

Comments

More from this blog

How to Deploy .NET Applications Continuously in Docker Without Pipelines

Useful Scripts for managing multiple Docker containers on a single host

Docker Folder Setup on Homelab (with Scripts)

Data Migration/Extraction from Autodesk Vault

Introduction

Setup Overview

Traefik Dashboard

Adding Containers

Tags

Non-standard port mappings

Certificate Management

Extra Security (ssllab test)

Middleware

Docker-compose.yml file

Rules.yaml file

Troubleshooting Steps

Final Thoughts

Command Palette

Comments

More from this blog

Introduction

Setup Overview

Traefik Dashboard

Adding Containers

Tags

Non-standard port mappings

Certificate Management

Extra Security (ssllab test)

Middleware

Docker-compose.yml file

Rules.yaml file

Troubleshooting Steps

Final Thoughts