I have a lab environment running VMware Cloud Director 10.x and wanted to leverage my existing Traefik server for SSL, Reverse Proxy, and Load Balancing my 3 server nodes. This article steps through my configuration and provides an example, working (anonymized) dynamic file based Traefik configuration file.
Overview
The lab environment consiste of the following:
- PhotonOS VM (Updated docker, installed docker-compose)
- Firewall Rule allowing ports 443 and 8443
- NAT Rule forwarding ports 443 and 8443 to internal IP address of the PhotonOS VM that runs Traefik
- 3 Node VMware Cloud Director 10.x using shared Internal issued SSL Certificate and connected to NFS for Transfer storage
Traefik v2 Server details
I like to run Traefik on a Docker host since it is super easy and fast to recover if anything were to go wrong. All I need is the info in this blog post to get the server back up and running. From my PhotonOS template to fully running/configured reverse proxy is measured in minutes 😄
I use Traefik static config to specify my entrypoints, enable the api and dashboard, set log file paths, enable healthcheck url, enable file and docker providers, configure my Let’s Encrypt resolver, and allow certificate verification to be skipped (IE: don’t object to self-signed or internal CA issued SSL certificates).
File System
Here’s what my folder tree looks like for this setup:
1❯ tree
2.
3├── config
4│ ├── acme.json <--- generated file
5│ ├── logs
6│ │ ├── access.log <--- generated file
7│ │ └── traefik.log <--- generated file
8│ ├── traefik.d
9│ │ └── 10-vcd.yml
10│ └── traefik.yaml
11├── docker-compose.yml
12└── env
13 └── aws.env
Note that a few files are generated. The others are files you must create. The generated files are:
- config/acme.json - contains the SSL Certificates that were issued by LetsEncrypt
- config/logs/access.log - Traefik web server access log
- config/logs/traefik.log - Traefik server log
The files you must create are listed below and sample contents are shown below the list:
- docker-compose.yml
- env/aws.env
- config/traefik.yaml
- config/traefik.d/10-vcd.yml
docker-compose.yml
Getting the right combination of switches and parameters can be confusing. There’s quite a bit of documentation though. Hopefully this cleaned version of my docker-compose.yml file can help you out.
1version: '3'
2
3services:
4 reverse-proxy:
5 # The official v2 Traefik docker image
6 image: traefik:v2.5.5
7 # If you don't specify an appropriate restart value, then upon host reboot, your container will not auto-start
8 restart: always
9 # Provide a friendly name for the container to allow for easier troubleshooting
10 container_name: traefik
11 # Since I use Route53 for DNS, I load up a restricted API account via environment variables and store them in a config file
12 env_file: ./env/aws.env
13 networks:
14 - proxy
15 ports:
16 # The HTTPS port for VMware Cloud Director and 8443 for the VM Console Proxy
17 - "443:443"
18 - "8443:8443"
19 # The Web UI (enabled by --api.insecure=true), port 8080 NOT permitted through Firewall
20 - "8080:8080"
21 labels:
22 - traefik.enable=true
23 volumes:
24 # So that Traefik can listen to the Docker events - helpful if you run additional containers
25 - /var/run/docker.sock:/var/run/docker.sock
26 - "./config:/etc/traefik"
27
28networks:
29 proxy:
30 external: true
Environment file containing DNS Keys (env/aws.env)
The aws.env file contains the necessary contents for use with the route53 DNS provider so that traefik can do the necessary DNS validation for generating the LetsEncrypt Wildcard SSL Certificates using the required DNS-01 Challenge.
1AWS_ACCESS_KEY_ID=(insert your AWS Access key ID here)
2AWS_SECRET_ACCESS_KEY=(Insert your AWS Secret Access Key here)
3AWS_PROPAGATION_TIMEOUT=300
4AWS_POLLING_INTERVAL=60
Static Traefik config (config/traefik.yaml)
Be sure to review the comments within the following yaml, and adjust to meet your environment needs.
1global:
2 checkNewVersion: true
3 sendAnonymousUsage: true
4
5serversTransport:
6 # Allow self-signed/Internal CA Issued Certs to be used easily for back-end connections
7 insecureSkipVerify: true
8
9api:
10 insecure: true
11 dashboard: true
12 debug: false
13
14log:
15 filePath: "/etc/traefik/logs/traefik.log"
16 level: INFO
17 format: common
18 # format: json
19
20accessLog:
21 filePath: "/etc/traefik/logs/access.log"
22 format: json
23 filters:
24 statusCodes:
25 - "200"
26 - "300-302"
27 retryAttempts: true
28 minDuration: "10ms"
29
30ping:
31 entryPoint: "traefik"
32
33entryPoints:
34 web:
35 address: ":80"
36 http:
37 redirections:
38 entryPoint:
39 to: secureweb
40 scheme: https
41 proxyProtocol:
42 insecure: true
43 forwardedHeaders:
44 insecure: true
45 secureweb:
46 address: ":443"
47 vcd-console:
48 address: ":8443"
49
50providers:
51 # Enable the file provider to define routers / middlewares / services in file
52 # Each *.yml file placed in this directory will be dynamically read and applied by Traefik!
53 file:
54 directory: /etc/traefik/traefik.d
55 watch: true
56 debugLogGeneratedTemplate: true
57 docker:
58 exposedbydefault: false
59 network: proxy
60
61certificatesResolvers:
62 cert-resolver:
63 acme:
64 email: "your-email@example.com"
65 storage: "/etc/traefik/acme.json"
66 # Staging Server: Keep the staging server enabled until your config is validated and working. Then comment it and uncomment the production server
67 caServer: "https://acme-staging-v02.api.letsencrypt.org/directory"
68 # Production Server: Keep this commented until you have validated with the staging server above
69 # caServer: https://acme-v02.api.letsencrypt.org/directory
70 # For wildcard cert, must use dnsChallenge
71 dnsChallenge:
72 provider: "route53"
73 delayBeforeCheck: 60
74 disablePropagationCheck: false
75 # Check your DNS settings to get the authoritative
76 # DNS Server IPs for your domain and fill in the IP
77 # Addresses here
78 resolvers:
79 - "dns.server.1.ip:53"
80 - "dns.server.2.ip:53"
81 - "dns.server.3.ip:53"
82 - "dns.server.4.ip:53"
Full Dynamic Config for VMware Cloud Director (config/traefik.d/10-vcd.yml)
Once you have the static configuration setup, you can move on to the dynamic file based configuration. I like this option because I can easily add new config files to generate new routers to direct traffic to additional internal sites as needed without restarting Traefik :)
Be sure to review the comments within the following yaml, and adjust to meet your environment needs.
1# tcp routing section - needed for the VMware Cloud Director Console Proxy
2tcp:
3 routers:
4 tcp-console:
5 entryPoints: ["vcd-console"]
6 rule: "HostSNI(`vcd.example.com`)"
7 # The vCD Console requires passthrough TLS as it is required to handle its own Cert.
8 tls:
9 passthrough: true
10 service: "vcd-console"
11 services:
12 vcd-console:
13 loadBalancer:
14 servers:
15 - address: "192.168.110.156:8443"
16 - address: "192.168.110.158:8443"
17 - address: "192.168.110.160:8443"
18# http routing section
19http:
20 routers:
21 vcd-ssl:
22 entryPoints: ["secureweb"]
23 # Lower priority is processed last. Setting another config to 1
24 # Allows for specific routers to be processed first as long
25 # as their priority is greater than 1
26 priority: 10
27 rule: "HostRegexp(`vcd.example.com`)"
28 tls:
29 certResolver: cert-resolver
30 # Note that the credentials provided in the env/aws.env must have the necessary permissions to create a DNS record in the Domain specified below.
31 domains:
32 - main: "*.example.com"
33 sans: ["example.com"]
34 service: "vcloud-director"
35
36 services:
37 vcloud-director:
38 loadBalancer:
39 servers:
40 - url: "https://192.168.110.156"
41 - url: "https://192.168.110.158"
42 - url: "https://192.168.110.160"
43 healthCheck:
44 path: /cloud/server_status
45 port: 443
46 interval: "10s"
47 timeout: "3s"
48 passHostHeader: true
49 responseForwarding:
50 flushInterval: "3s"
As time permits, I will update/revise this article to make it flow a bit better. I hope you found the article helpful!
As a follow-on to setting this up, I would highly encourage you to also learn How to enable logrotate on PhotonOS.