# Wildcard SSL with Cloudflare + Coolify/Traefik

This guide documents how to get automatic wildcard SSL certificates for `*.testbed.mk` using Cloudflare DNS and Traefik's DNS-01 challenge.

## Why This Is Needed

Traefik v3 removed the `onDemand` TLS option. `HostRegexp()` routers (used for wildcard subdomain matching) don't trigger automatic per-domain certificate provisioning. Only `Host()` rules do.

The solution: use a **DNS-01 challenge** with a **wildcard certificate**. This provisions a single `*.testbed.mk` cert that covers all memorial subdomains automatically.

DNS-01 requires a DNS provider with an API. Cloudflare is free and fully supported by Traefik.

## Architecture

```
Internet
  │
  ├── *.testbed.mk ──► Cloudflare DNS ──► VPS IP (62.84.176.135)
  │
  └── :443 ──► Traefik (coolify-proxy)
                ├── Wildcard cert: *.testbed.mk (via Cloudflare DNS-01)
                ├── Host(testbed.mk) → app:3000 (Coolify default router)
                └── HostRegexp(*.testbed.mk) → app:3000 (custom wildcard router)
```

## Step 1: Migrate DNS from Contabo to Cloudflare

### 1.1 Create a Cloudflare Account
- Go to https://dash.cloudflare.com
- Sign up (free plan is sufficient)

### 1.2 Add `testbed.mk` to Cloudflare
- Click **Add a Site**
- Enter `testbed.mk`
- Select the **Free** plan
- Cloudflare will scan existing DNS records

### 1.3 Verify DNS Records
Ensure these records exist in Cloudflare:
| Type | Name | Content | Proxy |
|------|------|---------|-------|
| A | `@` | `62.84.176.135` | DNS only (gray cloud) |
| A | `*` | `62.84.176.135` | DNS only (gray cloud) |
| CNAME | `accounts` | `accounts.clerk.services` | DNS only |
| CNAME | `clerk` | `frontend-api.clerk.services` | DNS only |
| CNAME | `clk._domainkey` | `dkim1.t15ssxc9pnhh.clerk.services` | DNS only |
| CNAME | `clk2._domainkey` | `dkim2.t15ssxc9pnhh.clerk.services` | DNS only |
| CNAME | `clkmail` | `mail.t15ssxc9pnhh.clerk.services` | DNS only |

**Important:** All records must be **DNS only** (gray cloud, not proxied). Proxied records will break Traefik's HTTP-01 challenge and cause SSL errors.

### 1.4 Update Nameservers
Cloudflare will display two nameservers (e.g., `xxx.ns.cloudflare.com`). Update them at your current registrar (MKHOST):

1. Log into MKHOST (or wherever `testbed.mk` is registered)
2. Find DNS/nameserver settings
3. Replace Contabo nameservers (`ns1.contabo.net`, `ns2.contabo.net`, `ns3.contabo.net`) with Cloudflare's nameservers
4. Save

DNS propagation may take up to 24-48 hours (usually faster). Check with:
```bash
dig +short NS testbed.mk
# Should show xxx.ns.cloudflare.com
```

### 1.5 Create a Cloudflare API Token

1. Go to Cloudflare → **My Profile** → **API Tokens**
2. Click **Create Token**
3. Use the **Edit zone DNS** template (or create custom):
   - Permissions: `Zone` → `DNS` → `Edit`
   - Zone Resources: `Include` → `Specific zone` → `testbed.mk`
4. Copy the token — you'll need it for Traefik

## Step 2: Configure Traefik for DNS-01 Wildcard Cert

### 2.1 Edit Traefik's docker-compose

On the VPS:
```bash
nano /data/coolify/proxy/docker-compose.yml
```

Replace the `certificatesresolvers` section. Change from HTTP-01 to DNS-01:

```yaml
# REMOVE these lines:
      - '--certificatesresolvers.letsencrypt.acme.httpchallenge=true'
      - '--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=http'

# ADD these lines:
      - '--certificatesresolvers.letsencrypt.acme.dnschallenge=true'
      - '--certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare'
```

Add Cloudflare credentials as environment variables in the `traefik` service:
```yaml
    environment:
      - CLOUDFLARE_DNS_API_TOKEN=YOUR_CLOUDFLARE_API_TOKEN_HERE
```

**Note:** Keep `--certificatesresolvers.letsencrypt.acme.storage=/traefik/acme.json` — this preserves existing certs.

### 2.2 Add Wildcard Certificate Config

Create a dynamic config file:
```bash
cat > /data/coolify/proxy/dynamic/wildcard-cert.yaml << 'EOF'
tls:
  certificates:
    - certResolver: letsencrypt
      domains:
        - main: "testbed.mk"
          sans:
            - "*.testbed.mk"
EOF
```

This tells Traefik to request a wildcard certificate covering `testbed.mk` and `*.testbed.mk` using the DNS-01 challenge via Cloudflare.

### 2.3 Restart Traefik

```bash
cd /data/coolify/proxy && docker compose up -d
```

### 2.4 Verify Cert Provisioning

```bash
# Check Traefik logs for cert provisioning
docker logs coolify-proxy 2>&1 | grep -i "certificate\|acme\|cloudflare" | tail -20

# Test the cert
curl -vI https://goce.testbed.mk 2>&1 | grep -E "subject:|issuer:"
# Should show: subject: CN=*.testbed.mk  (or similar, issued by Let's Encrypt)
```

### 2.5 Update docker-compose.yaml in the Project

The existing Traefik labels in `docker-compose.yaml` can now use the wildcard cert:

```yaml
services:
  app:
    labels:
      - traefik.enable=true
      - traefik.http.routers.spomeniqr-wildcard.rule=HostRegexp(`^[a-z0-9-]+\.testbed\.mk$$`)
      - traefik.http.routers.spomeniqr-wildcard.entryPoints=http,https
      - traefik.http.routers.spomeniqr-wildcard.service=spomeniqr-svc
      - traefik.http.routers.spomeniqr-wildcard.tls=true
      - traefik.http.routers.spomeniqr-wildcard.tls.certresolver=letsencrypt
      - traefik.http.services.spomeniqr-svc.loadbalancer.server.port=3000
      - traefik.docker.network=coolify
```

The `tls.certresolver=letsencrypt` label will now use the DNS-01 challenge. The wildcard cert covers all subdomains automatically.

## Step 3: Handle Coolify Updates

Coolify may overwrite `/data/coolify/proxy/docker-compose.yml` on updates. To persist the DNS-01 configuration:

### Option A: Re-apply after updates
Keep a backup of the modified file:
```bash
cp /data/coolify/proxy/docker-compose.yml /data/coolify/proxy/docker-compose.yml.dns01-backup
```

After a Coolify update, diff and re-apply the DNS-01 changes:
```bash
diff /data/coolify/proxy/docker-compose.yml /data/coolify/proxy/docker-compose.yml.dns01-backup
# Manually re-apply the dnschallenge lines
```

### Option B: Use Coolify's proxy customization
Check if Coolify v4.1+ supports custom proxy config injection:
- Coolify → Server → Proxy → Custom Configuration
- Add the DNS-01 challenge settings there

### Option C: Script-based persistence
Create a script that auto-applies the changes:
```bash
cat > /data/coolify/proxy/apply-dns01.sh << 'SCRIPT'
#!/bin/bash
# Re-apply DNS-01 challenge config after Coolify updates
sed -i 's|--certificatesresolvers.letsencrypt.acme.httpchallenge=true|--certificatesresolvers.letsencrypt.acme.dnschallenge=true|' /data/coolify/proxy/docker-compose.yml
sed -i 's|--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=http|--certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare|' /data/coolify/proxy/docker-compose.yml

# Add CLOUDFLARE_DNS_API_TOKEN env if not present
grep -q CLOUDFLARE_DNS_API_TOKEN /data/coolify/proxy/docker-compose.yml || \
  sed -i '/container_name: coolify-proxy/a\    environment:\n      - CLOUDFLARE_DNS_API_TOKEN=YOUR_TOKEN' /data/coolify/proxy/docker-compose.yml

cd /data/coolify/proxy && docker compose up -d
SCRIPT
chmod +x /data/coolify/proxy/apply-dns01.sh
```

## Step 4: Migrate Existing HTTP-01 Certs

Existing domains (like `testbed.mk` itself) already have HTTP-01 certs in `acme.json`. The DNS-01 resolver will coexist — new certs use DNS-01, existing ones continue working until renewal.

When existing certs renew, they'll switch to DNS-01 automatically (since that's the only resolver now). This is seamless.

## Step 5: Update Clerk Domains

After DNS migration to Cloudflare:
1. Go to Clerk Dashboard → **Domains**
2. Verify `testbed.mk` is still listed
3. Re-run DNS verification for the Clerk CNAME records (they should still work — Cloudflare proxies DNS-only CNAMEs)
4. If Clerk verification fails, ensure CNAME records in Cloudflare are **DNS only** (not proxied)

## Troubleshooting

### Cert not provisioning
```bash
# Check Traefik logs
docker logs coolify-proxy 2>&1 | grep -i error | tail -20

# Common issues:
# - CLOUDFLARE_DNS_API_TOKEN is wrong or expired
# - Token doesn't have Edit DNS permission for testbed.mk zone
# - Nameservers haven't fully propagated to Cloudflare yet
```

### Subdomain still shows TRAEFIK DEFAULT CERT
```bash
# Check if the wildcard cert is loaded
docker exec coolify-proxy cat /traefik/acme.json | python3 -c "
import sys, json
data = json.load(sys.stdin)
for resolver, certs in data.items():
    for cert in certs.get('Certificates', []):
        domain = cert.get('domain', {}).get('main', 'unknown')
        print(f'{resolver}: {domain}')
"

# Force cert renewal by deleting the acme.json entry
# (CAREFUL - only do this if needed)
```

### DNS not resolving after migration
```bash
# Check nameservers
dig +short NS testbed.mk
# Should show Cloudflare nameservers

# Check A record for wildcard
dig +short test.testbed.mk
# Should show 62.84.176.135
```

## Summary

| Aspect | Contabo DNS (current) | Cloudflare DNS (recommended) |
|--------|----------------------|-------------------------------|
| API access | No | Yes (free) |
| Wildcard SSL | Manual per-domain | Automatic wildcard cert |
| DNS-01 challenge | Not possible | Supported |
| SSL for memorials | Manual/complex | Fully automatic |
| Cost | Free | Free |
| Migration effort | — | ~30 min |
| Maintenance | Per-memorial cert management | Zero-touch after setup |