Rspamd web ui behind Apache Reverse Proxy

[Atlasfreak]

Hi,
I am trying to put the rspamd web ui behind an apache reverse proxy but I am unable to get it to work no matter what I do. The only error I get is (104)Connection reset by peer: [remote <IP>] AH01102: error reading status line from remote server 127.0.0.1:9989.
My Apache Config is the following:

ServerName <domain>

ErrorLog ${APACHE_LOG_DIR}/webseiten/<domain>.error.log
CustomLog ${APACHE_LOG_DIR}/webseiten/<domain>.access.log combined

Include /etc/letsencrypt/options-ssl-apache.conf

Include /etc/letsencrypt/ssl-certificate.conf

SSLProxyEngine On
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off

ProxyPreserveHost On
ProxyPass / http://127.0.0.1:9989/
ProxyPassReverse / http://127.0.0.1:9989/
ProxyAddHeaders On

RemoteIPHeader X-Forwarded-For
RemoteIPInternalProxy 127.0.0.1
SetEnvIf X-Forwarded-Proto "^https$" HTTPS=on

RequestHeader set X-Forwarded-Proto expr=%{REQUEST_SCHEME}
RequestHeader set X-Forwarded-SSL expr=%{HTTPS}

and my docker compose config:

services:
  mailserver:
    image: ghcr.io/docker-mailserver/docker-mailserver:latest
    container_name: mailserver
    # Provide the FQDN of your mail server here (Your DNS MX record should point to this value)
    hostname: <mail-domain>
    env_file: mailserver.env
    # More information about the mail-server ports:
    # https://docker-mailserver.github.io/docker-mailserver/latest/config/security/understanding-the-ports/
    # To avoid conflicts with yaml base-60 float, DO NOT remove the quotation marks.
    ports:
      - "25:25"    # SMTP  (explicit TLS => STARTTLS, Authentication is DISABLED => use port 465/587 instead)
      - "143:143"  # IMAP4 (explicit TLS => STARTTLS)
      - "465:465"  # ESMTP (implicit TLS)
      - "587:587"  # ESMTP (explicit TLS => STARTTLS)
      - "993:993"  # IMAP4 (implicit TLS)
      - "4190:4190" # Managesieve
      - "127.0.0.1:9989:11334" # Rspamd web ui
    volumes:
      - ./docker-data/dms/mail-data/:/var/mail/
      - ./docker-data/dms/mail-state/:/var/mail-state/
      - ./docker-data/dms/mail-logs/:/var/log/mail/
      - ./docker-data/dms/config/:/tmp/docker-mailserver/
      - /etc/localtime:/etc/localtime:ro
    restart: always
    stop_grace_period: 1m
    # Uncomment if using `ENABLE_FAIL2BAN=1`:
    cap_add:
       - NET_ADMIN
    healthcheck:
      test: "ss --listening --tcp | grep -P 'LISTEN.+:smtp' || exit 1"
      timeout: 3s
      retries: 0

I couldn't find anything in the rspamd.log that corresponds to the web interface (no timestamp matched when I was trying to access the web interface). When exposing the web interface directly to the internet I was able to connect. So it does work just the reverse proxy refuses to work.

PS: I know Apache isn't the best reverse proxy but I haven't come around to migrating to a different reverse proxy.

[casperklein]

Independent from apache, does access from the host work? curl http://127.0.0.1:9989

[Atlasfreak]

Interestingly no, which confuses me a lot because it did work when it was directly exposed to the internet. Never had such problems with docker. The site is reachable within the container.

[Atlasfreak]

Ok it seems like the network is completely misconfigured. I am also no longer able to connect from different containers to the mailserver.
I did set the PERMIT_DOCKER environment variable to connected-networks and that always worked but now it seems to no longer work for some reason.

[Atlasfreak]

Turns out fail2ban banned the docker container gateway...

[polarathene]

Turns out fail2ban banned the docker container gateway...

Well that was an unexpected one :)

Always good to troubleshoot by reproducing with a clean state if you run into this sort of issue in future.

[Atlasfreak]

Well I whitelisted it now so it shouldn't happen again. Fail2ban is probably the thing that caused the most trouble working with the mailserver ^^

[polarathene]

This was intended as a working example of proxying the web UI, but it seems you've already tracked down the actual cause 👍

---

References:

• https://docker-mailserver.github.io/docker-mailserver/latest/config/security/rspamd/#web-interface
• https://rspamd.com/webui/
• https://rspamd.com/doc/faq.html#how-to-use-the-webui-behind-a-proxy-server

I'm not sure where one can set what IP to bind to for the rspamd web-ui, but I can see it's already 0.0.0.0 in this case which is what we want:

$ ss -tlpn | grep rspamd

LISTEN 0      4096       127.0.0.1:11332      0.0.0.0:*    users:(("rspamd",pid=597,fd=8))
LISTEN 0      4096         0.0.0.0:11334      0.0.0.0:*    users:(("rspamd",pid=597,fd=12))

Here is a working example you can copy/paste and docker compose up to access reverse proxy to rspamd with Caddy instead of Apache:

services:
  reverse-proxy:
    image: lucaslorentz/caddy-docker-proxy:2.9
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  dms:
    image: mailserver/docker-mailserver:14
    hostname: mail.example.localhost
    # Caddy-Docker-Proxy container will read these labels to proxy the service:
    # - Provide the FQDN to use with HTTPS and a certificate will be provisioned and managed for you.
    # - The `reverse_proxy` line configures the port in the container to proxy requests for that FQDN to.
    labels:
      caddy: rspamd.example.localhost
      caddy.reverse_proxy: "{{ upstreams 11334 }}"
    # Enable rspamd:
    environment:
      ENABLE_RSPAMD: 1
      # Enabling rspamd replaces these services, disable them:
      ENABLE_AMAVIS: 0
      ENABLE_OPENDKIM: 0
      ENABLE_OPENDMARC: 0
      ENABLE_POLICYD_SPF: 0
    # These files would normally be provided via `volumes`,
    # The `configs` feature allows to embed these configs in `compose.yaml` for this example.
    configs:
      - source: dms-accounts
        target: /tmp/docker-mailserver/postfix-accounts.cf
      - source: rspamd-custom-commands
        target: /tmp/docker-mailserver/rspamd/custom-commands.conf

# This uses the Docker Compose `configs` feature to simplify running this quick example with copy/paste
# - The feature is compatible with any Compose release since 2024.
# - The character `$` below is changed to `$$` to opt-out of accidental variable interpolation (Compose feature).
# Both passwords hashed below are `secret`.
configs:
  # Account provisioned already:
  dms-accounts:
    content: |
      [email protected]|{SHA512-CRYPT}$$6$$sbgFRCmQ.KWS5ryb$$EsWrlYosiadgdUOxCBHY0DQ3qFbeudDhNMqHs6jZt.8gmxUwiLVy738knqkHD4zj4amkb296HFqQ3yDq4UXt8.

  # Get the bcrypt password hash from `docker compose run --rm -it dms rspamadm pw`:
  rspamd-custom-commands:
    content: |
      set-option-for-controller password "$$2$$bp33ciwq5g5x7gw6sxfngrjuqfrqo3hc$$dwjz8osmrj3m8q34e93myugg3x8s95kui76mk6trqkf5mozz9sxy"

Caddy recognizes .localhost for the FQDN so it will provision that certificate as a self-signed one. If this were a public TLD instead, then it would provision with LetsEncrypt.

This example uses Caddy Docker Proxy with the Docker socket to monitor containers running on the same container network that caddy service has, then check the labels and manage the config and certs for you. You could use a separate compose.yaml and a custom bridge network that is shared between the two services and it'll work just the same.

Likewise you can use Caddy directly and manually manage it's config. Caddy is nicer to work with than nginx and apache in my experience 😅

[Atlasfreak]

I'll keep it in mind when I eventually come around to killing my apache instance