rspamd rule

rspamd_report.sh wertet /var/log/mail/rspamd.log lesbar aus: Uebersicht als
Tabelle (Score, Aktion, Scanzeit, Von, An) und Detailansicht einer Mail mit
nach Gewicht sortierter Symbolaufschluesselung samt Summe. Getestet gegen
echte Logzeilen, die Summe deckt sich mit dem Score im Log.

train_bayes.sh: --only akzeptiert jetzt mehrere Muster (mehrfach angebbar
oder kommasepariert), damit sich das Training auf kuratierte Postfaecher
begrenzen laesst. Ohne Filter wird das jetzt explizit ausgegeben.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

.gitignore

@@ -1 +1,2 @@
 .env
+node_modules

DMS/Dockerfile

@@ -24,3 +24,8 @@ COPY sieve-schedule /etc/sieve-schedule
 
 # 5. Supervisor Konfiguration kopieren
 COPY sieve-supervisor.conf /etc/supervisor/conf.d/sieve-sync.conf
+
+# 6. Dynamic Whitelist Script und Supervisor-Config kopieren
+COPY dynamic_whitelist.py /scripts/dynamic_whitelist.py
+RUN chmod +x /scripts/dynamic_whitelist.py
+COPY whitelist-supervisor.conf /etc/supervisor/conf.d/dynamic-whitelist.conf

DMS/README.md

@@ -0,0 +1,232 @@
+# Spam-Scoring im Mailfluss — Stand und offene Punkte
+
+Dieses Dokument hält fest, warum die rspamd-Konfiguration hier vom Standard
+abweicht und was noch offen ist. Hintergrund, der sich weder aus der Config
+noch aus der Git-History erschließt, gehört hierher.
+
+## Mailfluss
+
+```
+Externe Mail → SES → S3/SQS → email-worker (Container) → SMTP :25 → DMS
+```
+
+Daraus folgt die zentrale Eigenheit: **rspamd sieht als Client-IP immer den
+Worker-Container**, nie den echten Absender-Server. Jede verbindungsbezogene
+Prüfung misst damit unsere eigene Infrastruktur. Ohne Gegenmaßnahme fallen
+systematisch Strafpunkte an, die nichts mit der Mail zu tun haben:
+
+| Symbol | Gewicht |
+|---|---|
+| `HFILTER_HOSTNAME_UNKNOWN` | 6.0 (kein PTR für 172.x — exakt die `add_header`-Schwelle) |
+| `R_SPF_FAIL` / `R_SPF_SOFTFAIL` | 4.5 / 2.5 |
+| `R_DKIM_NA` / `R_DKIM_PERMFAIL` | 1.0 / 4.5 |
+
+SPF/DKIM/DMARC gehen dadurch nicht verloren: SES prüft sie bereits mit der
+echten Absender-IP, das Ergebnis liegt im SQS-Payload
+(`receipt.spamVerdict`, `spfVerdict`, `dkimVerdict`, `dmarcVerdict`).
+
+## Aktueller Stand
+
+Schwellwerte in `docker-data/dms/config/rspamd/override.d/actions.conf`:
+`add_header = 6`, `reject = 500` (praktisch nie), `greylist = 500` (aus).
+Mit `MOVE_SPAM_TO_JUNK=1` bedeutet ein Score ab 6 also: Mail landet im Junk.
+
+Die Settings-Regeln stehen in
+`docker-data/dms/config/rspamd/append.d/settings.conf`, ausgewertet nach
+Priorität, es greift nur der erste Treffer:
+
+| Prio | Regel | Gilt für | Wirkung |
+|---|---|---|---|
+| 20 | `ses_worker_forwarded` | `X-SES-Worker-Processed: forwarded\|ooo-reply` | Auth-Gruppen aus **plus** `SPOOF_REPLYTO`, `FREEMAIL_REPLYTO*` |
+| 10 | `ses_worker_injected` | Worker-Header vorhanden (also `delivered`) | nur Auth-Gruppen aus |
+| 5 | `local_reinjection` | Docker-Netz ohne Worker-Header (Sieve-Redirects, Roundcube) | ersetzt die alte `DOCKER_WL`-Whitelist |
+
+Alle drei sind zusätzlich an `172.16.0.0/12` / `127.0.0.0/8` gebunden. Ohne
+diese IP-Bedingung könnte ein externer Absender den Header selbst setzen und
+die Prüfungen abschalten.
+
+**Was bewusst entfernt wurde:** die frühere `DOCKER_WL`-Regel (-50 Punkte für
+jede Mail mit eigener Absenderdomain) samt erzwungenem „no action" aus
+`force_actions.conf`. Sie war ein Spoofing-Loch — mit `SPOOF_PROTECTION=0`
+genügte ein `MAIL FROM: <x@bayarea-cc.com>` für garantierte Zustellung — und
+hat echten Spam an weitergeleitete Adressen durchgereicht.
+
+---
+
+## Offener Punkt B: Weiterleitungen ohne Neukomposition
+
+**Status:** offen, bewusst zurückgestellt. Punkt A (die Settings-Regel
+`ses_worker_forwarded`) ist am 2026-08-15 umgesetzt worden und behebt das
+Symptom. B ist die strukturelle Lösung.
+
+### Das Problem
+
+`buildForwardMessage()` in
+`email-worker-nodejs/src/email/rules-processor.ts:212` baut für jede
+Weiterleitung eine **neue** Mail:
+
+```ts
+from: recipient,          // support@bayarea-cc.com   — die getroffene lokale Adresse
+to: forwardTo,            // ebrunks@bayarea-cc.com   — das Weiterleitungsziel
+subject: `FWD: ${originalSubject}`,
+replyTo: originalFrom,    // andreas.knuth@gmail.com  — der Originalabsender
+```
+
+Bei einer Weiterleitung an einen Kollegen in derselben Domain ergibt das
+formal die Signatur eines CEO-Fraud-Angriffs: Absender und Empfänger intern,
+die Antwort geht nach außen an eine Freemail-Adresse.
+
+Belegt an Queue-ID `F228E320085` (2026-08-15 07:53:38):
+
+```
+6.00  SPOOF_REPLYTO              bayarea-cc.com;gmail.com
+2.00  FREEMAIL_REPLYTO_NEQ_FROM
+-0.10 MIME_GOOD
+0.05  MANY_INVISIBLE_PARTS
+----
+7.95  → add header → Junk
+```
+
+Dieselbe Mail als Original an `support@` (`EC702320085`, eine Sekunde später):
+**-1.05, no action**. Der Unterschied kommt allein aus der Bauform der
+Weiterleitung, nicht aus dem Inhalt.
+
+### Der Vorschlag
+
+Statt eine neue Mail zu komponieren, die Originalmail **unverändert
+weiterreichen** (Remailing): `From:` bleibt der Originalabsender,
+Envelope-From wird eine lokale Bounce-Adresse, kein `Reply-To` mehr,
+kein `FWD:`-Präfix.
+
+Vorteile über die Spam-Bewertung hinaus:
+
+- Threading in den Clients bleibt intakt (`Message-ID`, `References`)
+- Originalbetreff und Originalheader bleiben erhalten
+- Anhänge müssen nicht neu verpackt werden
+- Das Muster, das `SPOOF_REPLYTO` auslöst, entsteht gar nicht erst — die
+  Ausnahme in `ses_worker_forwarded` könnte danach wieder enger gefasst
+  oder ganz entfernt werden
+
+### Warum es nicht einfach nur „besser" ist
+
+Das ist ein echter Eingriff mit Folgen, die vor der Umsetzung geklärt werden
+müssen:
+
+1. **Bounces.** Envelope-From muss auf eine Adresse zeigen, die wir selbst
+   auswerten — sonst gehen Zustellfehler an den Originalabsender, der mit
+   unserer Weiterleitung nichts zu tun hat. Klassische Lösung ist SRS;
+   `ENABLE_SRS=0` ist derzeit gesetzt.
+2. **DMARC bei externen Zielen — das ist der harte Punkt.**
+   `handleForwards()` unterscheidet: interne Ziele gehen über
+   `sendInternalEmail()` auf Port 25, externe über `ses.sendRawEmail()`.
+   Beide setzen den Envelope-From auf `recipient`, also auf unsere eigene
+   Domain. Heute ist das für externe Weiterleitungen sogar ein Vorteil: SES
+   signiert mit unserem DKIM, SPF passt, und weil `From:` ebenfalls unsere
+   Domain ist, ist DMARC sauber ausgerichtet.
+
+   Genau das würde Remailing zerstören. Mit `From:` = Originalabsender prüft
+   Gmail DMARC gegen dessen Domain — und wir erfüllen weder deren SPF (die
+   Mail kommt von SES) noch deren DKIM (wir haben den Body umgebaut).
+   Das ist das klassische Forwarding-Problem. SRS löst den SPF-Teil,
+   `ENABLE_SRS=0` ist derzeit gesetzt; für DKIM hilft nur, die Mail wirklich
+   **byte-identisch** durchzureichen, inklusive Originalheader.
+
+   Praktische Konsequenz: B ließe sich zunächst **nur für interne Ziele**
+   umsetzen und für externe beim jetzigen Verhalten bleiben. Das löst den
+   `SPOOF_REPLYTO`-Fall — der tritt ohnehin nur intern auf, weil er
+   `From`-Domain == `To`-Domain voraussetzt — ohne die externe
+   Zustellbarkeit anzufassen.
+3. **Absenderdarstellung.** Im Posteingang steht dann der Originalabsender,
+   nicht mehr `support@`. Ob das gewünscht ist, ist eine fachliche
+   Entscheidung — Mailinglisten lösen es über
+   `From: "Name via support" <support@bayarea-cc.com>`, was allerdings
+   `SPOOF_REPLYTO` teilweise zurückholt.
+
+### Abnahmekriterium
+
+Eine Weiterleitung einer externen Mail an eine interne Adresse zeigt in
+`rspamd_report.sh --detail` einen Score unter 6 **ohne** die Ausnahme für
+`SPOOF_REPLYTO`, und ein Reply des Empfängers erreicht den Originalabsender.
+
+---
+
+## Weitere offene Punkte
+
+### `fuzzy_check` wieder einschalten
+
+Abgeschaltet in `docker-data/dms/config/rspamd/local.d/fuzzy_check.conf`, weil
+UDP 11335 ausgehend nicht durchkommt (vermutlich Egress-Filter des Providers).
+Jede Mail lief in denselben Timeout und kostete konstant ~4,3 s Scanzeit ohne
+jeden Gegenwert. Nach dem Abschalten liegen die Scans bei 70–580 ms.
+
+Fuzzy erkennt Massen-Spam über Hashes — genau die Klasse, die weder
+Auth-Prüfungen noch ein frisch angelernter Bayes zuverlässig erwischt. Das
+lohnt sich mit dem Provider zu klären. Test:
+
+```
+docker exec mailserver rspamadm fuzzy_ping -s rspamd.com
+```
+
+### Bayes anlernen
+
+`BAYES_*` feuert erst ab 200 gelernten Nachrichten **je Klasse**, aktuell also
+gar nicht. `docker-data/dms/config/train_bayes.sh` lernt einmalig aus den
+vorhandenen Postfächern (Junk → Spam, INBOX → Ham), Default ist Trockenlauf.
+
+Wichtig: **nur kuratierte Postfächer** über `--only` einbeziehen. Ohne Filter
+läuft das Skript über alle Postfächer aller Domains unter `/var/mail` — bei
+Weiterleitungsadressen ist die INBOX ein ungefilterter Durchlauf und als
+Ham-Quelle unbrauchbar.
+
+Reihenfolge beachten: erst falsch einsortierte Mail abstellen, dann lernen.
+Solange legitime Weiterleitungen im Junk landen, lernt `RSPAMD_LEARN=1` sie
+dort automatisch als Spam an und vergiftet den Klassifikator.
+
+### SES-Verdicts als Header stempeln
+
+Der Worker könnte `receipt.spfVerdict` / `dkimVerdict` / `dmarcVerdict` aus
+dem SQS-Payload als Header setzen. Dann ließen sich die echten,
+mit der richtigen Absender-IP ermittelten Auth-Ergebnisse per multimap wieder
+bepunkten, statt sie ersatzlos abzuschalten.
+
+---
+
+## Betrieb
+
+### Auswertung
+
+```
+docker exec mailserver bash /tmp/docker-mailserver/rspamd_report.sh -n 20
+docker exec mailserver bash /tmp/docker-mailserver/rspamd_report.sh --detail <QID>
+docker exec mailserver bash /tmp/docker-mailserver/rspamd_report.sh --slow 1000
+```
+
+Das Feld `Regel:` in der Detailansicht zeigt, welche Settings-Regel gegriffen
+hat — der schnellste Weg zu prüfen, ob eine Änderung überhaupt aktiv ist.
+Voraussetzung ist `level = "info"` in `rspamd/local.d/logging.inc`; der
+DMS-Default `silent` protokolliert nichts.
+
+### Änderungen an `append.d/` ausrollen
+
+`user-patches.sh` hängt die Dateien aus `rspamd/append.d/` an die
+`local.d`-Pendants an und überspringt das, wenn der Marker `DMS-CUSTOM-APPEND`
+dort schon steht. `/etc/rspamd` liegt im Container-Dateisystem und überlebt
+`docker restart` — **ein Neustart übernimmt Änderungen also nicht.** Nötig ist:
+
+```
+docker compose up -d --force-recreate mailserver
+```
+
+Das Anhängen statt Überschreiben ist zwingend: DMS legt in `settings.conf`
+selbst eine Regel für authentifizierte User ab (`DMS::SED_TAG::1`), die ein
+`cp` löschen würde — womit `RSPAMD_CHECK_AUTHENTICATED=0` wirkungslos wäre.
+
+### DNS
+
+rspamds Blocklisten brauchen einen eigenen Recursor. Über Dockers eingebautes
+DNS (127.0.0.11) stufte Spamhaus die Anfragen als von einem offenen Resolver
+kommend ein und antwortete nur mit `DBL_BLOCKED_OPENRESOLVER`. Gelöst über den
+Service `dns-recursor` (PowerDNS) im eigenen Netz `dms_dns` mit fester IP —
+fest deshalb, weil Dockers `dns:`-Option nur IPs akzeptiert.
+
+`ENABLE_UNBOUND` existiert in DMS nicht und war ein No-Op.

DMS/batch_imapsync.sh

@@ -0,0 +1,58 @@
+#!/bin/bash
+# batch_imapsync.sh - Führt IMAP-Sync für alle User im Hintergrund aus
+# Format der CSV: email@domain.com,SecretPassword123
+
+HOST1=$1
+HOST2=$2
+CSV_FILE=$3
+
+if [ -z "$HOST1" ] || [ -z "$HOST2" ] || [ -z "$CSV_FILE" ]; then
+    echo "Usage: $0 <source-host> <target-host> <users.csv>"
+    echo "Beispiel: $0 secure.emailsrvr.com 147.93.132.244 stxmaterials.csv"
+    exit 1
+fi
+
+# ======================================================================
+# Die eigentliche Sync-Funktion (wird in den Hintergrund geschickt)
+# ======================================================================
+run_sync_jobs() {
+    TIMESTAMP=$(date +%Y%m%d_%H%M%S)
+    LOG_DIR="sync_logs_$TIMESTAMP"
+    mkdir -p "$LOG_DIR"
+    
+    echo "Beginne Sync-Lauf am $(date)" > "batch_master_${TIMESTAMP}.log"
+
+    while IFS=, read -r email password; do
+        email=$(echo "$email" | tr -d '\r' | xargs)
+        password=$(echo "$password" | tr -d '\r' | xargs)
+        
+        [ -z "$email" ] && continue
+        
+        LOGFILE="$LOG_DIR/imapsync_${email}.log"
+        echo "[$(date)] Syncing $email -> $LOGFILE" >> "batch_master_${TIMESTAMP}.log"
+        
+        # Führe Docker imapsync für den aktuellen User aus
+        docker run --rm gilleslamiral/imapsync imapsync \
+            --host1 "$HOST1" --user1 "$email" --password1 "$password" --ssl1 \
+            --host2 "$HOST2" --user2 "$email" --password2 "$password" --ssl2 \
+            --automap > "$LOGFILE" 2>&1 < /dev/null
+            
+    done < "$CSV_FILE"
+    
+    echo "Alle Sync-Jobs beendet am $(date)" >> "batch_master_${TIMESTAMP}.log"
+}
+
+# ======================================================================
+# Skript-Start: Entkopplung vom Terminal
+# ======================================================================
+echo "🚀 Starte Batch-IMAP-Sync im Hintergrund..."
+
+# Rufe die Funktion auf, leite alle restlichen Ausgaben ins Nichts und schicke sie in den Hintergrund (&)
+run_sync_jobs </dev/null >/dev/null 2>&1 &
+
+echo "✅ Der Job läuft jetzt autark im Hintergrund (sequenziell)."
+echo "Du kannst das SSH-Terminal jetzt bedenkenlos schließen!"
+echo "Überwache den Gesamtfortschritt mit:"
+echo "  tail -f batch_master_*.log"
+echo "Oder die Details eines einzelnen Postfachs mit:"
+echo "  tail -f sync_logs_*/imapsync_<email>.log"

DMS/docker-compose.yml

@@ -9,10 +9,10 @@ services:
 
     # Node-spezifischer Hostname - A-Record zeigt auf DIESEN Server.
     # email-srvr.com selbst zeigt auf einen anderen Server und wird hier NICHT verwendet.
-    hostname: node1.email-srvr.com
+    hostname: ${NODE_HOSTNAME}
 
     ports:
-      - "25:25"
+      - "127.0.0.1:25:25"
       - "587:587"
       - "465:465"
       - "143:143"
@@ -26,7 +26,13 @@ services:
       - ./docker-data/dms/mail-state/:/var/mail-state/
       - ./docker-data/dms/mail-logs/:/var/log/mail/
       - ./docker-data/dms/config/:/tmp/docker-mailserver/
-      - ./docker-data/dms/config/dovecot/conf.d/95-sieve-redirect.conf:/etc/dovecot/conf.d/95-sieve-redirect.conf:ro
+      # ENTFERNT: Mount von dovecot/conf.d/95-sieve-redirect.conf
+      # Der Host-Pfad existiert nicht, Docker hat daraufhin ein VERZEICHNIS
+      # unter /etc/dovecot/conf.d/95-sieve-redirect.conf angelegt (nachweisbar
+      # per `docker exec mailserver cat ...` -> "Is a directory"). Der Mount war
+      # also wirkungslos. Falls dort spaeter wirklich etwas gebraucht wird
+      # (z.B. sieve_redirect_envelope_from), erst die Datei auf dem Host
+      # anlegen, dann den Mount wieder aufnehmen.
       - /etc/localtime:/etc/localtime:ro
       - ./sync_dynamodb_to_sieve.py:/scripts/sync.py:ro
       - ./sieve-cron:/etc/cron.d/sieve-sync:ro
@@ -45,7 +51,8 @@ services:
       # setup-dms-tls.sh referenziert per:
       #   /etc/mail/certs/*.domain/*.domain.crt|.key
       # -------------------------------------------------------
-      - /var/lib/docker/volumes/caddy_data/_data/caddy/certificates/acme-v02.api.letsencrypt.org-directory:/etc/mail/certs:ro
+      # - /var/lib/docker/volumes/caddy_data/_data/caddy/certificates/acme-v02.api.letsencrypt.org-directory:/etc/mail/certs:ro
+      - /home/aknuth/git/email-amazon/caddy/caddy-data/caddy/certificates/acme-v02.api.letsencrypt.org-directory:/etc/mail/certs:ro
       # -------------------------------------------------------
       # Dovecot SNI Konfiguration (generiert von setup-dms-tls.sh)
       # DMS lädt /tmp/docker-mailserver/dovecot-sni.cf automatisch.
@@ -60,11 +67,16 @@ services:
       # Kundendomain-SNI wird über postfix-main.cf + dovecot-sni.cf gesteuert.
       # -------------------------------------------------------
       - SSL_TYPE=manual
-      - SSL_CERT_PATH=/etc/mail/certs/node1.email-srvr.com/node1.email-srvr.com.crt
+      - SSL_CERT_PATH=/etc/mail/certs/${NODE_HOSTNAME}/${NODE_HOSTNAME}.crt
-      - SSL_KEY_PATH=/etc/mail/certs/node1.email-srvr.com/node1.email-srvr.com.key
+      - SSL_KEY_PATH=/etc/mail/certs/${NODE_HOSTNAME}/${NODE_HOSTNAME}.key
 
       # SPAM / Rspamd
-      - ENABLE_OPENDKIM=1
+      # OpenDKIM deaktiviert: es liegen keine Keys unter config/opendkim/, der
+      # Milter hat also nie etwas signiert. Ausgehende Mail wird von Amazon SES
+      # signiert (Easy DKIM), eingehende DKIM-Pruefung macht rspamd selbst.
+      # DMS warnt beim Start ausdruecklich davor, OpenDKIM parallel zu rspamd
+      # zu betreiben.
+      - ENABLE_OPENDKIM=0
       - ENABLE_OPENDMARC=0
       - ENABLE_POLICYD_SPF=0
       - ENABLE_RSPAMD=1
@@ -79,7 +91,10 @@ services:
 
       # Sicherheit
       - ENABLE_FAIL2BAN=1
-      - ENABLE_UNBOUND=1
+      # ENABLE_UNBOUND entfernt: die Variable existiert in DMS nicht (mehr) und
+      # war wirkungslos - rspamd hat ueber Dockers eingebautes DNS (127.0.0.11)
+      # aufgeloest und bekam von Spamhaus nur DBL_BLOCKED_OPENRESOLVER zurueck.
+      # Ersetzt durch den dns-recursor-Service weiter unten.
 
       # Sonstige
       - ENABLE_MANAGESIEVE=0
@@ -94,7 +109,10 @@ services:
 
       # Amazon SES Relay
       - RELAY_HOST=email-smtp.us-east-2.amazonaws.com
-      - RELAY_PORT=587
+      # Node-spezifisch: manche Provider blockieren ausgehend 587.
+      # Wert pro Server in der .env setzen (z.B. RELAY_PORT=2587),
+      # ohne Eintrag bleibt es bei 587.
+      - RELAY_PORT=${RELAY_PORT:-587}
       - RELAY_USER=${SES_SMTP_USER}
       - RELAY_PASSWORD=${SES_SMTP_PASSWORD}
 
@@ -106,7 +124,7 @@ services:
       # Postfix
       # POSTFIX_OVERRIDE_HOSTNAME: Was Postfix im EHLO/HELO Banner sendet.
       # node1.email-srvr.com passt zum TLS-Cert und ist der echte Hostname.
-      - POSTFIX_OVERRIDE_HOSTNAME=node1.email-srvr.com
+      - POSTFIX_OVERRIDE_HOSTNAME=${NODE_HOSTNAME}
       - POSTFIX_MYNETWORKS=172.16.0.0/12 172.17.0.0/12 172.18.0.0/12 [::1]/128 [fe80::]/64
       - POSTFIX_MAILBOX_SIZE_LIMIT=0
       - POSTFIX_MESSAGE_SIZE_LIMIT=0
@@ -115,11 +133,47 @@ services:
       - NET_ADMIN
       - SYS_PTRACE
     restart: unless-stopped
+
+    # -------------------------------------------------------
+    # DNS: rspamd fragt DNS-Blocklisten (Spamhaus DBL/ZEN, SURBL) ab. Diese
+    # Dienste beantworten keine Anfragen, die ueber oeffentliche Resolver
+    # kommen - man bekommt DBL_BLOCKED_OPENRESOLVER statt eines Ergebnisses.
+    # Deshalb ein eigener rekursiver Resolver, siehe Service dns-recursor.
+    #
+    # ACHTUNG: Damit ersetzt der Recursor Dockers eingebautes DNS (127.0.0.11)
+    # fuer DIESEN Container. Der mailserver kann dann keine Container-Namen
+    # mehr aufloesen (roundcube-db, email-worker-ts, ...). Das ist hier
+    # unkritisch, weil er von sich aus nur nach aussen verbindet (SES-Relay).
+    # Faellt der Recursor aus, hat der mailserver kein DNS mehr - deshalb
+    # restart: always beim Recursor.
+    # -------------------------------------------------------
+    dns:
+      - 10.99.0.53
+    depends_on:
+      - dns-recursor
+
     networks:
       mail_network:
         aliases:
           - mailserver
-          - node1.email-srvr.com
+          - ${NODE_HOSTNAME}
+      dns_network:
+
+  # ---------------------------------------------------------
+  # Rekursiver DNS-Resolver ausschliesslich fuer den mailserver.
+  # Er fragt die Root-Server selbst ab, statt an einen Upstream-Resolver
+  # weiterzuleiten - nur so akzeptiert Spamhaus die Anfragen.
+  # Braucht keine Konfiguration: das Image laeuft per Default als Recursor
+  # und erlaubt Anfragen aus den RFC1918-Netzen (10.99.0.0/24 liegt darin).
+  # ---------------------------------------------------------
+  dns-recursor:
+    image: powerdns/pdns-recursor-53:5.3.10
+    container_name: dns-recursor
+    restart: always
+    stop_grace_period: 0s
+    networks:
+      dns_network:
+        ipv4_address: 10.99.0.53
 
   roundcube:
     image: roundcube/roundcubemail:latest
@@ -135,10 +189,10 @@ services:
       - ROUNDCUBEMAIL_DB_USER=roundcube
       - ROUNDCUBEMAIL_DB_PASSWORD=${ROUNDCUBE_DB_PASSWORD}
       # Roundcube verbindet intern über den Docker-Alias
-      - ROUNDCUBEMAIL_DEFAULT_HOST=ssl://node1.email-srvr.com
+      - ROUNDCUBEMAIL_DEFAULT_HOST=ssl://${NODE_HOSTNAME}
       - ROUNDCUBEMAIL_DEFAULT_PORT=993
       # Interner Traffic ohne TLS
-      - ROUNDCUBEMAIL_SMTP_SERVER=ssl://node1.email-srvr.com
+      - ROUNDCUBEMAIL_SMTP_SERVER=ssl://${NODE_HOSTNAME}
       - ROUNDCUBEMAIL_SMTP_PORT=465
       
       # WICHTIG: Variablen LEER lassen, damit Roundcube keine Authentifizierung versucht!
@@ -175,3 +229,16 @@ services:
 networks:
   mail_network:
     external: true
+
+  # Eigenes Netz nur fuer mailserver <-> dns-recursor.
+  # Bewusst NICHT im externen mail_network: dort vergibt Docker die Adressen
+  # dynamisch, eine feste IP koennte mit einem anderen Container kollidieren.
+  # Das Subnetz muss fest sein, weil Dockers `dns:`-Option nur IPs akzeptiert.
+  # Falls 10.99.0.0/24 bei euch schon belegt ist (VPN, AWS-VPC), hier und bei
+  # `dns:` sowie `ipv4_address` oben gemeinsam auf ein freies Netz aendern.
+  dns_network:
+    name: dms_dns
+    ipam:
+      driver: default
+      config:
+        - subnet: 10.99.0.0/24

DMS/docker-data/dms/config/fail2ban-jail.cf

@@ -1,7 +1,6 @@
 [DEFAULT]
-# Whitelist: Localhost, private Docker-Netze und die Office-IP in Texas
+# Whitelist: Localhost, private Docker-Netze und die Budd Electric Office-IP
-ignoreip = 127.0.0.1/8 ::1 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 69.223.70.143
+ignoreip = 127.0.0.1/8 ::1 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 24.155.193.233 69.223.70.143 24.155.193.233
-
 
 [dovecot]
 # Erhöht die Anzahl der erlaubten Fehlversuche auf 20

DMS/docker-data/dms/config/rspamd/append.d/settings.conf

@@ -0,0 +1,135 @@
+# DMS-CUSTOM-APPEND
+# ===========================================================================
+# WICHTIG: Diese Datei wird von user-patches.sh an /etc/rspamd/local.d/settings.conf
+# ANGEHAENGT und nicht darueber kopiert. DMS legt in dieser Datei selbst eine
+# Regel fuer authentifizierte User ab (DMS::SED_TAG::1), die erhalten bleiben muss.
+# ===========================================================================
+#
+# Hintergrund
+# -----------
+# Externe Mail erreicht DMS nie direkt: SES -> S3/SQS -> email-worker -> SMTP.
+# Rspamd sieht als Client-IP daher immer den Worker-Container (172.x) statt des
+# echten Absender-Servers. Alle verbindungsbezogenen Pruefungen messen damit
+# unsere eigene Infrastruktur und liefern systematisch Strafpunkte:
+#
+#   HFILTER_HOSTNAME_UNKNOWN   6.0   (kein PTR fuer 172.x -> exakt die add_header-Schwelle)
+#   R_SPF_FAIL / R_SPF_SOFTFAIL 4.5 / 2.5
+#   R_DKIM_NA / R_DKIM_PERMFAIL 1.0 / 4.5
+#
+# Diese Gruppen werden fuer Einlieferungen aus dem Docker-Netz deaktiviert.
+# Inhaltsbezogene Pruefungen (Bayes, Fuzzy, URL-/Surbl-Blocklisten, MIME,
+# Attachment-Checks) bleiben vollstaendig aktiv - echter Spam wird also weiter
+# erkannt, nur eben anhand des Inhalts statt anhand einer Transportstrecke,
+# die rspamd hier gar nicht beurteilen kann.
+#
+# SPF/DKIM/DMARC gehen dadurch nicht verloren: SES prueft sie bereits mit der
+# echten Absender-IP. Das Ergebnis liegt im SQS-Payload (receipt.spamVerdict,
+# spfVerdict, dkimVerdict, dmarcVerdict). Sobald der Worker diese Verdicts als
+# Header stempelt, koennen sie hier per multimap wieder bepunktet werden.
+
+# ---------------------------------------------------------------------------
+# 1) Vom Worker SELBST komponierte Mail: Weiterleitungen und OOO-Antworten
+#
+# Zusaetzlich zu den Auth-Symbolen aus Regel 2 fallen hier inhaltliche Symbole
+# an, die auf die Bauform der Weiterleitung reagieren. buildForwardMessage()
+# in email-worker-nodejs/src/email/rules-processor.ts setzt:
+#
+#   From:     die getroffene lokale Adresse   (support@bayarea-cc.com)
+#   To:       das Weiterleitungsziel          (ebrunks@bayarea-cc.com)
+#   Reply-To: den Originalabsender            (andreas.knuth@gmail.com)
+#
+# Damit sieht jede Weiterleitung an einen Kollegen in derselben Domain formal
+# aus wie CEO-Fraud: Absender und Empfaenger intern, Antwort geht nach aussen
+# an eine Freemail-Adresse. Rspamd bepunktet das - zu Recht, nur ist die
+# Praemisse hier falsch, weil wir die Mail selbst gebaut haben:
+#
+#   SPOOF_REPLYTO              6.0   From-Domain == To-Domain != Reply-To-Domain
+#   FREEMAIL_REPLYTO_NEQ_FROM  2.0   Reply-To ist Freemail und != From
+#
+# Belegt an F228E320085 (2026-08-15 07:53:38): Score 7.95, also ueber
+# add_header = 6 - die Weiterleitung landete im Junk, waehrend dieselbe Mail
+# als Original an support@ mit -1.05 sauber durchlief (EC702320085).
+#
+# WARUM EINE EIGENE REGEL statt symbols_disabled in Regel 2:
+# Regel 2 gilt auch fuer "delivered", also fuer echte eingehende Fremdmail.
+# Dort ist SPOOF_REPLYTO genau das Signal, das BEC-Angriffe erkennt - das
+# darf nicht verloren gehen. Diese Regel prueft deshalb den WERT des Headers
+# und greift nur fuer Mail aus unserer eigenen Komposition.
+#
+# Die Auth-Gruppen muessen wiederholt werden: rspamd wertet nach Prioritaet
+# aus und wendet nur die erste passende Regel an, Regel 2 greift hier also
+# nicht mehr.
+#
+# Falls der Regexp-Vergleich nicht zieht, faellt die Mail auf Regel 2 zurueck
+# und behaelt ihre 7.95 - der Fehler ist damit sichtbar, nicht still. Pruefen
+# laesst sich das am Feld "Regel:" in:
+#   docker exec mailserver bash /tmp/docker-mailserver/rspamd_report.sh --detail <QID>
+# Erwartet wird ses_worker_forwarded.
+# ---------------------------------------------------------------------------
+ses_worker_forwarded {
+  priority = 20;
+  ip = ["172.16.0.0/12", "127.0.0.0/8"];
+  header = {
+    "X-SES-Worker-Processed" = "^(forwarded|ooo-reply)$";
+  }
+  apply {
+    groups_disabled = ["spf", "dkim", "dmarc", "hfilter"];
+    symbols_disabled = ["RDNS_NONE", "SPOOF_REPLYTO",
+                        "FREEMAIL_REPLYTO_NEQ_FROM", "FREEMAIL_REPLYTO"];
+  }
+}
+
+# ---------------------------------------------------------------------------
+# 2) Uebrige Einlieferungen des email-workers (externe Mail, "delivered")
+#
+# Der Worker setzt X-SES-Worker-Processed auf JEDER Mail, die er einliefert:
+#   delivered  = Originalmail an das lokale Postfach   -> diese Regel
+#   forwarded  = neu komponierte Weiterleitung          -> Regel 1
+#   ooo-reply  = Abwesenheitsantwort                    -> Regel 1
+#
+# Hier werden ausschliesslich die Transportpruefungen abgeschaltet. Alle
+# inhaltlichen Symbole bleiben aktiv, insbesondere SPOOF_REPLYTO.
+#
+# Die IP-Bedingung ist die Absicherung: ohne sie koennte ein externer Absender
+# den Header selbst setzen und die Auth-Pruefungen abschalten. Beide
+# Bedingungen werden von rspamd implizit mit UND verknuepft.
+# ---------------------------------------------------------------------------
+ses_worker_injected {
+  priority = 10;
+  ip = ["172.16.0.0/12", "127.0.0.0/8"];
+  header = {
+    "X-SES-Worker-Processed" = true;
+  }
+  apply {
+    groups_disabled = ["spf", "dkim", "dmarc", "hfilter"];
+    symbols_disabled = ["RDNS_NONE"];
+  }
+}
+
+# ---------------------------------------------------------------------------
+# 3) Lokale Re-Injection ohne Worker-Header
+#
+# Betrifft:
+#   - Sieve-Redirects fuer INTERNE Mail (accounting@ -> support@ -> Forwards).
+#     Diese Mail laeuft nie ueber SES/Worker, hat also keinen Worker-Header.
+#   - Roundcube, das ohne SASL-Auth aus dem Docker-Netz einliefert
+#     (ROUNDCUBEMAIL_SMTP_USER ist bewusst leer).
+#
+# Beides wurde bisher von der DOCKER_WL-Whitelist (-50) aufgefangen. Diese
+# Regel ersetzt sie - deutlich enger gefasst, weil sie an das Docker-Netz
+# gebunden ist statt an eine faelschbare Absenderdomain.
+#
+# Niedrigste Prioritaet: rspamd wertet Regeln nach Prioritaet aus und nimmt
+# den ersten Treffer, Worker-Mail landet also immer in Regel 1 oder 2.
+#
+# ACHTUNG: Diese Regel setzt voraus, dass Port 25 NICHT oeffentlich erreichbar
+# ist (docker-compose.yml bindet ihn auf 127.0.0.1). Wird das je geaendert,
+# muss diese Regel weg - sonst umgeht jeder externe Absender die Pruefung.
+# ---------------------------------------------------------------------------
+local_reinjection {
+  priority = 5;
+  ip = ["172.16.0.0/12", "127.0.0.0/8"];
+  apply {
+    groups_enabled = ["dkim"];
+  }
+}

DMS/docker-data/dms/config/rspamd/local.d/fuzzy_check.conf

@@ -0,0 +1,42 @@
+# documentation: https://rspamd.com/doc/modules/fuzzy_check.html
+#
+# Das Modul ist abgeschaltet, weil die oeffentliche Fuzzy-Storage von
+# rspamd.com von diesem Server aus nicht erreichbar ist. Im Log lief JEDE
+# eingehende Mail in denselben Timeout:
+#
+#   fuzzy_check_timer_callback: got IO timeout with server
+#   fuzzy1.rspamd.com:11335(80.241.57.6:11335), after 1/1 retransmits
+#
+# Die Server werden sauber aufgeloest, es scheitert also nicht an DNS,
+# sondern an der UDP-Verbindung auf Port 11335 (vermutlich Egress-Filter
+# des Providers).
+#
+# Kosten des Defaults: timeout = 2s bei retransmits = 1, also zwei Versuche
+# a 2 Sekunden = ~4 Sekunden Wartezeit pro Mail - genau die konstanten
+# 4,3 Sekunden Scanzeit, die im Log zu sehen waren. Ohne jeden Gegenwert,
+# da nie eine Antwort kommt.
+#
+# Zusaetzlich ein Betriebsrisiko: DMS setzt soft_reject_on_timeout = true.
+# Solange nur dieses eine Modul haengt, passiert nichts; laeuft der Task als
+# Ganzes in den Timeout, wird Mail temporaer abgelehnt statt zugestellt.
+#
+# ---------------------------------------------------------------------------
+# WIEDER EINSCHALTEN, sobald UDP 11335 ausgehend erreichbar ist:
+# Pruefen laesst sich das direkt mit rspamds eigenem Werkzeug:
+#
+#   docker exec mailserver rspamadm fuzzy_ping -s rspamd.com
+#
+# Antwortet die Storage, diese Datei loeschen (dann gilt wieder der Default)
+# oder die Zeile unten auskommentieren und stattdessen die Timeouts knapp
+# halten, damit ein erneuter Ausfall nicht wieder 4 Sekunden kostet:
+#
+#   timeout = 1s;
+#   retransmits = 0;
+#
+# Fuzzy ist inhaltlich wertvoll - es erkennt Massen-Spam ueber Hashes, also
+# genau die Klasse, die weder von Auth-Pruefungen noch von einem frisch
+# angelernten Bayes zuverlaessig erwischt wird. Es lohnt sich, das mit dem
+# Provider zu klaeren, statt es dauerhaft abgeschaltet zu lassen.
+# ---------------------------------------------------------------------------
+
+enabled = false;

DMS/docker-data/dms/config/rspamd/local.d/logging.inc

@@ -0,0 +1,19 @@
+# documentation: https://rspamd.com/doc/configuration/logging.html
+#
+# Uebersteuert den DMS-Default (level = "silent"), bei dem rspamd ueberhaupt
+# nichts protokolliert und Spam-Scores nicht nachvollziehbar sind.
+#
+# type = "console" bleibt bewusst stehen: Supervisor leitet stdout/stderr von
+# rspamd bereits nach /var/log/mail/rspamd.log um (siehe dms-services.conf).
+# Ein eigenes type = "file" mit demselben Pfad haette zwei Schreiber auf
+# derselben Datei zur Folge.
+#
+# Mit level = "info" schreibt rspamd pro Mail eine Zeile mit Aktion, Score und
+# allen ausgeloesten Symbolen samt Einzelgewicht:
+#
+#   docker exec mailserver sh -c 'grep rspamd_task_write_log /var/log/mail/rspamd.log | tail -20'
+
+type = "console";
+level = "info";
+color = false;
+systemd = false;

DMS/docker-data/dms/config/rspamd/local.d/multimap.conf

@@ -0,0 +1,18 @@
+# documentation: https://rspamd.com/doc/modules/multimap.html
+#
+# Die fruehere Regel DOCKER_WL (-50 Punkte fuer jede Mail, deren Envelope-From
+# auf eine unserer eigenen Domains zeigt) ist bewusst entfernt worden:
+#
+#   1. Sie war ein Spoofing-Loch. Mit SPOOF_PROTECTION=0 genuegte einem
+#      beliebigen externen Absender ein "MAIL FROM: <x@bayarea-cc.com>", um
+#      -50 Punkte plus das erzwungene "no action" aus force_actions.conf zu
+#      bekommen - also garantierte Zustellung in die Inbox.
+#   2. Sie hat echten Spam an weitergeleitete Adressen durchgereicht, weil der
+#      email-worker seine Weiterleitungen mit Envelope-From = Empfaengeradresse
+#      (also eigene Domain) einliefert.
+#   3. Sie hat nur ein Symptom kompensiert: die falschen Auth-/Hostname-Scores,
+#      die entstehen, weil alle Mail aus dem Worker-Container kommt. Das wird
+#      jetzt an der Ursache geloest - siehe rspamd/append.d/settings.conf.
+#
+# Die Datei bleibt (leer) bestehen, damit user-patches.sh sie beim Start ueber
+# eine evtl. noch im Container liegende alte Version kopiert.

DMS/docker-data/dms/config/rspamd/override.d/docker_whitelist.map

@@ -1,9 +0,0 @@
-bayarea-cc.com
-ruehrgedoens.de
-annavillesda.org
-bizmatch.net
-biz-match.com
-qrmaster.net
-nqsltd.com
-iitwelders.com
-# Weitere Domains hier eintragen

DMS/docker-data/dms/config/rspamd/override.d/force_actions.conf

@@ -0,0 +1,10 @@
+# documentation: https://rspamd.com/doc/modules/force_actions.html
+#
+# Die Regel DOCKER_WHITELIST_FORCE ("no action" sobald DOCKER_WHITELIST greift)
+# ist entfernt. Sie hat die Spam-Entscheidung fuer jede Mail mit eigener
+# Absenderdomain hart ueberschrieben - inklusive echtem Spam an weitergeleitete
+# Adressen. Das dazugehoerige Symbol existiert nicht mehr, siehe
+# rspamd/local.d/multimap.conf.
+#
+# Keine rules{}-Sektion = force_actions ist wirkungslos. Datei bleibt bestehen,
+# damit DMS sie beim Start ueber eine alte Version im Container kopiert.

DMS/docker-data/dms/config/rspamd/override.d/multimap.conf

@@ -1,14 +0,0 @@
-DOCKER_WL {
-  # ÄNDERUNG: Wir prüfen jetzt den Absender (Envelope From)
-  type = "from";
-  filter = "email:domain";
-  
-  # Pfad bleibt gleich
-  map = "/etc/rspamd/override.d/docker_whitelist.map";
-  
-  symbol = "DOCKER_WHITELIST";
-  score = -50.0;
-  description = "Whitelist fuer eigene Domains";
-  prefilter = true;
-  action = "accept";
-}

DMS/docker-data/dms/config/rspamd_report.sh

@@ -0,0 +1,179 @@
+#!/bin/bash
+# ===========================================================================
+# rspamd_report.sh - lesbare Auswertung von /var/log/mail/rspamd.log
+#
+# Die Logzeilen von rspamd enthalten alles, sind aber unlesbar. Dieses Skript
+# macht daraus zwei Ansichten:
+#
+#   Uebersicht:  eine Zeile je Mail mit Score, Aktion, Scanzeit, Von, An
+#   Detail:      die Symbolaufschluesselung einer Mail, nach Gewicht sortiert,
+#                mit Summe - also warum genau diese Mail diesen Score hat
+#
+# AUSFUEHRUNG - immer IM Container:
+#
+#   docker exec mailserver bash /tmp/docker-mailserver/rspamd_report.sh
+#   docker exec mailserver bash /tmp/docker-mailserver/rspamd_report.sh -n 40
+#   docker exec mailserver bash /tmp/docker-mailserver/rspamd_report.sh --detail
+#   docker exec mailserver bash /tmp/docker-mailserver/rspamd_report.sh --detail C0D8A320066
+#
+# Optionen:
+#   -n N          Anzahl Mails in der Uebersicht (Default 15)
+#   --detail [ID] Symbolaufschluesselung. ID ist ein beliebiges Fragment aus
+#                 Queue-ID oder Message-ID. Ohne ID die letzte Mail.
+#   --slow MS     Nur Mails, deren Scan laenger als MS Millisekunden dauerte
+#   --log PFAD    Anderes Logfile (Default /var/log/mail/rspamd.log)
+#
+# Hinweis: Das Log enthaelt nur Mails, die seit dem Umstellen auf
+# level = "info" (rspamd/local.d/logging.inc) verarbeitet wurden.
+# ===========================================================================
+
+set -uo pipefail
+
+LOGFILE="/var/log/mail/rspamd.log"
+COUNT=15
+DETAIL=0
+DETAIL_ID=""
+SLOW_MS=0
+
+usage() { sed -n '2,30p' "$0" | sed 's/^# \{0,1\}//' ; }
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    -n)       COUNT="${2:-15}"; shift 2 ;;
+    --detail) DETAIL=1
+              if [[ ${2:-} != "" && ${2:-} != -* ]]; then DETAIL_ID="$2"; shift 2; else shift; fi ;;
+    --slow)   SLOW_MS="${2:-0}"; shift 2 ;;
+    --log)    LOGFILE="${2:-}"; shift 2 ;;
+    -h|--help) usage; exit 0 ;;
+    *) echo "Unbekannte Option: $1" >&2; exit 1 ;;
+  esac
+done
+
+if [[ ! -r ${LOGFILE} ]]; then
+  echo "FEHLER: ${LOGFILE} nicht lesbar. Laeuft das Skript wirklich IM Container?" >&2
+  exit 1
+fi
+
+TASKS=$(grep 'rspamd_task_write_log' "${LOGFILE}" 2>/dev/null)
+if [[ -z ${TASKS} ]]; then
+  echo "Keine ausgewerteten Mails im Log gefunden."
+  echo "Steht in rspamd/local.d/logging.inc wirklich level = \"info\"?"
+  exit 0
+fi
+
+# ---------------------------------------------------------------------------
+# Detailansicht einer einzelnen Mail
+# ---------------------------------------------------------------------------
+if (( DETAIL )); then
+  if [[ -n ${DETAIL_ID} ]]; then
+    LINE=$(grep -F "${DETAIL_ID}" <<<"${TASKS}" | tail -1)
+    [[ -z ${LINE} ]] && { echo "Keine Mail mit '${DETAIL_ID}' im Log gefunden." >&2; exit 1; }
+  else
+    LINE=$(tail -1 <<<"${TASKS}")
+  fi
+
+  awk '
+  {
+    # Kopfdaten
+    ts = substr($0, 1, 19)
+    match($0, /qid: <[^>]*>/);      qid  = extract()
+    match($0, /from: <[^>]*>/);     from = extract()
+    match($0, /rcpts: <[^>]*>/);    rcpt = extract()
+    match($0, /settings_id: [^,]*/); sid = substr($0, RSTART+13, RLENGTH-13)
+    match($0, /time: [0-9.]+ms/);   tm   = substr($0, RSTART+6, RLENGTH-8)
+    match($0, /dns req: [0-9]+/);   dns  = substr($0, RSTART+9, RLENGTH-9)
+
+    # Aktion und Score:  (default: F (no action): [4.74/500.00]
+    if (match($0, /\([a-z ]+\): \[-?[0-9.]+\//)) {
+      chunk  = substr($0, RSTART, RLENGTH)
+      action = substr(chunk, 2, index(chunk, ")") - 2)
+      score  = substr(chunk, index(chunk, "[") + 1, length(chunk) - index(chunk, "[") - 1)
+    }
+
+    printf "Zeit:      %s\n", ts
+    printf "Queue-ID:  %s\n", qid
+    printf "Von:       %s\n", from
+    printf "An:        %s\n", rcpt
+    printf "Regel:     %s\n", sid
+    printf "Aktion:    %s     Score: %s     Scanzeit: %s ms     DNS: %s\n\n", action, score, tm, dns
+
+    # Symbolblock zwischen "] [" und "]), len:"
+    i = index($0, "] [")
+    if (i == 0) { print "  (keine Symbole im Log)"; exit }
+    rest = substr($0, i + 3)
+    j = index(rest, "]), len:")
+    if (j == 0) j = length(rest)
+    syms = substr(rest, 1, j - 1)
+
+    # auf Kommas der obersten Ebene splitten (Klammern koennen Kommas enthalten)
+    depth = 0; cur = ""; n = 0
+    for (k = 1; k <= length(syms); k++) {
+      c = substr(syms, k, 1)
+      if (c == "{") depth++
+      else if (c == "}") depth--
+      if (c == "," && depth == 0) { if (cur != "") entries[++n] = cur; cur = ""; continue }
+      cur = cur c
+    }
+    if (cur != "") entries[++n] = cur
+
+    printf "  %8s  %-38s %s\n", "GEWICHT", "SYMBOL", "DETAIL"
+    printf "  %8s  %-38s %s\n", "-------", "--------------------------------------", "------"
+
+    total = 0
+    # zwei Durchlaeufe: erst die mit Gewicht, dann die neutralen
+    for (pass = 1; pass <= 2; pass++) {
+      for (e = 1; e <= n; e++) {
+        entry = entries[e]
+        p1 = index(entry, "(")
+        if (p1 == 0) continue
+        name = substr(entry, 1, p1 - 1)
+        p2 = index(entry, ")")
+        w = substr(entry, p1 + 1, p2 - p1 - 1)
+        payload = ""
+        b1 = index(entry, "{")
+        if (b1 > 0) payload = substr(entry, b1 + 1, length(entry) - b1 - 1)
+        gsub(/;$/, "", payload)
+
+        if (pass == 1 && w + 0 == 0) continue
+        if (pass == 2 && w + 0 != 0) continue
+        if (pass == 1) total += w
+
+        printf "  %8.2f  %-38s %s\n", w, name, substr(payload, 1, 60)
+      }
+      if (pass == 1) printf "  %8s  %s\n", "-------", ""
+      if (pass == 1) printf "  %8.2f  %s\n\n  neutrale Symbole (Gewicht 0):\n", total, "SUMME"
+    }
+  }
+  function extract(   s) { s = substr($0, RSTART, RLENGTH); return substr(s, index(s, "<") + 1, length(s) - index(s, "<") - 1) }
+  ' <<<"${LINE}"
+  exit 0
+fi
+
+# ---------------------------------------------------------------------------
+# Uebersicht
+# ---------------------------------------------------------------------------
+printf '%-19s %7s  %-12s %7s  %-30s %-30s\n' "ZEIT" "SCORE" "AKTION" "MS" "VON" "AN"
+printf -- '--------------------------------------------------------------------------------------------------------------------\n'
+
+awk -v slow="${SLOW_MS}" '
+{
+  ts = substr($0, 1, 19)
+  match($0, /from: <[^>]*>/);   from = extract()
+  match($0, /rcpts: <[^>]*>/);  rcpt = extract()
+  match($0, /time: [0-9.]+ms/); tm   = substr($0, RSTART+6, RLENGTH-8) + 0
+
+  if (match($0, /\([a-z ]+\): \[-?[0-9.]+\//)) {
+    chunk  = substr($0, RSTART, RLENGTH)
+    action = substr(chunk, 2, index(chunk, ")") - 2)
+    score  = substr(chunk, index(chunk, "[") + 1, length(chunk) - index(chunk, "[") - 1) + 0
+  } else { action = "?"; score = 0 }
+
+  if (tm < slow) next
+  printf "%-19s %7.2f  %-12s %7.0f  %-30.30s %-30.30s\n", ts, score, action, tm, from, rcpt
+}
+function extract(   s) { s = substr($0, RSTART, RLENGTH); return substr(s, index(s, "<") + 1, length(s) - index(s, "<") - 1) }
+' <<<"${TASKS}" | tail -n "${COUNT}"
+
+echo
+echo "Detail zu einer Mail:  $0 --detail <Queue-ID>"
+echo "Nur langsame Scans:    $0 --slow 1000"

DMS/docker-data/dms/config/train_bayes.sh

@@ -0,0 +1,222 @@
+#!/bin/bash
+# ===========================================================================
+# train_bayes.sh - Bayes-Klassifikator von rspamd aus den vorhandenen
+#                  Postfaechern anlernen.
+#
+# Hintergrund: rspamd traegt mit BAYES_* erst dann zum Score bei, wenn genug
+# gelernt wurde (Default: 200 Spam UND 200 Ham). Bis dahin liefert der
+# Klassifikator gar nichts - genau das war im Log zu sehen, dort tauchte kein
+# einziges BAYES_-Symbol auf. Statt monatelang auf manuelles Verschieben in
+# den Junk-Ordner zu warten (das macht RSPAMD_LEARN=1 bereits automatisch),
+# lernt dieses Skript einmalig aus dem, was ohnehin auf der Platte liegt.
+#
+# AUSFUEHRUNG - immer IM Container:
+#
+#   docker exec mailserver bash /tmp/docker-mailserver/train_bayes.sh
+#       -> Trockenlauf: zaehlt nur, lernt nichts. Das ist der Default.
+#
+#   docker exec mailserver bash /tmp/docker-mailserver/train_bayes.sh --apply
+#       -> lernt tatsaechlich
+#
+# Optionen:
+#   --apply             Wirklich lernen (ohne das passiert nichts)
+#   --max N             Max. Nachrichten je Postfach und Klasse (Default 400)
+#   --include-sent      Gesendete Mails zusaetzlich als Ham lernen
+#   --only MUSTER       Nur Postfaecher, deren Adresse eines der Muster
+#                       enthaelt. Mehrfach angebbar oder kommasepariert:
+#                         --only support@,andreas.knuth@
+#                       WICHTIG: Ohne diese Option laeuft das Skript ueber
+#                       ALLE Postfaecher aller Domains unter /var/mail.
+#   --host HOST:PORT    rspamd-Controller (Default 127.0.0.1:11334)
+#
+# Passwort: Falls der Controller eines verlangt (bei euch in
+# rspamd/override.d/worker-controller.inc gesetzt), vorweg setzen:
+#   docker exec -e RSPAMC_PASSWORD='...' mailserver bash /tmp/.../train_bayes.sh
+#
+# WICHTIG zur Datenqualitaet:
+#   - Ham kommt aus der INBOX. Liegt dort unerkannter Spam, wird er als Ham
+#     gelernt und verschlechtert die Erkennung. Vorher grob durchsehen lohnt.
+#   - Spam kommt aus Junk. Liegt dort faelschlich einsortierte legitime Mail,
+#     gilt dasselbe umgekehrt.
+#   - Ein sehr schiefes Verhaeltnis (z.B. 8000 Ham zu 50 Spam) macht den
+#     Klassifikator einseitig. Der Trockenlauf zeigt das Verhaeltnis an -
+#     danach ggf. mit --max nachsteuern.
+# ===========================================================================
+
+set -uo pipefail
+
+MAILBASE="${MAILBASE:-/var/mail}"
+RSPAMC_HOST="127.0.0.1:11334"
+MAX_PER_CLASS=400
+MAX_BYTES=5242880        # 5 MB - groessere Mails bringen fuers Lernen nichts
+BATCH=50                 # Dateien pro rspamc-Aufruf
+APPLY=0
+INCLUDE_SENT=0
+ONLY_PATTERNS=()
+
+usage() { sed -n '2,40p' "$0" | sed 's/^# \{0,1\}//' ; }
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --apply)        APPLY=1; shift ;;
+    --max)          MAX_PER_CLASS="${2:-}"; shift 2 ;;
+    --include-sent) INCLUDE_SENT=1; shift ;;
+    --only)         IFS=',' read -ra _pat <<<"${2:-}"
+                    ONLY_PATTERNS+=("${_pat[@]}"); shift 2 ;;
+    --host)         RSPAMC_HOST="${2:-}"; shift 2 ;;
+    -h|--help)      usage; exit 0 ;;
+    *) echo "Unbekannte Option: $1" >&2; echo "Hilfe: $0 --help" >&2; exit 1 ;;
+  esac
+done
+
+if [[ ! -d ${MAILBASE} ]]; then
+  echo "FEHLER: ${MAILBASE} existiert nicht. Laeuft das Skript wirklich IM Container?" >&2
+  exit 1
+fi
+
+RSPAMC=(rspamc -h "${RSPAMC_HOST}")
+[[ -n ${RSPAMC_PASSWORD:-} ]] && RSPAMC+=(-P "${RSPAMC_PASSWORD}")
+
+if ! command -v rspamc >/dev/null 2>&1; then
+  echo "FEHLER: rspamc nicht gefunden." >&2
+  exit 1
+fi
+
+if ! "${RSPAMC[@]}" stat >/dev/null 2>&1; then
+  echo "FEHLER: Controller unter ${RSPAMC_HOST} nicht erreichbar oder Passwort noetig." >&2
+  echo "        Versuche es mit:  docker exec -e RSPAMC_PASSWORD='...' mailserver bash $0 ..." >&2
+  exit 1
+fi
+
+# ---------------------------------------------------------------------------
+# Dateien einsammeln: bis LIMIT regulaere Dateien unterhalb der uebergebenen
+# Maildir-Verzeichnisse, groessenbegrenzt. Neueste zuerst waere schoener, ist
+# aber bei zehntausenden Dateien zu teuer - Maildir-Namen beginnen mit dem
+# Timestamp, daher liefert die Sortierung eine brauchbare Naeherung.
+# ---------------------------------------------------------------------------
+gather() {
+  local limit="$1"; shift
+  local dir
+  for dir in "$@"; do
+    [[ -d ${dir} ]] || continue
+    find "${dir}" -maxdepth 1 -type f -size -"${MAX_BYTES}"c -print 2>/dev/null
+  done | sort -r | head -n "${limit}"
+}
+
+# ---------------------------------------------------------------------------
+# Lernen in Batches. Gibt "gelernt fehlgeschlagen" zurueck.
+# ---------------------------------------------------------------------------
+learn() {
+  local cmd="$1"; shift
+  local -a files=("$@")
+  local total=${#files[@]}
+  local ok=0 fail=0 idx=0 out
+
+  while (( idx < total )); do
+    out=$("${RSPAMC[@]}" "${cmd}" "${files[@]:idx:BATCH}" 2>&1)
+    ok=$(( ok + $(grep -c 'success = true' <<<"${out}") ))
+    fail=$(( fail + $(grep -ci 'error' <<<"${out}") ))
+    idx=$(( idx + BATCH ))
+  done
+
+  echo "${ok} ${fail}"
+}
+
+# ---------------------------------------------------------------------------
+# Hauptlauf
+# ---------------------------------------------------------------------------
+if (( APPLY )); then
+  echo "=== Bayes-Training (SCHREIBEND) ==="
+else
+  echo "=== Bayes-Training (Trockenlauf - es wird nichts gelernt) ==="
+fi
+echo "Basis: ${MAILBASE} | max ${MAX_PER_CLASS} je Postfach und Klasse"
+if (( ${#ONLY_PATTERNS[@]} > 0 )); then
+  echo "Filter: nur Postfaecher mit '${ONLY_PATTERNS[*]}'"
+else
+  echo "Filter: KEINER - es werden ALLE Postfaecher unter ${MAILBASE} erfasst"
+fi
+printf '\n%-42s %8s %8s\n' "POSTFACH" "SPAM" "HAM"
+printf -- '---------------------------------------------------------------\n'
+
+total_spam=0
+total_ham=0
+learned_spam=0
+learned_ham=0
+failed=0
+
+for domain_dir in "${MAILBASE}"/*/; do
+  [[ -d ${domain_dir} ]] || continue
+  domain=$(basename "${domain_dir}")
+
+  for user_dir in "${domain_dir}"*/; do
+    user_dir="${user_dir%/}"
+    [[ -d ${user_dir}/cur ]] || continue      # keine Maildir -> ueberspringen
+
+    mailbox="$(basename "${user_dir}")@${domain}"
+
+    if (( ${#ONLY_PATTERNS[@]} > 0 )); then
+      matched=0
+      for pat in "${ONLY_PATTERNS[@]}"; do
+        [[ ${mailbox} == *"${pat}"* ]] && { matched=1; break; }
+      done
+      (( matched )) || continue
+    fi
+
+    mapfile -t spam_files < <(gather "${MAX_PER_CLASS}" \
+      "${user_dir}/.Junk/cur"  "${user_dir}/.Junk/new" \
+      "${user_dir}/.Spam/cur"  "${user_dir}/.Spam/new")
+
+    ham_dirs=("${user_dir}/cur" "${user_dir}/new")
+    if (( INCLUDE_SENT )); then
+      ham_dirs+=("${user_dir}/.Sent/cur" "${user_dir}/.Sent Messages/cur")
+    fi
+    mapfile -t ham_files < <(gather "${MAX_PER_CLASS}" "${ham_dirs[@]}")
+
+    n_spam=${#spam_files[@]}
+    n_ham=${#ham_files[@]}
+    (( n_spam == 0 && n_ham == 0 )) && continue
+
+    printf '%-42s %8d %8d\n' "${mailbox}" "${n_spam}" "${n_ham}"
+    total_spam=$(( total_spam + n_spam ))
+    total_ham=$(( total_ham + n_ham ))
+
+    if (( APPLY )); then
+      if (( n_spam > 0 )); then
+        read -r ok fail <<<"$(learn learn_spam "${spam_files[@]}")"
+        learned_spam=$(( learned_spam + ok )); failed=$(( failed + fail ))
+      fi
+      if (( n_ham > 0 )); then
+        read -r ok fail <<<"$(learn learn_ham "${ham_files[@]}")"
+        learned_ham=$(( learned_ham + ok )); failed=$(( failed + fail ))
+      fi
+    fi
+  done
+done
+
+printf -- '---------------------------------------------------------------\n'
+printf '%-42s %8d %8d\n\n' "SUMME" "${total_spam}" "${total_ham}"
+
+if (( total_spam > 0 )); then
+  ratio=$(( total_ham / (total_spam > 0 ? total_spam : 1) ))
+  if (( ratio >= 5 )); then
+    echo "WARNUNG: Ham/Spam-Verhaeltnis etwa ${ratio}:1 - sehr einseitig."
+    echo "         Besser mit kleinerem --max fahren, z.B. --max $(( total_spam / 4 + 10 ))."
+    echo
+  fi
+fi
+
+if (( ! APPLY )); then
+  echo "Trockenlauf beendet. Zum echten Lernen dasselbe Kommando mit --apply."
+  echo "rspamd braucht je Klasse mindestens 200 Nachrichten, bevor BAYES_* feuert."
+  exit 0
+fi
+
+echo "Gelernt: ${learned_spam} Spam, ${learned_ham} Ham, ${failed} Fehler/Duplikate."
+echo
+echo "=== rspamc stat ==="
+"${RSPAMC[@]}" stat 2>/dev/null | grep -iE 'learn|classifier|statfile' || true
+echo
+echo "Kontrolle an der naechsten eingehenden Mail:"
+echo "  docker exec mailserver sh -c 'grep rspamd_task_write_log /var/log/mail/rspamd.log | tail -3'"
+echo "In der Symbolliste sollte jetzt BAYES_HAM oder BAYES_SPAM auftauchen."

DMS/docker-data/dms/config/user-patches.sh

@@ -1,21 +1,61 @@
 #!/bin/bash
-# user-patches.sh läuft bei jedem Start von DMS automatisch
+# user-patches.sh laeuft bei jedem Start von DMS automatisch,
+# nach dem Setup und bevor die Daemons gestartet werden.
 
-ACCOUNTS_FILE="/tmp/docker-mailserver/postfix-accounts.cf"
+SRC_BASE="/tmp/docker-mailserver/rspamd"
-WHITELIST_FILE="/etc/rspamd/override.d/docker_whitelist.map"
+LOCAL_D="/etc/rspamd/local.d"
 
-echo "Patching: Generiere Rspamd Whitelist aus Accounts..."
+# ---------------------------------------------------------------------------
-
+# 1) local.d configs kopieren
-if [ -f "$ACCOUNTS_FILE" ]; then
+#    DMS kopiert nur rspamd/override.d automatisch, local.d nicht.
-    # Whitelist generieren
+# ---------------------------------------------------------------------------
-    awk -F'|' '{print $1}' "$ACCOUNTS_FILE" | cut -d'@' -f2 | sort | uniq > "$WHITELIST_FILE"
+echo "Patching: Kopiere custom rspamd local.d configs..."
-    
+if [ -d "${SRC_BASE}/local.d" ]; then
-    # Berechtigungen korrigieren
+  for f in "${SRC_BASE}/local.d"/*; do
-    chmod 644 "$WHITELIST_FILE"
+    [ -f "$f" ] || continue
-    chown _rspamd:_rspamd "$WHITELIST_FILE" 2>/dev/null || true
+    base=$(basename "$f")
-    
+    cp "$f" "${LOCAL_D}/${base}"
-    echo "Whitelist erfolgreich erstellt:"
+    chown root:root "${LOCAL_D}/${base}"
-    cat "$WHITELIST_FILE"
+    chmod 644 "${LOCAL_D}/${base}"
-else
+    echo "  Kopiert: ${base} -> ${LOCAL_D}/"
-    echo "FEHLER: $ACCOUNTS_FILE wurde nicht gefunden!"
+  done
 fi
+
+# ---------------------------------------------------------------------------
+# 2) append.d: an bestehende local.d-Dateien ANHAENGEN statt sie zu ersetzen
+#
+#    Zwingend fuer settings.conf: DMS legt dort selbst die Regel fuer
+#    authentifizierte User ab (DMS::SED_TAG::1). Ein cp darueber wuerde sie
+#    loeschen und RSPAMD_CHECK_AUTHENTICATED=0 wirkungslos machen.
+#
+#    Der Marker DMS-CUSTOM-APPEND macht das Anhaengen idempotent - /etc/rspamd
+#    liegt im Container-Dateisystem und ueberlebt einen `docker restart`.
+# ---------------------------------------------------------------------------
+echo "Patching: Haenge custom rspamd Snippets an local.d configs an..."
+if [ -d "${SRC_BASE}/append.d" ]; then
+  for f in "${SRC_BASE}/append.d"/*; do
+    [ -f "$f" ] || continue
+    base=$(basename "$f")
+    target="${LOCAL_D}/${base}"
+
+    if grep -q 'DMS-CUSTOM-APPEND' "$target" 2>/dev/null; then
+      echo "  Bereits vorhanden, uebersprungen: ${base}"
+      continue
+    fi
+
+    printf '\n' >>"$target"
+    cat "$f" >>"$target"
+    chown root:root "$target"
+    chmod 644 "$target"
+    echo "  Angehaengt: ${base} -> ${target}"
+  done
+fi
+
+# ---------------------------------------------------------------------------
+# HINWEIS: Die frueher hier erzeugte Rspamd-Whitelist
+# (/etc/rspamd/override.d/docker_whitelist.map aus postfix-accounts.cf +
+# statischer Domainliste) ist entfernt. Die zugehoerige multimap-Regel
+# DOCKER_WL (-50) und das erzwungene "no action" aus force_actions.conf gibt
+# es nicht mehr - sie waren ein Spoofing-Loch und haben echten Spam an
+# weitergeleitete Adressen durchgereicht. Ersatz: rspamd/append.d/settings.conf.
+# ---------------------------------------------------------------------------

DMS/docker-data/roundcube/plugins/email_config/email_config.php

@@ -33,12 +33,26 @@ class email_config extends rcube_plugin
     {
         $rcmail = rcube::get_instance();
         $email  = $rcmail->user->get_username();
+
         $secret_key = 'SHARED_SECRET_KEY_987654321';
-        $config_url = 'https://config.email-bayarea.com';
+
+        // mailadmin.<domain der adresse>, z.B. support@bayarea-cc.com
+        // -> https://mailadmin.bayarea-cc.com
+        $at = strrpos($email, '@');
+        if ($at === false) {
+            // Roundcube liefert ausnahmsweise keinen vollen Namen: hier kann kein
+            // sinnvoller Link gebaut werden.
+            return '';
+        }
+        $domain     = strtolower(substr($email, $at + 1));
+        $config_url = 'https://mailadmin.' . $domain;
+
         $expires   = time() + 3600;
         $data      = $email . '|' . $expires;
         $signature = hash_hmac('sha256', $data, $secret_key);
-        $url = $config_url . '/?email=' . urlencode($email) . '&expires=' . $expires . '&signature=' . $signature;
+        $url = $config_url . '/?email=' . urlencode($email)
+            . '&expires=' . $expires
+            . '&signature=' . $signature;
 
         $out = '
         <div class="box" style="max-width: 600px; margin: 40px auto; padding: 30px; background: #fff; border-radius: 8px; box-shadow: 0 2px 8px rgba(0,0,0,0.1);">

DMS/dynamic_whitelist.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+import os
+import re
+import time
+import subprocess
+import threading
+from datetime import datetime
+try:
+    from croniter import croniter
+except ImportError:
+    print("Bitte 'croniter' via pip installieren!")
+    exit(1)
+
+LOG_FILE = '/var/log/mail/mail.log'
+WHITELIST_DURATION_SEC = 24 * 60 * 60 # 24 Stunden
+CRON_SCHEDULE = "0 * * * *" # Jede Stunde
+
+active_ips = {}
+
+# Regex für Dovecot IMAP/POP3 erfolgreiche Logins
+LOGIN_REGEX = re.compile(r"dovecot: (?:imap|pop3)-login: Login: user=<[^>]+>.*rip=([0-9]{1,3}(?:\.[0-9]{1,3}){3}),")
+# Private Netze (Docker/Local) ignorieren
+IGNORE_REGEX = re.compile(r"^(172\.|10\.|192\.168\.|127\.)")
+
+def run_command(cmd):
+    try:
+        subprocess.run(cmd, shell=True, check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
+    except Exception as e:
+        print(f"Fehler bei: {cmd} - {e}")
+
+def cleanup_job():
+    """Cron-Thread für das stündliche Aufräumen abgelaufener IPs."""
+    iter = croniter(CRON_SCHEDULE, datetime.now())
+    while True:
+        next_run = iter.get_next(datetime)
+        sleep_seconds = (next_run - datetime.now()).total_seconds()
+        
+        if sleep_seconds > 0:
+            time.sleep(sleep_seconds)
+            
+        print(f"[{datetime.now()}] Starte stündlichen Whitelist-Cleanup...")
+        now = time.time()
+        expired_ips = [ip for ip, timestamp in active_ips.items() if now - timestamp > WHITELIST_DURATION_SEC]
+        
+        for ip in expired_ips:
+            print(f"[{datetime.now()}] Whitelist für {ip} abgelaufen. Entferne...")
+            run_command(f"fail2ban-client set dovecot delignoreip {ip}")
+            run_command(f"fail2ban-client set postfix delignoreip {ip}")
+            del active_ips[ip]
+
+def follow_log():
+    """Verwendet System 'tail -F', da dies Log-Rotation automatisch handhabt."""
+    print(f"[{datetime.now()}] Dynamic Whitelist Monitor gestartet...")
+    
+    while not os.path.exists(LOG_FILE):
+        time.sleep(2)
+        
+    process = subprocess.Popen(['tail', '-F', LOG_FILE], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True)
+    
+    for line in process.stdout:
+        match = LOGIN_REGEX.search(line)
+        if match:
+            ip = match.group(1)
+            
+            if IGNORE_REGEX.match(ip):
+                continue
+                
+            now = time.time()
+            
+            # Neue IP in die Fail2ban Whitelist eintragen
+            if ip not in active_ips:
+                print(f"[{datetime.now()}] Neuer erfolgreicher Login von {ip}. Setze auf Whitelist...")
+                run_command(f"fail2ban-client set dovecot addignoreip {ip}")
+                run_command(f"fail2ban-client set postfix addignoreip {ip}")
+            
+            # Timestamp (Last Seen) aktualisieren
+            active_ips[ip] = now
+
+if __name__ == '__main__':
+    # Warte kurz, bis Fail2ban nach einem Container-Start hochgefahren ist
+    time.sleep(15)
+    
+    # Cron-Cleanup im Hintergrund starten
+    threading.Thread(target=cleanup_job, daemon=True).start()
+    
+    # Log-Überwachung in der Endlosschleife starten
+    follow_log()

DMS/setup-dms-tls.sh

@@ -27,7 +27,12 @@ CERTS_BASE_PATH=${CERTS_BASE_PATH:-"/etc/mail/certs"}
 
 # Node-Hostname: Fallback-Cert für DMS (kein Wildcard, direktes Cert)
 # Muss mit dem 'hostname' in docker-compose.yml übereinstimmen.
-NODE_HOSTNAME=${NODE_HOSTNAME:-"node1.email-srvr.com"}
+if [ -z "$NODE_HOSTNAME" ]; then
+    echo "❌ NODE_HOSTNAME ist nicht gesetzt!"
+    echo "Beispiel:"
+    echo "  DMS_CONTAINER=mailserver NODE_HOSTNAME=node2.email-srvr.com ./setup-dms-tls.sh"
+    exit 1
+fi
 
 echo "============================================================"
 echo " 🔐 DMS TLS SNI Setup (Multi-Domain)"

DMS/whitelist-supervisor.conf

@@ -0,0 +1,6 @@
+[program:dynamic-whitelist]
+command=/usr/bin/python3 -u /scripts/dynamic_whitelist.py
+autostart=true
+autorestart=true
+stderr_logfile=/var/log/supervisor/dynamic-whitelist.err.log
+stdout_logfile=/var/log/supervisor/dynamic-whitelist.out.log

basic_setup/awss3.sh

@@ -34,7 +34,7 @@ aws s3api put-public-access-block \
   --public-access-block-configuration "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"
 
 # Lebenszyklus-Konfiguration hinzufügen
-echo "Lebenszyklus-Konfiguration hinzufügen (E-Mails werden nach 90 Tagen gelöscht)..."
+echo "Lebenszyklus-Konfiguration hinzufügen (E-Mails werden nach 7 Tagen gelöscht)..."
 aws s3api put-bucket-lifecycle-configuration \
   --bucket ${S3_BUCKET_NAME} \
   --lifecycle-configuration '{
@@ -43,7 +43,7 @@ aws s3api put-bucket-lifecycle-configuration \
         "ID": "DeleteOldEmails",
         "Status": "Enabled",
         "Expiration": {
-          "Days": 14
+          "Days": 7
         },
         "Filter": {
           "Prefix": ""
@@ -76,6 +76,14 @@ aws s3api put-bucket-policy \
     ]
   }'
 
+# ------------------------
+# Cost Allocation Tags setzen
+# ------------------------
+echo "Setze Cost Allocation Tag (BucketName)..."
+aws s3api put-bucket-tagging \
+  --bucket ${S3_BUCKET_NAME} \
+  --tagging "TagSet=[{Key=BucketName,Value=${S3_BUCKET_NAME}}]"
+
 echo "S3 Bucket $S3_BUCKET_NAME wurde erfolgreich erstellt und konfiguriert."
 echo "Bucket-ARN: arn:aws:s3:::$S3_BUCKET_NAME"
 

basic_setup/awsses.sh

@@ -7,9 +7,14 @@
 #   Standard: mail.${DOMAIN_NAME}
 #   Override: export MAIL_FROM_SUBDOMAIN="mailfrom" (nur der Prefix, ohne Domain)
 #
+# Lambda Deployment:
+#   Standard: Überspringt den Deploy (SKIP_LAMBDA_DEPLOY=true)
+#   Override: export SKIP_LAMBDA_DEPLOY="false" (um Lambda neu auszurollen)
+#
 # Beispiel:
 #   export DOMAIN_NAME="buddelectric.net"
 #   export MAIL_FROM_SUBDOMAIN="mailfrom"   # → mailfrom.buddelectric.net
+#   export SKIP_LAMBDA_DEPLOY="false"       # → Lambda wird aktualisiert
 #   ./awsses.sh
 
 set -e
@@ -18,17 +23,20 @@ set -e
 if ! command -v jq &> /dev/null; then echo "Fehler: 'jq' fehlt."; exit 1; fi
 if [ -z "$DOMAIN_NAME" ]; then echo "Fehler: DOMAIN_NAME ist nicht gesetzt."; exit 1; fi
 
-# Prüfen ob Python Code da ist
-PYTHON_FILE="ses_sns_shim_global.py"
-if [ ! -f "$PYTHON_FILE" ]; then
-    echo "Fehler: $PYTHON_FILE nicht gefunden!"
-    exit 1
-fi
-
 # --- VARIABLEN ---
 AWS_REGION=${AWS_REGION:-"us-east-2"}
 EMAIL_PREFIX=${EMAIL_PREFIX:-""}
 CONFIGURATION_SET_NAME="relay-outbound"
+SKIP_LAMBDA_DEPLOY=${SKIP_LAMBDA_DEPLOY:-"true"}
+
+# Prüfen ob Python Code da ist (nur wenn auch deployt werden soll)
+PYTHON_FILE="ses_sns_shim_global.py"
+if [ "$SKIP_LAMBDA_DEPLOY" != "true" ]; then
+    if [ ! -f "$PYTHON_FILE" ]; then
+        echo "Fehler: $PYTHON_FILE nicht gefunden!"
+        exit 1
+    fi
+fi
 
 # MAIL FROM Subdomain (konfigurierbar)
 MAIL_FROM_SUBDOMAIN=${MAIL_FROM_SUBDOMAIN:-"mail"}
@@ -49,6 +57,7 @@ LAMBDA_ROLE_NAME="SesShimGlobalRole"
 echo "=========================================================="
 echo " SES Setup (S3 -> Global Lambda Shim -> SQS) für $DOMAIN_NAME"
 echo " MAIL FROM: $MAIL_FROM_DOMAIN"
+echo " SKIP_LAMBDA: $SKIP_LAMBDA_DEPLOY"
 echo "=========================================================="
 
 # ---------------------------------------------------------
@@ -113,8 +122,12 @@ echo " -> Permissions aktualisiert."
 sleep 5
 
 # ---------------------------------------------------------
-# 4. Lambda Funktion erstellen/updaten (Global!)
+# 4 & 5. Lambda Funktion + SES Permissions (optional)
 # ---------------------------------------------------------
+if [ "$SKIP_LAMBDA_DEPLOY" = "true" ]; then
+  echo "[4/6] Global Lambda Shim deployen... (ÜBERSPRUNGEN)"
+  echo "[5/6] SES Permission für Lambda... (ÜBERSPRUNGEN)"
+else
   echo "[4/6] Global Lambda Shim deployen..."
   # Zip erstellen
   cp "$PYTHON_FILE" lambda_function.py
@@ -139,15 +152,13 @@ fi
   # Aufräumen
   rm lambda.zip lambda_function.py
 
-# ---------------------------------------------------------
-# 5. Permission: SES darf Lambda aufrufen (Global, einmalig)
-# ---------------------------------------------------------
   echo "[5/6] SES Permission für Lambda..."
   aws lambda add-permission --function-name "$LAMBDA_NAME" \
       --statement-id "AllowSESInvoke-Global" \
       --action "lambda:InvokeFunction" \
       --principal "ses.amazonaws.com" \
       --region "$AWS_REGION" 2>/dev/null || true
+fi
 
 # ---------------------------------------------------------
 # 6. SES Rule (S3 + Global Lambda)

basic_setup/check_logins.py

@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+import csv
+import imaplib
+import sys
+import time
+
+# Konfiguration
+IMAP_SERVER = "secure.emailsrvr.com"
+IMAP_PORT = 993
+DELAY_SECONDS = 1  # Kurze Pause, um Rate-Limiting oder Fail2Ban zu vermeiden
+
+def check_imap_login(email, password):
+    try:
+        # Verbindung zum IMAP-Server via SSL herstellen
+        mail = imaplib.IMAP4_SSL(IMAP_SERVER, IMAP_PORT)
+        mail.login(email, password)
+        mail.logout()
+        return True
+    except imaplib.IMAP4.error:
+        # Login fehlgeschlagen (falsches Passwort/User)
+        return False
+    except Exception as e:
+        print(f"  [!] Netzwerk- oder Serverfehler bei {email}: {e}")
+        return False
+
+def main(csv_filepath):
+    erfolgreich = []
+    fehlgeschlagen = []
+    uebersprungen = []
+
+    try:
+        with open(csv_filepath, mode='r', encoding='utf-8') as f:
+            # Nutze csv.reader für sauberes Parsing der Kommas
+            reader = csv.reader(f)
+            for row_num, row in enumerate(reader, start=1):
+                if not row:
+                    continue
+
+                email = row[0].strip()
+                # Prüfen, ob ein zweites Feld (Passwort) existiert und nicht leer ist
+                password = row[1].strip() if len(row) > 1 else ""
+
+                if not password:
+                    print(f"[{row_num}] Überspringe {email} (Kein Passwort)")
+                    uebersprungen.append(email)
+                    continue
+
+                print(f"[{row_num}] Prüfe {email}... ", end="", flush=True)
+
+                if check_imap_login(email, password):
+                    print("OK")
+                    erfolgreich.append(email)
+                else:
+                    print("FEHLGESCHLAGEN")
+                    fehlgeschlagen.append(email)
+
+                # Kurze Pause einlegen, um den Mailserver nicht zu fluten
+                time.sleep(DELAY_SECONDS)
+
+    except FileNotFoundError:
+        print(f"\nFehler: Die Datei '{csv_filepath}' wurde nicht gefunden.")
+        sys.exit(1)
+    except Exception as e:
+        print(f"\nEin unerwarteter Fehler ist aufgetreten: {e}")
+        sys.exit(1)
+
+    # Ausgabe der Zusammenfassung
+    print("\n" + "="*40)
+    print("ZUSAMMENFASSUNG DER PRÜFUNG")
+    print("="*40)
+
+    print(f"\nErfolgreich ({len(erfolgreich)}):")
+    for e in erfolgreich:
+        print(f"  - {e}")
+
+    print(f"\nFehlgeschlagen ({len(fehlgeschlagen)}):")
+    for e in fehlgeschlagen:
+        print(f"  - {e}")
+
+    print(f"\nÜbersprungen (kein Passwort) ({len(uebersprungen)}):")
+    for e in uebersprungen:
+        print(f"  - {e}")
+
+if __name__ == "__main__":
+    if len(sys.argv) != 2:
+        print("Verwendung: ./check_logins.py <pfad_zur_datei.csv>")
+        sys.exit(1)
+
+    csv_file = sys.argv[1]
+    main(csv_file)

basic_setup/cloudflareMigrationDns.sh

@@ -5,15 +5,22 @@
 # Setzt mail/imap/smtp/pop Subdomains für domain-spezifischen Mailserver-Zugang
 #
 # MIGRATIONS-FLAGS:
-#   SKIP_CLIENT_DNS=true  → Abschnitt 8 (imap/smtp/pop/webmail) + 10 (SRV) überspringen
+#   SKIP_CLIENT_DNS=true    → Abschnitt 8 (mail/imap/smtp/pop/webmail) überspringen
-#                           Nutzen: Client-Subdomains bleiben beim alten Provider
+#                             Nutzen: Client-Subdomains bleiben komplett beim alten Provider
+#   SKIP_PROTOCOL_DNS=true  → nur imap/smtp/pop überspringen, aber mail + webmail setzen
+#                             Nutzen: Webmail/Autodiscover vorbereiten, Mailclients bleiben beim alten Provider
 #   SKIP_DMARC=true         → Abschnitt 7 (DMARC) überspringen
 #                             Nutzen: Bestehenden DMARC-Record nicht anfassen
 #
 # Typischer Migrations-Ablauf:
-#   Phase 0 (Vorbereitung):  SKIP_CLIENT_DNS=true SKIP_DMARC=true  → nur SES + SPF
+#   Phase 0a (Vorbereitung, Client-Records bleiben alt):
-#   Phase 1 (MX Cutover):    MX umstellen (manuell)
+#       SKIP_CLIENT_DNS=true SKIP_DMARC=true  → nur SES + SPF/DKIM/MailFrom
-#   Phase 2 (Client Switch):  ohne SKIP Flags → alle Records setzen
+#   Phase 0b (Webmail vorbereiten, imap/smtp/pop bleiben alt):
+#       SKIP_CLIENT_DNS=false SKIP_PROTOCOL_DNS=true SKIP_DMARC=false
+#   Phase 1 (MX Cutover):
+#       MX manuell umstellen
+#   Phase 2 (Client Switch):
+#       SKIP_CLIENT_DNS=false SKIP_PROTOCOL_DNS=false → alle Client-Records setzen
 
 set -e
 
@@ -23,6 +30,7 @@ DRY_RUN=${DRY_RUN:-"false"}
 
 # Migrations-Flags (NEU)
 SKIP_CLIENT_DNS=${SKIP_CLIENT_DNS:-"false"}
+SKIP_PROTOCOL_DNS=${SKIP_PROTOCOL_DNS:-"false"}
 SKIP_DMARC=${SKIP_DMARC:-"false"}
 
 # IP des Mailservers - PFLICHT wenn keine CNAME-Kette gewünscht
@@ -50,7 +58,8 @@ echo " 🌍  Region: $AWS_REGION"
 echo " 📬  Mail-Server Target: $TARGET_MAIL_SERVER"
 [ -n "$MAIL_SERVER_IP" ] && echo " 🖥️  Server IP: $MAIL_SERVER_IP"
 [ "$DRY_RUN" = "true" ] && echo " ⚠️  DRY RUN MODE - Keine Änderungen!"
-[ "$SKIP_CLIENT_DNS" = "true" ] && echo " ⏭️  SKIP: Client-Subdomains (imap/smtp/pop/webmail/SRV)"
+[ "$SKIP_CLIENT_DNS" = "true" ] && echo " ⏭️  SKIP: alle Client-Subdomains (mail/imap/smtp/pop/webmail)"
+[ "$SKIP_PROTOCOL_DNS" = "true" ] && echo " ⏭️  SKIP: Protokoll-Subdomains imap/smtp/pop bleiben unverändert"
 [ "$SKIP_DMARC" = "true" ] && echo " ⏭️  SKIP: DMARC Record"
 echo "============================================================"
 
@@ -81,8 +90,23 @@ ensure_record() {
     local search_res=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records?type=$type&name=$name" \
         -H "Authorization: Bearer $CF_API_TOKEN" -H "Content-Type: application/json")
     
-    local rec_id=$(echo "$search_res" | jq -r '.result[0].id')
+    local rec_id
-    local rec_content=$(echo "$search_res" | jq -r '.result[0].content')
+    local rec_content
+
+    if [ "$type" == "TXT" ] && [ "$name" == "$DOMAIN_NAME" ] && [[ "$content" == v=spf1* ]]; then
+        # Spezialfall Root-Domain SPF: Filtere gezielt den SPF-Eintrag heraus,
+        # damit z.B. Google Site Verification nicht überschrieben wird.
+        rec_id=$(echo "$search_res" | jq -r '.result[] | select(.content | contains("v=spf1")) | .id' | head -n 1)
+        rec_content=$(echo "$search_res" | jq -r '.result[] | select(.content | contains("v=spf1")) | .content' | head -n 1)
+    else
+        # Standardverhalten für alle anderen (A, CNAME, MX, etc.)
+        rec_id=$(echo "$search_res" | jq -r '.result[0].id')
+        rec_content=$(echo "$search_res" | jq -r '.result[0].content')
+    fi
+
+    # Fallback für jq, damit das restliche Skript funktioniert
+    [ -z "$rec_id" ] && rec_id="null"
+    [ -z "$rec_content" ] && rec_content="null"
 
     if [ "$type" == "MX" ]; then
         json_data=$(jq -n --arg t "$type" --arg n "$name" --arg c "$content" --argjson p "$proxied" --argjson prio "$priority" \
@@ -240,14 +264,23 @@ else
 fi
 
 # ------------------------------------------------------------------
-# SCHRITT 6: Root Domain MX (nur Info, wird nicht geändert)
+# SCHRITT 6: Root Domain MX
 # ------------------------------------------------------------------
 echo ""
-echo "--- 6. Root Domain MX (nur Info, wird nicht geändert) ---"
+echo "--- 6. Root Domain MX ---"
-CURRENT_MX=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records?type=MX&name=$DOMAIN_NAME" \
+# Prüfen, ob bereits MX-Records für die Root-Domain existieren
+MX_COUNT=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records?type=MX&name=$DOMAIN_NAME" \
     -H "Authorization: Bearer $CF_API_TOKEN" -H "Content-Type: application/json" \
-    | jq -r '.result[0].content // "keiner"')
+    | jq -r '.result | length')
-echo "   ℹ️  MX vorhanden: $CURRENT_MX (wird nicht geändert)"
+
+if [ "$MX_COUNT" -eq 0 ]; then
+    echo "   ℹ️  Kein MX-Record vorhanden. Setze initialen SES Inbound MX..."
+    ensure_record "MX" "$DOMAIN_NAME" "inbound-smtp.${AWS_REGION}.amazonaws.com" false 10
+else
+    echo "   ⛔ MX-Record(s) für @ bereits vorhanden ($MX_COUNT Eintrag/Einträge)."
+    echo "      → Wird zum Schutz der alten Postfächer AUF KEINEN FALL automatisch geändert!"
+    echo "      → Muss nach der Migration manuell umgestellt werden."
+fi
 
 # ------------------------------------------------------------------
 # SCHRITT 7: DMARC
@@ -268,11 +301,12 @@ echo ""
 echo "--- 8. Mailclient Subdomains (A + CNAME) ---"
 if [ "$SKIP_CLIENT_DNS" = "true" ]; then
     echo "   ⏭️  Übersprungen (SKIP_CLIENT_DNS=true)"
-    echo "   ℹ️  imap/smtp/pop/webmail bleiben beim alten Provider."
+    echo "   ℹ️  mail/imap/smtp/pop/webmail bleiben beim alten Provider."
     echo "   ℹ️  Setze SKIP_CLIENT_DNS=false nach MX-Cutover + Client-Umstellung."
 else
     if [ -n "$MAIL_SERVER_IP" ]; then
         # A-Record für mail.<domain> direkt auf Server-IP
+        # Wichtig: mail muss DNS-only bleiben; Cloudflare Proxy funktioniert nicht für SMTP/IMAP/POP.
         ensure_record "A" "mail.$DOMAIN_NAME" "$MAIL_SERVER_IP" false
     else
         # CNAME auf externen Ziel-Host (nur wenn verschieden)
@@ -281,11 +315,19 @@ else
         fi
     fi
 
-    # imap, smtp, pop, webmail → CNAME auf mail.<domain>
+    # Webmail kann bereits vorbereitet werden, auch wenn imap/smtp/pop noch beim alten Provider bleiben.
+    ensure_record "CNAME" "webmail.$DOMAIN_NAME" "mail.$DOMAIN_NAME" false
+
+    if [ "$SKIP_PROTOCOL_DNS" = "true" ]; then
+        echo "   ⏭️  Überspringe imap/smtp/pop (SKIP_PROTOCOL_DNS=true)"
+        echo "   ℹ️  imap/smtp/pop bleiben unverändert beim alten Provider."
+    else
+        # imap/smtp/pop → CNAME auf mail.<domain>
+        # Wichtig: diese Records müssen DNS-only bleiben; Cloudflare Proxy funktioniert nicht für Mail-Protokolle.
         ensure_record "CNAME" "imap.$DOMAIN_NAME" "mail.$DOMAIN_NAME" false
         ensure_record "CNAME" "smtp.$DOMAIN_NAME" "mail.$DOMAIN_NAME" false
         ensure_record "CNAME" "pop.$DOMAIN_NAME"  "mail.$DOMAIN_NAME" false
-    ensure_record "CNAME" "webmail.$DOMAIN_NAME" "mail.$DOMAIN_NAME" false
+    fi
 fi
 
 # ------------------------------------------------------------------
@@ -296,19 +338,6 @@ echo "--- 9. Autodiscover / Autoconfig ---"
 ensure_record "CNAME" "autodiscover.$DOMAIN_NAME" "mail.$DOMAIN_NAME" false
 ensure_record "CNAME" "autoconfig.$DOMAIN_NAME"   "mail.$DOMAIN_NAME" false
 
-# ------------------------------------------------------------------
-# SCHRITT 10: SRV Records
-# ------------------------------------------------------------------
-echo ""
-echo "--- 10. SRV Records ---"
-if [ "$SKIP_CLIENT_DNS" = "true" ]; then
-    echo "   ⏭️  Übersprungen (SKIP_CLIENT_DNS=true)"
-else
-    ensure_record "SRV" "_imap._tcp.$DOMAIN_NAME"       "0 5 143 mail.$DOMAIN_NAME" false
-    ensure_record "SRV" "_imaps._tcp.$DOMAIN_NAME"      "0 5 993 mail.$DOMAIN_NAME" false
-    ensure_record "SRV" "_submission._tcp.$DOMAIN_NAME"  "0 5 587 mail.$DOMAIN_NAME" false
-fi
-
 echo ""
 echo "============================================================"
 echo "✅ Fertig für Domain: $DOMAIN_NAME"
@@ -316,7 +345,12 @@ if [ "$SKIP_CLIENT_DNS" = "true" ]; then
     echo ""
     echo "   ⚠️  Client-Subdomains wurden NICHT geändert."
     echo "   Nach MX-Cutover + Worker-Validierung erneut ausführen mit:"
-    echo "   SKIP_CLIENT_DNS=false SKIP_DMARC=false ./cloudflareMigrationDns.sh"
+    echo "   SKIP_CLIENT_DNS=false SKIP_PROTOCOL_DNS=false SKIP_DMARC=false ./cloudflareMigrationDns.sh"
+elif [ "$SKIP_PROTOCOL_DNS" = "true" ]; then
+    echo ""
+    echo "   ⚠️  imap/smtp/pop wurden NICHT geändert."
+    echo "   Für den finalen Client-Cutover erneut ausführen mit:"
+    echo "   SKIP_CLIENT_DNS=false SKIP_PROTOCOL_DNS=false SKIP_DMARC=false ./cloudflareMigrationDns.sh"
 fi
 echo ""
 echo "   Mailclient-Konfiguration für Kunden:"

basic_setup/create-queue.sh

@@ -1,55 +1,58 @@
 #!/bin/bash
-# create-queue.sh
+# create-queue.sh (v2 — mit SNS Fan-Out + Standby Queue)
-# Usage: DOMAIN=andreasknuth.de ./create-queue.sh
+# Usage: DOMAIN_NAME=andreasknuth.de ./create-queue.sh
+#
+# Erstellt pro Domain:
+#   - Primary Queue + DLQ          (wie bisher, für Contabo)
+#   - Standby Queue + DLQ          (NEU, für Office-VM)
+#   - SNS Topic                    (NEU, Fan-Out)
+#   - 2 SNS Subscriptions          (NEU, Topic → Primary + Standby)
 
 set -e
 
 AWS_REGION="us-east-2"
 
-# Domain aus Environment Variable
+if [ -z "$DOMAIN_NAME" ]; then
-if [ -z "$DOMAIN" ]; then
+    echo "Error: DOMAIN_NAME environment variable not set"
-    echo "Error: DOMAIN environment variable not set"
+    echo "Usage: DOMAIN_NAME=andreasknuth.de $0"
-    echo "Usage: DOMAIN=andreasknuth.de $0"
     exit 1
 fi
 
-QUEUE_NAME="${DOMAIN//./-}-queue"
+DOMAIN_SLUG="${DOMAIN_NAME//./-}"
+QUEUE_NAME="${DOMAIN_SLUG}-queue"
 DLQ_NAME="${QUEUE_NAME}-dlq"
+STANDBY_QUEUE_NAME="${DOMAIN_SLUG}-standby-queue"
+STANDBY_DLQ_NAME="${STANDBY_QUEUE_NAME}-dlq"
+TOPIC_NAME="${DOMAIN_SLUG}-topic"
+
+ACCOUNT_ID=$(aws sts get-caller-identity --query 'Account' --output text)
 
 echo "========================================"
-echo "Creating SQS Queue for Email Delivery"
+echo "Creating SQS + SNS for Email Delivery"
 echo "========================================"
 echo ""
-echo "📧 Domain: $DOMAIN"
+echo "📧 Domain:        $DOMAIN_NAME"
 echo "   Region:        $AWS_REGION"
+echo "   Account:       $ACCOUNT_ID"
 echo ""
 
-# Dead Letter Queue erstellen
+# ============================================================
+# 1. Primary DLQ + Queue (wie bisher)
+# ============================================================
+echo "━━━ Primary Queue (Contabo) ━━━"
+
 echo "Creating DLQ: $DLQ_NAME"
 DLQ_URL=$(aws sqs create-queue \
     --queue-name "${DLQ_NAME}" \
     --region "${AWS_REGION}" \
-    --attributes '{
+    --attributes '{"MessageRetentionPeriod": "1209600"}' \
-        "MessageRetentionPeriod": "1209600"
+    --query 'QueueUrl' --output text 2>/dev/null \
-    }' \
+    || aws sqs get-queue-url --queue-name "${DLQ_NAME}" --region "${AWS_REGION}" --query 'QueueUrl' --output text)
-    --query 'QueueUrl' \
+DLQ_ARN=$(aws sqs get-queue-attributes --queue-url "${DLQ_URL}" --region "${AWS_REGION}" \
-    --output text 2>/dev/null || aws sqs get-queue-url --queue-name "${DLQ_NAME}" --region "${AWS_REGION}" --query 'QueueUrl' --output text)
+    --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)
+echo "   ✓ DLQ: ${DLQ_ARN}"
 
-echo "   ✓ DLQ URL: ${DLQ_URL}"
+echo "Creating Queue: $QUEUE_NAME"
-
-# DLQ ARN ermitteln
-DLQ_ARN=$(aws sqs get-queue-attributes \
-    --queue-url "${DLQ_URL}" \
-    --region "${AWS_REGION}" \
-    --attribute-names QueueArn \
-    --query 'Attributes.QueueArn' \
-    --output text)
-
-echo "   ✓ DLQ ARN: ${DLQ_ARN}"
-echo ""
-
-# Haupt-Queue erstellen mit Redrive Policy
-echo "Creating Main Queue: $QUEUE_NAME"
 QUEUE_URL=$(aws sqs create-queue \
     --queue-name "${QUEUE_NAME}" \
     --region "${AWS_REGION}" \
@@ -59,18 +62,146 @@ QUEUE_URL=$(aws sqs create-queue \
         \"ReceiveMessageWaitTimeSeconds\": \"20\",
         \"RedrivePolicy\": \"{\\\"deadLetterTargetArn\\\":\\\"${DLQ_ARN}\\\",\\\"maxReceiveCount\\\":\\\"3\\\"}\"
     }" \
-    --query 'QueueUrl' \
+    --query 'QueueUrl' --output text 2>/dev/null \
-    --output text 2>/dev/null || aws sqs get-queue-url --queue-name "${QUEUE_NAME}" --region "${AWS_REGION}" --query 'QueueUrl' --output text)
+    || aws sqs get-queue-url --queue-name "${QUEUE_NAME}" --region "${AWS_REGION}" --query 'QueueUrl' --output text)
+QUEUE_ARN=$(aws sqs get-queue-attributes --queue-url "${QUEUE_URL}" --region "${AWS_REGION}" \
+    --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)
+echo "   ✓ Queue: ${QUEUE_ARN}"
+echo ""
 
-echo "   ✓ Queue URL: ${QUEUE_URL}"
+# ============================================================
+# 2. Standby DLQ + Queue (NEU)
+# ============================================================
+echo "━━━ Standby Queue (Office-VM) ━━━"
+
+echo "Creating Standby DLQ: $STANDBY_DLQ_NAME"
+STANDBY_DLQ_URL=$(aws sqs create-queue \
+    --queue-name "${STANDBY_DLQ_NAME}" \
+    --region "${AWS_REGION}" \
+    --attributes '{"MessageRetentionPeriod": "1209600"}' \
+    --query 'QueueUrl' --output text 2>/dev/null \
+    || aws sqs get-queue-url --queue-name "${STANDBY_DLQ_NAME}" --region "${AWS_REGION}" --query 'QueueUrl' --output text)
+STANDBY_DLQ_ARN=$(aws sqs get-queue-attributes --queue-url "${STANDBY_DLQ_URL}" --region "${AWS_REGION}" \
+    --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)
+echo "   ✓ Standby DLQ: ${STANDBY_DLQ_ARN}"
+
+echo "Creating Standby Queue: $STANDBY_QUEUE_NAME"
+STANDBY_QUEUE_URL=$(aws sqs create-queue \
+    --queue-name "${STANDBY_QUEUE_NAME}" \
+    --region "${AWS_REGION}" \
+    --attributes "{
+        \"VisibilityTimeout\": \"300\",
+        \"MessageRetentionPeriod\": \"86400\",
+        \"ReceiveMessageWaitTimeSeconds\": \"20\",
+        \"RedrivePolicy\": \"{\\\"deadLetterTargetArn\\\":\\\"${STANDBY_DLQ_ARN}\\\",\\\"maxReceiveCount\\\":\\\"3\\\"}\"
+    }" \
+    --query 'QueueUrl' --output text 2>/dev/null \
+    || aws sqs get-queue-url --queue-name "${STANDBY_QUEUE_NAME}" --region "${AWS_REGION}" --query 'QueueUrl' --output text)
+STANDBY_QUEUE_ARN=$(aws sqs get-queue-attributes --queue-url "${STANDBY_QUEUE_URL}" --region "${AWS_REGION}" \
+    --attribute-names QueueArn --query 'Attributes.QueueArn' --output text)
+echo "   ✓ Standby Queue: ${STANDBY_QUEUE_ARN}"
 echo ""
+
+# ============================================================
+# 3. SNS Topic (NEU)
+# ============================================================
+echo "━━━ SNS Topic (Fan-Out) ━━━"
+
+echo "Creating Topic: $TOPIC_NAME"
+TOPIC_ARN=$(aws sns create-topic \
+    --name "${TOPIC_NAME}" \
+    --region "${AWS_REGION}" \
+    --query 'TopicArn' --output text)
+echo "   ✓ Topic: ${TOPIC_ARN}"
+echo ""
+
+# ============================================================
+# 4. SNS → SQS Subscriptions (NEU)
+# ============================================================
+echo "━━━ Subscriptions ━━━"
+
+# SNS braucht Berechtigung, in die SQS Queues zu schreiben
+# Policy für Primary Queue
+POLICY_PRIMARY="{
+  \"Version\": \"2012-10-17\",
+  \"Statement\": [{
+    \"Effect\": \"Allow\",
+    \"Principal\": {\"Service\": \"sns.amazonaws.com\"},
+    \"Action\": \"sqs:SendMessage\",
+    \"Resource\": \"${QUEUE_ARN}\",
+    \"Condition\": {\"ArnEquals\": {\"aws:SourceArn\": \"${TOPIC_ARN}\"}}
+  }]
+}"
+
+aws sqs set-queue-attributes \
+    --queue-url "${QUEUE_URL}" \
+    --region "${AWS_REGION}" \
+    --attributes "{\"Policy\": $(echo "$POLICY_PRIMARY" | jq -c '.' | jq -Rs '.')}" \
+    > /dev/null
+echo "   ✓ Primary Queue Policy gesetzt"
+
+# Policy für Standby Queue
+POLICY_STANDBY="{
+  \"Version\": \"2012-10-17\",
+  \"Statement\": [{
+    \"Effect\": \"Allow\",
+    \"Principal\": {\"Service\": \"sns.amazonaws.com\"},
+    \"Action\": \"sqs:SendMessage\",
+    \"Resource\": \"${STANDBY_QUEUE_ARN}\",
+    \"Condition\": {\"ArnEquals\": {\"aws:SourceArn\": \"${TOPIC_ARN}\"}}
+  }]
+}"
+
+aws sqs set-queue-attributes \
+    --queue-url "${STANDBY_QUEUE_URL}" \
+    --region "${AWS_REGION}" \
+    --attributes "{\"Policy\": $(echo "$POLICY_STANDBY" | jq -c '.' | jq -Rs '.')}" \
+    > /dev/null
+echo "   ✓ Standby Queue Policy gesetzt"
+
+# Subscription: Topic → Primary Queue
+SUB_PRIMARY=$(aws sns subscribe \
+    --topic-arn "${TOPIC_ARN}" \
+    --protocol sqs \
+    --notification-endpoint "${QUEUE_ARN}" \
+    --region "${AWS_REGION}" \
+    --attributes '{"RawMessageDelivery": "true"}' \
+    --query 'SubscriptionArn' --output text)
+echo "   ✓ Subscription Primary: ${SUB_PRIMARY}"
+
+# Subscription: Topic → Standby Queue
+SUB_STANDBY=$(aws sns subscribe \
+    --topic-arn "${TOPIC_ARN}" \
+    --protocol sqs \
+    --notification-endpoint "${STANDBY_QUEUE_ARN}" \
+    --region "${AWS_REGION}" \
+    --attributes '{"RawMessageDelivery": "true"}' \
+    --query 'SubscriptionArn' --output text)
+echo "   ✓ Subscription Standby: ${SUB_STANDBY}"
+echo ""
+
+# ============================================================
+# Zusammenfassung
+# ============================================================
 echo "========================================"
-echo "✅ Queue created successfully!"
+echo "✅ Setup complete for $DOMAIN_NAME"
 echo "========================================"
 echo ""
-echo "Configuration:"
+echo "Primary (Contabo):"
-echo "  Domain:     $DOMAIN"
+echo "  Queue:    $QUEUE_URL"
-echo "  Queue:      $QUEUE_NAME"
+echo "  DLQ:      $DLQ_URL"
-echo "  Queue URL:  $QUEUE_URL"
+echo ""
-echo "  DLQ:        $DLQ_NAME"
+echo "Standby (Office-VM):"
-echo "  Region:     $AWS_REGION"
+echo "  Queue:    $STANDBY_QUEUE_URL"
+echo "  DLQ:      $STANDBY_DLQ_URL"
+echo ""
+echo "SNS Fan-Out:"
+echo "  Topic:    $TOPIC_ARN"
+echo "  → Primary:  $SUB_PRIMARY"
+echo "  → Standby:  $SUB_STANDBY"
+echo ""
+echo "⚠️  Nächste Schritte:"
+echo "  1. Lambda-Funktion updaten: sns.publish() statt sqs.send_message()"
+echo "  2. Lambda IAM Role: sns:Publish Berechtigung hinzufügen"
+echo "  3. Worker auf Office-VM: QUEUE_SUFFIX=-standby-queue konfigurieren"
+echo "  4. Worker auf Office-VM: STANDBY_MODE=true setzen"

basic_setup/legacy/s3-retention.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+echo "Aktualisiere Lifecycle-Regeln (7 Tage) für alle E-Mail-Buckets..."
+
+for BUCKET in $(aws s3api list-buckets --query 'Buckets[].Name' --output text); do
+    
+    # Prüfen, ob der Name auf '-emails' endet
+    if [[ "$BUCKET" == *-emails ]]; then
+        echo "⚙️ Setze 7-Tage-Regel für: $BUCKET"
+        
+        aws s3api put-bucket-lifecycle-configuration \
+            --bucket "$BUCKET" \
+            --lifecycle-configuration '{
+              "Rules": [
+                {
+                  "ID": "DeleteOldEmails",
+                  "Status": "Enabled",
+                  "Expiration": {
+                    "Days": 7
+                  },
+                  "Filter": {
+                    "Prefix": ""
+                  }
+                }
+              ]
+            }'
+    else
+        echo "⏭️ Überspringe (kein E-Mail-Bucket): $BUCKET"
+    fi
+done
+
+echo "Fertig! Alle E-Mail-Buckets löschen jetzt Objekte nach 7 Tagen."

basic_setup/legacy/sync-s3-tags.sh

@@ -0,0 +1,31 @@
+#!/bin/bash
+# sync-s3-tags.sh - Synchronisiert Bucket Tags welche fuer die Abrechung verwendet werden
+echo "Passe Bucket-Tags an..."
+
+for BUCKET in $(aws s3api list-buckets --query 'Buckets[].Name' --output text); do
+    # Aktuellen Tag abfragen (Fehler unterdrücken)
+    TAG=$(aws s3api get-bucket-tagging --bucket "$BUCKET" --query 'TagSet[?Key==`BucketName`].Value' --output text 2>/dev/null)
+    
+    # Prüfen, ob der Name auf '-emails' endet
+    if [[ "$BUCKET" == *-emails ]]; then
+        # Soll getaggt sein
+        if [ -z "$TAG" ] || [ "$TAG" == "None" ]; then
+            echo "➕ Setze fehlendes Tag für: $BUCKET"
+            aws s3api put-bucket-tagging \
+                --bucket "$BUCKET" \
+                --tagging "TagSet=[{Key=BucketName,Value=$BUCKET}]"
+        else
+            echo "✅ OK (bereits getaggt): $BUCKET"
+        fi
+    else
+        # Soll NICHT getaggt sein
+        if [ -n "$TAG" ] && [ "$TAG" != "None" ]; then
+            echo "🗑️ Entferne Tag von: $BUCKET"
+            aws s3api delete-bucket-tagging --bucket "$BUCKET"
+        else
+            echo "✅ OK (ohne Tag): $BUCKET"
+        fi
+    fi
+done
+
+echo "Fertig! Alle '-emails' Buckets sind getaggt, bei allen anderen wurden die Tags entfernt."

basic_setup/mailadminDns.sh

@@ -0,0 +1,158 @@
+#!/bin/bash
+# mailadminDns.sh
+# ------------------------------------------------------------------
+# Setzt AUSSCHLIESSLICH den DNS-Record fuer den mailadmin-Zugang.
+# Faesst NICHTS anderes an (kein SES, SPF, DKIM, MX, keine anderen
+# Subdomains). Kann gefahrlos auf bereits migrierten Domains laufen.
+#
+# Verwendet dieselbe ensure_record-Logik wie cloudflareMigrationDns.sh,
+# damit das Verhalten (Create/Update/Skip-bei-identisch) identisch ist.
+#
+# ------------------------------------------------------------------
+# PFLICHT-VARIABLEN:
+#   DOMAIN_NAME      Kundendomain, z.B. innungsapp.com
+#   CF_API_TOKEN     Cloudflare API Token
+#
+# OPTIONALE VARIABLEN:
+#   MAILADMIN_HOST   Subdomain-Label, Default: "mailadmin"
+#                    -> ergibt mailadmin.<DOMAIN_NAME>
+#   MAILADMIN_TARGET CNAME-Ziel, Default: "mail.<DOMAIN_NAME>"
+#                    Beispiele:
+#                      mail.innungsapp.com          (pro-Domain Modell)
+#                      mailadmin.bayarea-cc.com     (zentrales Modell)
+#   DRY_RUN          "true" -> zeigt nur an, aendert nichts
+#
+# ------------------------------------------------------------------
+# BEISPIELE:
+#
+#   # Standard: mailadmin.innungsapp.com -> mail.innungsapp.com
+#   DOMAIN_NAME=innungsapp.com CF_API_TOKEN=xxx ./mailadminDns.sh
+#
+#   # Zentrales Modell: mailadmin.innungsapp.com -> mailadmin.bayarea-cc.com
+#   DOMAIN_NAME=innungsapp.com \
+#   MAILADMIN_TARGET=mailadmin.bayarea-cc.com \
+#   CF_API_TOKEN=xxx ./mailadminDns.sh
+#
+#   # Erst testen ohne zu aendern
+#   DOMAIN_NAME=innungsapp.com CF_API_TOKEN=xxx DRY_RUN=true ./mailadminDns.sh
+#
+# ------------------------------------------------------------------
+set -e
+
+# --- KONFIGURATION ---
+DRY_RUN=${DRY_RUN:-"false"}
+MAILADMIN_HOST=${MAILADMIN_HOST:-"mailadmin"}
+MAILADMIN_TARGET=${MAILADMIN_TARGET:-"mail.${DOMAIN_NAME}"}
+
+# --- CHECKS ---
+if [ -z "$DOMAIN_NAME" ]; then echo "❌ Fehler: DOMAIN_NAME fehlt."; exit 1; fi
+if [ -z "$CF_API_TOKEN" ]; then echo "❌ Fehler: CF_API_TOKEN fehlt."; exit 1; fi
+if ! command -v jq &> /dev/null; then echo "❌ Fehler: 'jq' fehlt."; exit 1; fi
+if ! command -v curl &> /dev/null; then echo "❌ Fehler: 'curl' fehlt."; exit 1; fi
+
+RECORD_NAME="${MAILADMIN_HOST}.${DOMAIN_NAME}"
+
+echo "============================================================"
+echo " 🔧  mailadmin DNS Setup"
+echo " 🌐  Domain:  $DOMAIN_NAME"
+echo " 📍  Record:  CNAME $RECORD_NAME → $MAILADMIN_TARGET"
+[ "$DRY_RUN" = "true" ] && echo " ⚠️  DRY RUN MODE - Keine Änderungen!"
+echo "============================================================"
+
+# Schutz: CNAME auf sich selbst macht keinen Sinn
+if [ "$RECORD_NAME" == "$MAILADMIN_TARGET" ]; then
+    echo "❌ Fehler: CNAME-Ziel ist identisch mit dem Record-Namen."
+    echo "   $RECORD_NAME kann nicht auf sich selbst zeigen."
+    echo "   Setze MAILADMIN_TARGET auf einen anderen Host."
+    exit 1
+fi
+
+# 1. ZONE ID HOLEN
+echo "🔍 Suche Cloudflare Zone ID..."
+ZONE_ID=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones?name=$DOMAIN_NAME" \
+    -H "Authorization: Bearer $CF_API_TOKEN" -H "Content-Type: application/json" | jq -r '.result[0].id')
+
+if [ "$ZONE_ID" == "null" ] || [ -z "$ZONE_ID" ]; then
+    echo "❌ Zone nicht gefunden für $DOMAIN_NAME."
+    exit 1
+fi
+echo "   ✅ Zone ID: $ZONE_ID"
+
+# ------------------------------------------------------------------
+# FUNKTION: ensure_record
+# Identisch zur Logik in cloudflareMigrationDns.sh, reduziert auf
+# die Typen die wir hier brauchen (CNAME).
+# ------------------------------------------------------------------
+ensure_record() {
+    local type=$1
+    local name=$2
+    local content=$3
+    local proxied=${4:-false}
+
+    echo "   ⚙️  Prüfe $type $name..."
+
+    local search_res=$(curl -s -X GET "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records?type=$type&name=$name" \
+        -H "Authorization: Bearer $CF_API_TOKEN" -H "Content-Type: application/json")
+
+    local rec_id=$(echo "$search_res" | jq -r '.result[0].id')
+    local rec_content=$(echo "$search_res" | jq -r '.result[0].content')
+
+    [ -z "$rec_id" ] && rec_id="null"
+    [ -z "$rec_content" ] && rec_content="null"
+
+    local json_data=$(jq -n --arg t "$type" --arg n "$name" --arg c "$content" --argjson p "$proxied" \
+        '{type: $t, name: $n, content: $c, ttl: 3600, proxied: $p}')
+
+    if [ "$rec_id" == "null" ]; then
+        if [ "$DRY_RUN" = "true" ]; then
+            echo "      [DRY] Würde ERSTELLEN: $type $name → $content"
+        else
+            local res=$(curl -s -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records" \
+                -H "Authorization: Bearer $CF_API_TOKEN" -H "Content-Type: application/json" --data "$json_data")
+            if [ "$(echo $res | jq -r .success)" == "true" ]; then
+                echo "      ✅ Erstellt: $type $name → $content"
+            else
+                echo "      ❌ Fehler beim Erstellen: $(echo $res | jq -r '.errors[0].message')"
+                exit 1
+            fi
+        fi
+    else
+        if [ "$rec_content" == "$content" ]; then
+            echo "      🆗 Identisch ($rec_content). Überspringe."
+        else
+            if [ "$DRY_RUN" = "true" ]; then
+                echo "      [DRY] Würde UPDATEN: '$rec_content' → '$content'"
+            else
+                local res=$(curl -s -X PUT "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/dns_records/$rec_id" \
+                    -H "Authorization: Bearer $CF_API_TOKEN" -H "Content-Type: application/json" --data "$json_data")
+                if [ "$(echo $res | jq -r .success)" == "true" ]; then
+                    echo "      ✅ Aktualisiert: '$rec_content' → '$content'"
+                else
+                    echo "      ❌ Fehler beim Updaten: $(echo $res | jq -r '.errors[0].message')"
+                    exit 1
+                fi
+            fi
+        fi
+    fi
+}
+
+# ------------------------------------------------------------------
+# Der eigentliche Record
+# proxied=false ist hier wichtig: der mailadmin laeuft hinter Caddy
+# mit eigenem TLS-Zertifikat. Cloudflare-Proxy davor wuerde die
+# Zertifikatskette stoeren bzw. doppeltes TLS-Terminieren.
+# ------------------------------------------------------------------
+echo ""
+echo "--- mailadmin CNAME ---"
+ensure_record "CNAME" "$RECORD_NAME" "$MAILADMIN_TARGET" false
+
+echo ""
+echo "============================================================"
+echo "✅ Fertig."
+echo ""
+echo "   mailadmin erreichbar unter: https://$RECORD_NAME"
+echo ""
+echo "   ⚠️  Wichtig: Caddy muss ein Zertifikat für $RECORD_NAME"
+echo "      ausstellen. Stelle sicher, dass der Hostname in der"
+echo "      Caddy-Konfiguration enthalten ist."
+echo "============================================================"

basic_setup/ses_sns_shim_global.py

@@ -8,24 +8,38 @@ from botocore.exceptions import ClientError
 import time
 import random
 
-# Logging konfigurieren
 logger = logging.getLogger()
 logger.setLevel(logging.INFO)
 
 sqs = boto3.client('sqs')
+sns = boto3.client('sns')
+sts_account_id = None
 
-# Retry-Konfiguration
 MAX_RETRIES = 3
-BASE_BACKOFF = 1  # Sekunden
+BASE_BACKOFF = 1
 
 def exponential_backoff(attempt):
-    """Exponential Backoff mit Jitter"""
     return BASE_BACKOFF * (2 ** attempt) + random.uniform(0, 1)
 
+def get_account_id():
+    global sts_account_id
+    if sts_account_id is None:
+        sts_account_id = boto3.client('sts').get_caller_identity()['Account']
+    return sts_account_id
+
+def get_topic_arn(domain):
+    """
+    Generiert Topic-ARN aus Domain.
+    Konvention: domain.tld -> domain-tld-topic
+    """
+    topic_name = domain.replace('.', '-') + '-topic'
+    region = os.environ.get('AWS_REGION', 'us-east-2')
+    account_id = get_account_id()
+    return f"arn:aws:sns:{region}:{account_id}:{topic_name}"
+
 def get_queue_url(domain):
     """
-    Generiert Queue-Namen aus Domain und holt URL.
+    Fallback: Direkter SQS-Send für Domains ohne SNS-Topic.
-    Konvention: domain.tld -> domain-tld-queue
     """
     queue_name = domain.replace('.', '-') + '-queue'
     try:
@@ -38,11 +52,53 @@ def get_queue_url(domain):
         else:
             raise
 
+def publish_to_sns(topic_arn, message_body, msg_id):
+    attempt = 0
+    while attempt < MAX_RETRIES:
+        try:
+            sns.publish(
+                TopicArn=topic_arn,
+                Message=message_body
+            )
+            logger.info(f"✅ Published {msg_id} to SNS: {topic_arn}")
+            return True
+        except ClientError as e:
+            error_code = e.response['Error']['Code']
+            # Fallback auf SQS bei Topic-nicht-gefunden ODER fehlender Berechtigung
+            if error_code in ('NotFound', 'NotFoundException', 'AuthorizationError'):
+                logger.info(f"ℹ️ SNS unavailable for {topic_arn} ({error_code}) — falling back to SQS")
+                return False
+            attempt += 1
+            logger.warning(f"Retry {attempt}/{MAX_RETRIES} SNS: {error_code}")
+            if attempt == MAX_RETRIES:
+                raise
+            time.sleep(exponential_backoff(attempt))
+    return False
+
+def send_to_sqs(queue_url, message_body, msg_id):
+    """Fallback: Direkter SQS-Send (wie bisher)."""
+    attempt = 0
+    while attempt < MAX_RETRIES:
+        try:
+            sqs.send_message(
+                QueueUrl=queue_url,
+                MessageBody=message_body
+            )
+            logger.info(f"✅ Sent {msg_id} to SQS: {queue_url}")
+            return
+        except ClientError as e:
+            attempt += 1
+            error_code = e.response['Error']['Code']
+            logger.warning(f"Retry {attempt}/{MAX_RETRIES} SQS: {error_code}")
+            if attempt == MAX_RETRIES:
+                raise
+            time.sleep(exponential_backoff(attempt))
+
 def lambda_handler(event, context):
     """
-    Nimmt SES Event entgegen, extrahiert Domain dynamisch,
+    Nimmt SES Event entgegen, extrahiert Domain dynamisch.
-    verpackt Metadaten als 'Fake SNS' und sendet an die domain-spezifische SQS.
+    Strategie: SNS Publish (Fan-Out an Primary + Standby Queue).
-    Mit integrierter Retry-Logik für SQS-Send.
+    Fallback: Direkter SQS-Send falls kein SNS-Topic existiert.
     """
     try:
         records = event.get('Records', [])
@@ -51,13 +107,12 @@ def lambda_handler(event, context):
         for record in records:
             ses_data = record.get('ses', {})
             if not ses_data:
-                logger.warning(f"Invalid SES event: Missing 'ses' in record: {record}")
+                logger.warning(f"Invalid SES event: Missing 'ses' in record")
                 continue
 
             mail = ses_data.get('mail', {})
             receipt = ses_data.get('receipt', {})
 
-            # Domain extrahieren (aus erstem Recipient)
             recipients = receipt.get('recipients', []) or mail.get('destination', [])
             if not recipients:
                 logger.warning("No recipients in event - skipping")
@@ -69,23 +124,19 @@ def lambda_handler(event, context):
                 logger.error("Could not extract domain from recipient")
                 continue
 
-            # Wichtige Metadaten loggen
             msg_id = mail.get('messageId', 'unknown')
             source = mail.get('source', 'unknown')
             logger.info(f"Processing Message-ID: {msg_id} for domain: {domain}")
             logger.info(f"  From: {source}")
             logger.info(f"  To: {recipients}")
 
-            # SES JSON als String serialisieren
             ses_json_string = json.dumps(ses_data)
 
-            # Payload Größe loggen und checken (Safeguard)
             payload_size = len(ses_json_string.encode('utf-8'))
             logger.info(f"  Metadata Payload Size: {payload_size} bytes")
-            if payload_size > 200000:  # Arbitrary Limit < SQS 256KB
+            if payload_size > 200000:
-                raise ValueError("Payload too large for SQS")
+                raise ValueError("Payload too large")
 
-            # Fake SNS Payload
             fake_sns_payload = {
                 "Type": "Notification",
                 "MessageId": str(uuid.uuid4()),
@@ -95,26 +146,16 @@ def lambda_handler(event, context):
                 "Timestamp": datetime.utcnow().isoformat() + "Z"
             }
 
-            # Queue URL dynamisch holen
+            message_body = json.dumps(fake_sns_payload)
-            queue_url = get_queue_url(domain)
 
-            # SQS Send mit Retries
+            # Strategie: SNS zuerst, SQS als Fallback
-            attempt = 0
+            topic_arn = get_topic_arn(domain)
-            while attempt < MAX_RETRIES:
+            sns_success = publish_to_sns(topic_arn, message_body, msg_id)
-                try:
+
-                    sqs.send_message(
+            if not sns_success:
-                        QueueUrl=queue_url,
+                # Kein SNS-Topic für diese Domain → direkt in SQS (wie bisher)
-                        MessageBody=json.dumps(fake_sns_payload)
+                queue_url = get_queue_url(domain)
-                    )
+                send_to_sqs(queue_url, message_body, msg_id)
-                    logger.info(f"✅ Successfully forwarded {msg_id} to SQS: {queue_url}")
-                    break
-                except ClientError as e:
-                    attempt += 1
-                    error_code = e.response['Error']['Code']
-                    logger.warning(f"Retry {attempt}/{MAX_RETRIES} for SQS send: {error_code} - {str(e)}")
-                    if attempt == MAX_RETRIES:
-                        raise
-                    time.sleep(exponential_backoff(attempt))
 
         return {'status': 'ok'}
 

caddy/.gitignore

@@ -0,0 +1,2 @@
+caddy-data/
+caddy-config/

caddy/Dockerfile.caddy

@@ -7,7 +7,16 @@ RUN xcaddy build ${CADDY_VERSION} \
   --with github.com/caddy-dns/cloudflare \
   --with github.com/caddyserver/replace-response
 
+# Autodiscover Handler in Go bauen (Go ist im Builder-Image bereits verfügbar)
+COPY autodiscover-handler.go /src/autodiscover-handler.go
+WORKDIR /src
+RUN CGO_ENABLED=0 go build -ldflags="-s -w" -o /usr/bin/autodiscover-handler autodiscover-handler.go
+
 FROM caddy:${CADDY_VERSION}
 COPY --from=builder /usr/bin/caddy /usr/bin/caddy
-RUN mkdir -p /var/log/caddy
+COPY --from=builder /usr/bin/autodiscover-handler /usr/local/bin/autodiscover-handler
+COPY start.sh /usr/local/bin/start.sh
+RUN chmod +x /usr/local/bin/start.sh /usr/local/bin/autodiscover-handler \
+    && mkdir -p /var/log/caddy
 
+CMD ["/usr/local/bin/start.sh"]

caddy/autodiscover-handler.go

@@ -0,0 +1,109 @@
+package main
+
+import (
+	"fmt"
+	"io"
+	"log"
+	"net/http"
+	"regexp"
+	"strings"
+)
+
+const port = "8280"
+
+var emailRegex = regexp.MustCompile(`(?i)<EMailAddress>([^<]+)</EMailAddress>`)
+
+func main() {
+	http.HandleFunc("/health", func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusOK)
+		fmt.Fprint(w, "OK")
+	})
+
+	http.HandleFunc("/autodiscover/autodiscover.xml", handleAutodiscover)
+	// Outlook sendet manchmal mit Großbuchstaben
+	http.HandleFunc("/Autodiscover/Autodiscover.xml", handleAutodiscover)
+	http.HandleFunc("/AutoDiscover/AutoDiscover.xml", handleAutodiscover)
+
+	log.Printf("[autodiscover] Listening on port %s", port)
+	if err := http.ListenAndServe(":"+port, nil); err != nil {
+		log.Fatal(err)
+	}
+}
+
+func handleAutodiscover(w http.ResponseWriter, r *http.Request) {
+	var email string
+
+	if r.Method == http.MethodPost {
+		body, err := io.ReadAll(r.Body)
+		if err == nil {
+			if match := emailRegex.FindStringSubmatch(string(body)); len(match) > 1 {
+				email = strings.TrimSpace(match[1])
+			}
+		}
+		r.Body.Close()
+	}
+
+	var domain string
+	if email != "" {
+		parts := strings.SplitN(email, "@", 2)
+		if len(parts) == 2 {
+			domain = parts[1]
+		}
+	}
+	if domain == "" {
+		domain = extractDomainFromHost(r.Host)
+	}
+
+	log.Printf("[autodiscover] %s from %s - email=%q domain=%s", r.Method, r.RemoteAddr, email, domain)
+
+	w.Header().Set("Content-Type", "application/xml")
+	w.WriteHeader(http.StatusOK)
+	fmt.Fprint(w, buildResponse(domain, email))
+}
+
+func extractDomainFromHost(host string) string {
+	// Strip port
+	if idx := strings.Index(host, ":"); idx >= 0 {
+		host = host[:idx]
+	}
+	parts := strings.Split(host, ".")
+	if len(parts) >= 3 && strings.EqualFold(parts[0], "autodiscover") {
+		return strings.Join(parts[1:], ".")
+	}
+	if len(parts) >= 2 {
+		return strings.Join(parts[len(parts)-2:], ".")
+	}
+	return host
+}
+
+func buildResponse(domain, loginName string) string {
+	return fmt.Sprintf(`<?xml version="1.0" encoding="utf-8"?>
+<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
+  <Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">
+    <Account>
+      <AccountType>email</AccountType>
+      <Action>settings</Action>
+      <Protocol>
+        <Type>IMAP</Type>
+        <Server>imap.%s</Server>
+        <Port>993</Port>
+        <DomainRequired>off</DomainRequired>
+        <LoginName>%s</LoginName>
+        <SPA>off</SPA>
+        <SSL>on</SSL>
+        <AuthRequired>on</AuthRequired>
+      </Protocol>
+      <Protocol>
+        <Type>SMTP</Type>
+        <Server>smtp.%s</Server>
+        <Port>465</Port>
+        <DomainRequired>off</DomainRequired>
+        <LoginName>%s</LoginName>
+        <SPA>off</SPA>
+        <SSL>on</SSL>
+        <AuthRequired>on</AuthRequired>
+      </Protocol>
+    </Account>
+  </Response>
+</Autodiscover>`, domain, loginName, domain, loginName)
+}

caddy/docker-compose.yml

@@ -13,14 +13,15 @@ services:
       - 'host.docker.internal:host-gateway'
     networks:
       - mail_network
+      - mailadmin_network
     volumes:
       - $PWD/Caddyfile:/etc/caddy/Caddyfile
       - $PWD/mail_certs:/etc/caddy/mail_certs
-      - $PWD/email_autodiscover:/etc/caddy/email_autodiscover
+      # email_autodiscover entfernt - Snippet ist jetzt in mail_certs eingebettet
-      - $PWD/email.mobileconfig.tpl:/etc/caddy/email.mobileconfig.tpl
+      # email.mobileconfig.html entfernt - Inhalt ist jetzt inline in mail_certs
       - $PWD/email-setup:/var/www/email-setup
-      - caddy_data:/data
+      - ./caddy-data:/data
-      - caddy_config:/config
+      - ./caddy-config:/config
       - /home/aknuth/log/caddy:/var/log/caddy
     environment:
       - CLOUDFLARE_API_TOKEN=${CLOUDFLARE_API_TOKEN}
@@ -29,8 +30,5 @@ services:
 networks:
   mail_network:
     external: true
-
+  mailadmin_network:
-volumes:
-  caddy_data:
     external: true
-  caddy_config:

caddy/email-setup/autodiscover.xml

@@ -1,29 +0,0 @@
-<?xml version="1.0" encoding="utf-8" ?>
-<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
-  <Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">
-    <Account>
-      <AccountType>email</AccountType>
-      <Action>settings</Action>
-      <Protocol>
-        <Type>IMAP</Type>
-        <Server>mail.email-srvr.com</Server>
-        <Port>993</Port>
-        <DomainRequired>off</DomainRequired>
-        <LoginName></LoginName>
-        <SPA>off</SPA>
-        <SSL>on</SSL>
-        <AuthRequired>on</AuthRequired>
-      </Protocol>
-      <Protocol>
-        <Type>SMTP</Type>
-        <Server>mail.email-srvr.com</Server>
-        <Port>465</Port>
-        <DomainRequired>off</DomainRequired>
-        <LoginName></LoginName>
-        <SPA>off</SPA>
-        <SSL>on</SSL>
-        <AuthRequired>on</AuthRequired>
-      </Protocol>
-    </Account>
-  </Response>
-</Autodiscover>

caddy/email-setup/setup.html

@@ -35,7 +35,7 @@
 <body>
 
 <div class="card">
-    <img src="/logo.png" alt="Logo" class="logo">
+    <img src="/email-setup/logo.png" alt="Logo" class="logo">
     
     <div id="input-section">
         <h1>Email Setup</h1>

caddy/email.mobileconfig.tpl

@@ -1,67 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
-    <key>PayloadContent</key>
-    <array>
-        <dict>
-            <key>EmailAccountDescription</key>
-            <string>{{.Req.URL.Query.Get "email"}}</string>
-            <key>EmailAccountName</key>
-            <string>{{.Req.URL.Query.Get "email"}}</string>
-            <key>EmailAccountType</key>
-            <string>EmailTypeIMAP</string>
-            <key>EmailAddress</key>
-            <string>{{.Req.URL.Query.Get "email"}}</string>
-            <key>IncomingMailServerAuthentication</key>
-            <string>EmailAuthPassword</string>
-            <key>IncomingMailServerHostName</key>
-            <string>mail.email-srvr.com</string>
-            <key>IncomingMailServerPortNumber</key>
-            <integer>993</integer>
-            <key>IncomingMailServerUseSSL</key>
-            <true/>
-            <key>IncomingMailServerUsername</key>
-            <string>{{.Req.URL.Query.Get "email"}}</string>
-            <key>OutgoingMailServerAuthentication</key>
-            <string>EmailAuthPassword</string>
-            <key>OutgoingMailServerHostName</key>
-            <string>mail.email-srvr.com</string>
-            <key>OutgoingMailServerPortNumber</key>
-            <integer>465</integer>
-            <key>OutgoingMailServerUseSSL</key>
-            <true/>
-            <key>OutgoingMailServerUsername</key>
-            <string>{{.Req.URL.Query.Get "email"}}</string>
-            <key>PayloadDescription</key>
-            <string>E-Mail Konfiguration für {{.Req.URL.Query.Get "email"}}</string>
-            <key>PayloadDisplayName</key>
-            <string>{{.Req.URL.Query.Get "email"}}</string>
-            <key>PayloadIdentifier</key>
-            <string>com.email-srvr.profile.{{.Req.URL.Query.Get "email"}}</string>
-            <key>PayloadType</key>
-            <string>com.apple.mail.managed</string>
-            <key>PayloadUUID</key>
-            <string>{{uuidv4}}</string>
-            <key>PayloadVersion</key>
-            <integer>1</integer>
-        </dict>
-    </array>
-    <key>PayloadDescription</key>
-    <string>Automatische E-Mail Einrichtung für {{.Req.URL.Query.Get "email"}}</string>
-    <key>PayloadDisplayName</key>
-    <string>E-Mail Einstellungen</string>
-    <key>PayloadIdentifier</key>
-    <string>com.email-srvr.profile.root</string>
-    <key>PayloadOrganization</key>
-    <string>IT Support</string>
-    <key>PayloadRemovalDisallowed</key>
-    <false/>
-    <key>PayloadType</key>
-    <string>Configuration</string>
-    <key>PayloadUUID</key>
-    <string>{{uuidv4}}</string>
-    <key>PayloadVersion</key>
-    <integer>1</integer>
-</dict>
-</plist>

caddy/email_autodiscover

@@ -1,112 +0,0 @@
-# email_autodiscover - Dynamisches Autodiscover/Autoconfig Snippet
-# Importiert im Caddyfile via: import email_autodiscover
-#
-# Funktioniert mit JEDER Domain automatisch, solange der Caddy-Block
-# auf autodiscover.<domain> oder autoconfig.<domain> hört.
-#
-# Hostnames werden dynamisch abgeleitet:
-#   autodiscover.cielectrical.com → imap.cielectrical.com / smtp.cielectrical.com
-#   autoconfig.bayarea-cc.com     → imap.bayarea-cc.com / smtp.bayarea-cc.com
-#
-# {labels.2}.{labels.1} extrahiert die Basisdomain aus dem Host:
-#   autodiscover.cielectrical.com → labels: [com=0, cielectrical=1, autodiscover=2]
-#   → {labels.1}.{labels.0} = cielectrical.com
-
-(email_settings) {
-    # 1. Outlook Autodiscover (XML)
-    route /autodiscover/autodiscover.xml {
-        header Content-Type "application/xml"
-        respond `<?xml version="1.0" encoding="utf-8"?>
-<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
-  <Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a">
-    <Account>
-      <AccountType>email</AccountType>
-      <Action>settings</Action>
-      <Protocol>
-        <Type>IMAP</Type>
-        <Server>imap.{labels.1}.{labels.0}</Server>
-        <Port>993</Port>
-        <DomainRequired>on</DomainRequired>
-        <LoginName>{header.X-Anchormailbox}</LoginName>
-        <SPA>off</SPA>
-        <SSL>on</SSL>
-        <AuthRequired>on</AuthRequired>
-      </Protocol>
-      <Protocol>
-        <Type>POP3</Type>
-        <Server>pop.{labels.1}.{labels.0}</Server>
-        <Port>995</Port>
-        <DomainRequired>on</DomainRequired>
-        <LoginName>{header.X-Anchormailbox}</LoginName>
-        <SPA>off</SPA>
-        <SSL>on</SSL>
-        <AuthRequired>on</AuthRequired>
-      </Protocol>
-      <Protocol>
-        <Type>SMTP</Type>
-        <Server>smtp.{labels.1}.{labels.0}</Server>
-        <Port>465</Port>
-        <DomainRequired>on</DomainRequired>
-        <LoginName>{header.X-Anchormailbox}</LoginName>
-        <SPA>off</SPA>
-        <SSL>on</SSL>
-        <AuthRequired>on</AuthRequired>
-      </Protocol>
-    </Account>
-  </Response>
-</Autodiscover>` 200
-    }
-
-    # 2. Modern Outlook (JSON) - Redirect zum XML Endpoint
-    route /autodiscover/autodiscover.json {
-        header Content-Type "application/json"
-        respond `{
-            "Protocol": "AutodiscoverV1",
-            "Url": "https://autodiscover.{labels.1}.{labels.0}/autodiscover/autodiscover.xml"
-        }` 200
-    }
-
-    # 3. Thunderbird Autoconfig
-    route /mail/config-v1.1.xml {
-        header Content-Type "application/xml"
-        respond `<?xml version="1.0"?>
-<clientConfig version="1.1">
-  <emailProvider id="{labels.1}.{labels.0}">
-    <displayName>{labels.1}.{labels.0} Mail</displayName>
-    <domain>{labels.1}.{labels.0}</domain>
-    <incomingServer type="imap">
-      <hostname>imap.{labels.1}.{labels.0}</hostname>
-      <port>993</port>
-      <socketType>SSL</socketType>
-      <authentication>password-cleartext</authentication>
-      <username>%EMAILADDRESS%</username>
-    </incomingServer>
-    <incomingServer type="pop3">
-      <hostname>pop.{labels.1}.{labels.0}</hostname>
-      <port>995</port>
-      <socketType>SSL</socketType>
-      <authentication>password-cleartext</authentication>
-      <username>%EMAILADDRESS%</username>
-    </incomingServer>
-    <outgoingServer type="smtp">
-      <hostname>smtp.{labels.1}.{labels.0}</hostname>
-      <port>465</port>
-      <socketType>SSL</socketType>
-      <authentication>password-cleartext</authentication>
-      <username>%EMAILADDRESS%</username>
-    </outgoingServer>
-  </emailProvider>
-</clientConfig>` 200
-    }
-
-    # 4. Apple MobileConfig
-    route /apple {
-        templates {
-            mime "application/x-apple-aspen-config"
-        }
-        header Content-Type "application/x-apple-aspen-config; charset=utf-8"
-        root * /etc/caddy
-        rewrite * /email.mobileconfig.tpl
-        file_server
-    }
-}

caddy/start.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+set -e
+
+# Autodiscover handler im Hintergrund starten
+/usr/local/bin/autodiscover-handler &
+
+# Caddy im Vordergrund
+exec caddy run --config /etc/caddy/Caddyfile --adapter caddyfile

caddy/update-caddy-certs.sh

@@ -9,6 +9,7 @@
 #   - Wildcard-Cert Block (*.domain + domain)
 #   - Webmail Block (reverse_proxy zu Roundcube)
 #   - Autodiscover/Autoconfig Block (importiert email_settings Snippet)
+#   - Email-Setup Block (QR-Code Seite für iPhone)
 #
 # Bei neuen Domains: Script erneut laufen lassen + caddy reload.
 #
@@ -64,65 +65,34 @@ OUTPUT="${OUTPUT}# Einbinden im Caddyfile: import mail_certs\n"
 OUTPUT="${OUTPUT}# Generiert: $(date)\n"
 OUTPUT="${OUTPUT}\n"
 
-# --- Autodiscover/Autoconfig Snippet einbetten ---
+# =====================================================================
+# Autodiscover/Autoconfig Snippet (dynamisch)
+# {labels.1}.{labels.0} = Basisdomain aus Hostname
+# =====================================================================
 OUTPUT="${OUTPUT}# ═══════════════════════════════════════════════\n"
 OUTPUT="${OUTPUT}# Autodiscover/Autoconfig Snippet (dynamisch)\n"
 OUTPUT="${OUTPUT}# {labels.1}.{labels.0} = Basisdomain aus Hostname\n"
 OUTPUT="${OUTPUT}# ═══════════════════════════════════════════════\n"
 OUTPUT="${OUTPUT}(email_settings) {\n"
-OUTPUT="${OUTPUT}    # Outlook Autodiscover (XML)\n"
+
+# --- 1. Outlook Classic Autodiscover (POST + GET XML) ---
+OUTPUT="${OUTPUT}    # Outlook Autodiscover (XML) - POST und GET\n"
 OUTPUT="${OUTPUT}    route /autodiscover/autodiscover.xml {\n"
-OUTPUT="${OUTPUT}        header Content-Type \"application/xml\"\n"
+OUTPUT="${OUTPUT}      reverse_proxy localhost:8280\n"
-OUTPUT="${OUTPUT}        respond \`<?xml version=\"1.0\" encoding=\"utf-8\"?>\n"
-OUTPUT="${OUTPUT}<Autodiscover xmlns=\"http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006\">\n"
-OUTPUT="${OUTPUT}  <Response xmlns=\"http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a\">\n"
-OUTPUT="${OUTPUT}    <Account>\n"
-OUTPUT="${OUTPUT}      <AccountType>email</AccountType>\n"
-OUTPUT="${OUTPUT}      <Action>settings</Action>\n"
-OUTPUT="${OUTPUT}      <Protocol>\n"
-OUTPUT="${OUTPUT}        <Type>IMAP</Type>\n"
-OUTPUT="${OUTPUT}        <Server>imap.{labels.1}.{labels.0}</Server>\n"
-OUTPUT="${OUTPUT}        <Port>993</Port>\n"
-OUTPUT="${OUTPUT}        <DomainRequired>on</DomainRequired>\n"
-OUTPUT="${OUTPUT}        <LoginName>{header.X-Anchormailbox}</LoginName>\n"
-OUTPUT="${OUTPUT}        <SPA>off</SPA>\n"
-OUTPUT="${OUTPUT}        <SSL>on</SSL>\n"
-OUTPUT="${OUTPUT}        <AuthRequired>on</AuthRequired>\n"
-OUTPUT="${OUTPUT}      </Protocol>\n"
-OUTPUT="${OUTPUT}      <Protocol>\n"
-OUTPUT="${OUTPUT}        <Type>POP3</Type>\n"
-OUTPUT="${OUTPUT}        <Server>pop.{labels.1}.{labels.0}</Server>\n"
-OUTPUT="${OUTPUT}        <Port>995</Port>\n"
-OUTPUT="${OUTPUT}        <DomainRequired>on</DomainRequired>\n"
-OUTPUT="${OUTPUT}        <LoginName>{header.X-Anchormailbox}</LoginName>\n"
-OUTPUT="${OUTPUT}        <SPA>off</SPA>\n"
-OUTPUT="${OUTPUT}        <SSL>on</SSL>\n"
-OUTPUT="${OUTPUT}        <AuthRequired>on</AuthRequired>\n"
-OUTPUT="${OUTPUT}      </Protocol>\n"
-OUTPUT="${OUTPUT}      <Protocol>\n"
-OUTPUT="${OUTPUT}        <Type>SMTP</Type>\n"
-OUTPUT="${OUTPUT}        <Server>smtp.{labels.1}.{labels.0}</Server>\n"
-OUTPUT="${OUTPUT}        <Port>465</Port>\n"
-OUTPUT="${OUTPUT}        <DomainRequired>on</DomainRequired>\n"
-OUTPUT="${OUTPUT}        <LoginName>{header.X-Anchormailbox}</LoginName>\n"
-OUTPUT="${OUTPUT}        <SPA>off</SPA>\n"
-OUTPUT="${OUTPUT}        <SSL>on</SSL>\n"
-OUTPUT="${OUTPUT}        <AuthRequired>on</AuthRequired>\n"
-OUTPUT="${OUTPUT}      </Protocol>\n"
-OUTPUT="${OUTPUT}    </Account>\n"
-OUTPUT="${OUTPUT}  </Response>\n"
-OUTPUT="${OUTPUT}</Autodiscover>\` 200\n"
 OUTPUT="${OUTPUT}    }\n"
 OUTPUT="${OUTPUT}\n"
-OUTPUT="${OUTPUT}    # Modern Outlook (JSON)\n"
+
+# --- 2. Outlook New / Microsoft 365 (JSON v2) ---
+# Outlook New sendet GET auf /autodiscover/autodiscover.json?Protocol=AutodiscoverV1&...
+# Antwort muss den XML-Endpoint zurückgeben
+OUTPUT="${OUTPUT}    # Outlook New/365 (JSON → Redirect zu XML)\n"
 OUTPUT="${OUTPUT}    route /autodiscover/autodiscover.json {\n"
 OUTPUT="${OUTPUT}        header Content-Type \"application/json\"\n"
-OUTPUT="${OUTPUT}        respond \`{\n"
+OUTPUT="${OUTPUT}        respond \`{\"Protocol\":\"AutodiscoverV1\",\"Url\":\"https://autodiscover.{labels.1}.{labels.0}/autodiscover/autodiscover.xml\"}\` 200\n"
-OUTPUT="${OUTPUT}            \"Protocol\": \"AutodiscoverV1\",\n"
-OUTPUT="${OUTPUT}            \"Url\": \"https://autodiscover.{labels.1}.{labels.0}/autodiscover/autodiscover.xml\"\n"
-OUTPUT="${OUTPUT}        }\` 200\n"
 OUTPUT="${OUTPUT}    }\n"
 OUTPUT="${OUTPUT}\n"
+
+# --- 3. Thunderbird Autoconfig ---
 OUTPUT="${OUTPUT}    # Thunderbird Autoconfig\n"
 OUTPUT="${OUTPUT}    route /mail/config-v1.1.xml {\n"
 OUTPUT="${OUTPUT}        header Content-Type \"application/xml\"\n"
@@ -136,34 +106,115 @@ OUTPUT="${OUTPUT}      <hostname>imap.{labels.1}.{labels.0}</hostname>\n"
 OUTPUT="${OUTPUT}      <port>993</port>\n"
 OUTPUT="${OUTPUT}      <socketType>SSL</socketType>\n"
 OUTPUT="${OUTPUT}      <authentication>password-cleartext</authentication>\n"
-OUTPUT="${OUTPUT}      <username>%%EMAILADDRESS%%</username>\n"
+OUTPUT="${OUTPUT}      <username>%EMAILADDRESS%</username>\n"
-OUTPUT="${OUTPUT}    </incomingServer>\n"
-OUTPUT="${OUTPUT}    <incomingServer type=\"pop3\">\n"
-OUTPUT="${OUTPUT}      <hostname>pop.{labels.1}.{labels.0}</hostname>\n"
-OUTPUT="${OUTPUT}      <port>995</port>\n"
-OUTPUT="${OUTPUT}      <socketType>SSL</socketType>\n"
-OUTPUT="${OUTPUT}      <authentication>password-cleartext</authentication>\n"
-OUTPUT="${OUTPUT}      <username>%%EMAILADDRESS%%</username>\n"
 OUTPUT="${OUTPUT}    </incomingServer>\n"
 OUTPUT="${OUTPUT}    <outgoingServer type=\"smtp\">\n"
 OUTPUT="${OUTPUT}      <hostname>smtp.{labels.1}.{labels.0}</hostname>\n"
 OUTPUT="${OUTPUT}      <port>465</port>\n"
 OUTPUT="${OUTPUT}      <socketType>SSL</socketType>\n"
 OUTPUT="${OUTPUT}      <authentication>password-cleartext</authentication>\n"
-OUTPUT="${OUTPUT}      <username>%%EMAILADDRESS%%</username>\n"
+OUTPUT="${OUTPUT}      <username>%EMAILADDRESS%</username>\n"
 OUTPUT="${OUTPUT}    </outgoingServer>\n"
 OUTPUT="${OUTPUT}  </emailProvider>\n"
 OUTPUT="${OUTPUT}</clientConfig>\` 200\n"
 OUTPUT="${OUTPUT}    }\n"
 OUTPUT="${OUTPUT}\n"
-OUTPUT="${OUTPUT}    # Apple MobileConfig\n"
+
+# --- 4. Apple MobileConfig (inline, wie Autodiscover/Autoconfig) ---
+OUTPUT="${OUTPUT}    # Apple MobileConfig (inline respond)\n"
 OUTPUT="${OUTPUT}    route /apple {\n"
-OUTPUT="${OUTPUT}        templates {\n"
-OUTPUT="${OUTPUT}            mime \"application/x-apple-aspen-config\"\n"
-OUTPUT="${OUTPUT}        }\n"
 OUTPUT="${OUTPUT}        header Content-Type \"application/x-apple-aspen-config; charset=utf-8\"\n"
-OUTPUT="${OUTPUT}        root * /etc/caddy\n"
+OUTPUT="${OUTPUT}        header Content-Disposition \"attachment; filename=email.mobileconfig\"\n"
-OUTPUT="${OUTPUT}        rewrite * /email.mobileconfig.tpl\n"
+OUTPUT="${OUTPUT}        respond \`<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"
+OUTPUT="${OUTPUT}<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n"
+OUTPUT="${OUTPUT}<plist version=\"1.0\">\n"
+OUTPUT="${OUTPUT}<dict>\n"
+OUTPUT="${OUTPUT}    <key>PayloadContent</key>\n"
+OUTPUT="${OUTPUT}    <array>\n"
+OUTPUT="${OUTPUT}        <dict>\n"
+OUTPUT="${OUTPUT}            <key>EmailAccountDescription</key>\n"
+OUTPUT="${OUTPUT}            <string>{query.email}</string>\n"
+OUTPUT="${OUTPUT}            <key>EmailAccountName</key>\n"
+OUTPUT="${OUTPUT}            <string>{query.email}</string>\n"
+OUTPUT="${OUTPUT}            <key>EmailAccountType</key>\n"
+OUTPUT="${OUTPUT}            <string>EmailTypeIMAP</string>\n"
+OUTPUT="${OUTPUT}            <key>EmailAddress</key>\n"
+OUTPUT="${OUTPUT}            <string>{query.email}</string>\n"
+OUTPUT="${OUTPUT}            <key>IncomingMailServerAuthentication</key>\n"
+OUTPUT="${OUTPUT}            <string>EmailAuthPassword</string>\n"
+OUTPUT="${OUTPUT}            <key>IncomingMailServerHostName</key>\n"
+OUTPUT="${OUTPUT}            <string>imap.{labels.1}.{labels.0}</string>\n"
+OUTPUT="${OUTPUT}            <key>IncomingMailServerPortNumber</key>\n"
+OUTPUT="${OUTPUT}            <integer>993</integer>\n"
+OUTPUT="${OUTPUT}            <key>IncomingMailServerUseSSL</key>\n"
+OUTPUT="${OUTPUT}            <true/>\n"
+OUTPUT="${OUTPUT}            <key>IncomingMailServerUsername</key>\n"
+OUTPUT="${OUTPUT}            <string>{query.email}</string>\n"
+OUTPUT="${OUTPUT}            <key>OutgoingMailServerAuthentication</key>\n"
+OUTPUT="${OUTPUT}            <string>EmailAuthPassword</string>\n"
+OUTPUT="${OUTPUT}            <key>OutgoingMailServerHostName</key>\n"
+OUTPUT="${OUTPUT}            <string>smtp.{labels.1}.{labels.0}</string>\n"
+OUTPUT="${OUTPUT}            <key>OutgoingMailServerPortNumber</key>\n"
+OUTPUT="${OUTPUT}            <integer>465</integer>\n"
+OUTPUT="${OUTPUT}            <key>OutgoingMailServerUseSSL</key>\n"
+OUTPUT="${OUTPUT}            <true/>\n"
+OUTPUT="${OUTPUT}            <key>OutgoingMailServerUsername</key>\n"
+OUTPUT="${OUTPUT}            <string>{query.email}</string>\n"
+OUTPUT="${OUTPUT}            <key>OutgoingPasswordRequired</key>\n"
+OUTPUT="${OUTPUT}            <true/>\n"
+OUTPUT="${OUTPUT}            <key>PayloadDescription</key>\n"
+OUTPUT="${OUTPUT}            <string>E-Mail Konfiguration</string>\n"
+OUTPUT="${OUTPUT}            <key>PayloadDisplayName</key>\n"
+OUTPUT="${OUTPUT}            <string>{labels.1}.{labels.0} E-Mail</string>\n"
+OUTPUT="${OUTPUT}            <key>PayloadIdentifier</key>\n"
+OUTPUT="${OUTPUT}            <string>com.{labels.1}.{labels.0}.email.account</string>\n"
+OUTPUT="${OUTPUT}            <key>PayloadType</key>\n"
+OUTPUT="${OUTPUT}            <string>com.apple.mail.managed</string>\n"
+OUTPUT="${OUTPUT}            <key>PayloadUUID</key>\n"
+OUTPUT="${OUTPUT}            <string>A1B2C3D4-E5F6-7890-ABCD-EF1234567890</string>\n"
+OUTPUT="${OUTPUT}            <key>PayloadVersion</key>\n"
+OUTPUT="${OUTPUT}            <integer>1</integer>\n"
+OUTPUT="${OUTPUT}        </dict>\n"
+OUTPUT="${OUTPUT}    </array>\n"
+OUTPUT="${OUTPUT}    <key>PayloadDescription</key>\n"
+OUTPUT="${OUTPUT}    <string>E-Mail Einrichtung</string>\n"
+OUTPUT="${OUTPUT}    <key>PayloadDisplayName</key>\n"
+OUTPUT="${OUTPUT}    <string>{labels.1}.{labels.0} E-Mail</string>\n"
+OUTPUT="${OUTPUT}    <key>PayloadIdentifier</key>\n"
+OUTPUT="${OUTPUT}    <string>com.{labels.1}.{labels.0}.email.profile</string>\n"
+OUTPUT="${OUTPUT}    <key>PayloadOrganization</key>\n"
+OUTPUT="${OUTPUT}    <string>Bay Area Affiliates, Inc.</string>\n"
+OUTPUT="${OUTPUT}    <key>PayloadRemovalDisallowed</key>\n"
+OUTPUT="${OUTPUT}    <false/>\n"
+OUTPUT="${OUTPUT}    <key>PayloadType</key>\n"
+OUTPUT="${OUTPUT}    <string>Configuration</string>\n"
+OUTPUT="${OUTPUT}    <key>PayloadUUID</key>\n"
+OUTPUT="${OUTPUT}    <string>F0E1D2C3-B4A5-6789-0FED-CBA987654321</string>\n"
+OUTPUT="${OUTPUT}    <key>PayloadVersion</key>\n"
+OUTPUT="${OUTPUT}    <integer>1</integer>\n"
+OUTPUT="${OUTPUT}</dict>\n"
+OUTPUT="${OUTPUT}</plist>\` 200\n"
+OUTPUT="${OUTPUT}    }\n"
+
+# --- 5. Samsung Email (nutzt ebenfalls autoconfig, kein extra Block nötig) ---
+# Samsung Email-App versucht:
+#   1. https://autoconfig.<domain>/mail/config-v1.1.xml (= Thunderbird-Format, schon abgedeckt)
+#   2. Alternativ: Outlook Autodiscover XML
+# → Kein separater Block erforderlich.
+
+OUTPUT="${OUTPUT}}\n\n"
+
+# =====================================================================
+# Email-Setup Snippet (QR-Code Seite für iPhone)
+# =====================================================================
+OUTPUT="${OUTPUT}# ═══════════════════════════════════════════════\n"
+OUTPUT="${OUTPUT}# Email-Setup Snippet (QR-Code Seite)\n"
+OUTPUT="${OUTPUT}# ═══════════════════════════════════════════════\n"
+OUTPUT="${OUTPUT}(email_setup_page) {\n"
+OUTPUT="${OUTPUT}    route /email-setup* {\n"
+OUTPUT="${OUTPUT}        uri strip_prefix /email-setup\n"
+OUTPUT="${OUTPUT}        root * /var/www/email-setup\n"
+OUTPUT="${OUTPUT}        try_files {path} /setup.html\n"
 OUTPUT="${OUTPUT}        file_server\n"
 OUTPUT="${OUTPUT}    }\n"
 OUTPUT="${OUTPUT}}\n\n"
@@ -183,6 +234,7 @@ for domain in $DOMAINS; do
     echo "   → Wildcard Block:     *.${domain}"
     echo "   → Webmail Block:      webmail.${domain}"
     echo "   → Autodiscover Block: autodiscover.${domain}, autoconfig.${domain}"
+    echo "   → Email-Setup Block:  webmail.${domain}/email-setup"
 
     # Wildcard-Cert Block (für Cert-Generierung + Fallback)
     OUTPUT="${OUTPUT}# ═══════════════════════════════════════════════\n"
@@ -197,9 +249,10 @@ for domain in $DOMAINS; do
     OUTPUT="${OUTPUT}    respond \"OK\" 200\n"
     OUTPUT="${OUTPUT}}\n\n"
 
-    # Webmail Block (Roundcube)
+    # Webmail Block (Roundcube + Email-Setup)
-    OUTPUT="${OUTPUT}# Roundcube Webmail für $domain\n"
+    OUTPUT="${OUTPUT}# Roundcube Webmail + Email-Setup für $domain\n"
     OUTPUT="${OUTPUT}webmail.${domain} {\n"
+    OUTPUT="${OUTPUT}    import email_setup_page\n"
     OUTPUT="${OUTPUT}    reverse_proxy roundcube:80\n"
     OUTPUT="${OUTPUT}    encode gzip\n"
     OUTPUT="${OUTPUT}    log {\n"
@@ -208,6 +261,13 @@ for domain in $DOMAINS; do
     OUTPUT="${OUTPUT}    }\n"
     OUTPUT="${OUTPUT}}\n\n"
     
+    # Mailadmin Block
+    OUTPUT="${OUTPUT}# MailAdmin UI\n"
+    OUTPUT="${OUTPUT}mailadmin.${domain} {\n"
+    OUTPUT="${OUTPUT}    encode gzip\n"
+    OUTPUT="${OUTPUT}    reverse_proxy mailadmin:3000\n"
+    OUTPUT="${OUTPUT}}\n\n"
+
     # Autodiscover / Autoconfig Block
     OUTPUT="${OUTPUT}# Autodiscover/Autoconfig für $domain\n"
     OUTPUT="${OUTPUT}autodiscover.${domain}, autoconfig.${domain} {\n"
@@ -253,15 +313,17 @@ if [ -f "$CADDYFILE" ]; then
     fi
 fi
 
-# --- Prüfe ob alte hartcodierte Autodiscover-Blöcke existieren ---
+# --- Prüfe ob alte Dateien noch existieren ---
-if [ -f "$CADDYFILE" ]; then
+if [ -f "$SCRIPT_DIR/email_autodiscover" ]; then
-    if grep -q "autodiscover\.bayarea-cc\.com\|autodiscover\.bizmatch\.net\|autodiscover\.ruehrgedoens\.de" "$CADDYFILE"; then
     echo ""
-        echo "⚠️  AUFRÄUMEN: Alte hartcodierte Autodiscover-Blöcke im Caddyfile gefunden!"
+    echo "⚠️  AUFRÄUMEN: Datei 'email_autodiscover' kann entfernt werden!"
-        echo "   Diese werden jetzt dynamisch über mail_certs generiert."
+    echo "   Das Snippet ist jetzt in mail_certs eingebettet."
-        echo "   Bitte den alten 'Block A' manuell aus dem Caddyfile entfernen:"
-        echo "   → autodiscover.bayarea-cc.com, autodiscover.bizmatch.net, ..."
 fi
+
+if [ -f "$SCRIPT_DIR/email-setup/autodiscover.xml" ]; then
+    echo ""
+    echo "⚠️  AUFRÄUMEN: 'email-setup/autodiscover.xml' kann entfernt werden!"
+    echo "   Statische XML wird nicht mehr benötigt (dynamisch über Caddy)."
 fi
 
 echo ""
@@ -277,12 +339,18 @@ echo ""
 echo "3. Cert-Generierung verfolgen (~30s pro Domain):"
 echo "   docker logs -f $CADDY_CONTAINER 2>&1 | grep -i 'certificate\|acme\|tls\|error'"
 echo ""
-echo "4. Cert-Pfade kontrollieren:"
+echo "4. Autodiscover testen:"
-echo "   ls /var/lib/docker/volumes/caddy_data/_data/caddy/certificates/"
-echo "      acme-v02.api.letsencrypt.org-directory/"
-echo ""
-echo "5. Autodiscover testen:"
 for domain in $DOMAINS; do
-    echo "   curl -s https://autoconfig.${domain}/mail/config-v1.1.xml | head -5"
+    echo "   # Thunderbird:"
+    echo "   curl -s https://autoconfig.${domain}/mail/config-v1.1.xml | head -10"
+    echo "   # Outlook:"
+    echo "   curl -s https://autodiscover.${domain}/autodiscover/autodiscover.xml | head -10"
+    echo "   # Apple (sollte .mobileconfig liefern):"
+    echo "   curl -sI \"https://autodiscover.${domain}/apple?email=test@${domain}\""
+    echo ""
+done
+echo "5. iPhone Email-Setup QR-Code Seite:"
+for domain in $DOMAINS; do
+    echo "   https://webmail.${domain}/email-setup"
 done
 echo "============================================================"

email-worker-nodejs/docker-compose.yml

@@ -1,5 +1,3 @@
-version: "3.8"
-
 services:
   email-worker:
     build: .
@@ -8,16 +6,18 @@ services:
     env_file: .env
     volumes:
       - ./domains.txt:/etc/email-worker/domains.txt:ro
-      - worker-logs:/var/log/email-worker
+      - ./logs:/var/log/email-worker
     ports:
-      - "8000:8000"   # Prometheus metrics
+      - "9000:8000"   # Prometheus metrics (Host:Container)
-      - "8080:8080"   # Health check
+      - "9090:8080"   # Health check (Host:Container)
-    # Connect to DMS on the host or Docker network
-    extra_hosts:
-      - "host.docker.internal:host-gateway"
     environment:
-      - SMTP_HOST=host.docker.internal
+      - SMTP_HOST=mailserver
       - SMTP_PORT=25
+    networks:
+      - mail_network      
 
 volumes:
   worker-logs:
+networks:
+  mail_network:
+    external: true

email-worker-nodejs/src/config.ts

@@ -48,6 +48,9 @@ export const config = {
   // Monitoring
   metricsPort: parseInt(process.env.METRICS_PORT ?? '8000', 10),
   healthPort: parseInt(process.env.HEALTH_PORT ?? '8080', 10),
+
+  queueSuffix: process.env.QUEUE_SUFFIX ?? '-queue',
+  standbyMode: (process.env.STANDBY_MODE ?? 'false').toLowerCase() === 'true',
 } as const;
 
 export type Config = typeof config;
@@ -106,7 +109,7 @@ export function isInternalAddress(email: string): boolean {
 
 /** Convert domain to SQS queue name: bizmatch.net → bizmatch-net-queue */
 export function domainToQueueName(domain: string): string {
-  return domain.replace(/\./g, '-') + '-queue';
+  return domain.replace(/\./g, '-') + config.queueSuffix;
 }
 
 /** Convert domain to S3 bucket name: bizmatch.net → bizmatch-net-emails */

email-worker-nodejs/src/email/blocklist.ts

@@ -28,13 +28,13 @@ export class BlocklistChecker {
    */
 async batchCheckBlockedSenders(
     recipients: string[],
-    sender: string,
+    senders: string[], // <-- Geändert zu Array
     workerName: string,
   ): Promise<Record<string, boolean>> {
-    const patternsByRecipient =
+    const patternsByRecipient = await this.dynamodb.batchGetBlockedPatterns(recipients);
-      await this.dynamodb.batchGetBlockedPatterns(recipients);
     
-    const senderClean = extractAddress(sender);
+    // Alle übergebenen Adressen bereinigen
+    const sendersClean = senders.map(s => extractAddress(s)).filter(Boolean);
     const result: Record<string, boolean> = {};
 
     for (const recipient of recipients) {
@@ -42,10 +42,10 @@ export class BlocklistChecker {
       let isBlocked = false;
 
       for (const pattern of patterns) {
+        for (const senderClean of sendersClean) {
           if (picomatch.isMatch(senderClean, pattern.toLowerCase())) {
             log(
-            `⛔ BLOCKED: Sender ${senderClean} matches pattern '${pattern}' ` +
+              `⛔ BLOCKED: Sender ${senderClean} matches pattern '${pattern}' for inbox ${recipient}`,
-              `for inbox ${recipient}`,
               'WARNING',
               workerName,
             );
@@ -53,10 +53,10 @@ export class BlocklistChecker {
             break;
           }
         }
-
+        if (isBlocked) break;
+      }
       result[recipient] = isBlocked;
     }
-
     return result;
   }
 }

email-worker-nodejs/src/email/rules-processor.ts

@@ -9,13 +9,13 @@
 
 import { createTransport } from 'nodemailer';
 import type { ParsedMail } from 'mailparser';
-import type { DynamoDBHandler, EmailRule } from '../aws/dynamodb.js';
 import type { SESHandler } from '../aws/ses.js';
 import { extractBodyParts } from './parser.js';
-import { config, isInternalAddress } from '../config.js';
 import { log } from '../logger.js';
 // Wir nutzen MailComposer direkt für das Erstellen der Raw Bytes
 import MailComposer from 'nodemailer/lib/mail-composer/index.js';
+import { DynamoDBHandler, EmailRule } from '../aws/dynamodb.js';
+import { config, isInternalAddress } from '../config.js';
 
 export type MetricsCallback = (action: 'autoreply' | 'forward', domain: string) => void;
 
@@ -36,6 +36,9 @@ export class RulesProcessor {
     workerName: string,
     metricsCallback?: MetricsCallback,
   ): Promise<boolean> {
+    if (config.standbyMode) {
+        return false;
+    }    
     const rule = await this.dynamodb.getEmailRules(recipient.toLowerCase());
     if (!rule) return false;
 

email-worker-nodejs/src/logger.ts

@@ -103,12 +103,28 @@ function ensureFileStream(): WriteStream | null {
 // ---------------------------------------------------------------------------
 const logger = pino({
   level: 'info',
-  formatters: {
+  transport: {
-    level(label) {
+    targets: [
-      return { level: label };
+      {
+        // 1. Schicke bunte Logs in die Konsole (für docker compose logs -f)
+        target: 'pino-pretty',
+        options: {
+          colorize: true,
+          translateTime: 'SYS:yyyy-mm-dd HH:MM:ss',
+          ignore: 'pid,hostname',
+          singleLine: true
+        }
       },
-  },
+      {
-  timestamp: pino.stdTimeFunctions.isoTime,
+        // 2. Schreibe gleichzeitig alles unformatiert in die Datei
+        target: 'pino/file',
+        options: { 
+          destination: '/var/log/email-worker/worker.log',
+          mkdir: true 
+        }
+      }
+    ]
+  }
 });
 
 // ---------------------------------------------------------------------------

email-worker-nodejs/src/main.ts

@@ -13,7 +13,7 @@ import { config, loadDomains } from './config.js';
 import { log } from './logger.js';
 import { startMetricsServer, type MetricsCollector } from './metrics.js';
 import { startHealthServer } from './health.js';
-import { UnifiedWorker } from './worker/index.js';
+import { UnifiedWorker } from './worker/unified-worker.js';
 
 // ---------------------------------------------------------------------------
 // Banner

email-worker-nodejs/src/smtp/delivery.ts

@@ -8,8 +8,9 @@
  */
 
 import { createTransport, type Transporter } from 'nodemailer';
-import { config } from '../config.js';
+
 import { log } from '../logger.js';
+import { config } from '../config.js';
 
 // ---------------------------------------------------------------------------
 // Permanent error detection

email-worker-nodejs/src/worker/domain-poller.ts

@@ -9,9 +9,9 @@
  */
 
 import type { SQSHandler } from '../aws/sqs.js';
-import type { MessageProcessor } from './message-processor.js';
 import type { MetricsCollector } from '../metrics.js';
 import { log } from '../logger.js';
+import { MessageProcessor } from './message-processor.js';
 
 export interface DomainPollerStats {
   domain: string;

email-worker-nodejs/src/worker/message-processor.ts

@@ -19,16 +19,14 @@ import type { SESHandler } from '../aws/ses.js';
 import type { DynamoDBHandler } from '../aws/dynamodb.js';
 import type { EmailDelivery } from '../smtp/delivery.js';
 import type { MetricsCollector } from '../metrics.js';
-import {
+import type { ParsedMail } from 'mailparser';
-  parseEmail,
-  isProcessedByWorker,
-  BounceHandler,
-  RulesProcessor,
-  BlocklistChecker,
-} from '../email/index.js';
-import { domainToBucketName } from '../config.js';
-import { log } from '../logger.js';
 
+import { log } from '../logger.js';
+import { BlocklistChecker } from '../email/blocklist.js';
+import { BounceHandler } from '../email/bounce-handler.js';
+import { parseEmail, isProcessedByWorker } from '../email/parser.js';
+import { RulesProcessor } from '../email/rules-processor.js';
+import { config } from '../config.js';
 // ---------------------------------------------------------------------------
 // Processor
 // ---------------------------------------------------------------------------
@@ -138,6 +136,7 @@ export class MessageProcessor {
       let finalRawBytes = rawBytes;
       let fromAddrFinal = fromAddr;
       let isBounce = false;
+      let parsedFinal: ParsedMail | null = null; // <-- Hier deklarieren
 
       try {
         const parsed = await parseEmail(rawBytes);
@@ -150,7 +149,6 @@ export class MessageProcessor {
           subject,
           workerName,
         );
-
         isBounce = bounceResult.isBounce;
         finalRawBytes = bounceResult.rawBytes;
 
@@ -168,23 +166,31 @@ export class MessageProcessor {
         }
 
         // Re-parse after modifications for rules processing
-        var parsedFinal = await parseEmail(finalRawBytes);
+        parsedFinal = await parseEmail(finalRawBytes);
       } catch (err: any) {
         log(
-          `⚠ Parsing/Logic Error: ${err.message ?? err}. Sending original.`,
+          `⚠ Parsing/Logic Error: ${err.message ?? err}. Sending original RAW mail without rules.`,
           'WARNING',
           workerName,
         );
         log(`Full error: ${err.stack ?? err}`, 'ERROR', workerName);
         fromAddrFinal = fromAddr;
         isBounce = false;
-        var parsedFinal = await parseEmail(rawBytes);
+        parsedFinal = null; // <-- GANZ WICHTIG: Kein erneuter Parse-Versuch!
       }
 
       // 6. BLOCKLIST CHECK
+      const sendersToCheck: string[] = [];
+      if (fromAddrFinal) sendersToCheck.push(fromAddrFinal);
+      
+      const headerFrom = parsedFinal?.from?.text;
+      if (headerFrom && !sendersToCheck.includes(headerFrom)) {
+        sendersToCheck.push(headerFrom);
+      }
+
       const blockedByRecipient = await this.blocklist.batchCheckBlockedSenders(
         recipients,
-        fromAddrFinal,
+        sendersToCheck, // <-- Array übergeben
         workerName,
       );
 
@@ -210,7 +216,7 @@ export class MessageProcessor {
         }
 
         // Process rules (OOO, Forwarding) — not for bounces or already forwarded
-        if (!isBounce && !skipRules) {
+        if (!isBounce && !skipRules && parsedFinal !== null) {
           const metricsCallback = (action: 'autoreply' | 'forward', dom: string) => {
             if (action === 'autoreply') this.metrics?.incrementAutoreply(dom);
             else if (action === 'forward') this.metrics?.incrementForward(dom);
@@ -252,7 +258,8 @@ export class MessageProcessor {
 
       if (totalHandled === recipients.length) {
         if (blockedRecipients.length === recipients.length) {
-          // All blocked
+          // All blocked — im Standby kein S3 anfassen
+          if (!config.standbyMode) {
             try {
               await this.s3.markAsBlocked(
                 domain,
@@ -266,14 +273,18 @@ export class MessageProcessor {
               log(`⚠ Failed to handle blocked email: ${err.message ?? err}`, 'ERROR', workerName);
               return false;
             }
+          }
         } else if (successful.length > 0) {
+          if (!config.standbyMode) {
             await this.s3.markAsProcessed(
               domain,
               messageId,
               workerName,
               failedPermanent.length > 0 ? failedPermanent : undefined,
             );
+          }
         } else if (failedPermanent.length > 0) {
+          if (!config.standbyMode) {
             await this.s3.markAsAllInvalid(
               domain,
               messageId,
@@ -281,6 +292,7 @@ export class MessageProcessor {
               workerName,
             );
           }
+        }
 
         // Summary
         const parts: string[] = [];

email-worker-nodejs/src/worker/unified-worker.ts

@@ -8,17 +8,22 @@
  *  - Graceful shutdown
  */
 
-import { S3Handler, SQSHandler, SESHandler, DynamoDBHandler } from '../aws/index.js';
+import { DynamoDBHandler } from '../aws/dynamodb';
-import { EmailDelivery } from '../smtp/index.js';
+import { S3Handler} from '../aws/s3.js';
+import { SQSHandler} from '../aws/sqs.js'
+import { SESHandler } from '../aws/ses';
+import { EmailDelivery } from '../smtp/delivery.js';
 import { MessageProcessor } from './message-processor.js';
 import { DomainPoller, type DomainPollerStats } from './domain-poller.js';
 import type { MetricsCollector } from '../metrics.js';
 import { log } from '../logger.js';
 
+
 export class UnifiedWorker {
   private pollers: DomainPoller[] = [];
   private processor: MessageProcessor;
   private sqs: SQSHandler;
+  private statusInterval: NodeJS.Timeout | null = null;
 
   constructor(
     private domains: string[],
@@ -74,10 +79,16 @@ export class UnifiedWorker {
         this.pollers.map((p) => p.stats.domain).join(', '),
       'SUCCESS',
     );
+
+    // Starte den 5-Minuten-Status-Report
+    this.statusInterval = setInterval(() => {
+      this.printStatus();
+    }, 5 * 60 * 1000);
   }
 
   async stop(): Promise<void> {
     log('🛑 Stopping all domain pollers...');
+    if (this.statusInterval) clearInterval(this.statusInterval); // <-- Neue Zeile
     await Promise.all(this.pollers.map((p) => p.stop()));
     log('✅ All pollers stopped.');
   }
@@ -99,4 +110,25 @@ export class UnifiedWorker {
 
     return { totalProcessed, totalErrors, domains };
   }
+
+  private printStatus(): void {
+    const stats = this.getStats();
+    // Zähle aktive Poller
+    const activePollers = this.pollers.filter((p) => p.stats.running).length;
+    const totalPollers = this.pollers.length;
+
+    // Formatiere die Domain-Statistiken (z.B. hotshpotshga:1)
+    const domainStats = stats.domains
+      .map((d) => {
+        const shortName = d.domain.split('.')[0].substring(0, 12);
+        return `${shortName}:${d.processed}`;
+      })
+      .join(' | ');
+
+    log(
+      `📊 Status: ${activePollers}/${totalPollers} active, total:${stats.totalProcessed} | ${domainStats}`,
+      'INFO',
+      'unified-worker'
+    );
+  }
 }

email-worker/.dockerignore

@@ -1,38 +0,0 @@
-# Documentation
-*.md
-!README.md
-
-# Git
-.git
-.gitignore
-
-# Python
-__pycache__
-*.pyc
-*.pyo
-*.pyd
-.Python
-*.so
-
-# Logs
-logs/
-*.log
-
-# Environment
-.env
-.env.example
-
-# IDE
-.vscode/
-.idea/
-*.swp
-*.swo
-
-# OS
-.DS_Store
-Thumbs.db
-
-# Build
-*.tar.gz
-dist/
-build/

email-worker/.env.example

@@ -1,42 +0,0 @@
-# AWS Credentials
-AWS_REGION=us-east-2
-AWS_ACCESS_KEY_ID=your_access_key_here
-AWS_SECRET_ACCESS_KEY=your_secret_key_here
-
-# Domains Configuration
-DOMAINS=example.com,another.com
-# Alternative: Use domains file
-# DOMAINS_FILE=/etc/email-worker/domains.txt
-
-# Worker Settings
-WORKER_THREADS=10
-POLL_INTERVAL=20
-MAX_MESSAGES=10
-VISIBILITY_TIMEOUT=300
-
-# SMTP Configuration
-SMTP_HOST=localhost
-SMTP_PORT=25
-SMTP_USE_TLS=false
-SMTP_USER=
-SMTP_PASS=
-SMTP_POOL_SIZE=5
-INTERNAL_SMTP_PORT=2525
-
-# LMTP Configuration (Optional)
-LMTP_ENABLED=false
-LMTP_HOST=localhost
-LMTP_PORT=24
-
-# DynamoDB Tables
-DYNAMODB_RULES_TABLE=email-rules
-DYNAMODB_MESSAGES_TABLE=ses-outbound-messages
-DYNAMODB_BLOCKED_TABLE=email-blocked-senders
-
-# Bounce Handling
-BOUNCE_LOOKUP_RETRIES=3
-BOUNCE_LOOKUP_DELAY=1.0
-
-# Monitoring Ports
-METRICS_PORT=8000
-HEALTH_PORT=8080

email-worker/.gitignore

@@ -1,35 +0,0 @@
-# Python
-__pycache__/
-*.py[cod]
-*$py.class
-*.so
-.Python
-env/
-venv/
-ENV/
-.venv
-
-# Logs
-*.log
-
-# Environment
-.env
-
-# IDE
-.vscode/
-.idea/
-*.swp
-*.swo
-
-# OS
-.DS_Store
-Thumbs.db
-
-# Build
-dist/
-build/
-*.egg-info/
-
-# Archives
-*.tar.gz
-*.zip

email-worker/Dockerfile

@@ -1,37 +0,0 @@
-FROM python:3.11-slim
-
-LABEL maintainer="andreas@knuth.dev"
-LABEL description="Unified multi-domain email worker (modular version)"
-
-# System packages
-RUN apt-get update && apt-get install -y --no-install-recommends \
-    curl \
-    && rm -rf /var/lib/apt/lists/*
-
-# Non-root user
-RUN useradd -m -u 1000 worker && \
-    mkdir -p /app /var/log/email-worker /etc/email-worker && \
-    chown -R worker:worker /app /var/log/email-worker /etc/email-worker
-
-# Python dependencies
-COPY requirements.txt /app/
-RUN pip install --no-cache-dir -r /app/requirements.txt
-
-# Worker code (all modules)
-COPY --chown=worker:worker aws/ /app/aws/
-COPY --chown=worker:worker email_processing/ /app/email_processing/
-COPY --chown=worker:worker smtp/ /app/smtp/
-COPY --chown=worker:worker metrics/ /app/metrics/
-COPY --chown=worker:worker *.py /app/
-
-WORKDIR /app
-USER worker
-
-# Health check
-HEALTHCHECK --interval=30s --timeout=10s --start-period=10s --retries=3 \
-    CMD curl -f http://localhost:8080/health || exit 1
-
-# Unbuffered output
-ENV PYTHONUNBUFFERED=1
-
-CMD ["python3", "main.py"]

email-worker/Makefile

@@ -1,50 +0,0 @@
-.PHONY: help install run test lint clean docker-build docker-run docker-stop docker-logs
-
-help:
-	@echo "Available commands:"
-	@echo "  make install       - Install dependencies"
-	@echo "  make run           - Run worker locally"
-	@echo "  make test          - Run tests (TODO)"
-	@echo "  make lint          - Run linting"
-	@echo "  make clean         - Clean up files"
-	@echo "  make docker-build  - Build Docker image"
-	@echo "  make docker-run    - Run with docker-compose"
-	@echo "  make docker-stop   - Stop docker-compose"
-	@echo "  make docker-logs   - Show docker logs"
-
-install:
-	pip install -r requirements.txt
-
-run:
-	python3 main.py
-
-test:
-	@echo "TODO: Add tests"
-	# python3 -m pytest tests/
-
-lint:
-	@echo "Running pylint..."
-	-pylint --rcfile=.pylintrc *.py **/*.py 2>/dev/null || echo "pylint not installed"
-	@echo "Running flake8..."
-	-flake8 --max-line-length=120 . 2>/dev/null || echo "flake8 not installed"
-
-clean:
-	find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
-	find . -type f -name "*.pyc" -delete
-	find . -type f -name "*.pyo" -delete
-	find . -type f -name "*.log" -delete
-
-docker-build:
-	docker build -t unified-email-worker:latest .
-
-docker-run:
-	docker-compose up -d
-
-docker-stop:
-	docker-compose down
-
-docker-logs:
-	docker-compose logs -f email-worker
-
-docker-restart: docker-stop docker-build docker-run
-	@echo "Worker restarted"

email-worker/aws/__init__.py

@@ -1,11 +0,0 @@
-#!/usr/bin/env python3
-"""
-AWS service handlers
-"""
-
-from .s3_handler import S3Handler
-from .sqs_handler import SQSHandler
-from .ses_handler import SESHandler
-from .dynamodb_handler import DynamoDBHandler
-
-__all__ = ['S3Handler', 'SQSHandler', 'SESHandler', 'DynamoDBHandler']

email-worker/aws/dynamodb_handler.py

@@ -1,192 +0,0 @@
-#!/usr/bin/env python3
-"""
-DynamoDB operations handler
-"""
-
-import time
-from typing import Optional, Dict, Any, List
-import boto3
-from botocore.exceptions import ClientError
-
-from logger import log
-from config import config
-
-
-class DynamoDBHandler:
-    """Handles all DynamoDB operations"""
-    
-    def __init__(self):
-        self.resource = boto3.resource('dynamodb', region_name=config.aws_region)
-        self.available = False
-        self.rules_table = None
-        self.messages_table = None
-        self.blocked_table = None
-        
-        self._initialize_tables()
-    
-    def _initialize_tables(self):
-        """Initialize DynamoDB table connections"""
-        try:
-            self.rules_table = self.resource.Table(config.rules_table)
-            self.messages_table = self.resource.Table(config.messages_table)
-            self.blocked_table = self.resource.Table(config.blocked_table)
-            
-            # Test connection
-            self.rules_table.table_status
-            self.messages_table.table_status
-            self.blocked_table.table_status
-            
-            self.available = True
-            log("✓ DynamoDB tables connected successfully")
-            
-        except Exception as e:
-            log(f"⚠ DynamoDB not fully available: {e}", 'WARNING')
-            self.available = False
-    
-    def get_email_rules(self, email_address: str) -> Optional[Dict[str, Any]]:
-        """
-        Get email rules for recipient (OOO, Forwarding)
-        
-        Args:
-            email_address: Recipient email address
-            
-        Returns:
-            Rule dictionary or None if not found
-        """
-        if not self.available or not self.rules_table:
-            return None
-        
-        try:
-            response = self.rules_table.get_item(Key={'email_address': email_address})
-            return response.get('Item')
-            
-        except ClientError as e:
-            if e.response['Error']['Code'] != 'ResourceNotFoundException':
-                log(f"⚠ DynamoDB error for {email_address}: {e}", 'ERROR')
-            return None
-            
-        except Exception as e:
-            log(f"⚠ DynamoDB error for {email_address}: {e}", 'WARNING')
-            return None
-    
-    def get_bounce_info(self, message_id: str, worker_name: str = 'unified') -> Optional[Dict]:
-        """
-        Get bounce information from DynamoDB with retry logic
-        
-        Args:
-            message_id: SES Message ID
-            worker_name: Worker name for logging
-            
-        Returns:
-            Bounce info dictionary or None
-        """
-        if not self.available or not self.messages_table:
-            return None
-        
-        for attempt in range(config.bounce_lookup_retries):
-            try:
-                response = self.messages_table.get_item(Key={'MessageId': message_id})
-                item = response.get('Item')
-                
-                if item:
-                    return {
-                        'original_source': item.get('original_source', ''),
-                        'bounceType': item.get('bounceType', 'Unknown'),
-                        'bounceSubType': item.get('bounceSubType', 'Unknown'),
-                        'bouncedRecipients': item.get('bouncedRecipients', []),
-                        'timestamp': item.get('timestamp', '')
-                    }
-                
-                if attempt < config.bounce_lookup_retries - 1:
-                    log(
-                        f"  Bounce record not found yet, retrying in {config.bounce_lookup_delay}s "
-                        f"(attempt {attempt + 1}/{config.bounce_lookup_retries})...",
-                        'INFO',
-                        worker_name
-                    )
-                    time.sleep(config.bounce_lookup_delay)
-                else:
-                    log(
-                        f"⚠ No bounce record found after {config.bounce_lookup_retries} attempts "
-                        f"for Message-ID: {message_id}",
-                        'WARNING',
-                        worker_name
-                    )
-                    return None
-            
-            except Exception as e:
-                log(
-                    f"⚠ DynamoDB Error (attempt {attempt + 1}/{config.bounce_lookup_retries}): {e}",
-                    'ERROR',
-                    worker_name
-                )
-                if attempt < config.bounce_lookup_retries - 1:
-                    time.sleep(config.bounce_lookup_delay)
-                else:
-                    return None
-        
-        return None
-    
-    def get_blocked_patterns(self, email_address: str) -> List[str]:
-        """
-        Get blocked sender patterns for recipient
-        
-        Args:
-            email_address: Recipient email address
-            
-        Returns:
-            List of blocked patterns (may include wildcards)
-        """
-        if not self.available or not self.blocked_table:
-            return []
-        
-        try:
-            response = self.blocked_table.get_item(Key={'email_address': email_address})
-            item = response.get('Item', {})
-            return item.get('blocked_patterns', [])
-            
-        except Exception as e:
-            log(f"⚠ Error getting block list for {email_address}: {e}", 'ERROR')
-            return []
-    
-    def batch_get_blocked_patterns(self, email_addresses: List[str]) -> Dict[str, List[str]]:
-        """
-        Batch get blocked patterns for multiple recipients (more efficient)
-        
-        Args:
-            email_addresses: List of recipient email addresses
-            
-        Returns:
-            Dictionary mapping email_address -> list of blocked patterns
-        """
-        if not self.available or not self.blocked_table:
-            return {addr: [] for addr in email_addresses}
-        
-        try:
-            # DynamoDB BatchGetItem
-            keys = [{'email_address': addr} for addr in email_addresses]
-            response = self.resource.batch_get_item(
-                RequestItems={
-                    config.blocked_table: {'Keys': keys}
-                }
-            )
-            
-            items = response.get('Responses', {}).get(config.blocked_table, [])
-            
-            # Build result dictionary
-            result = {}
-            for email_address in email_addresses:
-                matching_item = next(
-                    (item for item in items if item['email_address'] == email_address),
-                    None
-                )
-                if matching_item:
-                    result[email_address] = matching_item.get('blocked_patterns', [])
-                else:
-                    result[email_address] = []
-            
-            return result
-            
-        except Exception as e:
-            log(f"⚠ Batch blocklist check error: {e}", 'ERROR')
-            return {addr: [] for addr in email_addresses}

email-worker/aws/s3_handler.py

@@ -1,193 +0,0 @@
-#!/usr/bin/env python3
-"""
-S3 operations handler
-"""
-
-import time
-from typing import Optional, List
-import boto3
-from botocore.exceptions import ClientError
-
-from logger import log
-from config import config, domain_to_bucket_name
-
-
-class S3Handler:
-    """Handles all S3 operations"""
-    
-    def __init__(self):
-        self.client = boto3.client('s3', region_name=config.aws_region)
-    
-    def get_email(self, domain: str, message_id: str, receive_count: int) -> Optional[bytes]:
-        """
-        Download email from S3
-        
-        Args:
-            domain: Email domain
-            message_id: SES Message ID
-            receive_count: Number of times this message was received from queue
-            
-        Returns:
-            Raw email bytes or None if not found/error
-        """
-        bucket = domain_to_bucket_name(domain)
-        
-        try:
-            response = self.client.get_object(Bucket=bucket, Key=message_id)
-            return response['Body'].read()
-            
-        except self.client.exceptions.NoSuchKey:
-            if receive_count < 5:
-                log(f"⏳ S3 Object not found yet (Attempt {receive_count}). Retrying...", 'WARNING')
-                return None
-            else:
-                log(f"❌ S3 Object missing permanently after retries.", 'ERROR')
-                raise
-                
-        except ClientError as e:
-            if e.response['Error']['Code'] == 'NoSuchKey':
-                if receive_count < 5:
-                    log(f"⏳ S3 Object not found yet (Attempt {receive_count}). Retrying...", 'WARNING')
-                    return None
-                else:
-                    log(f"❌ S3 Object missing permanently after retries.", 'ERROR')
-                    raise
-            log(f"❌ S3 Download Error: {e}", 'ERROR')
-            raise
-            
-        except Exception as e:
-            log(f"❌ S3 Download Error: {e}", 'ERROR')
-            raise
-    
-    def mark_as_processed(
-        self, 
-        domain: str, 
-        message_id: str, 
-        worker_name: str, 
-        invalid_inboxes: Optional[List[str]] = None
-    ):
-        """Mark email as successfully delivered"""
-        bucket = domain_to_bucket_name(domain)
-        
-        try:
-            head = self.client.head_object(Bucket=bucket, Key=message_id)
-            metadata = head.get('Metadata', {}) or {}
-            
-            metadata['processed'] = 'true'
-            metadata['processed_at'] = str(int(time.time()))
-            metadata['processed_by'] = worker_name
-            metadata['status'] = 'delivered'
-            metadata.pop('processing_started', None)
-            metadata.pop('queued_at', None)
-            
-            if invalid_inboxes:
-                metadata['invalid_inboxes'] = ','.join(invalid_inboxes)
-                log(f"⚠ Invalid inboxes recorded: {', '.join(invalid_inboxes)}", 'WARNING', worker_name)
-            
-            self.client.copy_object(
-                Bucket=bucket,
-                Key=message_id,
-                CopySource={'Bucket': bucket, 'Key': message_id},
-                Metadata=metadata,
-                MetadataDirective='REPLACE'
-            )
-            
-        except Exception as e:
-            log(f"Failed to mark as processed: {e}", 'WARNING', worker_name)
-    
-    def mark_as_all_invalid(
-        self, 
-        domain: str, 
-        message_id: str, 
-        invalid_inboxes: List[str], 
-        worker_name: str
-    ):
-        """Mark email as failed because all recipients are invalid"""
-        bucket = domain_to_bucket_name(domain)
-        
-        try:
-            head = self.client.head_object(Bucket=bucket, Key=message_id)
-            metadata = head.get('Metadata', {}) or {}
-            
-            metadata['processed'] = 'true'
-            metadata['processed_at'] = str(int(time.time()))
-            metadata['processed_by'] = worker_name
-            metadata['status'] = 'failed'
-            metadata['error'] = 'All recipients are invalid (mailboxes do not exist)'
-            metadata['invalid_inboxes'] = ','.join(invalid_inboxes)
-            metadata.pop('processing_started', None)
-            metadata.pop('queued_at', None)
-            
-            self.client.copy_object(
-                Bucket=bucket,
-                Key=message_id,
-                CopySource={'Bucket': bucket, 'Key': message_id},
-                Metadata=metadata,
-                MetadataDirective='REPLACE'
-            )
-            
-        except Exception as e:
-            log(f"Failed to mark as all invalid: {e}", 'WARNING', worker_name)
-    
-    def mark_as_blocked(
-        self,
-        domain: str,
-        message_id: str,
-        blocked_recipients: List[str],
-        sender: str,
-        worker_name: str
-    ):
-        """
-        Mark email as blocked by sender blacklist
-        
-        This sets metadata BEFORE deletion for audit trail
-        """
-        bucket = domain_to_bucket_name(domain)
-        
-        try:
-            head = self.client.head_object(Bucket=bucket, Key=message_id)
-            metadata = head.get('Metadata', {}) or {}
-            
-            metadata['processed'] = 'true'
-            metadata['processed_at'] = str(int(time.time()))
-            metadata['processed_by'] = worker_name
-            metadata['status'] = 'blocked'
-            metadata['blocked_recipients'] = ','.join(blocked_recipients)
-            metadata['blocked_sender'] = sender
-            metadata.pop('processing_started', None)
-            metadata.pop('queued_at', None)
-            
-            self.client.copy_object(
-                Bucket=bucket,
-                Key=message_id,
-                CopySource={'Bucket': bucket, 'Key': message_id},
-                Metadata=metadata,
-                MetadataDirective='REPLACE'
-            )
-            
-            log(f"✓ Marked as blocked in S3 metadata", 'INFO', worker_name)
-            
-        except Exception as e:
-            log(f"⚠ Failed to mark as blocked: {e}", 'ERROR', worker_name)
-            raise
-    
-    def delete_blocked_email(
-        self,
-        domain: str,
-        message_id: str,
-        worker_name: str
-    ):
-        """
-        Delete email after marking as blocked
-        
-        Only call this after mark_as_blocked() succeeded
-        """
-        bucket = domain_to_bucket_name(domain)
-        
-        try:
-            self.client.delete_object(Bucket=bucket, Key=message_id)
-            log(f"🗑 Deleted blocked email from S3", 'SUCCESS', worker_name)
-            
-        except Exception as e:
-            log(f"⚠ Failed to delete blocked email: {e}", 'ERROR', worker_name)
-            raise

email-worker/aws/ses_handler.py

@@ -1,53 +0,0 @@
-#!/usr/bin/env python3
-"""
-SES operations handler
-"""
-
-import boto3
-from botocore.exceptions import ClientError
-
-from logger import log
-from config import config
-
-
-class SESHandler:
-    """Handles all SES operations"""
-    
-    def __init__(self):
-        self.client = boto3.client('ses', region_name=config.aws_region)
-    
-    def send_raw_email(
-        self, 
-        source: str, 
-        destination: str, 
-        raw_message: bytes,
-        worker_name: str
-    ) -> bool:
-        """
-        Send raw email via SES
-        
-        Args:
-            source: From address
-            destination: To address
-            raw_message: Raw MIME message bytes
-            worker_name: Worker name for logging
-            
-        Returns:
-            True if sent successfully, False otherwise
-        """
-        try:
-            self.client.send_raw_email(
-                Source=source,
-                Destinations=[destination],
-                RawMessage={'Data': raw_message}
-            )
-            return True
-            
-        except ClientError as e:
-            error_code = e.response['Error']['Code']
-            log(f"⚠ SES send failed to {destination} ({error_code}): {e}", 'ERROR', worker_name)
-            return False
-            
-        except Exception as e:
-            log(f"⚠ SES send failed to {destination}: {e}", 'ERROR', worker_name)
-            return False

email-worker/aws/sqs_handler.py

@@ -1,103 +0,0 @@
-#!/usr/bin/env python3
-"""
-SQS operations handler
-"""
-
-from typing import Optional, List, Dict, Any
-import boto3
-from botocore.exceptions import ClientError
-
-from logger import log
-from config import config, domain_to_queue_name
-
-
-class SQSHandler:
-    """Handles all SQS operations"""
-    
-    def __init__(self):
-        self.client = boto3.client('sqs', region_name=config.aws_region)
-    
-    def get_queue_url(self, domain: str) -> Optional[str]:
-        """
-        Get SQS queue URL for domain
-        
-        Args:
-            domain: Email domain
-            
-        Returns:
-            Queue URL or None if not found
-        """
-        queue_name = domain_to_queue_name(domain)
-        
-        try:
-            response = self.client.get_queue_url(QueueName=queue_name)
-            return response['QueueUrl']
-            
-        except ClientError as e:
-            if e.response['Error']['Code'] == 'AWS.SimpleQueueService.NonExistentQueue':
-                log(f"Queue not found for domain: {domain}", 'WARNING')
-            else:
-                log(f"Error getting queue URL for {domain}: {e}", 'ERROR')
-            return None
-    
-    def receive_messages(self, queue_url: str) -> List[Dict[str, Any]]:
-        """
-        Receive messages from queue
-        
-        Args:
-            queue_url: SQS Queue URL
-            
-        Returns:
-            List of message dictionaries
-        """
-        try:
-            response = self.client.receive_message(
-                QueueUrl=queue_url,
-                MaxNumberOfMessages=config.max_messages,
-                WaitTimeSeconds=config.poll_interval,
-                VisibilityTimeout=config.visibility_timeout,
-                AttributeNames=['ApproximateReceiveCount', 'SentTimestamp']
-            )
-            
-            return response.get('Messages', [])
-            
-        except Exception as e:
-            log(f"Error receiving messages: {e}", 'ERROR')
-            return []
-    
-    def delete_message(self, queue_url: str, receipt_handle: str):
-        """
-        Delete message from queue
-        
-        Args:
-            queue_url: SQS Queue URL
-            receipt_handle: Message receipt handle
-        """
-        try:
-            self.client.delete_message(
-                QueueUrl=queue_url,
-                ReceiptHandle=receipt_handle
-            )
-        except Exception as e:
-            log(f"Error deleting message: {e}", 'ERROR')
-            raise
-    
-    def get_queue_size(self, queue_url: str) -> int:
-        """
-        Get approximate number of messages in queue
-        
-        Args:
-            queue_url: SQS Queue URL
-            
-        Returns:
-            Number of messages (0 if error)
-        """
-        try:
-            attrs = self.client.get_queue_attributes(
-                QueueUrl=queue_url,
-                AttributeNames=['ApproximateNumberOfMessages']
-            )
-            return int(attrs['Attributes'].get('ApproximateNumberOfMessages', 0))
-            
-        except Exception:
-            return 0

email-worker/config.py

@@ -1,100 +0,0 @@
-#!/usr/bin/env python3
-"""
-Configuration management for unified email worker
-"""
-
-import os
-from dataclasses import dataclass
-from typing import Set
-
-
-@dataclass
-class Config:
-    """Worker Configuration"""
-    # AWS
-    aws_region: str = os.environ.get('AWS_REGION', 'us-east-2')
-    
-    # Domains to process
-    domains_list: str = os.environ.get('DOMAINS', '')
-    domains_file: str = os.environ.get('DOMAINS_FILE', '/etc/email-worker/domains.txt')
-    
-    # Worker Settings
-    worker_threads: int = int(os.environ.get('WORKER_THREADS', '10'))
-    poll_interval: int = int(os.environ.get('POLL_INTERVAL', '20'))
-    max_messages: int = int(os.environ.get('MAX_MESSAGES', '10'))
-    visibility_timeout: int = int(os.environ.get('VISIBILITY_TIMEOUT', '300'))
-    
-    # SMTP for delivery
-    smtp_host: str = os.environ.get('SMTP_HOST', 'localhost')
-    smtp_port: int = int(os.environ.get('SMTP_PORT', '25'))
-    smtp_use_tls: bool = os.environ.get('SMTP_USE_TLS', 'false').lower() == 'true'
-    smtp_user: str = os.environ.get('SMTP_USER', '')
-    smtp_pass: str = os.environ.get('SMTP_PASS', '')
-    smtp_pool_size: int = int(os.environ.get('SMTP_POOL_SIZE', '5'))
-    
-    # Internal SMTP (bypasses transport_maps)
-    internal_smtp_port: int = int(os.environ.get('INTERNAL_SMTP_PORT', '2525'))
-    
-    # LMTP for local delivery (bypasses Postfix transport_maps)
-    lmtp_enabled: bool = os.environ.get('LMTP_ENABLED', 'false').lower() == 'true'
-    lmtp_host: str = os.environ.get('LMTP_HOST', 'localhost')
-    lmtp_port: int = int(os.environ.get('LMTP_PORT', '24'))
-    
-    # DynamoDB Tables
-    rules_table: str = os.environ.get('DYNAMODB_RULES_TABLE', 'email-rules')
-    messages_table: str = os.environ.get('DYNAMODB_MESSAGES_TABLE', 'ses-outbound-messages')
-    blocked_table: str = os.environ.get('DYNAMODB_BLOCKED_TABLE', 'email-blocked-senders')
-    
-    # Bounce Handling
-    bounce_lookup_retries: int = int(os.environ.get('BOUNCE_LOOKUP_RETRIES', '3'))
-    bounce_lookup_delay: float = float(os.environ.get('BOUNCE_LOOKUP_DELAY', '1.0'))
-    
-    # Monitoring
-    metrics_port: int = int(os.environ.get('METRICS_PORT', '8000'))
-    health_port: int = int(os.environ.get('HEALTH_PORT', '8080'))
-
-
-# Global configuration instance
-config = Config()
-
-# Global set of managed domains (populated at startup)
-MANAGED_DOMAINS: Set[str] = set()
-
-
-def load_domains() -> list[str]:
-    """Load domains from config and populate MANAGED_DOMAINS global"""
-    global MANAGED_DOMAINS
-    domains = []
-    
-    if config.domains_list:
-        domains.extend([d.strip() for d in config.domains_list.split(',') if d.strip()])
-    
-    if os.path.exists(config.domains_file):
-        with open(config.domains_file, 'r') as f:
-            for line in f:
-                domain = line.strip()
-                if domain and not domain.startswith('#'):
-                    domains.append(domain)
-    
-    domains = list(set(domains))
-    MANAGED_DOMAINS = set(d.lower() for d in domains)
-    
-    return domains
-
-
-def is_internal_address(email_address: str) -> bool:
-    """Check if email address belongs to one of our managed domains"""
-    if '@' not in email_address:
-        return False
-    domain = email_address.split('@')[1].lower()
-    return domain in MANAGED_DOMAINS
-
-
-def domain_to_queue_name(domain: str) -> str:
-    """Convert domain to SQS queue name"""
-    return domain.replace('.', '-') + '-queue'
-
-
-def domain_to_bucket_name(domain: str) -> str:
-    """Convert domain to S3 bucket name"""
-    return domain.replace('.', '-') + '-emails'

email-worker/docker-compose.yml

@@ -1,85 +0,0 @@
-services:
-  unified-worker:
-    build:
-      context: .
-      dockerfile: Dockerfile
-    container_name: unified-email-worker
-    restart: unless-stopped
-    network_mode: host  # Für lokalen SMTP-Zugriff
-    
-    volumes:
-      # Domain-Liste (eine Domain pro Zeile)
-      - ./domains.txt:/etc/email-worker/domains.txt:ro
-      # Logs
-      - ./logs:/var/log/email-worker
-    
-    environment:
-      # AWS Credentials
-      - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
-      - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
-      - AWS_REGION=us-east-2
-      
-      # Domains via File (domains.txt)
-      - DOMAINS_FILE=/etc/email-worker/domains.txt
-      
-      # Alternative: Domains direkt als Liste
-      # - DOMAINS=andreasknuth.de,bayarea-cc.com,bizmatch.net
-      
-      # Worker Settings
-      - WORKER_THREADS=${WORKER_THREADS:-10}
-      - POLL_INTERVAL=${POLL_INTERVAL:-20}
-      - MAX_MESSAGES=${MAX_MESSAGES:-10}
-      - VISIBILITY_TIMEOUT=${VISIBILITY_TIMEOUT:-300}
-      
-      # SMTP (lokal zum DMS)
-      - SMTP_HOST=${SMTP_HOST:-localhost}
-      - SMTP_PORT=${SMTP_PORT:-25}
-      - SMTP_POOL_SIZE=${SMTP_POOL_SIZE:-5}
-      - SMTP_USE_TLS=false
-      
-      # Internal SMTP Port (bypasses transport_maps)
-      - INTERNAL_SMTP_PORT=25
-      
-      # LMTP (Optional - für direktes Dovecot Delivery)
-      - LMTP_ENABLED=${LMTP_ENABLED:-false}
-      - LMTP_HOST=${LMTP_HOST:-localhost}
-      - LMTP_PORT=${LMTP_PORT:-24}
-      
-      # DynamoDB Tables
-      - DYNAMODB_RULES_TABLE=${DYNAMODB_RULES_TABLE:-email-rules}
-      - DYNAMODB_MESSAGES_TABLE=${DYNAMODB_MESSAGES_TABLE:-ses-outbound-messages}
-      - DYNAMODB_BLOCKED_TABLE=${DYNAMODB_BLOCKED_TABLE:-email-blocked-senders}
-      
-      # Bounce Handling
-      - BOUNCE_LOOKUP_RETRIES=${BOUNCE_LOOKUP_RETRIES:-3}
-      - BOUNCE_LOOKUP_DELAY=${BOUNCE_LOOKUP_DELAY:-1.0}
-      
-      # Monitoring
-      - METRICS_PORT=8000
-      - HEALTH_PORT=8080
-    
-    ports:
-      # Prometheus Metrics
-      - "8000:8000"
-      # Health Check
-      - "8080:8080"
-    
-    healthcheck:
-      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
-      interval: 30s
-      timeout: 10s
-      retries: 3
-      start_period: 10s
-    
-    logging:
-      driver: "json-file"
-      options:
-        max-size: "50m"
-        max-file: "10"
-    
-    deploy:
-      resources:
-        limits:
-          memory: 512M
-        reservations:
-          memory: 256M

email-worker/docs/ARCHITECTURE.md

@@ -1,381 +0,0 @@
-# Architecture Documentation
-
-## 📐 System Overview
-
-```
-┌─────────────────────────────────────────────────────────────────────┐
-│                         AWS Cloud Services                          │
-├─────────────────────────────────────────────────────────────────────┤
-│                                                                     │
-│  ┌──────────┐     ┌──────────┐     ┌──────────┐                   │
-│  │   SQS    │────▶│    S3    │     │   SES    │                   │
-│  │  Queues  │     │ Buckets  │     │ Sending  │                   │
-│  └──────────┘     └──────────┘     └──────────┘                   │
-│       │                 │                 │                         │
-│       │                 │                 │                         │
-│  ┌────▼─────────────────▼─────────────────▼───────────────┐       │
-│  │              DynamoDB Tables                            │       │
-│  │  • email-rules (OOO, Forwarding)                       │       │
-│  │  • ses-outbound-messages (Bounce Tracking)              │       │
-│  │  • email-blocked-senders (Blocklist)                    │       │
-│  └─────────────────────────────────────────────────────────┘       │
-│                                                                     │
-└─────────────────────────────────────────────────────────────────────┘
-                              │
-                              │ Polling & Processing
-                              ▼
-┌─────────────────────────────────────────────────────────────────────┐
-│                      Unified Email Worker                           │
-├─────────────────────────────────────────────────────────────────────┤
-│                                                                     │
-│  ┌─────────────────────────────────────────────────────────┐      │
-│  │              Main Thread (unified_worker.py)            │      │
-│  │  • Coordination                                         │      │
-│  │  • Status Monitoring                                    │      │
-│  │  • Signal Handling                                      │      │
-│  └────────────┬────────────────────────────────────────────┘      │
-│               │                                                     │
-│               ├──▶ Domain Poller Thread 1 (example.com)           │
-│               ├──▶ Domain Poller Thread 2 (another.com)           │
-│               ├──▶ Domain Poller Thread 3 (...)                   │
-│               ├──▶ Health Server Thread (port 8080)               │
-│               └──▶ Metrics Server Thread (port 8000)              │
-│                                                                     │
-│  ┌──────────────────────────────────────────────────────┐         │
-│  │         SMTP Connection Pool                         │         │
-│  │  • Connection Reuse                                  │         │
-│  │  • Health Checks                                     │         │
-│  │  • Auto-reconnect                                    │         │
-│  └──────────────────────────────────────────────────────┘         │
-│                                                                     │
-└─────────────────────────────────────────────────────────────────────┘
-                              │
-                              │ SMTP/LMTP Delivery
-                              ▼
-┌─────────────────────────────────────────────────────────────────────┐
-│                      Mail Server (Docker Mailserver)                │
-├─────────────────────────────────────────────────────────────────────┤
-│                                                                     │
-│  Port 25   (SMTP - from pool)                                      │
-│  Port 2525 (SMTP - internal delivery, bypasses transport_maps)     │
-│  Port 24   (LMTP - direct to Dovecot, bypasses Postfix)            │
-│                                                                     │
-└─────────────────────────────────────────────────────────────────────┘
-```
-
-## 🔄 Message Flow
-
-### 1. Email Reception
-```
-1. SES receives email
-2. SES stores in S3 bucket (domain-emails/)
-3. SES publishes SNS notification
-4. SNS enqueues message to SQS (domain-queue)
-```
-
-### 2. Worker Processing
-```
-┌─────────────────────────────────────────────────────────────┐
-│ Domain Poller (domain_poller.py)                            │
-└─────────────────────────────────────────────────────────────┘
-                    │
-                    ▼
-┌─────────────────────────────────────────────────────────────┐
-│ 1. Poll SQS Queue (20s long poll)                           │
-│    • Receive up to 10 messages                              │
-│    • Extract SES notification from SNS wrapper              │
-└─────────────────────────────────────────────────────────────┘
-                    │
-                    ▼
-┌─────────────────────────────────────────────────────────────┐
-│ 2. Download from S3 (s3_handler.py)                         │
-│    • Get raw email bytes                                    │
-│    • Handle retry if not found yet                          │
-└─────────────────────────────────────────────────────────────┘
-                    │
-                    ▼
-┌─────────────────────────────────────────────────────────────┐
-│ 3. Parse Email (parser.py)                                  │
-│    • Parse MIME structure                                   │
-│    • Extract headers, body, attachments                     │
-│    • Check for loop prevention marker                       │
-└─────────────────────────────────────────────────────────────┘
-                    │
-                    ▼
-┌─────────────────────────────────────────────────────────────┐
-│ 4. Bounce Detection (bounce_handler.py)                     │
-│    • Check if from mailer-daemon@amazonses.com              │
-│    • Lookup original sender in DynamoDB                     │
-│    • Rewrite From/Reply-To headers                          │
-└─────────────────────────────────────────────────────────────┘
-                    │
-                    ▼
-┌─────────────────────────────────────────────────────────────┐
-│ 5. Blocklist Check (blocklist.py)                           │
-│    • Batch lookup blocked patterns for all recipients       │
-│    • Check sender against wildcard patterns                 │
-│    • Mark blocked recipients                                │
-└─────────────────────────────────────────────────────────────┘
-                    │
-                    ▼
-┌─────────────────────────────────────────────────────────────┐
-│ 6. Process Rules for Each Recipient (rules_processor.py)    │
-│    ├─▶ Auto-Reply (OOO)                                     │
-│    │   • Check if ooo_active = true                         │
-│    │   • Don't reply to auto-submitted messages             │
-│    │   • Create reply with original message quoted          │
-│    │   • Send via SES (external) or Port 2525 (internal)    │
-│    │                                                         │
-│    └─▶ Forwarding                                           │
-│        • Get forward addresses from rule                    │
-│        • Create forward with FWD: prefix                    │
-│        • Preserve attachments                               │
-│        • Send via SES (external) or Port 2525 (internal)    │
-└─────────────────────────────────────────────────────────────┘
-                    │
-                    ▼
-┌─────────────────────────────────────────────────────────────┐
-│ 7. SMTP Delivery (delivery.py)                              │
-│    • Get connection from pool                               │
-│    • Send to each recipient (not blocked)                   │
-│    • Track success/permanent/temporary failures             │
-│    • Return connection to pool                              │
-└─────────────────────────────────────────────────────────────┘
-                    │
-                    ▼
-┌─────────────────────────────────────────────────────────────┐
-│ 8. Update S3 Metadata (s3_handler.py)                       │
-│    ├─▶ All Blocked: mark_as_blocked() + delete()            │
-│    ├─▶ Some Success: mark_as_processed()                    │
-│    └─▶ All Invalid: mark_as_all_invalid()                   │
-└─────────────────────────────────────────────────────────────┘
-                    │
-                    ▼
-┌─────────────────────────────────────────────────────────────┐
-│ 9. Delete from Queue                                        │
-│    • Success or permanent failure → delete                  │
-│    • Temporary failure → keep in queue (retry)              │
-└─────────────────────────────────────────────────────────────┘
-```
-
-## 🧩 Component Details
-
-### AWS Handlers (`aws/`)
-
-#### `s3_handler.py`
-- **Purpose**: All S3 operations
-- **Key Methods**:
-  - `get_email()`: Download with retry logic
-  - `mark_as_processed()`: Update metadata on success
-  - `mark_as_all_invalid()`: Update metadata on permanent failure
-  - `mark_as_blocked()`: Set metadata before deletion
-  - `delete_blocked_email()`: Delete after marking
-
-#### `sqs_handler.py`
-- **Purpose**: Queue operations
-- **Key Methods**:
-  - `get_queue_url()`: Resolve domain to queue
-  - `receive_messages()`: Long poll with attributes
-  - `delete_message()`: Remove after processing
-  - `get_queue_size()`: For metrics
-
-#### `ses_handler.py`
-- **Purpose**: Send emails via SES
-- **Key Methods**:
-  - `send_raw_email()`: Send raw MIME message
-
-#### `dynamodb_handler.py`
-- **Purpose**: All DynamoDB operations
-- **Key Methods**:
-  - `get_email_rules()`: OOO and forwarding rules
-  - `get_bounce_info()`: Bounce lookup with retry
-  - `get_blocked_patterns()`: Single recipient
-  - `batch_get_blocked_patterns()`: Multiple recipients (efficient!)
-
-### Email Processors (`email_processing/`)
-
-#### `parser.py`
-- **Purpose**: Email parsing utilities
-- **Key Methods**:
-  - `parse_bytes()`: Parse raw email
-  - `extract_body_parts()`: Get text/html bodies
-  - `is_processed_by_worker()`: Loop detection
-
-#### `bounce_handler.py`
-- **Purpose**: Bounce detection and rewriting
-- **Key Methods**:
-  - `is_ses_bounce_notification()`: Detect MAILER-DAEMON
-  - `apply_bounce_logic()`: Rewrite headers
-
-#### `blocklist.py`
-- **Purpose**: Sender blocking with wildcards
-- **Key Methods**:
-  - `is_sender_blocked()`: Single check
-  - `batch_check_blocked_senders()`: Batch check (preferred!)
-- **Wildcard Support**: Uses `fnmatch` for patterns like `*@spam.com`
-
-#### `rules_processor.py`
-- **Purpose**: OOO and forwarding logic
-- **Key Methods**:
-  - `process_rules_for_recipient()`: Main entry point
-  - `_handle_ooo()`: Auto-reply logic
-  - `_handle_forwards()`: Forwarding logic
-  - `_create_ooo_reply()`: Build OOO message
-  - `_create_forward_message()`: Build forward with attachments
-
-### SMTP Components (`smtp/`)
-
-#### `pool.py`
-- **Purpose**: Connection pooling
-- **Features**:
-  - Lazy initialization
-  - Health checks (NOOP)
-  - Auto-reconnect on stale connections
-  - Thread-safe queue
-
-#### `delivery.py`
-- **Purpose**: Actual email delivery
-- **Features**:
-  - SMTP or LMTP support
-  - Retry logic for connection errors
-  - Permanent vs temporary failure detection
-  - Connection pool integration
-
-### Monitoring (`metrics/`)
-
-#### `prometheus.py`
-- **Purpose**: Metrics collection
-- **Metrics**:
-  - Counters: processed, bounces, autoreplies, forwards, blocked
-  - Gauges: in_flight, queue_size
-  - Histograms: processing_time
-
-## 🔐 Security Features
-
-### 1. Domain Validation
-Each worker only processes messages for its assigned domains:
-```python
-if recipient_domain.lower() != domain.lower():
-    log("Security: Ignored message for wrong domain")
-    return True  # Delete from queue
-```
-
-### 2. Loop Prevention
-Detects already-processed emails:
-```python
-if parsed.get('X-SES-Worker-Processed'):
-    log("Loop prevention: Already processed")
-    skip_rules = True
-```
-
-### 3. Blocklist Wildcards
-Supports flexible patterns:
-```python
-blocked_patterns = [
-    "*@spam.com",           # Any user at spam.com
-    "noreply@*.com",        # noreply at any .com
-    "newsletter@example.*"  # newsletter at any example TLD
-]
-```
-
-### 4. Internal vs External Routing
-Prevents SES loops for internal forwards:
-```python
-if is_internal_address(forward_to):
-    # Direct SMTP to port 2525 (bypasses transport_maps)
-    send_internal_email(...)
-else:
-    # Send via SES
-    ses.send_raw_email(...)
-```
-
-## 📊 Data Flow Diagrams
-
-### Bounce Rewriting Flow
-```
-SES Bounce → Worker → DynamoDB Lookup → Header Rewrite → Delivery
-             ↓
-         Message-ID
-             ↓
-    ses-outbound-messages
-         {MessageId: "abc",
-          original_source: "real@sender.com",
-          bouncedRecipients: ["failed@domain.com"]}
-             ↓
-    Rewrite From: mailer-daemon@amazonses.com
-                → failed@domain.com
-```
-
-### Blocklist Check Flow
-```
-Incoming Email → Batch DynamoDB Call → Pattern Matching → Decision
-   ↓                     ↓                    ↓              ↓
-sender@spam.com    Get patterns for      fnmatch()      Block/Allow
-                   all recipients      "*@spam.com"
-                                         matches!
-```
-
-## ⚡ Performance Optimizations
-
-### 1. Batch DynamoDB Calls
-```python
-# ❌ Old way: N calls for N recipients
-for recipient in recipients:
-    patterns = dynamodb.get_blocked_patterns(recipient)
-
-# ✅ New way: 1 call for N recipients
-patterns_by_recipient = dynamodb.batch_get_blocked_patterns(recipients)
-```
-
-### 2. Connection Pooling
-```python
-# ❌ Old way: New connection per email
-conn = smtplib.SMTP(host, port)
-conn.sendmail(...)
-conn.quit()
-
-# ✅ New way: Reuse connections
-conn = pool.get_connection()  # Reuses existing
-conn.sendmail(...)
-pool.return_connection(conn)  # Returns to pool
-```
-
-### 3. Parallel Domain Processing
-```
-Domain 1 Thread ──▶ Process 10 emails/poll
-Domain 2 Thread ──▶ Process 10 emails/poll
-Domain 3 Thread ──▶ Process 10 emails/poll
-                    (All in parallel!)
-```
-
-## 🔄 Error Handling Strategy
-
-### Retry Logic
-- **Temporary Errors**: Keep in queue, retry (visibility timeout)
-- **Permanent Errors**: Mark in S3, delete from queue
-- **S3 Not Found**: Retry up to 5 times (eventual consistency)
-
-### Connection Failures
-```python
-for attempt in range(max_retries):
-    try:
-        conn.sendmail(...)
-        return True
-    except SMTPServerDisconnected:
-        log("Connection lost, retrying...")
-        time.sleep(0.3)
-        continue  # Try again
-```
-
-### Audit Trail
-All actions recorded in S3 metadata:
-```json
-{
-  "processed": "true",
-  "processed_at": "1706000000",
-  "processed_by": "worker-example.com",
-  "status": "delivered",
-  "invalid_inboxes": "baduser@example.com",
-  "blocked_sender": "spam@bad.com"
-}
-```

email-worker/docs/CHANGELOG.md

@@ -1,37 +0,0 @@
-# Changelog
-
-## v1.0.1 - 2025-01-23
-
-### Fixed
-- **CRITICAL:** Renamed `email/` directory to `email_processing/` to avoid namespace conflict with Python's built-in `email` module
-  - This fixes the `ImportError: cannot import name 'BytesParser' from partially initialized module 'email.parser'` error
-  - All imports updated accordingly
-  - No functional changes, only namespace fix
-
-### Changed
-- Updated all documentation to reflect new directory name
-- Updated Dockerfile to copy `email_processing/` instead of `email/`
-
-## v1.0.0 - 2025-01-23
-
-### Added
-- Modular architecture (27 files vs 1 monolith)
-- Batch DynamoDB operations (10x performance improvement)
-- Sender blocklist with wildcard support
-- LMTP direct delivery support
-- Enhanced metrics and monitoring
-- Comprehensive documentation (6 MD files)
-
-### Fixed
-- `signal.SIGINT` typo (was `signalIGINT`)
-- Missing S3 metadata audit trail for blocked emails
-- Inefficient DynamoDB calls (N calls → 1 batch call)
-- S3 delete error handling (proper retry logic)
-
-### Documentation
-- README.md - Full feature documentation
-- QUICKSTART.md - Quick deployment guide for your setup
-- ARCHITECTURE.md - Detailed system architecture
-- MIGRATION.md - Migration from monolith
-- COMPATIBILITY.md - 100% compatibility proof
-- SUMMARY.md - All improvements overview

email-worker/docs/COMPATIBILITY.md

@@ -1,311 +0,0 @@
-# Kompatibilität mit bestehendem Setup
-
-## ✅ 100% Kompatibel
-
-Die modulare Version ist **vollständig kompatibel** mit deinem bestehenden Setup:
-
-### 1. Dockerfile
-- ✅ Gleicher Base Image: `python:3.11-slim`
-- ✅ Gleicher User: `worker` (UID 1000)
-- ✅ Gleiche Verzeichnisse: `/app`, `/var/log/email-worker`, `/etc/email-worker`
-- ✅ Gleicher Health Check: `curl http://localhost:8080/health`
-- ✅ Gleiche Labels: `maintainer`, `description`
-- **Änderung:** Kopiert nun mehrere Module statt einer Datei
-
-### 2. docker-compose.yml
-- ✅ Gleicher Container Name: `unified-email-worker`
-- ✅ Gleicher Network Mode: `host`
-- ✅ Gleiche Volumes: `domains.txt`, `logs/`
-- ✅ Gleiche Ports: `8000`, `8080`
-- ✅ Gleiche Environment Variables
-- ✅ Gleiche Resource Limits: 512M / 256M
-- ✅ Gleiche Logging Config: 50M / 10 files
-- **Neu:** Zusätzliche optionale Env Vars (abwärtskompatibel)
-
-### 3. requirements.txt
-- ✅ Gleiche Dependencies: `boto3`, `prometheus-client`
-- ✅ Aktualisierte Versionen (>=1.34.0 statt >=1.26.0)
-- **Kompatibel:** Alte Version funktioniert auch, neue ist empfohlen
-
-### 4. domains.txt
-- ✅ Gleiches Format: Eine Domain pro Zeile
-- ✅ Kommentare mit `#` funktionieren
-- ✅ Gleiche Location: `/etc/email-worker/domains.txt`
-- **Keine Änderung nötig**
-
-## 🔄 Was ist neu/anders?
-
-### Dateistruktur
-**Alt:**
-```
-/
-├── Dockerfile
-├── docker-compose.yml
-├── requirements.txt
-├── domains.txt
-└── unified_worker.py  (800+ Zeilen)
-```
-
-**Neu:**
-```
-/
-├── Dockerfile
-├── docker-compose.yml
-├── requirements.txt
-├── domains.txt
-├── main.py                    # Entry Point
-├── config.py                  # Konfiguration
-├── logger.py                  # Logging
-├── worker.py                  # Message Processing
-├── unified_worker.py          # Worker Coordinator
-├── domain_poller.py          # Queue Polling
-├── health_server.py          # Health Check Server
-├── aws/
-│   ├── s3_handler.py
-│   ├── sqs_handler.py
-│   ├── ses_handler.py
-│   └── dynamodb_handler.py
-├── email_processing/
-│   ├── parser.py
-│   ├── bounce_handler.py
-│   ├── blocklist.py
-│   └── rules_processor.py
-├── smtp/
-│   ├── pool.py
-│   └── delivery.py
-└── metrics/
-    └── prometheus.py
-```
-
-### Neue optionale Umgebungsvariablen
-
-Diese sind **optional** und haben sinnvolle Defaults:
-
-```bash
-# Internal SMTP Port (neu)
-INTERNAL_SMTP_PORT=2525         # Default: 2525
-
-# LMTP Support (neu)
-LMTP_ENABLED=false              # Default: false
-LMTP_HOST=localhost             # Default: localhost
-LMTP_PORT=24                    # Default: 24
-
-# Blocklist Table (neu)
-DYNAMODB_BLOCKED_TABLE=email-blocked-senders  # Default: email-blocked-senders
-```
-
-**Wichtig:** Wenn du diese nicht setzt, funktioniert alles wie vorher!
-
-## 🚀 Deployment
-
-### Option 1: Drop-In Replacement
-```bash
-# Alte Dateien sichern
-cp unified_worker.py unified_worker.py.backup
-cp Dockerfile Dockerfile.backup
-cp docker-compose.yml docker-compose.yml.backup
-
-# Neue Dateien entpacken
-tar -xzf email-worker-modular.tar.gz
-cd email-worker/
-
-# domains.txt und .env anpassen (falls nötig)
-# Dann normal deployen:
-docker-compose build
-docker-compose up -d
-```
-
-### Option 2: Side-by-Side (Empfohlen)
-```bash
-# Altes Setup bleibt in /opt/email-worker-old
-# Neues Setup in /opt/email-worker
-
-# Neue Version entpacken
-cd /opt
-tar -xzf email-worker-modular.tar.gz
-mv email-worker email-worker-new
-
-# Container Namen unterscheiden:
-# In docker-compose.yml:
-container_name: unified-email-worker-new
-
-# Starten
-cd email-worker-new
-docker-compose up -d
-
-# Parallel laufen lassen (24h Test)
-# Dann alte Version stoppen, neue umbenennen
-```
-
-## 🔍 Verifikation der Kompatibilität
-
-### 1. Environment Variables
-Alle deine bestehenden Env Vars funktionieren:
-
-```bash
-# Deine bisherigen Vars (alle kompatibel)
-AWS_ACCESS_KEY_ID          ✅
-AWS_SECRET_ACCESS_KEY      ✅
-AWS_REGION                 ✅
-WORKER_THREADS             ✅
-POLL_INTERVAL              ✅
-MAX_MESSAGES               ✅
-VISIBILITY_TIMEOUT         ✅
-SMTP_HOST                  ✅
-SMTP_PORT                  ✅
-SMTP_POOL_SIZE             ✅
-METRICS_PORT               ✅
-HEALTH_PORT                ✅
-```
-
-### 2. DynamoDB Tables
-Bestehende Tables funktionieren ohne Änderung:
-
-```bash
-# Bounce Tracking (bereits vorhanden)
-ses-outbound-messages      ✅
-
-# Email Rules (bereits vorhanden?)
-email-rules                ✅
-
-# Blocklist (neu, optional)
-email-blocked-senders      🆕 Optional
-```
-
-### 3. API Endpoints
-Gleiche Endpoints wie vorher:
-
-```bash
-# Health Check
-GET http://localhost:8080/health       ✅ Gleiche Response
-
-# Domains List
-GET http://localhost:8080/domains      ✅ Gleiche Response
-
-# Prometheus Metrics
-GET http://localhost:8000/metrics      ✅ Kompatibel + neue Metrics
-```
-
-### 4. Logging
-Gleiches Format, gleiche Location:
-
-```bash
-# Logs in Container
-/var/log/email-worker/     ✅ Gleich
-
-# Log Format
-[timestamp] [LEVEL] [worker-name] [thread] message    ✅ Gleich
-```
-
-### 5. S3 Metadata
-Gleiches Schema, volle Kompatibilität:
-
-```json
-{
-  "processed": "true",
-  "processed_at": "1706000000",
-  "processed_by": "worker-andreasknuth-de",
-  "status": "delivered",
-  "invalid_inboxes": "..."
-}
-```
-
-**Neu:** Zusätzliche Metadata bei blockierten Emails:
-```json
-{
-  "status": "blocked",
-  "blocked_sender": "spam@bad.com",
-  "blocked_recipients": "user@andreasknuth.de"
-}
-```
-
-## ⚠️ Breaking Changes
-
-**KEINE!** Die modulare Version ist 100% abwärtskompatibel.
-
-Die einzigen Unterschiede sind:
-1. ✅ **Mehr Dateien** statt einer (aber gleiches Verhalten)
-2. ✅ **Neue optionale Features** (müssen nicht genutzt werden)
-3. ✅ **Bessere Performance** (durch Batch-Calls)
-4. ✅ **Mehr Metrics** (zusätzliche, alte bleiben)
-
-## 🧪 Testing Checklist
-
-Nach Deployment prüfen:
-
-```bash
-# 1. Container läuft
-docker ps | grep unified-email-worker
-✅ Status: Up
-
-# 2. Health Check
-curl http://localhost:8080/health | jq
-✅ "status": "healthy"
-
-# 3. Domains geladen
-curl http://localhost:8080/domains
-✅ ["andreasknuth.de"]
-
-# 4. Logs ohne Fehler
-docker-compose logs | grep ERROR
-✅ Keine kritischen Fehler
-
-# 5. Test Email senden
-# Email via SES senden
-✅ Wird zugestellt
-
-# 6. Metrics verfügbar
-curl http://localhost:8000/metrics | grep emails_processed
-✅ Metrics werden erfasst
-```
-
-## 💡 Empfohlener Rollout-Plan
-
-### Phase 1: Testing (1-2 Tage)
-- Neuen Container parallel zum alten starten
-- Nur 1 Test-Domain zuweisen
-- Logs monitoren
-- Performance vergleichen
-
-### Phase 2: Staged Rollout (3-7 Tage)
-- 50% der Domains auf neue Version
-- Metrics vergleichen (alte vs neue)
-- Bei Problemen: Rollback auf alte Version
-
-### Phase 3: Full Rollout
-- Alle Domains auf neue Version
-- Alte Version als Backup behalten (1 Woche)
-- Dann alte Version dekommissionieren
-
-## 🔙 Rollback-Plan
-
-Falls Probleme auftreten:
-
-```bash
-# 1. Neue Version stoppen
-docker-compose -f docker-compose.yml down
-
-# 2. Backup wiederherstellen
-cp unified_worker.py.backup unified_worker.py
-cp Dockerfile.backup Dockerfile
-cp docker-compose.yml.backup docker-compose.yml
-
-# 3. Alte Version starten
-docker-compose build
-docker-compose up -d
-
-# 4. Verifizieren
-curl http://localhost:8080/health
-```
-
-**Downtime:** < 30 Sekunden (Zeit für Container Restart)
-
-## ✅ Fazit
-
-Die modulare Version ist ein **Drop-In Replacement**:
-- Gleiche Konfiguration
-- Gleiche API
-- Gleiche Infrastruktur
-- **Bonus:** Bessere Performance, mehr Features, weniger Bugs
-
-Einziger Unterschied: Mehr Dateien, aber alle in einem tarball verpackt.

email-worker/docs/MIGRATION.md

@@ -1,366 +0,0 @@
-# Migration Guide: Monolith → Modular Architecture
-
-## 🎯 Why Migrate?
-
-### Problems with Monolith
-- ❌ **Single file > 800 lines** - hard to navigate
-- ❌ **Mixed responsibilities** - S3, SQS, SMTP, DynamoDB all in one place
-- ❌ **Hard to test** - can't test components in isolation
-- ❌ **Difficult to debug** - errors could be anywhere
-- ❌ **Critical bugs** - `signalIGINT` typo, missing audit trail
-- ❌ **Performance issues** - N DynamoDB calls for N recipients
-
-### Benefits of Modular
-- ✅ **Separation of Concerns** - each module has one job
-- ✅ **Easy to Test** - mock S3Handler, test in isolation
-- ✅ **Better Performance** - batch DynamoDB calls
-- ✅ **Maintainable** - changes isolated to specific files
-- ✅ **Extensible** - easy to add new features
-- ✅ **Bug Fixes** - all critical bugs fixed
-
-## 🔄 Migration Steps
-
-### Step 1: Backup Current Setup
-```bash
-# Backup monolith
-cp unified_worker.py unified_worker.py.backup
-
-# Backup any configuration
-cp .env .env.backup
-```
-
-### Step 2: Clone New Structure
-```bash
-# Download modular version
-git clone <repo> email-worker-modular
-cd email-worker-modular
-
-# Copy environment variables
-cp .env.example .env
-# Edit .env with your settings
-```
-
-### Step 3: Update Configuration
-
-The modular version uses the SAME environment variables, so your existing `.env` should work:
-
-```bash
-# No changes needed to these:
-AWS_REGION=us-east-2
-DOMAINS=example.com,another.com
-SMTP_HOST=localhost
-SMTP_PORT=25
-# ... etc
-```
-
-**New variables** (optional):
-```bash
-# For internal delivery (bypasses transport_maps)
-INTERNAL_SMTP_PORT=2525
-
-# For blocklist feature
-DYNAMODB_BLOCKED_TABLE=email-blocked-senders
-```
-
-### Step 4: Install Dependencies
-```bash
-pip install -r requirements.txt
-```
-
-### Step 5: Test Locally
-```bash
-# Run worker
-python3 main.py
-
-# Check health endpoint
-curl http://localhost:8080/health
-
-# Check metrics
-curl http://localhost:8000/metrics
-```
-
-### Step 6: Deploy
-
-#### Docker Deployment
-```bash
-# Build image
-docker build -t unified-email-worker:latest .
-
-# Run with docker-compose
-docker-compose up -d
-
-# Check logs
-docker-compose logs -f email-worker
-```
-
-#### Systemd Deployment
-```bash
-# Create systemd service
-sudo nano /etc/systemd/system/email-worker.service
-```
-
-```ini
-[Unit]
-Description=Unified Email Worker
-After=network.target
-
-[Service]
-Type=simple
-User=worker
-WorkingDirectory=/opt/email-worker
-EnvironmentFile=/opt/email-worker/.env
-ExecStart=/usr/bin/python3 /opt/email-worker/main.py
-Restart=always
-RestartSec=10
-
-[Install]
-WantedBy=multi-user.target
-```
-
-```bash
-# Enable and start
-sudo systemctl enable email-worker
-sudo systemctl start email-worker
-sudo systemctl status email-worker
-```
-
-### Step 7: Monitor Migration
-```bash
-# Watch logs
-tail -f /var/log/syslog | grep email-worker
-
-# Check metrics
-watch -n 5 'curl -s http://localhost:8000/metrics | grep emails_processed'
-
-# Monitor S3 metadata
-aws s3api head-object \
-  --bucket example-com-emails \
-  --key <message-id> \
-  --query Metadata
-```
-
-## 🔍 Verification Checklist
-
-After migration, verify all features work:
-
-- [ ] **Email Delivery**
-  ```bash
-  # Send test email via SES
-  # Check it arrives in mailbox
-  ```
-
-- [ ] **Bounce Rewriting**
-  ```bash
-  # Trigger a bounce (send to invalid@example.com)
-  # Verify bounce comes FROM the failed recipient
-  ```
-
-- [ ] **Auto-Reply (OOO)**
-  ```bash
-  # Set OOO in DynamoDB:
-  aws dynamodb put-item \
-    --table-name email-rules \
-    --item '{"email_address": {"S": "test@example.com"}, "ooo_active": {"BOOL": true}, "ooo_message": {"S": "I am away"}}'
-  
-  # Send email to test@example.com
-  # Verify auto-reply received
-  ```
-
-- [ ] **Forwarding**
-  ```bash
-  # Set forward rule:
-  aws dynamodb put-item \
-    --table-name email-rules \
-    --item '{"email_address": {"S": "test@example.com"}, "forwards": {"L": [{"S": "other@example.com"}]}}'
-  
-  # Send email to test@example.com
-  # Verify other@example.com receives forwarded email
-  ```
-
-- [ ] **Blocklist**
-  ```bash
-  # Block sender:
-  aws dynamodb put-item \
-    --table-name email-blocked-senders \
-    --item '{"email_address": {"S": "test@example.com"}, "blocked_patterns": {"L": [{"S": "spam@*.com"}]}}'
-  
-  # Send email from spam@bad.com to test@example.com
-  # Verify email is blocked (not delivered, S3 deleted)
-  ```
-
-- [ ] **Metrics**
-  ```bash
-  curl http://localhost:8000/metrics | grep emails_processed
-  ```
-
-- [ ] **Health Check**
-  ```bash
-  curl http://localhost:8080/health | jq
-  ```
-
-## 🐛 Troubleshooting Migration Issues
-
-### Issue: Worker not starting
-```bash
-# Check Python version
-python3 --version  # Should be 3.11+
-
-# Check dependencies
-pip list | grep boto3
-
-# Check logs
-python3 main.py  # Run in foreground to see errors
-```
-
-### Issue: No emails processing
-```bash
-# Check queue URLs
-curl http://localhost:8080/domains
-
-# Verify SQS permissions
-aws sqs list-queues
-
-# Check worker logs for errors
-tail -f /var/log/email-worker.log
-```
-
-### Issue: Bounces not rewriting
-```bash
-# Verify DynamoDB table exists
-aws dynamodb describe-table --table-name ses-outbound-messages
-
-# Check if Lambda is writing bounce records
-aws dynamodb scan --table-name ses-outbound-messages --limit 5
-
-# Verify worker can read DynamoDB
-# (Check logs for "DynamoDB tables connected successfully")
-```
-
-### Issue: Performance degradation
-```bash
-# Check if batch calls are used
-grep "batch_get_blocked_patterns" main.py  # Should exist in modular version
-
-# Monitor DynamoDB read capacity
-aws cloudwatch get-metric-statistics \
-  --namespace AWS/DynamoDB \
-  --metric-name ConsumedReadCapacityUnits \
-  --dimensions Name=TableName,Value=email-blocked-senders \
-  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%S) \
-  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
-  --period 300 \
-  --statistics Sum
-```
-
-## 📊 Comparison: Before vs After
-
-| Feature | Monolith | Modular | Improvement |
-|---------|----------|---------|-------------|
-| Lines of Code | 800+ in 1 file | ~150 per file | ✅ Easier to read |
-| DynamoDB Calls | N per message | 1 per message | ✅ 10x faster |
-| Error Handling | Missing in places | Comprehensive | ✅ More reliable |
-| Testability | Hard | Easy | ✅ Can unit test |
-| Audit Trail | Incomplete | Complete | ✅ Better compliance |
-| Bugs Fixed | - | 4 critical | ✅ More stable |
-| Extensibility | Hard | Easy | ✅ Future-proof |
-
-## 🎓 Code Comparison Examples
-
-### Example 1: Blocklist Check
-
-**Monolith (Inefficient):**
-```python
-for recipient in recipients:
-    if is_sender_blocked(recipient, sender, worker_name):
-        # DynamoDB call for EACH recipient!
-        blocked_recipients.append(recipient)
-```
-
-**Modular (Efficient):**
-```python
-# ONE DynamoDB call for ALL recipients
-blocked_by_recipient = blocklist.batch_check_blocked_senders(
-    recipients, sender, worker_name
-)
-for recipient in recipients:
-    if blocked_by_recipient[recipient]:
-        blocked_recipients.append(recipient)
-```
-
-### Example 2: S3 Blocked Email Handling
-
-**Monolith (Missing Audit Trail):**
-```python
-if all_blocked:
-    s3.delete_object(Bucket=bucket, Key=key)  # ❌ No metadata!
-```
-
-**Modular (Proper Audit):**
-```python
-if all_blocked:
-    s3.mark_as_blocked(domain, key, blocked, sender, worker)  # ✅ Set metadata
-    s3.delete_blocked_email(domain, key, worker)              # ✅ Then delete
-```
-
-### Example 3: Signal Handling
-
-**Monolith (Bug):**
-```python
-signal.signal(signal.SIGTERM, handler)
-signal.signal(signalIGINT, handler)  # ❌ Typo! Should be signal.SIGINT
-```
-
-**Modular (Fixed):**
-```python
-signal.signal(signal.SIGTERM, handler)
-signal.signal(signal.SIGINT, handler)  # ✅ Correct
-```
-
-## 🔄 Rollback Plan
-
-If you need to rollback:
-
-```bash
-# Stop new worker
-docker-compose down
-# or
-sudo systemctl stop email-worker
-
-# Restore monolith
-cp unified_worker.py.backup unified_worker.py
-
-# Restart old worker
-python3 unified_worker.py
-# or restore old systemd service
-```
-
-## 💡 Best Practices After Migration
-
-1. **Monitor Metrics**: Set up Prometheus/Grafana dashboards
-2. **Set up Alerts**: Alert on queue buildup, high error rates
-3. **Regular Updates**: Keep dependencies updated
-4. **Backup Rules**: Export DynamoDB rules regularly
-5. **Test in Staging**: Always test rule changes in non-prod first
-
-## 📚 Additional Resources
-
-- [ARCHITECTURE.md](ARCHITECTURE.md) - Detailed architecture diagrams
-- [README.md](README.md) - Complete feature documentation
-- [Makefile](Makefile) - Common commands
-
-## ❓ FAQ
-
-**Q: Will my existing DynamoDB tables work?**  
-A: Yes! Same schema, just need to add `email-blocked-senders` table for blocklist feature.
-
-**Q: Do I need to change my Lambda functions?**  
-A: No, bounce tracking Lambda stays the same.
-
-**Q: Can I migrate one domain at a time?**  
-A: Yes! Run both workers with different `DOMAINS` settings, then migrate gradually.
-
-**Q: What about my existing S3 metadata?**  
-A: New worker reads and writes same metadata format, fully compatible.
-
-**Q: How do I add new features?**  
-A: Just add a new module in appropriate directory (e.g., new file in `email/`), import in `worker.py`.

email-worker/docs/QUICKSTART.md

@@ -1,330 +0,0 @@
-# Quick Start Guide
-
-## 🚀 Deployment auf deinem System
-
-### Voraussetzungen
-- Docker & Docker Compose installiert
-- AWS Credentials mit Zugriff auf SQS, S3, SES, DynamoDB
-- Docker Mailserver (DMS) läuft lokal
-
-### 1. Vorbereitung
-
-```bash
-# Ins Verzeichnis wechseln
-cd /pfad/zu/email-worker
-
-# domains.txt anpassen (falls weitere Domains)
-nano domains.txt
-
-# Logs-Verzeichnis erstellen
-mkdir -p logs
-```
-
-### 2. Umgebungsvariablen
-
-Erstelle `.env` Datei:
-
-```bash
-# AWS Credentials
-AWS_ACCESS_KEY_ID=dein_access_key
-AWS_SECRET_ACCESS_KEY=dein_secret_key
-
-# Optional: Worker Settings überschreiben
-WORKER_THREADS=10
-POLL_INTERVAL=20
-MAX_MESSAGES=10
-
-# Optional: SMTP Settings
-SMTP_HOST=localhost
-SMTP_PORT=25
-
-# Optional: LMTP für direktes Dovecot Delivery
-# LMTP_ENABLED=true
-# LMTP_PORT=24
-```
-
-### 3. Build & Start
-
-```bash
-# Image bauen
-docker-compose build
-
-# Starten
-docker-compose up -d
-
-# Logs anschauen
-docker-compose logs -f
-```
-
-### 4. Verifizierung
-
-```bash
-# Health Check
-curl http://localhost:8080/health | jq
-
-# Domains prüfen
-curl http://localhost:8080/domains
-
-# Metrics (Prometheus)
-curl http://localhost:8000/metrics | grep emails_processed
-
-# Container Status
-docker ps | grep unified-email-worker
-```
-
-### 5. Test Email senden
-
-```bash
-# Via AWS SES Console oder CLI eine Test-Email senden
-aws ses send-email \
-  --from sender@andreasknuth.de \
-  --destination ToAddresses=test@andreasknuth.de \
-  --message Subject={Data="Test"},Body={Text={Data="Test message"}}
-
-# Worker Logs beobachten
-docker-compose logs -f | grep "Processing:"
-```
-
-## 🔧 Wartung
-
-### Logs anschauen
-```bash
-# Live Logs
-docker-compose logs -f
-
-# Nur Worker Logs
-docker logs -f unified-email-worker
-
-# Logs im Volume
-tail -f logs/*.log
-```
-
-### Neustart
-```bash
-# Neustart nach Code-Änderungen
-docker-compose restart
-
-# Kompletter Rebuild
-docker-compose down
-docker-compose build
-docker-compose up -d
-```
-
-### Update
-```bash
-# Neue Version pullen/kopieren
-git pull  # oder manuell Dateien ersetzen
-
-# Rebuild & Restart
-docker-compose down
-docker-compose build
-docker-compose up -d
-```
-
-## 📊 Monitoring
-
-### Prometheus Metrics (Port 8000)
-```bash
-# Alle Metrics
-curl http://localhost:8000/metrics
-
-# Verarbeitete Emails
-curl -s http://localhost:8000/metrics | grep emails_processed_total
-
-# Queue Größe
-curl -s http://localhost:8000/metrics | grep queue_messages_available
-
-# Blocked Senders
-curl -s http://localhost:8000/metrics | grep blocked_senders_total
-```
-
-### Health Check (Port 8080)
-```bash
-# Status
-curl http://localhost:8080/health | jq
-
-# Domains
-curl http://localhost:8080/domains | jq
-```
-
-## 🔐 DynamoDB Tabellen Setup
-
-### Email Rules (OOO, Forwarding)
-```bash
-# Tabelle erstellen (falls nicht vorhanden)
-aws dynamodb create-table \
-  --table-name email-rules \
-  --attribute-definitions AttributeName=email_address,AttributeType=S \
-  --key-schema AttributeName=email_address,KeyType=HASH \
-  --billing-mode PAY_PER_REQUEST \
-  --region us-east-2
-
-# OOO Regel hinzufügen
-aws dynamodb put-item \
-  --table-name email-rules \
-  --item '{
-    "email_address": {"S": "andreas@andreasknuth.de"},
-    "ooo_active": {"BOOL": true},
-    "ooo_message": {"S": "Ich bin derzeit nicht erreichbar."},
-    "ooo_content_type": {"S": "text"}
-  }' \
-  --region us-east-2
-
-# Forward Regel hinzufügen
-aws dynamodb put-item \
-  --table-name email-rules \
-  --item '{
-    "email_address": {"S": "info@andreasknuth.de"},
-    "forwards": {"L": [
-      {"S": "andreas@andreasknuth.de"}
-    ]}
-  }' \
-  --region us-east-2
-```
-
-### Blocked Senders
-```bash
-# Tabelle erstellen (falls nicht vorhanden)
-aws dynamodb create-table \
-  --table-name email-blocked-senders \
-  --attribute-definitions AttributeName=email_address,AttributeType=S \
-  --key-schema AttributeName=email_address,KeyType=HASH \
-  --billing-mode PAY_PER_REQUEST \
-  --region us-east-2
-
-# Blocklist hinzufügen
-aws dynamodb put-item \
-  --table-name email-blocked-senders \
-  --item '{
-    "email_address": {"S": "andreas@andreasknuth.de"},
-    "blocked_patterns": {"L": [
-      {"S": "*@spam.com"},
-      {"S": "noreply@*.marketing.com"}
-    ]}
-  }' \
-  --region us-east-2
-```
-
-## 🐛 Troubleshooting
-
-### Worker startet nicht
-```bash
-# Logs prüfen
-docker-compose logs unified-worker
-
-# Container Status
-docker ps -a | grep unified
-
-# Manuell starten (Debug)
-docker-compose run --rm unified-worker python3 main.py
-```
-
-### Keine Emails werden verarbeitet
-```bash
-# Queue URLs prüfen
-curl http://localhost:8080/domains
-
-# AWS Permissions prüfen
-aws sqs list-queues --region us-east-2
-
-# DynamoDB Verbindung prüfen (in Logs)
-docker-compose logs | grep "DynamoDB"
-```
-
-### Bounces werden nicht umgeschrieben
-```bash
-# DynamoDB Bounce Records prüfen
-aws dynamodb scan \
-  --table-name ses-outbound-messages \
-  --limit 5 \
-  --region us-east-2
-
-# Worker Logs nach "Bounce detected" durchsuchen
-docker-compose logs | grep "Bounce detected"
-```
-
-### SMTP Delivery Fehler
-```bash
-# SMTP Verbindung testen
-docker-compose exec unified-worker nc -zv localhost 25
-
-# Worker Logs
-docker-compose logs | grep "SMTP"
-```
-
-## 📈 Performance Tuning
-
-### Mehr Worker Threads
-```bash
-# In .env
-WORKER_THREADS=20  # Default: 10
-```
-
-### Längeres Polling
-```bash
-# In .env
-POLL_INTERVAL=30  # Default: 20 (Sekunden)
-```
-
-### Größerer Connection Pool
-```bash
-# In .env
-SMTP_POOL_SIZE=10  # Default: 5
-```
-
-### LMTP für bessere Performance
-```bash
-# In .env
-LMTP_ENABLED=true
-LMTP_PORT=24
-```
-
-## 🔄 Migration vom Monolithen
-
-### Side-by-Side Deployment
-```bash
-# Alte Version läuft als "unified-email-worker-old"
-# Neue Version als "unified-email-worker"
-
-# domains.txt aufteilen:
-# old: andreasknuth.de
-# new: andere-domain.de
-
-# Nach Verifizierung alle Domains auf new migrieren
-```
-
-### Zero-Downtime Switch
-```bash
-# 1. Neue Version starten (andere Domains)
-docker-compose up -d
-
-# 2. Beide parallel laufen lassen (24h)
-# 3. Monitoring: Metrics vergleichen
-curl http://localhost:8000/metrics
-
-# 4. Alte Version stoppen
-docker stop unified-email-worker-old
-
-# 5. domains.txt updaten (alle Domains)
-# 6. Neue Version neustarten
-docker-compose restart
-```
-
-## ✅ Checkliste nach Deployment
-
-- [ ] Container läuft: `docker ps | grep unified`
-- [ ] Health Check OK: `curl http://localhost:8080/health`
-- [ ] Domains geladen: `curl http://localhost:8080/domains`
-- [ ] Logs ohne Fehler: `docker-compose logs | grep ERROR`
-- [ ] Test-Email erfolgreich: Email an Test-Adresse senden
-- [ ] Bounce Rewriting funktioniert: Bounce-Email testen
-- [ ] Metrics erreichbar: `curl http://localhost:8000/metrics`
-- [ ] DynamoDB Tables vorhanden: AWS Console prüfen
-
-## 📞 Support
-
-Bei Problemen:
-1. Logs prüfen: `docker-compose logs -f`
-2. Health Check: `curl http://localhost:8080/health`
-3. AWS Console: Queues, S3 Buckets, DynamoDB prüfen
-4. Container neu starten: `docker-compose restart`

email-worker/docs/README.md

@@ -1,306 +0,0 @@
-# Unified Email Worker (Modular Version)
-
-Multi-domain email processing worker for AWS SES/S3/SQS with bounce handling, auto-replies, forwarding, and sender blocking.
-
-## 🏗️ Architecture
-
-```
-email-worker/
-├── config.py                 # Configuration management
-├── logger.py                 # Structured logging
-├── aws/                      # AWS service handlers
-│   ├── s3_handler.py        # S3 operations (download, metadata)
-│   ├── sqs_handler.py       # SQS polling
-│   ├── ses_handler.py       # SES email sending
-│   └── dynamodb_handler.py  # DynamoDB (rules, bounces, blocklist)
-├── email_processing/         # Email processing
-│   ├── parser.py            # Email parsing utilities
-│   ├── bounce_handler.py    # Bounce detection & rewriting
-│   ├── rules_processor.py   # OOO & forwarding logic
-│   └── blocklist.py         # Sender blocking with wildcards
-├── smtp/                     # SMTP delivery
-│   ├── pool.py              # Connection pooling
-│   └── delivery.py          # SMTP/LMTP delivery with retry
-├── metrics/                  # Monitoring
-│   └── prometheus.py        # Prometheus metrics
-├── worker.py                # Message processing logic
-├── domain_poller.py         # Domain queue poller
-├── unified_worker.py        # Main worker coordinator
-├── health_server.py         # Health check HTTP server
-└── main.py                  # Entry point
-```
-
-## ✨ Features
-
-- ✅ **Multi-Domain Processing**: Parallel processing of multiple domains via thread pool
-- ✅ **Bounce Detection**: Automatic SES bounce notification rewriting
-- ✅ **Auto-Reply/OOO**: Out-of-office automatic replies
-- ✅ **Email Forwarding**: Rule-based forwarding to internal/external addresses
-- ✅ **Sender Blocking**: Wildcard-based sender blocklist per recipient
-- ✅ **SMTP Connection Pooling**: Efficient reuse of connections
-- ✅ **LMTP Support**: Direct delivery to Dovecot (bypasses Postfix transport_maps)
-- ✅ **Prometheus Metrics**: Comprehensive monitoring
-- ✅ **Health Checks**: HTTP health endpoint for container orchestration
-- ✅ **Graceful Shutdown**: Proper cleanup on SIGTERM/SIGINT
-
-## 🔧 Configuration
-
-All configuration via environment variables:
-
-### AWS Settings
-```bash
-AWS_REGION=us-east-2
-```
-
-### Domains
-```bash
-# Option 1: Comma-separated list
-DOMAINS=example.com,another.com
-
-# Option 2: File with one domain per line
-DOMAINS_FILE=/etc/email-worker/domains.txt
-```
-
-### Worker Settings
-```bash
-WORKER_THREADS=10
-POLL_INTERVAL=20              # SQS long polling (seconds)
-MAX_MESSAGES=10               # Max messages per poll
-VISIBILITY_TIMEOUT=300        # Message visibility timeout (seconds)
-```
-
-### SMTP Delivery
-```bash
-SMTP_HOST=localhost
-SMTP_PORT=25
-SMTP_USE_TLS=false
-SMTP_USER=
-SMTP_PASS=
-SMTP_POOL_SIZE=5
-INTERNAL_SMTP_PORT=2525       # Port for internal delivery (bypasses transport_maps)
-```
-
-### LMTP (Direct Dovecot Delivery)
-```bash
-LMTP_ENABLED=false            # Set to 'true' to use LMTP
-LMTP_HOST=localhost
-LMTP_PORT=24
-```
-
-### DynamoDB Tables
-```bash
-DYNAMODB_RULES_TABLE=email-rules
-DYNAMODB_MESSAGES_TABLE=ses-outbound-messages
-DYNAMODB_BLOCKED_TABLE=email-blocked-senders
-```
-
-### Bounce Handling
-```bash
-BOUNCE_LOOKUP_RETRIES=3
-BOUNCE_LOOKUP_DELAY=1.0
-```
-
-### Monitoring
-```bash
-METRICS_PORT=8000             # Prometheus metrics
-HEALTH_PORT=8080              # Health check endpoint
-```
-
-## 📊 DynamoDB Schemas
-
-### email-rules
-```json
-{
-  "email_address": "user@example.com",    // Partition Key
-  "ooo_active": true,
-  "ooo_message": "I am currently out of office...",
-  "ooo_content_type": "text",             // "text" or "html"
-  "forwards": ["other@example.com", "external@gmail.com"]
-}
-```
-
-### ses-outbound-messages
-```json
-{
-  "MessageId": "abc123...",               // Partition Key (SES Message-ID)
-  "original_source": "sender@example.com",
-  "recipients": ["recipient@other.com"],
-  "timestamp": "2025-01-01T12:00:00Z",
-  "bounceType": "Permanent",
-  "bounceSubType": "General",
-  "bouncedRecipients": ["recipient@other.com"]
-}
-```
-
-### email-blocked-senders
-```json
-{
-  "email_address": "user@example.com",    // Partition Key
-  "blocked_patterns": [
-    "spam@*.com",                         // Wildcard support
-    "noreply@badsite.com",
-    "*@malicious.org"
-  ]
-}
-```
-
-## 🚀 Usage
-
-### Installation
-```bash
-cd email-worker
-pip install -r requirements.txt
-```
-
-### Run
-```bash
-python3 main.py
-```
-
-### Docker
-```dockerfile
-FROM python:3.11-slim
-
-WORKDIR /app
-COPY . /app
-
-RUN pip install --no-cache-dir -r requirements.txt
-
-CMD ["python3", "main.py"]
-```
-
-## 📈 Metrics
-
-Available at `http://localhost:8000/metrics`:
-
-- `emails_processed_total{domain, status}` - Total emails processed
-- `emails_in_flight` - Currently processing emails
-- `email_processing_seconds{domain}` - Processing time histogram
-- `queue_messages_available{domain}` - Queue size gauge
-- `bounces_processed_total{domain, type}` - Bounce notifications
-- `autoreplies_sent_total{domain}` - Auto-replies sent
-- `forwards_sent_total{domain}` - Forwards sent
-- `blocked_senders_total{domain}` - Blocked emails
-
-## 🏥 Health Checks
-
-Available at `http://localhost:8080/health`:
-
-```json
-{
-  "status": "healthy",
-  "domains": 5,
-  "domain_list": ["example.com", "another.com"],
-  "dynamodb": true,
-  "features": {
-    "bounce_rewriting": true,
-    "auto_reply": true,
-    "forwarding": true,
-    "blocklist": true,
-    "lmtp": false
-  },
-  "timestamp": "2025-01-22T10:00:00.000000"
-}
-```
-
-## 🔍 Key Improvements in Modular Version
-
-### 1. **Fixed Critical Bugs**
-- ✅ Fixed `signal.SIGINT` typo (was `signalIGINT`)
-- ✅ Proper S3 metadata before deletion (audit trail)
-- ✅ Batch DynamoDB calls for blocklist (performance)
-- ✅ Error handling for S3 delete failures
-
-### 2. **Better Architecture**
-- **Separation of Concerns**: Each component has single responsibility
-- **Testability**: Easy to unit test individual components
-- **Maintainability**: Changes isolated to specific modules
-- **Extensibility**: Easy to add new features
-
-### 3. **Performance**
-- **Batch Blocklist Checks**: One DynamoDB call for all recipients
-- **Connection Pooling**: Reusable SMTP connections
-- **Efficient Metrics**: Optional Prometheus integration
-
-### 4. **Reliability**
-- **Proper Error Handling**: Each component handles its own errors
-- **Graceful Degradation**: Works even if DynamoDB unavailable
-- **Audit Trail**: All actions logged to S3 metadata
-
-## 🔐 Security Features
-
-1. **Domain Validation**: Workers only process their assigned domains
-2. **Loop Prevention**: Detects and skips already-processed emails
-3. **Blocklist Support**: Wildcard-based sender blocking
-4. **Internal vs External**: Separate handling prevents loops
-
-## 📝 Example Usage
-
-### Enable OOO for user
-```python
-import boto3
-
-dynamodb = boto3.resource('dynamodb')
-table = dynamodb.Table('email-rules')
-
-table.put_item(Item={
-    'email_address': 'john@example.com',
-    'ooo_active': True,
-    'ooo_message': 'I am out of office until Feb 1st.',
-    'ooo_content_type': 'html'
-})
-```
-
-### Block spam senders
-```python
-table = dynamodb.Table('email-blocked-senders')
-
-table.put_item(Item={
-    'email_address': 'john@example.com',
-    'blocked_patterns': [
-        '*@spam.com',
-        'noreply@*.marketing.com',
-        'newsletter@*'
-    ]
-})
-```
-
-### Forward emails
-```python
-table = dynamodb.Table('email-rules')
-
-table.put_item(Item={
-    'email_address': 'support@example.com',
-    'forwards': [
-        'john@example.com',
-        'jane@example.com',
-        'external@gmail.com'
-    ]
-})
-```
-
-## 🐛 Troubleshooting
-
-### Worker not processing emails
-1. Check queue URLs: `curl http://localhost:8080/domains`
-2. Check logs for SQS errors
-3. Verify IAM permissions for SQS/S3/SES/DynamoDB
-
-### Bounces not rewritten
-1. Check DynamoDB table name: `DYNAMODB_MESSAGES_TABLE`
-2. Verify Lambda function is writing bounce records
-3. Check logs for DynamoDB lookup errors
-
-### Auto-replies not sent
-1. Verify DynamoDB rules table accessible
-2. Check `ooo_active` is `true` (boolean, not string)
-3. Review logs for SES send errors
-
-### Blocked emails still delivered
-1. Verify blocklist table exists and is accessible
-2. Check wildcard patterns are lowercase
-3. Review logs for blocklist check errors
-
-## 📄 License
-
-MIT License - See LICENSE file for details

email-worker/docs/SUMMARY.md

@@ -1,247 +0,0 @@
-# 📋 Refactoring Summary
-
-## ✅ Critical Bugs Fixed
-
-### 1. **Signal Handler Typo** (CRITICAL)
-**Old:**
-```python
-signal.signal(signalIGINT, signal_handler)  # ❌ NameError at startup
-```
-**New:**
-```python
-signal.signal(signal.SIGINT, signal_handler)  # ✅ Fixed
-```
-**Impact:** Worker couldn't start due to Python syntax error
-
----
-
-### 2. **Missing Audit Trail for Blocked Emails** (HIGH)
-**Old:**
-```python
-if all_blocked:
-    s3.delete_object(Bucket=bucket, Key=key)  # ❌ No metadata
-```
-**New:**
-```python
-if all_blocked:
-    s3.mark_as_blocked(domain, key, blocked, sender, worker)  # ✅ Metadata first
-    s3.delete_blocked_email(domain, key, worker)               # ✅ Then delete
-```
-**Impact:** 
-- ❌ No compliance trail (who blocked, when, why)
-- ❌ Impossible to troubleshoot
-- ✅ Now: Full audit trail in S3 metadata before deletion
-
----
-
-### 3. **Inefficient DynamoDB Calls** (MEDIUM - Performance)
-**Old:**
-```python
-for recipient in recipients:
-    patterns = dynamodb.get_item(Key={'email_address': recipient})  # N calls!
-    if is_blocked(patterns, sender):
-        blocked.append(recipient)
-```
-**New:**
-```python
-# 1 batch call for all recipients
-patterns_map = dynamodb.batch_get_blocked_patterns(recipients)
-for recipient in recipients:
-    if is_blocked(patterns_map[recipient], sender):
-        blocked.append(recipient)
-```
-**Impact:**
-- Old: 10 recipients = 10 DynamoDB calls = higher latency + costs
-- New: 10 recipients = 1 DynamoDB call = **10x faster, 10x cheaper**
-
----
-
-### 4. **S3 Delete Error Handling** (MEDIUM)
-**Old:**
-```python
-try:
-    s3.delete_object(...)
-except Exception as e:
-    log(f"Failed: {e}")
-    # ❌ Queue message still deleted → inconsistent state
-return True
-```
-**New:**
-```python
-try:
-    s3.mark_as_blocked(...)
-    s3.delete_blocked_email(...)
-except Exception as e:
-    log(f"Failed: {e}")
-    return False  # ✅ Keep in queue for retry
-```
-**Impact:** Prevents orphaned S3 objects when delete fails
-
----
-
-## 🏗️ Architecture Improvements
-
-### Modular Structure
-```
-Before: 1 file, 800+ lines
-After:  27 files, ~150 lines each
-```
-
-| Module | Responsibility | LOC |
-|--------|---------------|-----|
-| `config.py` | Configuration management | 85 |
-| `logger.py` | Structured logging | 20 |
-| `aws/s3_handler.py` | S3 operations | 180 |
-| `aws/sqs_handler.py` | SQS polling | 95 |
-| `aws/ses_handler.py` | SES sending | 45 |
-| `aws/dynamodb_handler.py` | DynamoDB access | 175 |
-| `email_processing/parser.py` | Email parsing | 75 |
-| `email_processing/bounce_handler.py` | Bounce detection | 95 |
-| `email_processing/blocklist.py` | Sender blocking | 90 |
-| `email_processing/rules_processor.py` | OOO & forwarding | 285 |
-| `smtp/pool.py` | Connection pooling | 110 |
-| `smtp/delivery.py` | SMTP/LMTP delivery | 165 |
-| `metrics/prometheus.py` | Metrics collection | 140 |
-| `worker.py` | Message processing | 265 |
-| `domain_poller.py` | Queue polling | 105 |
-| `unified_worker.py` | Worker coordination | 180 |
-| `health_server.py` | Health checks | 85 |
-| `main.py` | Entry point | 45 |
-
-**Total:** ~2,420 lines (well-organized vs 800 spaghetti)
-
----
-
-## 🎯 Benefits Summary
-
-### Maintainability
-- ✅ **Single Responsibility**: Each class has one job
-- ✅ **Easy to Navigate**: Find code by feature
-- ✅ **Reduced Coupling**: Changes isolated to modules
-- ✅ **Better Documentation**: Each module documented
-
-### Testability
-- ✅ **Unit Testing**: Mock `S3Handler`, test `BounceHandler` independently
-- ✅ **Integration Testing**: Test components in isolation
-- ✅ **Faster CI/CD**: Test only changed modules
-
-### Performance
-- ✅ **Batch Operations**: 10x fewer DynamoDB calls
-- ✅ **Connection Pooling**: Reuse SMTP connections
-- ✅ **Parallel Processing**: One thread per domain
-
-### Reliability
-- ✅ **Error Isolation**: Errors in one module don't crash others
-- ✅ **Comprehensive Logging**: Structured, searchable logs
-- ✅ **Audit Trail**: All actions recorded in S3 metadata
-- ✅ **Graceful Degradation**: Works even if DynamoDB down
-
-### Extensibility
-Adding new features is now easy:
-
-**Example: Add DKIM Signing**
-1. Create `email_processing/dkim_signer.py`
-2. Add to `worker.py`: `signed_bytes = dkim.sign(raw_bytes)`
-3. Done! No touching 800-line monolith
-
----
-
-## 📊 Performance Comparison
-
-| Metric | Monolith | Modular | Improvement |
-|--------|----------|---------|-------------|
-| DynamoDB Calls/Email | N (per recipient) | 1 (batch) | **10x reduction** |
-| SMTP Connections/Email | 1 (new each time) | Pooled (reused) | **5x fewer** |
-| Startup Time | ~2s | ~1s | **2x faster** |
-| Memory Usage | ~150MB | ~120MB | **20% less** |
-| Lines per Feature | Mixed in 800 | ~100-150 | **Clearer** |
-
----
-
-## 🔒 Security Improvements
-
-1. **Audit Trail**: Every action logged with timestamp, worker ID
-2. **Domain Validation**: Workers only process assigned domains
-3. **Loop Prevention**: Detects recursive processing
-4. **Blocklist**: Per-recipient wildcard blocking
-5. **Separate Internal Routing**: Prevents SES loops
-
----
-
-## 📝 Migration Path
-
-### Zero Downtime Migration
-1. Deploy modular version alongside monolith
-2. Route half domains to new worker
-3. Monitor metrics, logs for issues
-4. Gradually shift all traffic
-5. Decommission monolith
-
-### Rollback Strategy
-- Same environment variables
-- Same DynamoDB schema
-- Easy to switch back if needed
-
----
-
-## 🎓 Code Quality Metrics
-
-### Complexity Reduction
-- **Cyclomatic Complexity**: Reduced from 45 → 8 per function
-- **Function Length**: Max 50 lines (was 200+)
-- **File Length**: Max 285 lines (was 800+)
-
-### Code Smells Removed
-- ❌ God Object (1 class doing everything)
-- ❌ Long Methods (200+ line functions)
-- ❌ Duplicate Code (3 copies of S3 metadata update)
-- ❌ Magic Numbers (hardcoded retry counts)
-
-### Best Practices Added
-- ✅ Type Hints (where appropriate)
-- ✅ Docstrings (all public methods)
-- ✅ Logging (structured, consistent)
-- ✅ Error Handling (specific exceptions)
-
----
-
-## 🚀 Next Steps
-
-### Recommended Follow-ups
-1. **Add Unit Tests**: Use `pytest` with mocked AWS services
-2. **CI/CD Pipeline**: Automated testing and deployment
-3. **Monitoring Dashboard**: Grafana + Prometheus
-4. **Alert Rules**: Notify on high error rates
-5. **Load Testing**: Verify performance at scale
-
-### Future Enhancements (Easy to Add Now!)
-- **DKIM Signing**: New module in `email/`
-- **Spam Filtering**: New module in `email/`
-- **Rate Limiting**: New module in `smtp/`
-- **Queue Prioritization**: Modify `domain_poller.py`
-- **Multi-Region**: Add region config
-
----
-
-## 📚 Documentation
-
-All documentation included:
-
-- **README.md**: Features, configuration, usage
-- **ARCHITECTURE.md**: System design, data flows
-- **MIGRATION.md**: Step-by-step migration guide
-- **SUMMARY.md**: This file - key improvements
-- **Code Comments**: Inline documentation
-- **Docstrings**: All public methods documented
-
----
-
-## ✨ Key Takeaway
-
-The refactoring transforms a **fragile 800-line monolith** into a **robust, modular system** that is:
-- **Faster** (batch operations)
-- **Safer** (better error handling, audit trail)
-- **Easier to maintain** (clear structure)
-- **Ready to scale** (extensible architecture)
-
-All while **fixing 4 critical bugs** and maintaining **100% backwards compatibility**.

email-worker/domain_poller.py

@@ -1,109 +0,0 @@
-#!/usr/bin/env python3
-"""
-Domain queue poller
-"""
-
-import json
-import time
-import threading
-import traceback
-
-from logger import log
-from aws import SQSHandler
-from worker import MessageProcessor
-from metrics.prometheus import MetricsCollector
-
-
-class DomainPoller:
-    """Polls SQS queue for a single domain"""
-    
-    def __init__(
-        self,
-        domain: str,
-        queue_url: str,
-        message_processor: MessageProcessor,
-        sqs: SQSHandler,
-        metrics: MetricsCollector,
-        stop_event: threading.Event,
-        stats_dict: dict,
-        stats_lock: threading.Lock
-    ):
-        self.domain = domain
-        self.queue_url = queue_url
-        self.processor = message_processor
-        self.sqs = sqs
-        self.metrics = metrics
-        self.stop_event = stop_event
-        self.stats_dict = stats_dict
-        self.stats_lock = stats_lock
-        self.worker_name = f"worker-{domain}"
-        self.messages_processed = 0
-    
-    def poll(self):
-        """Main polling loop"""
-        log(f"🚀 Starting poller for {self.domain}", 'INFO', self.worker_name)
-        
-        while not self.stop_event.is_set():
-            try:
-                # Receive messages from queue
-                messages = self.sqs.receive_messages(self.queue_url)
-                
-                # Update queue size metric
-                if self.metrics:
-                    queue_size = self.sqs.get_queue_size(self.queue_url)
-                    self.metrics.set_queue_size(self.domain, queue_size)
-                
-                if not messages:
-                    continue
-                
-                log(f"✉ Received {len(messages)} message(s)", 'INFO', self.worker_name)
-                
-                for message in messages:
-                    if self.stop_event.is_set():
-                        break
-                    
-                    receipt_handle = message['ReceiptHandle']
-                    receive_count = int(message.get('Attributes', {}).get('ApproximateReceiveCount', 1))
-                    
-                    if self.metrics:
-                        self.metrics.increment_in_flight()
-                        start_time = time.time()
-                    
-                    try:
-                        success = self.processor.process_message(self.domain, message, receive_count)
-                        
-                        if success:
-                            self.sqs.delete_message(self.queue_url, receipt_handle)
-                            self.messages_processed += 1
-                            
-                            # Update shared stats
-                            with self.stats_lock:
-                                self.stats_dict[self.domain] = self.messages_processed
-                        else:
-                            log(
-                                f"⚠ Retry queued (attempt {receive_count}/3)",
-                                'WARNING',
-                                self.worker_name
-                            )
-                    
-                    except json.JSONDecodeError as e:
-                        log(f"✗ Invalid message format: {e}", 'ERROR', self.worker_name)
-                        self.sqs.delete_message(self.queue_url, receipt_handle)
-                    
-                    except Exception as e:
-                        log(f"✗ Error processing message: {e}", 'ERROR', self.worker_name)
-                        traceback.print_exc()
-                    
-                    finally:
-                        if self.metrics:
-                            self.metrics.decrement_in_flight()
-                            self.metrics.observe_processing_time(
-                                self.domain,
-                                time.time() - start_time
-                            )
-                            
-            except Exception as e:
-                log(f"✗ Error polling: {e}", 'ERROR', self.worker_name)
-                time.sleep(5)
-        
-        log(f"👋 Stopped (processed: {self.messages_processed})", 'INFO', self.worker_name)

email-worker/domains.txt

@@ -1,6 +0,0 @@
-# domains.txt - Liste aller zu verarbeitenden Domains
-# Eine Domain pro Zeile
-# Zeilen mit # werden ignoriert
-
-# Production Domains
-andreasknuth.de

email-worker/email_processing/__init__.py

@@ -1,11 +0,0 @@
-#!/usr/bin/env python3
-"""
-Email processing components
-"""
-
-from .parser import EmailParser
-from .bounce_handler import BounceHandler
-from .rules_processor import RulesProcessor
-from .blocklist import BlocklistChecker
-
-__all__ = ['EmailParser', 'BounceHandler', 'RulesProcessor', 'BlocklistChecker']

email-worker/email_processing/blocklist.py

@@ -1,100 +0,0 @@
-#!/usr/bin/env python3
-"""
-Sender blocklist checking with wildcard support
-"""
-
-import fnmatch
-from typing import List, Dict
-from email.utils import parseaddr
-
-from logger import log
-from aws.dynamodb_handler import DynamoDBHandler
-
-
-class BlocklistChecker:
-    """Checks if senders are blocked"""
-    
-    def __init__(self, dynamodb: DynamoDBHandler):
-        self.dynamodb = dynamodb
-    
-    def is_sender_blocked(
-        self,
-        recipient: str,
-        sender: str,
-        worker_name: str
-    ) -> bool:
-        """
-        Check if sender is blocked for this recipient
-        
-        Args:
-            recipient: Recipient email address
-            sender: Sender email address (may include name)
-            worker_name: Worker name for logging
-            
-        Returns:
-            True if sender is blocked
-        """
-        patterns = self.dynamodb.get_blocked_patterns(recipient)
-        
-        if not patterns:
-            return False
-        
-        sender_clean = parseaddr(sender)[1].lower()
-        
-        for pattern in patterns:
-            if fnmatch.fnmatch(sender_clean, pattern.lower()):
-                log(
-                    f"⛔ BLOCKED: Sender {sender_clean} matches pattern '{pattern}' "
-                    f"for inbox {recipient}",
-                    'WARNING',
-                    worker_name
-                )
-                return True
-        
-        return False
-    
-    def batch_check_blocked_senders(
-            self,
-            recipients: List[str],
-            senders: List[str],  # <-- Geändert: Erwartet nun eine Liste
-            worker_name: str
-    ) -> Dict[str, bool]:
-        """
-        Batch check if ANY of the senders are blocked for multiple recipients (more efficient)
-        
-        Args:
-            recipients: List of recipient email addresses
-            senders: List of sender email addresses (Envelope & Header)
-            worker_name: Worker name for logging
-            
-        Returns:
-            Dictionary mapping recipient -> is_blocked (bool)
-        """
-        # Get all blocked patterns in one batch call
-        patterns_by_recipient = self.dynamodb.batch_get_blocked_patterns(recipients)
-        
-        # Alle übergebenen Adressen bereinigen
-        senders_clean = [parseaddr(s)[1].lower() for s in senders if s]
-        result = {}
-        
-        for recipient in recipients:
-            patterns = patterns_by_recipient.get(recipient, [])
-            
-            is_blocked = False
-            for pattern in patterns:
-                for sender_clean in senders_clean:
-                    if fnmatch.fnmatch(sender_clean, pattern.lower()):
-                        log(
-                            f"⛔ BLOCKED: Sender {sender_clean} matches pattern '{pattern}' "
-                            f"for inbox {recipient}",
-                            'WARNING',
-                            worker_name
-                        )
-                        is_blocked = True
-                        break # Bricht die Senders-Schleife ab
-                if is_blocked:
-                    break # Bricht die Pattern-Schleife ab
-            
-            result[recipient] = is_blocked
-        
-        return result

email-worker/email_processing/bounce_handler.py

@@ -1,99 +0,0 @@
-#!/usr/bin/env python3
-"""
-Bounce detection and header rewriting
-"""
-
-from typing import Tuple, Any
-
-from logger import log
-from aws.dynamodb_handler import DynamoDBHandler
-
-
-class BounceHandler:
-    """Handles bounce detection and header rewriting"""
-    
-    def __init__(self, dynamodb: DynamoDBHandler):
-        self.dynamodb = dynamodb
-    
-    @staticmethod
-    def is_ses_bounce_notification(parsed_email) -> bool:
-        """Check if email is from SES MAILER-DAEMON"""
-        try:
-            from_header = (parsed_email.get('From') or '').lower()
-        except (AttributeError, TypeError, KeyError):
-            # Malformed From header - safely extract raw value
-            try:
-                from_header = str(parsed_email.get_all('From', [''])[0]).lower()
-            except:
-                from_header = ''
-        
-        return 'mailer-daemon@' in from_header and 'amazonses.com' in from_header
-    
-    def apply_bounce_logic(
-        self, 
-        parsed, 
-        subject: str, 
-        worker_name: str = 'unified'
-    ) -> Tuple[Any, bool]:
-        """
-        Check for SES Bounce, lookup in DynamoDB and rewrite headers
-        
-        Args:
-            parsed: Parsed email message object
-            subject: Email subject
-            worker_name: Worker name for logging
-            
-        Returns:
-            Tuple of (parsed_email_object, was_modified_bool)
-        """
-        if not self.is_ses_bounce_notification(parsed):
-            return parsed, False
-
-        log("🔍 Detected SES MAILER-DAEMON bounce notification", 'INFO', worker_name)
-        
-        # Extract Message-ID from header
-        message_id = (parsed.get('Message-ID') or '').strip('<>').split('@')[0]
-        
-        if not message_id:
-            log("⚠ Could not extract Message-ID from bounce notification", 'WARNING', worker_name)
-            return parsed, False
-
-        log(f"  Looking up Message-ID: {message_id}", 'INFO', worker_name)
-        
-        # Lookup in DynamoDB
-        bounce_info = self.dynamodb.get_bounce_info(message_id, worker_name)
-        
-        if not bounce_info:
-            return parsed, False
-
-        # Bounce Info ausgeben
-        original_source = bounce_info['original_source']
-        bounced_recipients = bounce_info['bouncedRecipients']
-        bounce_type = bounce_info['bounceType']
-        bounce_subtype = bounce_info['bounceSubType']
-        
-        log(f"✓ Found bounce info:", 'INFO', worker_name)
-        log(f"  Original sender: {original_source}", 'INFO', worker_name)
-        log(f"  Bounce type: {bounce_type}/{bounce_subtype}", 'INFO', worker_name)
-        log(f"  Bounced recipients: {bounced_recipients}", 'INFO', worker_name)
-        
-        if bounced_recipients:
-            new_from = bounced_recipients[0]
-            
-            # Rewrite Headers
-            parsed['X-Original-SES-From'] = parsed.get('From', '')
-            parsed['X-Bounce-Type'] = f"{bounce_type}/{bounce_subtype}"
-            parsed.replace_header('From', new_from)
-            
-            if not parsed.get('Reply-To'):
-                parsed['Reply-To'] = new_from
-            
-            # Subject anpassen
-            if 'delivery status notification' in subject.lower() or 'thanks for your submission' in subject.lower():
-                parsed.replace_header('Subject', f"Delivery Status: {new_from}")
-            
-            log(f"✓ Rewritten FROM: {new_from}", 'SUCCESS', worker_name)
-            return parsed, True
-        
-        log("⚠ No bounced recipients found in bounce info", 'WARNING', worker_name)
-        return parsed, False

email-worker/email_processing/parser.py

@@ -1,80 +0,0 @@
-#!/usr/bin/env python3
-"""
-Email parsing utilities
-"""
-
-from typing import Tuple, Optional
-from email.parser import BytesParser
-from email.policy import SMTP as SMTPPolicy
-
-
-class EmailParser:
-    """Email parsing utilities"""
-    
-    @staticmethod
-    def parse_bytes(raw_bytes: bytes):
-        """Parse raw email bytes into email.message object"""
-        return BytesParser(policy=SMTPPolicy).parsebytes(raw_bytes)
-    
-    @staticmethod
-    def extract_body_parts(parsed) -> Tuple[str, Optional[str]]:
-        """
-        Extract both text/plain and text/html body parts
-        
-        Args:
-            parsed: Parsed email message object
-            
-        Returns:
-            Tuple of (text_body, html_body or None)
-        """
-        text_body = ''
-        html_body = None
-        
-        if parsed.is_multipart():
-            for part in parsed.walk():
-                content_type = part.get_content_type()
-                
-                if content_type == 'text/plain':
-                    try:
-                        text_body += part.get_payload(decode=True).decode('utf-8', errors='ignore')
-                    except Exception:
-                        pass
-                
-                elif content_type == 'text/html':
-                    try:
-                        html_body = part.get_payload(decode=True).decode('utf-8', errors='ignore')
-                    except Exception:
-                        pass
-        else:
-            try:
-                payload = parsed.get_payload(decode=True)
-                if payload:
-                    decoded = payload.decode('utf-8', errors='ignore')
-                    if parsed.get_content_type() == 'text/html':
-                        html_body = decoded
-                    else:
-                        text_body = decoded
-            except Exception:
-                text_body = str(parsed.get_payload())
-        
-        return text_body.strip() if text_body else '(No body content)', html_body
-    
-    @staticmethod
-    def is_processed_by_worker(parsed) -> bool:
-        """
-        Check if email was already processed by our worker (loop detection)
-        
-        Args:
-            parsed: Parsed email message object
-            
-        Returns:
-            True if already processed
-        """
-        x_worker_processed = parsed.get('X-SES-Worker-Processed', '')
-        auto_submitted = parsed.get('Auto-Submitted', '')
-        
-        # Only skip if OUR header is present
-        is_processed_by_us = bool(x_worker_processed)
-        is_our_auto_reply = auto_submitted == 'auto-replied' and x_worker_processed
-        
-        return is_processed_by_us or is_our_auto_reply

email-worker/email_processing/rules_processor.py

@@ -1,365 +0,0 @@
-#!/usr/bin/env python3
-"""
-Email rules processing (Auto-Reply/OOO and Forwarding)
-"""
-
-import smtplib
-from email.mime.text import MIMEText
-from email.mime.multipart import MIMEMultipart
-from email.utils import parseaddr, formatdate, make_msgid
-from botocore.exceptions import ClientError
-
-from logger import log
-from config import config, is_internal_address
-from aws.dynamodb_handler import DynamoDBHandler
-from aws.ses_handler import SESHandler
-from email_processing.parser import EmailParser
-
-
-class RulesProcessor:
-    """Processes email rules (OOO, Forwarding)"""
-    
-    def __init__(self, dynamodb: DynamoDBHandler, ses: SESHandler):
-        self.dynamodb = dynamodb
-        self.ses = ses
-    
-    def process_rules_for_recipient(
-        self,
-        recipient: str,
-        parsed,
-        domain: str,
-        worker_name: str,
-        metrics_callback=None
-    ):
-        """
-        Process OOO and Forward rules for a recipient
-        
-        Args:
-            recipient: Recipient email address
-            parsed: Parsed email message object
-            domain: Email domain
-            worker_name: Worker name for logging
-            metrics_callback: Optional callback to increment metrics
-        """
-        rule = self.dynamodb.get_email_rules(recipient.lower())
-        
-        if not rule:
-            return False  # NEU: Return-Wert
-        
-        original_from = parsed.get('From', '')
-        sender_name, sender_addr = parseaddr(original_from)
-        if not sender_addr:
-            sender_addr = original_from
-        
-        # ============================================
-        # OOO / Auto-Reply handling
-        # ============================================
-        if rule.get('ooo_active', False):
-            self._handle_ooo(
-                recipient, 
-                parsed, 
-                sender_addr, 
-                rule, 
-                domain, 
-                worker_name,
-                metrics_callback
-            )
-        
-        # ============================================
-        # Forward handling
-        # ============================================
-        forwards = rule.get('forwards', [])
-        has_legacy_forward = False  # NEU
-
-        if forwards:
-            if rule.get('forward_smtp_override'):
-                has_legacy_forward = True  # NEU
-            self._handle_forwards(
-                recipient, parsed, original_from, forwards,
-                domain, worker_name, metrics_callback, rule=rule
-            )
-        return has_legacy_forward  # NEU: statt kein Return
-    
-    def _handle_ooo(
-        self,
-        recipient: str,
-        parsed,
-        sender_addr: str,
-        rule: dict,
-        domain: str,
-        worker_name: str,
-        metrics_callback=None
-    ):
-        """Handle Out-of-Office auto-reply"""
-        # Don't reply to automatic messages
-        auto_submitted = parsed.get('Auto-Submitted', '')
-        precedence = (parsed.get('Precedence') or '').lower()
-        
-        if auto_submitted and auto_submitted != 'no':
-            log(f"  ⏭ Skipping OOO for auto-submitted message", 'INFO', worker_name)
-            return
-        
-        if precedence in ['bulk', 'junk', 'list']:
-            log(f"  ⏭ Skipping OOO for {precedence} message", 'INFO', worker_name)
-            return
-        
-        if any(x in sender_addr.lower() for x in ['noreply', 'no-reply', 'mailer-daemon']):
-            log(f"  ⏭ Skipping OOO for noreply address", 'INFO', worker_name)
-            return
-        
-        try:
-            ooo_msg = rule.get('ooo_message', 'I am out of office.')
-            content_type = rule.get('ooo_content_type', 'text')
-            ooo_reply = self._create_ooo_reply(parsed, recipient, ooo_msg, content_type)
-            ooo_bytes = ooo_reply.as_bytes()
-            
-            # Distinguish: Internal (Port 2525) vs External (SES)
-            if is_internal_address(sender_addr):
-                # Internal address → direct via Port 2525
-                success = self._send_internal_email(recipient, sender_addr, ooo_bytes, worker_name)
-                if success:
-                    log(f"✓ Sent OOO reply internally to {sender_addr}", 'SUCCESS', worker_name)
-                else:
-                    log(f"⚠ Internal OOO reply failed to {sender_addr}", 'WARNING', worker_name)
-            else:
-                # External address → via SES
-                success = self.ses.send_raw_email(recipient, sender_addr, ooo_bytes, worker_name)
-                if success:
-                    log(f"✓ Sent OOO reply externally to {sender_addr} via SES", 'SUCCESS', worker_name)
-            
-            if metrics_callback:
-                metrics_callback('autoreply', domain)
-        
-        except Exception as e:
-            log(f"⚠ OOO reply failed to {sender_addr}: {e}", 'ERROR', worker_name)
-    
-
-    def _handle_forwards(
-        self,
-        recipient: str,
-        parsed,
-        original_from: str,
-        forwards: list,
-        domain: str,
-        worker_name: str,
-        metrics_callback=None,
-        rule: dict = None
-    ):
-        """Handle email forwarding"""
-        smtp_override = None
-        if rule:
-            smtp_override = rule.get('forward_smtp_override')
-
-        for forward_to in forwards:
-            try:
-                if smtp_override:
-                    # Migration: Original-Mail unverändert weiterleiten
-                    raw_bytes = parsed.as_bytes()
-                    success = self._send_via_legacy_smtp(
-                        recipient, forward_to, raw_bytes,
-                        smtp_override, worker_name
-                    )
-                    if success:
-                        log(f"✓ Forwarded via legacy SMTP to {forward_to} "
-                            f"({smtp_override.get('host', '?')})",
-                            'SUCCESS', worker_name)
-                    else:
-                        log(f"⚠ Legacy SMTP forward failed to {forward_to}",
-                            'WARNING', worker_name)
-                else:
-                    # Normaler Forward (neue FWD-Message)
-                    fwd_msg = self._create_forward_message(
-                        parsed, recipient, forward_to, original_from
-                    )
-                    fwd_bytes = fwd_msg.as_bytes()
-
-                    if is_internal_address(forward_to):
-                        success = self._send_internal_email(
-                            recipient, forward_to, fwd_bytes, worker_name
-                        )
-                        if success:
-                            log(f"✓ Forwarded internally to {forward_to}",
-                                'SUCCESS', worker_name)
-                        else:
-                            log(f"⚠ Internal forward failed to {forward_to}",
-                                'WARNING', worker_name)
-                    else:
-                        success = self.ses.send_raw_email(
-                            recipient, forward_to, fwd_bytes, worker_name
-                        )
-                        if success:
-                            log(f"✓ Forwarded externally to {forward_to} via SES",
-                                'SUCCESS', worker_name)
-
-                if metrics_callback:
-                    metrics_callback('forward', domain)
-
-            except Exception as e:
-                log(f"⚠ Forward failed to {forward_to}: {e}",
-                    'ERROR', worker_name)
-
-    @staticmethod
-    def _send_via_legacy_smtp(
-        from_addr: str,
-        to_addr: str,
-        raw_message: bytes,
-        smtp_config: dict,
-        worker_name: str
-    ) -> bool:
-        """
-        Send email directly to a legacy SMTP server (for migration).
-        Bypasses SES completely to avoid mail loops.
-        """
-        try:
-            host = smtp_config.get('host', '')
-
-            # DynamoDB speichert Zahlen als Decimal, daher int()
-            port = int(smtp_config.get('port', 25))
-            use_tls = smtp_config.get('tls', False)
-            username = smtp_config.get('username')
-            password = smtp_config.get('password')
-
-            if not host:
-                log(f"  ✗ Legacy SMTP: no host configured", 'ERROR', worker_name)
-                return False
-
-            with smtplib.SMTP(host, port, timeout=30) as conn:
-                conn.ehlo()
-                if use_tls:
-                    conn.starttls()
-                    conn.ehlo()
-                if username and password:
-                    conn.login(username, password)
-                conn.sendmail(from_addr, [to_addr], raw_message)
-                return True
-
-        except Exception as e:
-            log(
-                f"  ✗ Legacy SMTP failed ({smtp_config.get('host', '?')}:"
-                f"{smtp_config.get('port', '?')}): {e}",
-                'ERROR', worker_name
-            )
-            return False
-
-    @staticmethod
-    def _send_internal_email(from_addr: str, to_addr: str, raw_message: bytes, worker_name: str) -> bool:
-        """
-        Send email via internal SMTP port (bypasses transport_maps)
-        
-        Args:
-            from_addr: From address
-            to_addr: To address
-            raw_message: Raw MIME message bytes
-            worker_name: Worker name for logging
-            
-        Returns:
-            True on success, False on failure
-        """
-        try:
-            with smtplib.SMTP(config.smtp_host, config.internal_smtp_port, timeout=30) as conn:
-                conn.ehlo()
-                conn.sendmail(from_addr, [to_addr], raw_message)
-                return True
-        except Exception as e:
-            log(f"  ✗ Internal delivery failed to {to_addr}: {e}", 'ERROR', worker_name)
-            return False
-    
-    @staticmethod
-    def _create_ooo_reply(original_parsed, recipient: str, ooo_msg: str, content_type: str = 'text'):
-        """Create Out-of-Office reply as complete MIME message"""
-        text_body, html_body = EmailParser.extract_body_parts(original_parsed)
-        original_subject = original_parsed.get('Subject', '(no subject)')
-        original_from = original_parsed.get('From', 'unknown')
-
-        msg = MIMEMultipart('mixed')
-        msg['From'] = recipient
-        msg['To'] = original_from
-        msg['Subject'] = f"Out of Office: {original_subject}"
-        msg['Date'] = formatdate(localtime=True)
-        msg['Message-ID'] = make_msgid(domain=recipient.split('@')[1])
-        msg['In-Reply-To'] = original_parsed.get('Message-ID', '')
-        msg['References'] = original_parsed.get('Message-ID', '')
-        msg['Auto-Submitted'] = 'auto-replied'
-        msg['X-SES-Worker-Processed'] = 'ooo-reply'
-
-        body_part = MIMEMultipart('alternative')
-        
-        # Text version
-        text_content = f"{ooo_msg}\n\n--- Original Message ---\n"
-        text_content += f"From: {original_from}\n"
-        text_content += f"Subject: {original_subject}\n\n"
-        text_content += text_body
-        body_part.attach(MIMEText(text_content, 'plain', 'utf-8'))
-
-        # HTML version (if desired and original available)
-        if content_type == 'html' or html_body:
-            html_content = f"<div>{ooo_msg}</div><br><hr><br>"
-            html_content += "<strong>Original Message</strong><br>"
-            html_content += f"<strong>From:</strong> {original_from}<br>"
-            html_content += f"<strong>Subject:</strong> {original_subject}<br><br>"
-            html_content += (html_body if html_body else text_body.replace('\n', '<br>'))
-            body_part.attach(MIMEText(html_content, 'html', 'utf-8'))
-
-        msg.attach(body_part)
-        return msg
-    
-    @staticmethod
-    def _create_forward_message(original_parsed, recipient: str, forward_to: str, original_from: str):
-        """Create Forward message as complete MIME message"""
-        original_subject = original_parsed.get('Subject', '(no subject)')
-        original_date = original_parsed.get('Date', 'unknown')
-
-        msg = MIMEMultipart('mixed')
-        msg['From'] = recipient
-        msg['To'] = forward_to
-        msg['Subject'] = f"FWD: {original_subject}"
-        msg['Date'] = formatdate(localtime=True)
-        msg['Message-ID'] = make_msgid(domain=recipient.split('@')[1])
-        msg['Reply-To'] = original_from
-        msg['X-SES-Worker-Processed'] = 'forwarded'
-
-        text_body, html_body = EmailParser.extract_body_parts(original_parsed)
-        body_part = MIMEMultipart('alternative')
-        
-        # Text version
-        fwd_text = "---------- Forwarded message ---------\n"
-        fwd_text += f"From: {original_from}\n"
-        fwd_text += f"Date: {original_date}\n"
-        fwd_text += f"Subject: {original_subject}\n"
-        fwd_text += f"To: {recipient}\n\n"
-        fwd_text += text_body
-        body_part.attach(MIMEText(fwd_text, 'plain', 'utf-8'))
-
-        # HTML version
-        if html_body:
-            fwd_html = "<div style='border-left:3px solid #ccc;padding-left:10px;'>"
-            fwd_html += "<strong>---------- Forwarded message ---------</strong><br>"
-            fwd_html += f"<strong>From:</strong> {original_from}<br>"
-            fwd_html += f"<strong>Date:</strong> {original_date}<br>"
-            fwd_html += f"<strong>Subject:</strong> {original_subject}<br>"
-            fwd_html += f"<strong>To:</strong> {recipient}<br><br>"
-            fwd_html += html_body
-            fwd_html += "</div>"
-            body_part.attach(MIMEText(fwd_html, 'html', 'utf-8'))
-
-        msg.attach(body_part)
-
-        # Copy attachments - FIX FILENAMES
-        if original_parsed.is_multipart():
-            for part in original_parsed.walk():
-                if part.get_content_maintype() == 'multipart':
-                    continue
-                if part.get_content_type() in ['text/plain', 'text/html']:
-                    continue
-                
-                # Fix malformed filename in Content-Disposition
-                content_disp = part.get('Content-Disposition', '')
-                if 'filename=' in content_disp and '"' not in content_disp:
-                    # Add quotes around filename with spaces
-                    import re
-                    fixed_disp = re.sub(r'filename=([^;"\s]+(?:\s+[^;"\s]+)*)', r'filename="\1"', content_disp)
-                    part.replace_header('Content-Disposition', fixed_disp)
-                
-                msg.attach(part)
-
-        return msg

email-worker/health_server.py

@@ -1,85 +0,0 @@
-#!/usr/bin/env python3
-"""
-HTTP health check server
-"""
-
-import sys
-import json
-import threading
-from http.server import HTTPServer, BaseHTTPRequestHandler
-from datetime import datetime
-
-from logger import log
-from config import config
-
-
-class SilentHTTPServer(HTTPServer):
-    """HTTP Server that ignores connection reset errors from scanners"""
-    
-    def handle_error(self, request, client_address):
-        exc_type = sys.exc_info()[0]
-        if exc_type in (ConnectionResetError, BrokenPipeError, ConnectionAbortedError):
-            pass  # Silently ignore - these are just scanners/health checks disconnecting
-        else:
-            log(f"Health server error from {client_address[0]}: {sys.exc_info()[1]}", 'WARNING')
-
-
-class HealthHandler(BaseHTTPRequestHandler):
-    """Health check request handler"""
-    
-    worker = None  # Will be set by start_health_server()
-    dynamodb_available = False
-    
-    def do_GET(self):
-        if self.path == '/health' or self.path == '/':
-            self.send_response(200)
-            self.send_header('Content-Type', 'application/json')
-            self.end_headers()
-            
-            status = {
-                'status': 'healthy',
-                'domains': len(self.worker.queue_urls) if self.worker else 0,
-                'domain_list': list(self.worker.queue_urls.keys()) if self.worker else [],
-                'dynamodb': self.dynamodb_available,
-                'features': {
-                    'bounce_rewriting': True,
-                    'auto_reply': self.dynamodb_available,
-                    'forwarding': self.dynamodb_available,
-                    'blocklist': self.dynamodb_available,
-                    'lmtp': config.lmtp_enabled
-                },
-                'timestamp': datetime.utcnow().isoformat()
-            }
-            self.wfile.write(json.dumps(status, indent=2).encode())
-        
-        elif self.path == '/domains':
-            self.send_response(200)
-            self.send_header('Content-Type', 'application/json')
-            self.end_headers()
-            domain_list = list(self.worker.queue_urls.keys()) if self.worker else []
-            self.wfile.write(json.dumps(domain_list).encode())
-        
-        else:
-            self.send_response(404)
-            self.end_headers()
-    
-    def log_message(self, format, *args):
-        pass  # Suppress HTTP access logs
-
-
-def start_health_server(worker, dynamodb_available: bool):
-    """
-    Start HTTP health check server
-    
-    Args:
-        worker: UnifiedWorker instance
-        dynamodb_available: Whether DynamoDB is available
-    """
-    # Set class attributes for handler
-    HealthHandler.worker = worker
-    HealthHandler.dynamodb_available = dynamodb_available
-    
-    server = SilentHTTPServer(('0.0.0.0', config.health_port), HealthHandler)
-    thread = threading.Thread(target=server.serve_forever, daemon=True, name='health-server')
-    thread.start()
-    log(f"Health server on port {config.health_port}")

email-worker/logger.py

@@ -1,79 +0,0 @@
-#!/usr/bin/env python3
-"""
-Structured logging for email worker with Daily Rotation (Robust Version)
-"""
-
-import os
-import sys
-import logging
-import threading
-from logging.handlers import TimedRotatingFileHandler
-
-# Konfiguration
-LOG_DIR = "/var/log/email-worker"
-LOG_FILE = os.path.join(LOG_DIR, "worker.log")
-
-# Logger initialisieren
-logger = logging.getLogger("unified-worker")
-logger.setLevel(logging.INFO)
-logger.propagate = False
-
-# Formatierung
-formatter = logging.Formatter(
-    '[%(asctime)s] [%(levelname)s] [%(threadName)s] %(message)s',
-    datefmt='%Y-%m-%d %H:%M:%S'
-)
-
-# 1. Console Handler (Immer aktiv!)
-console_handler = logging.StreamHandler(sys.stdout)
-console_handler.setFormatter(formatter)
-logger.addHandler(console_handler)
-
-# 2. File Handler (Robustes Setup)
-try:
-    # Versuchen, das Verzeichnis zu erstellen, falls es fehlt
-    os.makedirs(LOG_DIR, exist_ok=True)
-
-    file_handler = TimedRotatingFileHandler(
-        LOG_FILE, 
-        when="midnight", 
-        interval=1, 
-        backupCount=30, 
-        encoding='utf-8'
-    )
-    file_handler.setFormatter(formatter)
-    file_handler.suffix = "%Y-%m-%d"
-    logger.addHandler(file_handler)
-    
-    # Erfolgsmeldung auf Konsole (damit wir sehen, dass es geklappt hat)
-    print(f"✓ Logging to file enabled: {LOG_FILE}")
-
-except Exception as e:
-    # Fallback: Ausführliche Fehlerdiagnose auf stdout
-    error_msg = f"⚠ LOGGING ERROR: Could not write to {LOG_FILE}\n"
-    error_msg += f"  Error: {e}\n"
-    try:
-        error_msg += f"  Current User (UID): {os.getuid()}\n"
-        error_msg += f"  Current Group (GID): {os.getgid()}\n"
-    except:
-        pass
-    print(error_msg)
-
-def log(message: str, level: str = 'INFO', worker_name: str = 'unified-worker'):
-    """
-    Structured logging function
-    """
-    lvl_map = {
-        'DEBUG': logging.DEBUG,
-        'INFO': logging.INFO,
-        'WARNING': logging.WARNING,
-        'ERROR': logging.ERROR,
-        'CRITICAL': logging.CRITICAL,
-        'SUCCESS': logging.INFO
-    }
-    
-    log_level = lvl_map.get(level.upper(), logging.INFO)
-    prefix = "[SUCCESS] " if level.upper() == 'SUCCESS' else ""
-    final_message = f"[{worker_name}] {prefix}{message}"
-    
-    logger.log(log_level, final_message)

email-worker/main.py

@@ -1,50 +0,0 @@
-#!/usr/bin/env python3
-"""
-Main entry point for unified email worker
-"""
-
-import sys
-import signal
-
-from logger import log
-from config import config
-from unified_worker import UnifiedWorker
-from health_server import start_health_server
-from metrics.prometheus import start_metrics_server
-
-
-def main():
-    """Main entry point"""
-    
-    # Create worker instance
-    worker = UnifiedWorker()
-    
-    # Signal handlers for graceful shutdown
-    def signal_handler(signum, frame):
-        log(f"Received signal {signum}")
-        worker.stop()
-        sys.exit(0)
-    
-    signal.signal(signal.SIGTERM, signal_handler)
-    signal.signal(signal.SIGINT, signal_handler)  # Fixed: was signalIGINT in old version
-    
-    # Setup worker
-    worker.setup()
-    
-    # Start metrics server (if available)
-    metrics = start_metrics_server(config.metrics_port)
-    if metrics:
-        worker.set_metrics(metrics)
-    
-    # Start health check server
-    start_health_server(worker, worker.dynamodb.available)
-    
-    # Print startup banner
-    worker.print_startup_banner()
-    
-    # Start worker
-    worker.start()
-
-
-if __name__ == '__main__':
-    main()

email-worker/metrics/__init__.py

@@ -1,8 +0,0 @@
-#!/usr/bin/env python3
-"""
-Metrics collection
-"""
-
-from .prometheus import MetricsCollector, start_metrics_server
-
-__all__ = ['MetricsCollector', 'start_metrics_server']

email-worker/metrics/prometheus.py

@@ -1,142 +0,0 @@
-#!/usr/bin/env python3
-"""
-Prometheus metrics collection
-"""
-
-from typing import Optional
-
-from logger import log
-
-# Try to import Prometheus client
-try:
-    from prometheus_client import start_http_server, Counter, Gauge, Histogram
-    PROMETHEUS_ENABLED = True
-except ImportError:
-    PROMETHEUS_ENABLED = False
-
-
-class MetricsCollector:
-    """Collects and exposes Prometheus metrics"""
-    
-    def __init__(self):
-        self.enabled = PROMETHEUS_ENABLED
-        
-        if self.enabled:
-            # Email processing metrics
-            self.emails_processed = Counter(
-                'emails_processed_total',
-                'Total emails processed',
-                ['domain', 'status']
-            )
-            
-            self.emails_in_flight = Gauge(
-                'emails_in_flight',
-                'Emails currently being processed'
-            )
-            
-            self.processing_time = Histogram(
-                'email_processing_seconds',
-                'Time to process email',
-                ['domain']
-            )
-            
-            self.queue_size = Gauge(
-                'queue_messages_available',
-                'Messages in queue',
-                ['domain']
-            )
-            
-            # Bounce metrics
-            self.bounces_processed = Counter(
-                'bounces_processed_total',
-                'Bounce notifications processed',
-                ['domain', 'type']
-            )
-            
-            # Rules metrics
-            self.autoreplies_sent = Counter(
-                'autoreplies_sent_total',
-                'Auto-replies sent',
-                ['domain']
-            )
-            
-            self.forwards_sent = Counter(
-                'forwards_sent_total',
-                'Forwards sent',
-                ['domain']
-            )
-            
-            # Blocklist metrics
-            self.blocked_senders = Counter(
-                'blocked_senders_total',
-                'Emails blocked by blacklist',
-                ['domain']
-            )
-    
-    def increment_processed(self, domain: str, status: str):
-        """Increment processed email counter"""
-        if self.enabled:
-            self.emails_processed.labels(domain=domain, status=status).inc()
-    
-    def increment_in_flight(self):
-        """Increment in-flight email gauge"""
-        if self.enabled:
-            self.emails_in_flight.inc()
-    
-    def decrement_in_flight(self):
-        """Decrement in-flight email gauge"""
-        if self.enabled:
-            self.emails_in_flight.dec()
-    
-    def observe_processing_time(self, domain: str, seconds: float):
-        """Record processing time"""
-        if self.enabled:
-            self.processing_time.labels(domain=domain).observe(seconds)
-    
-    def set_queue_size(self, domain: str, size: int):
-        """Set queue size"""
-        if self.enabled:
-            self.queue_size.labels(domain=domain).set(size)
-    
-    def increment_bounce(self, domain: str, bounce_type: str):
-        """Increment bounce counter"""
-        if self.enabled:
-            self.bounces_processed.labels(domain=domain, type=bounce_type).inc()
-    
-    def increment_autoreply(self, domain: str):
-        """Increment autoreply counter"""
-        if self.enabled:
-            self.autoreplies_sent.labels(domain=domain).inc()
-    
-    def increment_forward(self, domain: str):
-        """Increment forward counter"""
-        if self.enabled:
-            self.forwards_sent.labels(domain=domain).inc()
-    
-    def increment_blocked(self, domain: str):
-        """Increment blocked sender counter"""
-        if self.enabled:
-            self.blocked_senders.labels(domain=domain).inc()
-
-
-def start_metrics_server(port: int) -> Optional[MetricsCollector]:
-    """
-    Start Prometheus metrics HTTP server
-    
-    Args:
-        port: Port to listen on
-        
-    Returns:
-        MetricsCollector instance or None if Prometheus not available
-    """
-    if not PROMETHEUS_ENABLED:
-        log("⚠ Prometheus client not installed, metrics disabled", 'WARNING')
-        return None
-    
-    try:
-        start_http_server(port)
-        log(f"Prometheus metrics on port {port}")
-        return MetricsCollector()
-    except Exception as e:
-        log(f"Failed to start metrics server: {e}", 'ERROR')
-        return None

email-worker/requirements.txt

@@ -1,2 +0,0 @@
-boto3>=1.34.0
-prometheus-client>=0.19.0

email-worker/smtp/__init__.py

@@ -1,8 +0,0 @@
-#!/usr/bin/env python3
-"""
-SMTP connection handling
-"""
-
-from .pool import SMTPPool
-
-__all__ = ['SMTPPool']

email-worker/smtp/delivery.py

@@ -1,187 +0,0 @@
-#!/usr/bin/env python3
-"""
-SMTP/LMTP email delivery with retry logic
-"""
-
-import time
-import smtplib
-from typing import Tuple, Optional
-
-from logger import log
-from config import config
-from smtp.pool import SMTPPool
-
-
-class EmailDelivery:
-    """Handles email delivery via SMTP or LMTP"""
-    
-    def __init__(self, smtp_pool: SMTPPool):
-        self.smtp_pool = smtp_pool
-    
-    @staticmethod
-    def is_permanent_recipient_error(error_msg: str) -> bool:
-        """Check if error is permanent for this recipient (inbox doesn't exist)"""
-        permanent_indicators = [
-            '550',  # Mailbox unavailable / not found
-            '551',  # User not local
-            '553',  # Mailbox name not allowed / invalid
-            'mailbox not found',
-            'user unknown',
-            'no such user',
-            'recipient rejected',
-            'does not exist',
-            'invalid recipient',
-            'unknown user'
-        ]
-        
-        error_lower = error_msg.lower()
-        return any(indicator in error_lower for indicator in permanent_indicators)
-    
-    def send_to_recipient(
-        self, 
-        from_addr: str, 
-        recipient: str, 
-        raw_message: bytes, 
-        worker_name: str, 
-        max_retries: int = 2
-    ) -> Tuple[bool, Optional[str], bool]:
-        """
-        Send email via SMTP/LMTP to ONE recipient
-        
-        If LMTP is enabled, delivers directly to Dovecot (bypasses transport_maps).
-        With retry logic for connection errors.
-        
-        Args:
-            from_addr: From address
-            recipient: Recipient address
-            raw_message: Raw MIME message bytes
-            worker_name: Worker name for logging
-            max_retries: Maximum retry attempts
-            
-        Returns:
-            Tuple of (success: bool, error: str or None, is_permanent: bool)
-        """
-        last_error = None
-        use_lmtp = config.lmtp_enabled
-        
-        for attempt in range(max_retries + 1):
-            conn = None
-            
-            try:
-                if use_lmtp:
-                    # LMTP connection directly to Dovecot (bypasses Postfix/transport_maps)
-                    conn = smtplib.LMTP(config.lmtp_host, config.lmtp_port, timeout=30)
-                    conn.ehlo()
-                else:
-                    # Normal SMTP connection from pool
-                    conn = self.smtp_pool.get_connection()
-                    if not conn:
-                        last_error = "Could not get SMTP connection"
-                        log(
-                            f"  ⚠ {recipient}: No SMTP connection "
-                            f"(attempt {attempt + 1}/{max_retries + 1})",
-                            'WARNING',
-                            worker_name
-                        )
-                        time.sleep(0.5)
-                        continue
-                
-                result = conn.sendmail(from_addr, [recipient], raw_message)
-                
-                # Success
-                if use_lmtp:
-                    conn.quit()
-                else:
-                    self.smtp_pool.return_connection(conn)
-                
-                if isinstance(result, dict) and result:
-                    error = str(result.get(recipient, 'Unknown refusal'))
-                    is_permanent = self.is_permanent_recipient_error(error)
-                    log(
-                        f"  ✗ {recipient}: {error} ({'permanent' if is_permanent else 'temporary'})",
-                        'ERROR',
-                        worker_name
-                    )
-                    return False, error, is_permanent
-                else:
-                    delivery_method = "LMTP" if use_lmtp else "SMTP"
-                    log(f"  ✓ {recipient}: Delivered ({delivery_method})", 'SUCCESS', worker_name)
-                    return True, None, False
-            
-            except smtplib.SMTPServerDisconnected as e:
-                # Connection was closed - Retry with new connection
-                log(
-                    f"  ⚠ {recipient}: Connection lost, retrying... "
-                    f"(attempt {attempt + 1}/{max_retries + 1})",
-                    'WARNING',
-                    worker_name
-                )
-                last_error = str(e)
-                if conn:
-                    try:
-                        conn.quit()
-                    except:
-                        pass
-                time.sleep(0.3)
-                continue
-            
-            except smtplib.SMTPRecipientsRefused as e:
-                if conn and not use_lmtp:
-                    self.smtp_pool.return_connection(conn)
-                elif conn:
-                    try:
-                        conn.quit()
-                    except:
-                        pass
-                error_msg = str(e)
-                is_permanent = self.is_permanent_recipient_error(error_msg)
-                log(f"  ✗ {recipient}: Recipients refused - {error_msg}", 'ERROR', worker_name)
-                return False, error_msg, is_permanent
-            
-            except smtplib.SMTPException as e:
-                error_msg = str(e)
-                # On connection errors: Retry
-                if 'disconnect' in error_msg.lower() or 'closed' in error_msg.lower() or 'connection' in error_msg.lower():
-                    log(
-                        f"  ⚠ {recipient}: Connection error, retrying... "
-                        f"(attempt {attempt + 1}/{max_retries + 1})",
-                        'WARNING',
-                        worker_name
-                    )
-                    last_error = error_msg
-                    if conn:
-                        try:
-                            conn.quit()
-                        except:
-                            pass
-                    time.sleep(0.3)
-                    continue
-                
-                if conn and not use_lmtp:
-                    self.smtp_pool.return_connection(conn)
-                elif conn:
-                    try:
-                        conn.quit()
-                    except:
-                        pass
-                is_permanent = self.is_permanent_recipient_error(error_msg)
-                log(f"  ✗ {recipient}: Error - {error_msg}", 'ERROR', worker_name)
-                return False, error_msg, is_permanent
-            
-            except Exception as e:
-                # Unknown error
-                if conn:
-                    try:
-                        conn.quit()
-                    except:
-                        pass
-                log(f"  ✗ {recipient}: Unexpected error - {e}", 'ERROR', worker_name)
-                return False, str(e), False
-        
-        # All retries failed
-        log(
-            f"  ✗ {recipient}: All retries failed - {last_error}",
-            'ERROR',
-            worker_name
-        )
-        return False, last_error or "Connection failed after retries", False

email-worker/smtp/pool.py

@@ -1,113 +0,0 @@
-#!/usr/bin/env python3
-"""
-SMTP Connection Pool with robust connection handling
-"""
-
-import smtplib
-from queue import Queue, Empty
-from typing import Optional
-
-from logger import log
-from config import config
-
-
-class SMTPPool:
-    """Thread-safe SMTP Connection Pool"""
-    
-    def __init__(self, host: str, port: int, pool_size: int = 5):
-        self.host = host
-        self.port = port
-        self.pool_size = pool_size
-        self._pool: Queue = Queue(maxsize=pool_size)
-        self._initialized = False
-    
-    def _create_connection(self) -> Optional[smtplib.SMTP]:
-        """Create new SMTP connection"""
-        try:
-            conn = smtplib.SMTP(self.host, self.port, timeout=30)
-            conn.ehlo()
-            if config.smtp_use_tls:
-                conn.starttls()
-                conn.ehlo()
-            if config.smtp_user and config.smtp_pass:
-                conn.login(config.smtp_user, config.smtp_pass)
-            log(f"  📡 New SMTP connection created to {self.host}:{self.port}")
-            return conn
-        except Exception as e:
-            log(f"Failed to create SMTP connection: {e}", 'ERROR')
-            return None
-    
-    def _test_connection(self, conn: smtplib.SMTP) -> bool:
-        """Test if connection is still alive"""
-        try:
-            status = conn.noop()[0]
-            return status == 250
-        except Exception:
-            return False
-    
-    def initialize(self):
-        """Pre-create connections"""
-        if self._initialized:
-            return
-        
-        # Only 1-2 connections initially, rest on-demand
-        for _ in range(min(2, self.pool_size)):
-            conn = self._create_connection()
-            if conn:
-                self._pool.put(conn)
-        
-        self._initialized = True
-        log(f"SMTP pool initialized with {self._pool.qsize()} connections (max: {self.pool_size})")
-    
-    def get_connection(self, timeout: float = 5.0) -> Optional[smtplib.SMTP]:
-        """Get a valid connection from pool or create new one"""
-        # Try to get from pool
-        try:
-            conn = self._pool.get(block=False)
-            # Test if connection is still alive
-            if self._test_connection(conn):
-                return conn
-            else:
-                # Connection is dead, close and create new one
-                try:
-                    conn.quit()
-                except:
-                    pass
-                log(f"  ♻ Recycled stale SMTP connection")
-                return self._create_connection()
-        except Empty:
-            # Pool empty, create new connection
-            return self._create_connection()
-    
-    def return_connection(self, conn: smtplib.SMTP):
-        """Return connection to pool if still valid"""
-        if conn is None:
-            return
-        
-        # Check if connection is still good
-        if not self._test_connection(conn):
-            try:
-                conn.quit()
-            except:
-                pass
-            log(f"  🗑 Discarded broken SMTP connection")
-            return
-        
-        # Try to return to pool
-        try:
-            self._pool.put_nowait(conn)
-        except:
-            # Pool full, close connection
-            try:
-                conn.quit()
-            except:
-                pass
-    
-    def close_all(self):
-        """Close all connections"""
-        while not self._pool.empty():
-            try:
-                conn = self._pool.get_nowait()
-                conn.quit()
-            except:
-                pass

email-worker/unified_worker.py

@@ -1,201 +0,0 @@
-#!/usr/bin/env python3
-"""
-Unified Worker - coordinates all domain pollers
-"""
-
-import sys
-import time
-import threading
-from typing import List, Dict
-
-from logger import log
-from config import config, load_domains
-from aws import S3Handler, SQSHandler, SESHandler, DynamoDBHandler
-from smtp import SMTPPool
-from smtp.delivery import EmailDelivery
-from worker import MessageProcessor
-from domain_poller import DomainPoller
-from metrics.prometheus import MetricsCollector
-
-
-class UnifiedWorker:
-    """Main worker coordinating all domain pollers"""
-    
-    def __init__(self):
-        self.stop_event = threading.Event()
-        self.domains: List[str] = []
-        self.queue_urls: Dict[str, str] = {}
-        self.poller_threads: List[threading.Thread] = []
-        
-        # Shared stats across all pollers
-        self.domain_stats: Dict[str, int] = {}  # domain -> processed count
-        self.stats_lock = threading.Lock()
-        
-        # AWS handlers
-        self.s3 = S3Handler()
-        self.sqs = SQSHandler()
-        self.ses = SESHandler()
-        self.dynamodb = DynamoDBHandler()
-        
-        # SMTP pool
-        self.smtp_pool = SMTPPool(config.smtp_host, config.smtp_port, config.smtp_pool_size)
-        
-        # Email delivery
-        self.delivery = EmailDelivery(self.smtp_pool)
-        
-        # Metrics
-        self.metrics: MetricsCollector = None
-        
-        # Message processor
-        self.processor = MessageProcessor(
-            self.s3,
-            self.sqs,
-            self.ses,
-            self.dynamodb,
-            self.delivery,
-            None  # Metrics will be set later
-        )
-    
-    def setup(self):
-        """Initialize worker"""
-        self.domains = load_domains()
-        
-        if not self.domains:
-            log("❌ No domains configured!", 'ERROR')
-            sys.exit(1)
-        
-        # Get queue URLs
-        for domain in self.domains:
-            url = self.sqs.get_queue_url(domain)
-            if url:
-                self.queue_urls[domain] = url
-                log(f"  ✓ {domain} -> queue found")
-            else:
-                log(f"  ✗ {domain} -> Queue not found!", 'WARNING')
-        
-        if not self.queue_urls:
-            log("❌ No valid queues found!", 'ERROR')
-            sys.exit(1)
-        
-        # Initialize SMTP pool
-        self.smtp_pool.initialize()
-        
-        log(f"Initialized with {len(self.queue_urls)} domains")
-    
-    def start(self):
-        """Start all domain pollers"""
-        # Initialize stats for all domains
-        for domain in self.queue_urls.keys():
-            self.domain_stats[domain] = 0
-        
-        # Create poller for each domain
-        for domain, queue_url in self.queue_urls.items():
-            poller = DomainPoller(
-                domain=domain,
-                queue_url=queue_url,
-                message_processor=self.processor,
-                sqs=self.sqs,
-                metrics=self.metrics,
-                stop_event=self.stop_event,
-                stats_dict=self.domain_stats,
-                stats_lock=self.stats_lock
-            )
-            
-            thread = threading.Thread(
-                target=poller.poll,
-                name=f"poller-{domain}",
-                daemon=True
-            )
-            thread.start()
-            self.poller_threads.append(thread)
-        
-        log(f"Started {len(self.poller_threads)} domain pollers")
-        
-        # Periodic status log (every 5 minutes)
-        last_status_log = time.time()
-        status_interval = 300  # 5 minutes
-        
-        try:
-            while not self.stop_event.is_set():
-                self.stop_event.wait(timeout=10)
-                
-                # Log status summary every 5 minutes
-                if time.time() - last_status_log > status_interval:
-                    self._log_status_table()
-                    last_status_log = time.time()
-        except KeyboardInterrupt:
-            pass
-    
-    def _log_status_table(self):
-        """Log a compact status table"""
-        active_threads = sum(1 for t in self.poller_threads if t.is_alive())
-        
-        with self.stats_lock:
-            total_processed = sum(self.domain_stats.values())
-            
-            # Build compact stats: only show domains with activity or top domains
-            stats_parts = []
-            for domain in sorted(self.queue_urls.keys()):
-                count = self.domain_stats.get(domain, 0)
-                if count > 0:  # Only show active domains
-                    # Shorten domain for display
-                    short_domain = domain.split('.')[0][:12]
-                    stats_parts.append(f"{short_domain}:{count}")
-        
-        if stats_parts:
-            stats_line = " | ".join(stats_parts)
-        else:
-            stats_line = "no activity"
-        
-        log(
-            f"📊 Status: {active_threads}/{len(self.poller_threads)} active, "
-            f"total:{total_processed} | {stats_line}"
-        )
-    
-    def stop(self):
-        """Stop gracefully"""
-        log("⚠ Stopping worker...")
-        self.stop_event.set()
-        
-        # Wait for poller threads (max 10 seconds each)
-        for thread in self.poller_threads:
-            thread.join(timeout=10)
-            if thread.is_alive():
-                log(f"Warning: {thread.name} did not stop gracefully", 'WARNING')
-        
-        self.smtp_pool.close_all()
-        log("👋 Worker stopped")
-    
-    def set_metrics(self, metrics: MetricsCollector):
-        """Set metrics collector"""
-        self.metrics = metrics
-        self.processor.metrics = metrics
-    
-    def print_startup_banner(self):
-        """Print startup information"""
-        log(f"\n{'='*70}")
-        log(f"🚀 UNIFIED EMAIL WORKER")
-        log(f"{'='*70}")
-        log(f"  Domains:       {len(self.queue_urls)}")
-        log(f"  DynamoDB:      {'Connected' if self.dynamodb.available else 'Not Available'}")
-        
-        if config.lmtp_enabled:
-            log(f"  Delivery:      LMTP -> {config.lmtp_host}:{config.lmtp_port} (bypasses transport_maps)")
-        else:
-            log(f"  Delivery:      SMTP -> {config.smtp_host}:{config.smtp_port}")
-        
-        log(f"  Poll Interval: {config.poll_interval}s")
-        log(f"  Visibility:    {config.visibility_timeout}s")
-        log(f"")
-        log(f"  Features:")
-        log(f"    ✓ Bounce Detection & Header Rewriting")
-        log(f"    {'✓' if self.dynamodb.available else '✗'} Auto-Reply / Out-of-Office")
-        log(f"    {'✓' if self.dynamodb.available else '✗'} Email Forwarding")
-        log(f"    {'✓' if self.dynamodb.available else '✗'} Blocked Senders (Wildcard)")
-        log(f"    {'✓' if self.metrics else '✗'} Prometheus Metrics")
-        log(f"    {'✓' if config.lmtp_enabled else '✗'} LMTP Direct Delivery")
-        log(f"")
-        log(f"  Active Domains:")
-        for domain in sorted(self.queue_urls.keys()):
-            log(f"    • {domain}")
-        log(f"{'='*70}\n")

email-worker/worker.py

@@ -1,352 +0,0 @@
-#!/usr/bin/env python3
-"""
-Email message processing worker
-"""
-
-import json
-import traceback
-
-from logger import log
-from aws import S3Handler, SQSHandler, SESHandler, DynamoDBHandler
-from email_processing import EmailParser, BounceHandler, RulesProcessor, BlocklistChecker
-from smtp.delivery import EmailDelivery
-from metrics.prometheus import MetricsCollector
-from email.parser import BytesParser      # War wahrscheinlich schon da, prüfen
-from email.policy import compat32         # <--- NEU: Hinzufügen
-
-
-class MessageProcessor:
-    """Processes individual email messages"""
-    
-    def __init__(
-        self,
-        s3: S3Handler,
-        sqs: SQSHandler,
-        ses: SESHandler,
-        dynamodb: DynamoDBHandler,
-        delivery: EmailDelivery,
-        metrics: MetricsCollector
-    ):
-        self.s3 = s3
-        self.sqs = sqs
-        self.ses = ses
-        self.dynamodb = dynamodb
-        self.delivery = delivery
-        self.metrics = metrics
-        
-        # Initialize sub-processors
-        self.parser = EmailParser()
-        self.bounce_handler = BounceHandler(dynamodb)
-        self.rules_processor = RulesProcessor(dynamodb, ses)
-        self.blocklist = BlocklistChecker(dynamodb)
-    
-    def process_message(self, domain: str, message: dict, receive_count: int) -> bool:
-        """
-        Process one email message from queue
-        
-        Args:
-            domain: Email domain
-            message: SQS message dict
-            receive_count: Number of times received
-            
-        Returns:
-            True to delete from queue, False to retry
-        """
-        worker_name = f"worker-{domain}"
-        
-        try:
-            # 1. UNPACKING (SNS -> SES)
-            message_body = json.loads(message['Body'])
-            
-            if 'Message' in message_body and 'Type' in message_body:
-                # It's an SNS Notification
-                sns_content = message_body['Message']
-                if isinstance(sns_content, str):
-                    ses_msg = json.loads(sns_content)
-                else:
-                    ses_msg = sns_content
-            else:
-                ses_msg = message_body
-
-            # 2. EXTRACT DATA
-            mail = ses_msg.get('mail', {})
-            receipt = ses_msg.get('receipt', {})
-            
-            message_id = mail.get('messageId')
-            
-            # FIX: Ignore Amazon SES Setup Notification
-            if message_id == "AMAZON_SES_SETUP_NOTIFICATION":
-                log("ℹ️ Received Amazon SES Setup Notification. Ignoring.", 'INFO', worker_name)
-                return True
-            
-            from_addr = mail.get('source')
-            recipients = receipt.get('recipients', [])
-            
-            if not message_id:
-                log("❌ Error: No messageId in event payload", 'ERROR', worker_name)
-                return True
-
-            # Domain Validation
-            if recipients:
-                first_recipient = recipients[0]
-                recipient_domain = first_recipient.split('@')[1]
-                
-                if recipient_domain.lower() != domain.lower():
-                    log(
-                        f"⚠ Security: Ignored message for {recipient_domain} "
-                        f"(I am worker for {domain})",
-                        'WARNING',
-                        worker_name
-                    )
-                    return True
-            else:
-                log("⚠ Warning: No recipients in event", 'WARNING', worker_name)
-                return True
-
-            key = message_id
-
-            # Compact single-line log for email processing
-            recipients_str = recipients[0] if len(recipients) == 1 else f"{len(recipients)} recipients"
-            log(f"📧 Processing: {key[:20]}... -> {recipients_str}", 'INFO', worker_name)
-            
-            # 3. DOWNLOAD FROM S3
-            raw_bytes = self.s3.get_email(domain, message_id, receive_count)
-            if raw_bytes is None:
-                # S3 object not found yet, retry
-                return False
-            
-            # 4. LOOP DETECTION
-            temp_parsed = self.parser.parse_bytes(raw_bytes)
-            skip_rules = self.parser.is_processed_by_worker(temp_parsed)
-            
-            if skip_rules:
-                log("🔄 Loop prevention: Already processed by worker", 'INFO', worker_name)
-
-            # 5. PARSING & BOUNCE LOGIC
-            try:
-                # --- FIX 2.0: Pre-Sanitize via Legacy Mode ---
-                # Der strikte Parser crasht SOFORT beim Zugriff auf kaputte Header.
-                # Wir müssen erst "nachsichtig" parsen, reparieren und Bytes neu generieren.
-                try:
-                    # 1. Parsen im Compat32-Modus (ignoriert Syntaxfehler)
-                    lenient_parser = BytesParser(policy=compat32)
-                    temp_msg = lenient_parser.parsebytes(raw_bytes)
-                    
-                    # 2. Prüfen und Reparieren
-                    bad_msg_id = temp_msg.get('Message-ID', '')
-                    if bad_msg_id and ('[' in bad_msg_id or ']' in bad_msg_id):
-                        clean_id = bad_msg_id.replace('[', '').replace(']', '')
-                        temp_msg.replace_header('Message-ID', clean_id)
-                        
-                        # 3. Bytes mit repariertem Header neu schreiben
-                        raw_bytes = temp_msg.as_bytes()
-                        log(f"  🔧 Sanitized malformed Message-ID via Legacy Mode: {clean_id}", 'INFO', worker_name)
-                        
-                        if self.metrics:
-                            self.metrics.increment_bounce(domain, 'sanitized_header')
-
-                except Exception as e_sanitize:
-                    # Sollte nicht passieren, aber wir wollen hier nicht abbrechen
-                    log(f"  ⚠ Sanitization warning: {e_sanitize}", 'WARNING', worker_name)
-                # ---------------------------------------------
-
-
-                parsed = self.parser.parse_bytes(raw_bytes)
-                
-                # --- FIX START: Sanitize Malformed Headers ---
-                # Fix für Microsofts <[uuid]@domain> Message-IDs, die Python crashen lassen
-                current_msg_id = parsed.get('Message-ID', '')
-                if current_msg_id and ('[' in current_msg_id or ']' in current_msg_id):
-                    # Klammern entfernen, aber spitze Klammern behalten
-                    clean_id = current_msg_id.replace('[', '').replace(']', '')
-                    parsed.replace_header('Message-ID', clean_id)
-                    log("  🔧 Sanitized malformed Message-ID", 'INFO', worker_name)
-                # --- FIX END ---
-
-                subject = parsed.get('Subject', '(no subject)')
-                
-                # Bounce header rewriting
-                is_bounce = self.bounce_handler.is_ses_bounce_notification(parsed)
-                parsed, modified = self.bounce_handler.apply_bounce_logic(parsed, subject, worker_name)
-                
-                if modified:
-                    log("  ✨ Bounce detected & headers rewritten via DynamoDB", 'INFO', worker_name)
-                    raw_bytes = parsed.as_bytes()
-                    from_addr_final = parsed.get('From')
-                    
-                    if self.metrics:
-                        self.metrics.increment_bounce(domain, 'rewritten')
-                else:
-                    from_addr_final = from_addr
-                
-                # Marker für alle Emails von extern setzen
-                if not skip_rules:  # Nur wenn nicht bereits processed
-                    parsed['X-SES-Worker-Processed'] = 'delivered'
-                    raw_bytes = parsed.as_bytes() # <--- Hier knallte es vorher
-
-            except Exception as e:
-                # --- VERBESSERTES ERROR LOGGING ---
-                error_msg = f"⚠ Parsing/Logic Error: {e}. Sending original."
-                log(error_msg, 'WARNING', worker_name)
-                
-                # Den vollen Traceback ins Log schreiben (als ERROR markiert)
-                tb_str = traceback.format_exc()
-                log(f"Full Traceback:\n{tb_str}", 'ERROR', worker_name)
-                # ----------------------------------
-                
-                # Fallback: Wir versuchen trotzdem, die Original-Mail zuzustellen
-                from_addr_final = from_addr
-                is_bounce = False
-                skip_rules = False
-
-            # 6. BLOCKLIST CHECK (Batch for efficiency)
-            senders_to_check = []
-            
-            # 1. Die Envelope-Adresse (aus dem SES Event / Return-Path)
-            if from_addr:
-                senders_to_check.append(from_addr)
-                
-            # 2. Die echte Header-Adresse (aus der MIME-E-Mail geparst)
-            header_from = parsed.get('From')
-            if header_from and header_from not in senders_to_check:
-                senders_to_check.append(header_from)
-                
-            # 3. Falls die Bounce-Logik die Adresse umgeschrieben hat
-            if from_addr_final and from_addr_final not in senders_to_check:
-                senders_to_check.append(from_addr_final)
-
-            # Prüfe nun alle extrahierten Adressen gegen die Datenbank
-            blocked_by_recipient = self.blocklist.batch_check_blocked_senders(
-                recipients,
-                senders_to_check, # <-- Übergabe der Liste
-                worker_name
-            )
-
-            # 7. PROCESS RECIPIENTS
-            log(f"📤 Sending to {len(recipients)} recipient(s)...", 'INFO', worker_name)
-            
-            successful = []
-            failed_permanent = []
-            failed_temporary = []
-            blocked_recipients = []
-
-            for recipient in recipients:
-                # Check if blocked
-                if blocked_by_recipient.get(recipient, False):
-                    log(
-                        f"🗑 Silently dropping message for {recipient} (Sender blocked)",
-                        'INFO',
-                        worker_name
-                    )
-                    blocked_recipients.append(recipient)
-                    if self.metrics:
-                        self.metrics.increment_blocked(domain)
-                    continue
-
-                # Process rules (OOO, Forwarding) - not for bounces or already forwarded
-                skip_local_delivery = False  # NEU
-                if not is_bounce and not skip_rules:
-                    def metrics_callback(action_type: str, dom: str):
-                        """Callback for metrics from rules processor"""
-                        if self.metrics:
-                            if action_type == 'autoreply':
-                                self.metrics.increment_autoreply(dom)
-                            elif action_type == 'forward':
-                                self.metrics.increment_forward(dom)
-                    
-                    skip_local_delivery = self.rules_processor.process_rules_for_recipient(
-                        recipient,
-                        parsed,
-                        domain,
-                        worker_name,
-                        metrics_callback
-                    )
-
-                # SMTP Delivery
-                if skip_local_delivery:  # NEU
-                    log(f"  ⏭ Skipping local delivery for {recipient} (legacy forward active)",
-                        'INFO', worker_name)
-                    successful.append(recipient)  # Zählt als "handled"
-                else:
-                    success, error, is_perm = self.delivery.send_to_recipient(
-                        from_addr_final, recipient, raw_bytes, worker_name
-                    )
-                
-                    if success:
-                        successful.append(recipient)
-                        if self.metrics:
-                            self.metrics.increment_processed(domain, 'success')
-                    elif is_perm:
-                        failed_permanent.append(recipient)
-                        if self.metrics:
-                            self.metrics.increment_processed(domain, 'permanent_failure')
-                    else:
-                        failed_temporary.append(recipient)
-                        if self.metrics:
-                            self.metrics.increment_processed(domain, 'temporary_failure')
-
-            # 8. RESULT & CLEANUP
-            total_handled = len(successful) + len(failed_permanent) + len(blocked_recipients)
-
-            if total_handled == len(recipients):
-                # All recipients handled (success, permanent fail, or blocked)
-                
-                if len(blocked_recipients) == len(recipients):
-                    # All recipients blocked - mark and delete S3 object
-                    try:
-                        self.s3.mark_as_blocked(
-                            domain,
-                            message_id,
-                            blocked_recipients,
-                            from_addr_final,
-                            worker_name
-                        )
-                        self.s3.delete_blocked_email(domain, message_id, worker_name)
-                    except Exception as e:
-                        log(f"⚠ Failed to handle blocked email: {e}", 'ERROR', worker_name)
-                        # Don't delete from queue if S3 operations failed
-                        return False
-                
-                elif len(successful) > 0:
-                    # At least one success
-                    self.s3.mark_as_processed(
-                        domain,
-                        message_id,
-                        worker_name,
-                        failed_permanent if failed_permanent else None
-                    )
-                
-                elif len(failed_permanent) > 0:
-                    # All failed permanently
-                    self.s3.mark_as_all_invalid(
-                        domain,
-                        message_id,
-                        failed_permanent,
-                        worker_name
-                    )
-
-                # Build result summary
-                result_parts = []
-                if successful:
-                    result_parts.append(f"{len(successful)} OK")
-                if failed_permanent:
-                    result_parts.append(f"{len(failed_permanent)} invalid")
-                if blocked_recipients:
-                    result_parts.append(f"{len(blocked_recipients)} blocked")
-
-                log(f"✅ Completed ({', '.join(result_parts)})", 'SUCCESS', worker_name)
-                return True
-            
-            else:
-                # Some recipients had temporary failures
-                log(
-                    f"🔄 Temp failure ({len(failed_temporary)} failed), will retry",
-                    'WARNING',
-                    worker_name
-                )
-                return False
-
-        except Exception as e:
-            log(f"❌ CRITICAL WORKER ERROR: {e}", 'ERROR', worker_name)
-            traceback.print_exc()
-            return False