Dmytro Alieksieiev via Postfix-users:
> Hi Wietse, Viktor,
>
> I ran into a funky interaction between|recipient_canonical|header
> rewriting and a socketmap map (PostSRSd, doing reverse-SRS), where a
> single malformed/header/address can deterministically cancel an
> otherwise-deliverable message. I think I've narrowed the root cause down
> to something that postfix itself has no way to guard against, and I'd
> love your read on it and maybe propose a way to solve this issue.
>
>
> Setup
>
> An incoming relay accepts mail for forwarding and hands it to an
> outgoing relay. The outgoing relay does reverse-SRS via a socketmap and
> also canonicalizes header recipients (so the|To:|in a DSN matches the
> real MAIL TO, per RFC 3464):
>
> |recipient_canonical_maps = socketmap:inet:127.0.0.1:10003:reverse
> recipient_canonical_classes = envelope_recipient, header_recipient
> remote_header_rewrite_domain = domain.invalid # remote clients ->
> rewrite context = remote |
>
> The socketmap server is PostSRSd 2.x, but ? importantly ? it is*not*the
> culprit; any socketmap with a bounded request buffer behaves the same
> (see below).
Would a header_checks pattern work around with this, like replacing
long text with a shorter text?
I agree that a PERM error should not result in a 451 status.
Wietse
> Trigger
>
> A forwarded spam message (envelope recipient perfectly valid and
> deliverable) carries a*cosmetically mangled|To:|or|Cc:|header*. The
> interesting part is not the two-bare-|@|addr-spec ? that turns out to be
> harmless ? but an*unbalanced double-quote combined with a long folded
> continuation*:
>
> |Cc: [[email protected]]@host.example.org"
> <[email protected]@evil.example.net>
> AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
> AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
> ... (~2 KB of folded junk, no closing quote) ...
> AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA] |
>
> The|"|after|host.example.org|is never closed. So|tok822|treats
> everything that follows ? including the entire folded blob ? as*one
> quoted-string localpart*, and in the remote rewrite context
> appends|@domain.invalid|, producing a single ~2 KB address:
>
> |"<~2000 chars of junk>"@domain.invalid |
>
>
> What happens
>
> |cleanup(8)|issues that ~2 KB string as the canonical-map (socketmap)
> query. It exceeds the socketmap server's request buffer
> (PostSRSd's|PAYLOAD_SIZE|is 512 bytes), so the server returns a
> permanent error:
>
> |warning: socketmap:inet:127.0.0.1:10003:reverse socketmap server
> permanent error: Too big. warning: reverse lookup error for ""
> <[email protected]@evil.example.net> ...[AAAA...]"@domain.invalid"
> warning: ABCDEF: recipient_canonical_maps map lookup problem for ... --
> message not accepted, try again later ABCDEF: removed (canceled) |
>
> and the upstream sender gets:
>
> |451 4.3.0 Error: queue file write error (in reply to end of DATA command) |
>
> Because it's a 451, the upstream relay retries the message for its whole
> queue lifetime ? so one attacker-controlled cosmetic header turns into a
> message that can never drain and slowly fills a relay with deferred
> retries. (A different socketmap/version returns|PERM Invalid
> query.|instead of|Too big.|for the same input; same class, same outcome.)
>
>
> Root cause, and why I can't defend against it inside postfix
>
> The address/content/is fine ? the same recipient handed to the map as a
> normal-length key returns "not found" and delivers cleanly. What breaks
> is purely that*cleanup builds a canonical-map query that is too large to
> be represented*, and then treats the resulting|PERM|as fatal. Two
> separate things line up:
>
> 1. An*unbalanced quote*lets a folded header expand into one enormous
> address token. Postfix is arguably being lenient/correct here, but
> the result is a multi-KB "address".
> 2. *A failed lookup during/optional header/canonicalization is fatal to
> the whole message.*The junk address erroring is fine ?
> it/should/fail. What's disproportionate is the blast
> radius:|header_recipient|rewriting is a cosmetic, best-effort nicety
> (align a DSN|To:|for RFC 3464), yet its failure cancels a message
> whose envelope recipient is perfectly valid and deliverable. A
> cosmetic step failing shouldn't take down the delivery.
> 3. *A/permanent/map error (|PERM|) is reported to the client as
> a/transient/|451|.*This is what turns a one-off into a queue-wedge:
> the socketmap said/permanent/, but the upstream sees|451 queue file
> write error|, so it retries the identical, deterministically-failing
> message for the entire queue lifetime. Had the same condition
> surfaced as a|5xx|, the message would simply bounce and the queue
> would drain ? annoying, but not self-perpetuating.
>
> And here's the part that motivated writing to you:*postfix gives an
> operator no lever to prevent its own doomed query.*
>
> * There's no introspection point between "cleanup extracted this
> header address" and "cleanup sends it to the map" ? I can't see or
> cap the key that's about to be sent.
> * Canonical/socketmap lookups have*no key-length limit*; cleanup will
> happily emit a key it must know the socketmap can't accept.
> * |header_checks|match a*whole logical header*, not individual
> addresses, so I can't write a rule like "reject if any single
> address in To/Cc exceeds N bytes or X symbols." I can only
> pattern-match the header text as a blob, which is fragile against
> folding.
> * The failure surfaces as a generic|451 ... queue file write error|?
> indistinguishable from disk-full, a database outage, etc. ? so I
> can't even intercept/this specific/condition and handle it differently.
>
> Net: the only place I/can/currently block this is a milter/content
> filter in front of the relay (which is what I'll do), but that feels
> like papering over a postfix-side sharp edge ? and it's a leaky patch,
> because a milter parses headers/very/differently from postfix's|tok822|.
> The filter's address parser doesn't reproduce the
> unclosed-quote-swallows-the-fold behavior, so what the milter sees as
> the recipient set and what|cleanup|will actually hand the socketmap are
> two different things. I can approximate (flag unbalanced quotes /
> over-long tokens on the raw header), but I can't reliably pre-empt
> postfix's own tokenization from outside it.
>
>
> Questions / possible directions
>
> 1. Should a lookup failure during*optional header*canonicalization
> (|header_recipient|) be fatal at all? Logging and leaving the header
> untouched (continue, don't cancel) would avoid the whole class of
> problem for cosmetic rewrites.
> 2. Could the canonical/socketmap client*cap the key length*it will send
> ? e.g. if the rewritten address exceeds a representable/
> configurable maximum, treat it as "no match" (skip the rewrite)
> instead of emitting a query the server must reject? Postfix knows
> the key before it sends it.
> 3. Failing that, is there appetite for a*per-address length
> guard*operators could set (reject or skip addresses whose serialized
> form exceeds N), given|header_checks|can't express per-address
> limits today?
> 4. Would distinguishing a socketmap|PERM|on/header/canonicalization
> from a genuine transport/tempfail ? with its own status rather than
> the generic|queue file write error|? be reasonable, so operators can
> act on it?
> 5. Most impactful, and independent of the above: should a
> socketmap*|PERM|map to a permanent|5xx|*rather than a
> transient|451|? The server explicitly said/permanent/, yet the
> client is told to retry forever. Surfacing it as|5xx|(bounce)
> instead of|451|(retry) would, on its own, prevent the deterministic
> queue-wedge ? even if everything else stayed as-is.
>
>
> Minimal reproduction
>
> Stock postfix + any socketmap that bounds its request (e.g. PostSRSd,
> 512-byte payload):
>
> |recipient_canonical_maps = socketmap:inet:127.0.0.1:10003:reverse
> recipient_canonical_classes = envelope_recipient, header_recipient
> remote_header_rewrite_domain = domain.invalid
> local_header_rewrite_clients = <something that excludes the injecting
> client, so context = remote> |
>
> Inject (clean envelope, junk only in the header) a message
> whose|To:|or|Cc:|opens a double-quote that is never closed and is
> followed by ~2 KB of folded content. cleanup issues an oversized
> socketmap query, gets|PERM|, and cancels the message with|451 4.3.0
> Error: queue file write error|. Balancing/closing the quote (so the
> token stays a normal-length address) makes it deliver normally ?
> confirming size, not content, is the fault.
>
> Thanks for postfix, and for reading this far.
>
> --
> Regards,
> Dmytro Alieksieiev
> DevOps Engineer
> _______________________________________________
> Postfix-users mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
_______________________________________________
Postfix-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
