Re: Corner cases in SSL_shutdown.
On 2/2/21 12:39 PM, Leo Bicknell wrote: > In a message written on Tue, Feb 02, 2021 at 04:54:18PM +, Antonio Leding > wrote: >>You're not doin' well son...quit diggin' and go back to rethink your >>approach. I dare say at least a majority on this list, including >>myself, will trust Viktor et al a far bit more than someone coming in >>from the cold who freely admits the are not "well versed" in the app, >>nor a key protocol used by that app, but then still feels qualified to >>argue as to the (falsely) alleged flaws in that app... > > This is not the first place I've discussed this, and the usual > reception is some fascination at the interplay of the TLS library > and the application. I have found many opinions of the severity > or urgency, but I have yet in any previous community had anyone > argue that dropping the TLS connection was a good behavior. There are a few reasons (all mentioned earlier) why Postfix uses the approach it does: - Postfix is process-per-connection, so waiting for a timeout would consume nontrivial resources. - Postfix does not send QUIT until it has received responses to all pending commands. An SMTP server will not respond to a command until it has received all of the commands data. Together, this guarantees that when Postfix sends QUIT, the buffers on **both** sides of the connection are empty. - Postfix never downgrades a TLS connection to clear text. Therefore, any call to SSL_shutdown will soon be followed by closing the socket. >From a security perspective, it actually isn’t necessary for Postfix to send a close notify at all. Simply closing the TCP connection would work just as well, and might even be a bit faster. An FTPS implementation, however, absolutely must do a full bidirectional shutdown of the TLS connection. Since Postfix’s behavior as a client is valid, servers should not consider it an error condition. If a server does consider it an error, I would consider this a bug in the server implementation. Conversely, an FTPS implementation *should* consider lack of bidirectional shutdown to be an error. Sincerely, Demi
Re: Conditional relayhost based on message size
On 1/16/21 5:12 PM, Wietse Venema wrote: > Viktor Dukhovni: >> On Sat, Jan 16, 2021 at 04:48:22AM -0500, Viktor Dukhovni wrote: >> >>> On Sat, Jan 16, 2021 at 08:14:34AM +, Alexander wrote: >>> My goal is to conditionally select the relayhost based on the total size of the outgoing message. The rationale is that I'm using Amazon AWS SES for the most part. Alas, SES only accepts messages up to 10 MB in size (this includes images and attachments that are part of the message), and when the outgoing message exceeds that size, it will be bounced. Any tips for filters or tools that would help me define another relayhost based on the outgoing message size would be greatly appreciated! >>> >>> Since the queue manager has no mechanism for size-dependent selection of >>> a default transport (with the help of trivial-rewrite), the only way >>> handle this deterministically in Postfix is to shunt large messages into >>> a separate instance, which can be done in a number of ways, but is not >>> particularly simple or elegant. >> >> For this to be naturally supported in Postfix, we'd need a new feature, >> namely a variant of "FILTER" that overrides *just* the default_transport, >> making it possible to use more general criteria than "sender-dependent" >> for default_transport selection. >> >> Possible design: >> >> 1. New access(5) verb taking an associated transport value that >> triggers such an override. >> 2. New queue-file record type holding the default-transport >> override. >> 3. Queue manager loads this record into memory in the same way >> as "FILTER". If both are present, "FILTER" wins. >> 4. Queue manager sends its value along with all trivial-rewrite >> "resolve" requests. >> 5. trivial-rewrite uses this default transport rather than >> $default_transport when the destination domain is not >> in any of the more specific address classes, and there >> is no explicit transport(5) mapping for the recipient. >> >> This would be primarily useful only when using a relayhost, subject to >> secondary criteria to divert some messages to a different transport >> and/or nexthop based on ad-hoc message features detected during >> message reception. The difference from "FILTER" is that messages >> to local recipients are not affected. >> >> I am not sure whether adding such a feature motivated by just the >> presented use-case is warranted. > > My take: not if it only solves only one problem. I agree. In particular, we should avoid a pile of features with very narrow use-cases. > For example, I have several notes on making the scheduling (within > a delivery transport) dependent on a concept of 'cost', which could > be one "number of delivery requests with similar sender or domain", > "message size", "number of delivery requests with similar receiver > or domain", or some weighted combination. Just compute a cost and > schedule as appropriate for that cost bucket. > > If we make routing dependent on message properties, we could take > a page from trivial-rewrite which has two flavors of routing, one > for probes and one for non-probes. We could have N flavors of > routing and select the flavor based on the weighted computation of > something. > > Wietse Personally, I would like to be able to control this via a plugin of some sort. Declarative configuration is awesome, but it eventually reaches its limits. Would the existing milter system be a good fit for this? If not, I wonder if an embedded Lua interpreter would be a good idea. Lua is *tiny* (much smaller than Postfix itself), extremely portable, and reasonably fast for an interpreted language. There is also LuaJIT, which is extremely fast, while still being smaller than Postfix. Embedding an interpreter would allow users to express their business rules in a full programming language, rather than being limited to what Postfix supports natively. Sincerely, Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 10/25/20 2:46 PM, Wietse Venema wrote: > postfix-3.6-20201025 has a preliminary implementation to limit the > envelope senders that a local user may specify to the Postfix > sendmail (or postdrop) command. The real work is done in a library > module, so that similar functionality can later be added to the > Postfix SMTP daemon. > > Source: > http://ftp.porcupine.org/mirrors/postfix-release/index.html I looked at the source code, and all I can say is: Wow. Thank you, Wietse! Your implementation is indeed of very high quality. Certainly better than mine! > Manual: > http://www.porcupine.org/postfix-mirror/postconf.5.html#local_login_sender_maps Nit: Given the quoted localpart TODO, it might be a good idea to suggest limiting the character set that will be matched. On a system I ran, I would use: /etc/postfix/login_senders: # Allow both the bare username and user@domain forms. /([A-Za-z][A-Za-z0-9_-]*)$/iAE $1, $1...@example.com but the regex will of course be system-dependent. I say "might" because one could reasonably argue that if a user is allowed to login with a username containing a comma or space, something has already gone wrong. > I still need to add some credits to the HISTORY file, and update > the RELEASE_NOTES file with a feature summary. > > Wietse Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Limiting HELO spoofing in Postfix?
On 10/24/20 6:38 PM, Viktor Dukhovni wrote: > On Sat, Oct 24, 2020 at 03:22:28PM -0700, Rich Wales wrote: > >> From Viktor Dukhovni: >>> I don't recall whether you have as yet posted the requested (sans any >>> reformatting of line breaks) outputs of: >>> >>> $ postconf -Mf >>> $ postconf -nf >> >> See the attached text files. > > Thanks. Could you also post the output of: > > # netstat -anp --inet | grep LISTEN > # netstat -anp --inet6 | grep LISTEN > # iptables -n -L -v > # iptables -t nat -n -L -v > Also: # nft list ruleset OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: DMARC and security (was: sanity-check postfix XCLIENT usage ?)
On Fri, Oct 23, 2020 at 3:26 PM Demi M. Obenour wrote: > >> "p=quarantine" might be a better choice, but I do consider lack of > >> DMARC to be a security hole. I certainly don't want someone to be > >> able to forge mail that claims to be from me. There are all sorts of > >> nasty social engineering attacks someone could do with that ability, > >> many of which have real-world consequences. Mailing lists are the most important reason why DMARC is not terribly meaningful as a trust indicator for email from gmail.com. Anyone can claim an account that looks uncannily similar to an existing account, and would fool most users unless they are particularly attentive to minute details. I've heard reports that Paypal's transactional email benefits from DMARC and that phishing is less frequent as a result. Though I'm somewhat surprised this is effective, I'm willing to believe it is actually true (the fishers must just not care enough to make a convincing pitch from a different domain). But for the consumer email providers, attestation that the email was really from Gmail is not a very strong indicator that the message is legitimate. -- Not the same Demi M. Obenour.
Re: Postfix smtp gets stuck with XCLIENT when using smtps
On 10/22/20 12:47 PM, Aki Tuomi wrote: > Hi! > > I stumbled upon a possible bug with postfix. I am using postfix 3.4.14, and > when I use XCLIENT command over smtps (not starttls), the session gets stuck > until further input, which causes it to abort the connection due to > unexpected SSL packet. Just FYI, gmail marked this message as spam. Not sure why. Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: sanity-check postfix XCLIENT usage ?
On 10/22/20 3:35 PM, Bob Proulx wrote: > Demi M. Obenour wrote: >> Viktor Dukhovni wrote: >>>> Demi M. Obenour wrote: >>>> This is really a security hole in gmail. Given the popularity of >>>> gmail, however, I seriously suggest somehow treating gmail as if it >>>> had p=reject, as it should. >>> No it should not have "p=reject" that's only for sites that only send >>> "transactional" email. And lack of DMARC is not a "security hole". >> >> "p=quarantine" might be a better choice, but I do consider lack of >> DMARC to be a security hole. I certainly don't want someone to be >> able to forge mail that claims to be from me. There are all sorts of >> nasty social engineering attacks someone could do with that ability, >> many of which have real-world consequences. > > Such as your mail from Gmail through mailing lists such as this one? > DMARC breaks traditional mailing list usage because it focuses on the > header address not the envelope address. That's because MUAs display the From: header, not the envelope address. DMARC is aimed at preventing spoofing. If someone sends a message that claims to be from me, but is not, that could damage my reputation or worse. If GMail had p=reject, such a message would be dropped as a forgery. If a relative of mine gets a message that claims to be from me, but is actually from , they at least have a chance of knowing the message is bogus. > Sites with a strict DMARC policy require mailing lists to either > rewrite header addresses to avoid the breakage, or to drop the mail, > or other worse alternatives. Strict DMARC policy is why we are often > seeing "... via ..." in the From: addresses and the address rewritten > now when it is coming from a site that has set a strict DMARC policy. To me, that is a good thing. I *want* mailing lists to either relay the message without changes, or take ownership of the message body by changing the From: header. Otherwise, they are claiming that I sent a message that I never in fact sent, which is not okay. "... via ..." is what I want to see in a mailing list message. > Strict DMARC policy is suitable for banks and other direct mailing use > wishing higher security but is not suitable for a user's general email > where they want to send mail to mailing lists and have other > interactions with the community. If a mailing list relays mail without changing it, DMARC will pass, since the digital signature will still verify correctly. Changing the message without changing the From: header is spoofing, and mailing list software that does it is broken. > Bob Sincerely, Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: sanity-check postfix XCLIENT usage ?
On 10/22/20 12:25 PM, Viktor Dukhovni wrote: >> On Oct 22, 2020, at 2:11 PM, Demi M. Obenour wrote: >> >> I know :( >> >> This is really a security hole in gmail. Given the popularity of >> gmail, however, I seriously suggest somehow treating gmail as if it >> had p=reject, as it should. > No it should not have "p=reject" that's only for sites that only send > "transactional" email. And lack of DMARC is not a "security hole". "p=quarantine" might be a better choice, but I do consider lack of DMARC to be a security hole. I certainly don't want someone to be able to forge mail that claims to be from me. There are all sorts of nasty social engineering attacks someone could do with that ability, many of which have real-world consequences. Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: sanity-check postfix XCLIENT usage ?
On 10/22/20 3:23 AM, Bastian Blank wrote: > Hi name less > > On Wed, Oct 21, 2020 at 10:13:54AM -0700, PGNet Dev wrote: >> I've online-checked SPF/DMARC records for 'intuit.com'; all _seems_ to be ok. >> I've cranked up opendmarc logging level to >> MilterDebug 5 >> with that, on failed attempt, I see only an unhelpful >> Oct 21 09:43:39 mx.example.com opendmarc[7977]: 4CGbb3aX1Pz2N: >> intuit.com fail > > This is not Postfix! > >> Trying 1st from @gmail.com (or any domain i've tried _other_ than >> 'intuit.com') > > Please see the DMARC policy of gmail.com, especially the "none" policy: > > | _dmarc.gmail.com. IN TXT "v=DMARC1; p=none; sp=quarantine; […]" I know :( This is really a security hole in gmail. Given the popularity of gmail, however, I seriously suggest somehow treating gmail as if it had p=reject, as it should. Sincerely, Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Limiting HELO spoofing in Postfix?
On 10/20/20 8:20 PM, IL Ka wrote: >> > /index.php?s=index/\\think\\app/invokefunction&function=call_user_func_array&vars[0]=phpinfo&vars[1][]=1 > That is fine: networks are constantly scanned by bots. They are trying to > hack any site using well-known vulnerabilities. > > I have a lot of similar entries, although I do not have PHP on my site) > > I have never been hacked, but if I were, here is what I would do: > * Reformat drive and install the latest stable version of your favorite OS. > Be sure to upgrade it on the regular basis. Many OSes can do that using > cron. I agree, with the caveat that an attacker would need to have obtained root access to implant a rootkit. I consider the likelihood of this high enough that wiping and reinstalling is justified. > * Use the latest stable version of some mature framework and also update > it. If you aren't using one, then make sure you understand how to write > secure code and how to run it correctly > * Close all ports except http, https and ssh (which you should move away > from 22 port because 22 port is also scanned by bots). Disable password > authentication for ssh (use keys instead) If password and challenge-response authentication for SSH are disabled, it isn't necessary to move SSH off of port 22. SSH keys are not vulnerable to brute-force attack, and last pre-authentication vulnerability (other than denial of service) that I am aware of in OpenSSH was in 2003. Moving the SSH port can, however, reduce noise in your logs. fail2ban and friends can help as well. Sincerely, Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: multiple relay servers
On 10/21/20 11:16 AM, Fred Morris wrote: > If DNSSEC isn't required for the domain(s) in question (or at least postfix > in this specific case) you might look at RPZ as a way of rewriting just a > single record in the zone: https://www.dnsrpz.info/ You can also use a local validating recursive resolver (such as Unbound) and inject a fake record yourself. Postfix doesn't validate DNSSEC on its own. That said, I am not sure how to get Unbound to lie about the AD bit. Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Mail server recently became an open relay
On 10/19/20 3:29 PM, Jaroslaw Rafa wrote: > Dnia 19.10.2020 o godz. 21:12:20 John Fawcett pisze: >> Sorry not to be able to give a definitive answer. Typical mail injection >> via php will use a script that already calls the php mail function or >> similar functions that open the smtp connection. But there are other >> attack vectors that are possible that allow hackers to gain the >> privileges of the web server user. > > Very often hackers abuse web pages that allow users to upload files to the > web server. If the input is not correctly sanitized, it may be possible to > upload an arbitrary php script and get it executed. > > There were multiple attacks based on this scenario. Can this be mitigated by denying the PHP user write permission on any directory where PHP files will be executed? Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 10/17/20 6:25 PM, Wietse Venema wrote: > Demi M. Obenour: >>> BTW I realized that I swapped the semantics of smtpd_sender_login_maps >>> (a mapping from sender address to the login names that are allowed >>> to use that sender address) when we were discussing the postdrop >>> feature (a mapping from login name to the sender addresses that the >>> login name is allowed to use). >>> >>> If we stick with that model, then I think that the feature should be >>> renamed to local_login_sender_maps. >> >> I agree, although I wonder if it is better to be consistent and >> change the semantics to match those of smtpd_sender_login_maps. >> local_login_sender_maps was easier to implement and seemed more >> intuitive, but it is less consistent than local_sender_login_maps, >> which can in some cases be set as >> >> local_sender_login_maps = $smtpd_sender_login_maps > > This is not obviously right. In fact, I could argue for the opposite: > > local_login_sender_maps = $smtpd_login_sender_maps > > smtpd_sender_login_maps pre-dates the Postfix submission services > (submission + smtps) by three years. It solved a problem for an > SMTP server that receives authenticated and unauthenticated messages > on the same port. In that setting, the sender address had to be the > search key for policy lookup. > > In contrast, postdrop always knows a user ID; there is no legitimate > way to invoke postdrop anonymously. With the Postfix submission > services, SMTP submission has become more similar to postdrop > submission: Postfix always knows a user identity, thus the user > identity can become the search key for policy lookup. For the port > 25 MTA-to-MTA service one can then reject all mail from a remote > site that claims to be from a local user. > > Wietse I agree. local_login_sender_maps is far, *far* more intuitive. In my test instance (which uses my own patch, pending the incorporation of this into a release), I use the equivalent of the following: main.cf: local_login_sender_maps = pcre:/etc/postfix/senders /etc/postfix/senders: /^([A-Za-z_][A-Za-z0-9_.-]*)$/ $1, $1@localhost Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 10/17/20 6:42 PM, Wietse Venema wrote: > Jaroslaw Rafa: >> Dnia 17.10.2020 o godz. 18:25:13 Wietse Venema pisze: >>> For the port >>> 25 MTA-to-MTA service one can then reject all mail from a remote >>> site that claims to be from a local user. >> >> That's not a good idea. Assume domain.com is configured that way and some >> user on a completely different domain (us...@site.net) forwards their mail >> to us...@domain.com. Then what happens if some otheru...@domain.com sends >> mail to us...@site.net ? > > [historical scenario omitted] > > Exactly the scanario that SPF and the like are supposed to prevent. > > Wietse To elaborate, my understanding is that site.net should use MAIL FROM:, but leave the body unchanged. domain.com will then accept the message, as it is from an IP in site.net's SPF record, and DKIM ignores the envelope. Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: rbl check debug
Just FYI, GMail marked this mail as spam. Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 10/17/20 11:34 AM, Wietse Venema wrote: > Demi M. Obenour: >> Should I submit another patch? In addition to adding >> local_sender_login_maps, I have fixed what appeared to be a bug in >> the current postdrop and sendmail commands: root and $mail_owner were >> not automatically allowed to submit mail. Since this is inconsistent >> with similar checks elsewhere, I believe it is unintentional. > > No. I am working on a quality implementation, and that takes more > time than 'code that works'. More patches are not needed. Indeed it does. If I may ask, is there any way I could have made my patch better? I would like to know so that I can write better code in the future. > BTW I realized that I swapped the semantics of smtpd_sender_login_maps > (a mapping from sender address to the login names that are allowed > to use that sender address) when we were discussing the postdrop > feature (a mapping from login name to the sender addresses that the > login name is allowed to use). > > If we stick with that model, then I think that the feature should be > renamed to local_login_sender_maps. I agree, although I wonder if it is better to be consistent and change the semantics to match those of smtpd_sender_login_maps. local_login_sender_maps was easier to implement and seemed more intuitive, but it is less consistent than local_sender_login_maps, which can in some cases be set as local_sender_login_maps = $smtpd_sender_login_maps in main.cf. > Wietse Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: possible bottlenecks
On 10/17/20 1:23 AM, Viktor Dukhovni wrote: >> On Oct 17, 2020, at 3:09 AM, Demi M. Obenour wrote: >> >>> The practical limit to the deferred queue size is therefore ~2 days of >>> throughput, and depends heavily on the per-delivery latency. If >>> delivery failures are slow (tarpitting or otherwise slow destinations) >>> the impact is greater. >> >> Can the latency problems be worked around by increasing concurrency? > > Yes, so unsurprisingly, Postfix amortises latency via carefully managed > concurrency. > >> My understanding is that Postfix might have problems at very high >> concurrency due to using one process per connection, whereas some >> other servers are event-driven and can handle thousands of connections >> without using too much memory. > > Postfix reuses processes for multiple deliveries, so process creation > is effectively amortised. SMTP delivery being a rather expensive operation > (DNS lookups, connection setup, TLS handshakes, ...) the fractional (becase > shared across multiple deliveries) cost of process creation is dwarfed by > the actual SMTP transaction costs. > > On modern hardware (anything built in the last ~2 decades), you can run > thousands of concurrent SMTP delivery agents, without any difficulty, > their executables are loaded only once, and per connection memory utilisation > is modest. You run out of remote sites' willingness to receive your email > long before you run out of local capacity to send it. > > The event driven design mostly just makes those other servers more complex, > and more prone to security bugs. Postfix 3.4 and later grudgingly do some > event-driven work because TLS connection reuse with OpenSSL is not possible > out-of-process. So the tlsproxy(8) process context switches between multiple > TLS connections, but the rest of the SMTP delivery agent is one connection > per process and performs just fine. The architecture is however more robust > and secure. Good point. I have wondered if something like s2n would be a better choice, although I would probably use the OpenBSD Postfix packages built against LibreSSL. > Postfix is not an HTTP server handling tens to hundreds of thousands of > requests > per second, and does not benefit from the optimisations needed for those kinds > of workloads. Premature optimisations that sacrifice robustness and security > for little gain are not part of the design. I had not considered that, but you are correct. Email really is a much lower volume service than HTTP. If Postfix needed to handle hundreds of thousands of requests per second and concurrent connections, the process-per-connection model would obviously not work. But it does not need to handle that much load, so the simpler approach is better. Since Postfix is written in C, that simpler approach is to use one process per connection. If one is Google or Microsoft and need to process hundreds of millions of messages per day, then Postfix might not work. But if one needs to handle that much mail, then one can probably afford to write a bespoke MTA. Sincerely, Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: possible bottlenecks
On 10/16/20 9:24 PM, Viktor Dukhovni wrote: > The practical limit to the deferred queue size is therefore ~2 days of > throughput, and depends heavily on the per-delivery latency. If > delivery failures are slow (tarpitting or otherwise slow destinations) > the impact is greater. Can the latency problems be worked around by increasing concurrency? My understanding is that Postfix might have problems at very high concurrency due to using one process per connection, whereas some other servers are event-driven and can handle thousands of connections without using too much memory. > There is no magic that can make OpenSMTPD immune to the laws of > arithmetic. Indeed there is not. Any statement to that effect on my part was erroneous and based on a misunderstanding. Sincerely, Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
Should I submit another patch? In addition to adding local_sender_login_maps, I have fixed what appeared to be a bug in the current postdrop and sendmail commands: root and $mail_owner were not automatically allowed to submit mail. Since this is inconsistent with similar checks elsewhere, I believe it is unintentional. Sincerely, Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: possible bottlenecks
On 10/16/20 2:10 PM, Viktor Dukhovni wrote: >> On Oct 16, 2020, at 3:14 PM, Demi M. Obenour wrote: >> >> I don’t recommend stock OpenSMTPD for security reasons, although I >> have some patches that make it much better in this regard. However, >> all of those relate to local deliveries. If you can afford to disable >> local deliveries, OpenSMTPD is actually a good choice for this work. >> It can handle multi-million-message queues without any problems. > > Well, for good performance one should not have much of a queue at all, > the mail should go out as quickly as it comes in. If you're queueing > a lot of email, then your output is not keeping up with the input. Not necessarily. It is quite possible for the peak rate of incoming traffic to be greater than the average rate at which it can be delivered, even though the average rate of incoming mail is lower than the average delivery rate. That will result in queues forming and eventually draining. If the bursty traffic is distributed among multiple recipients with their own, separate rate limits, it is quite possible to have a queue of finite size at quasi-steady-state. > Unless there's a particularly good reason why you believe that OpenSMTPD > would do better than Postfix in bulk mail delivery performance, it is not > helpful to recommend it here. I misunderstood your previous message, sorry. I interpreted it as a statement that Postfix struggles with very large mail queues, and I know OpenSMTPD does not. Apologies, Demi M. Obenour OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: possible bottlenecks
On 10/16/20 8:57 AM, @lbutlr wrote: > On 13 Oct 2020, at 22:47, Zsombor B wrote: >> I know this is a complicated question but what/where do you see possible >> bottlenecks in postfix? >> Is it CPU? RAM? Disk IO? > > In theory? Sure, any of those could be a bottle neck. On actuality, the > bottles necks are processing spam if you receive mail and not appearing to be > a spammer. > >> I'm building an infra to send out ~3-5 million emails a day. > > If you pop onto the Internet all of a sudden sending 5 million emails a day > you better be sure that your DKIM SPF DMARC and DNS are perfect and that your > IP address has never been associated with a spammer. Because if there is one > thing that will cripple your mail server it is having mail sit in queue > because it's been throttled. The big email hosts do this a lot (especially > Outlook.com and yahoo.com). And if you get on their (automated) bad side, you > are well and thoroughly screwed. If you messages LOOK spammy enough that > users will mark them as spam, then you will, again, be completely hosed > whether the email is spam or not. > > Other than that, I think a raspberry pi 4 with a USB SSD might be able to > mange 5 million emails a day. I don’t recommend stock OpenSMTPD for security reasons, although I have some patches that make it much better in this regard. However, all of those relate to local deliveries. If you can afford to disable local deliveries, OpenSMTPD is actually a good choice for this work. It can handle multi-million-message queues without any problems. That said, you will run into numerous other problems. https://www.mail-archive.com/misc@opensmtpd.org/msg05153.html is a good introduction to them. Gilles Chehade (the author of that post, and formerly one of the two main developers of OpenSMTPD) is an expert on the subject, and I trust his recommendation. Sincerely, Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Occasional transient "Insufficient system storage" errors
On 10/15/20 3:44 AM, Dara Poon wrote: > (Well, that was embarrassing! I had a Spamassassin milter on outbound mail > that tagged my own message as a false positive. Sending it again for > readability. Sorry!) FYI, GMail considered both the original message and the resend to be spam as well. Sincerely, Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Sender restriction to reject message with multiple from addresses
On 10/9/20 11:06 PM, Viktor Dukhovni wrote: > On Fri, Oct 09, 2020 at 10:59:33PM -0400, Demi M. Obenour wrote: > >> I love DKIM, but it should have been on the Sender header and not >> the From header. However, for that to work, MUAs would have had to >> display something like "f...@example.com claims that this message >> is from f...@example.com and b...@example.com", and they do not. > > Actually, Outlook does exactly that, and other MUAs would have come on > board if there was good cause to do that. At this point however, nobody > is investing much many in MUA development. All the $$$ are going into > walled-garden cloud webmail systems. :-( Someone should probably file enhancement requests with other MUAs. And at least NeoMutt and Thunderbird are actively developed. >> That lead to the current design. > > You're perhaps confusing DKIM with DMARC. DKIM just signs the message > content and whatever headers it is configured to sign. It is mere > integrity protection, not policy. The signing domain is determined from > the selector and the "d" field in the DKIM header, and is not tied to > either From or Sender. > > DKIM is fine. The actual breakage is in DMARC. Sadly, it is too late to change DMARC. Hopefully we can add a new header that means what From once did. Doing away with DMARC isn't an option either, as it creates a massive security hole. Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Sender restriction to reject message with multiple from addresses
On 10/9/20 9:48 PM, Viktor Dukhovni wrote: >> What are the semantics of a From: header with multiple addresses? > The message purports to be the work of multiple authors. Such a message > is required to have a "Sender" header, but in most cases that constraint > is unlikely to be enforced. I love DKIM, but it should have been on the Sender header and not the From header. However, for that to work, MUAs would have had to display something like "f...@example.com claims that this message is from f...@example.com and b...@example.com", and they do not. That lead to the current design. Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Sender restriction to reject message with multiple from addresses
On 10/9/20 8:45 PM, Wietse Venema wrote: > Ron Wheeler: >> I am also the family genealogist and just moved to Gramps from FTM. >> >> I am not sure what "multiple from addresses" actually means. It is not >> possible for an email to come from more than one email address at a time >> in reality. > > Multiple addresses in one From: header are allowed by the RFC 5322 spec. > Multiple From: headers in a message are not OK. What are the semantics of a From: header with multiple addresses? Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 10/9/20 1:06 PM, Demi M. Obenour wrote:
> On 10/8/20 3:19 PM, Wietse Venema wrote:
>> Demi M. Obenour:
>>> On 10/8/20 8:25 AM, Wietse Venema wrote:
>>>> Demi M. Obenour:
>>>>> On 10/6/20 4:23 PM, Wietse Venema wrote:
>>>>>> If the feature is turned on then there should probably be a
>>>>>> default action for users not listed in the table (deny or allow).
>>>>>> Its not going to be pretty when only the numerical UID is avaialble
>>>>>> (a 1:1 mapping username->sender would not make sense).
>>>>>
>>>>> What about defaulting to allow if local_sender_login_maps has its
>>>>> default value, and deny otherwise? That keeps the current default
>>>>> behavior, while still allowing administrators to lock it down.
>>>>
>>>> The action (deny) for unmatched users should not depend on the
>>>> (non-empty) local_sender_login_maps value.
>>>
>>> Should this be a configuration option?
>>
>> It is not needed. If someone wants unmatched users to allow all,
>> just say so:
>>
>> local_sender_login_maps = address map> static:*
>>
>> This still enforces the allowed email addressesfor users that have
>> an entry.
>>
>> If local_sender_login_maps is turned on, indexing with "#" plus the
>> UID as a string would do the job. It is not like users can arbitrarily
>> remove their login name from the password file.
>
> Here is an updated diff which implements this idea. It also splits out
> the FROM access check into its own function, and uses var_rcpt_delim
> instead of hard-coding "+". $mail_owner is now allowed to send mail
> as any user.
Looks like Thunderbird mangled the whitespace. Here is another attempt:
diff -ur ../postfix-3.5.7-old/src/global/mail_params.h
../postfix-3.5.7/src/global/mail_params.h
--- ../postfix-3.5.7-old/src/global/mail_params.h 2020-10-05
18:52:49.51800 -0400
+++ ../postfix-3.5.7/src/global/mail_params.h 2020-10-05 18:10:32.64100
-0400
@@ -1667,6 +1667,10 @@
#define DEF_SMTPD_SASL_TYPEDEF_SERVER_SASL_TYPE
extern char *var_smtpd_sasl_type;
+#define VAR_LOCAL_SND_AUTH_MAPS"local_sender_login_maps"
+#define DEF_LOCAL_SND_AUTH_MAPS"static:*"
+extern char *var_local_snd_auth_maps;
+
#define VAR_SMTPD_SND_AUTH_MAPS"smtpd_sender_login_maps"
#define DEF_SMTPD_SND_AUTH_MAPS""
extern char *var_smtpd_snd_auth_maps;
diff -ur ../postfix-3.5.7-old/src/postdrop/postdrop.c
../postfix-3.5.7/src/postdrop/postdrop.c
--- ../postfix-3.5.7-old/src/postdrop/postdrop.c2020-10-05
18:52:49.51800 -0400
+++ ../postfix-3.5.7/src/postdrop/postdrop.c2020-10-09 12:53:43.24900
-0400
@@ -114,6 +114,7 @@
#include /* remove() */
#include
#include
+#include
#include
#include
#include
@@ -147,6 +148,10 @@
#include
#include
#include
+#include
+#include
+#include
+#include
/* Application-specific. */
@@ -167,9 +172,13 @@
* Local mail submission access list.
*/
char *var_submit_acl;
+char *var_local_snd_auth_maps;
+char *var_rcpt_delim;
static const CONFIG_STR_TABLE str_table[] = {
VAR_SUBMIT_ACL, DEF_SUBMIT_ACL, &var_submit_acl, 0, 0,
+VAR_LOCAL_SND_AUTH_MAPS, DEF_LOCAL_SND_AUTH_MAPS,
&var_local_snd_auth_maps, 0, 0,
+VAR_RCPT_DELIM, DEF_RCPT_DELIM, &var_rcpt_delim, 0, 0,
0,
};
@@ -220,6 +229,65 @@
postdrop_sig(0);
}
+/*
+ * postdrop_check_access - check if a user is authorized to send mail as
+ * the envelope sender
+ */
+static void postdrop_check_access(VSTRING *const buf, uid_t const uid) {
+MAPS*local_sender_login_maps;
+struct mypasswd *username;
+const char *owners;
+char *saved_owners, *cp, *name, *stripped, *stripped_copy,
username_buf[22];
+char *pw_name;
+int found = 0;
+
+/* Root and $mail_owner are always allowed to send as anyone. */
+if (uid == 0 || uid == var_owner_uid)
+ return;
+
+/*
+ * Get the username.
+ */
+if ((username = mypwuid(uid)) != NULL)
+ pw_name = username->pw_name;
+else {
+ int status = snprintf(username_buf, sizeof username_buf, "#%"PRIu64,
(uint64_t)uid);
+ if (status < 0 || status >= sizeof username_buf)
+ msg_fatal("snprintf");
+ pw_name = username_buf;
+}
+
+local_sender_login_maps = maps_create(VAR_SMTPD_SND_AUTH_MAPS,
+ var_local_snd_auth_maps,
+ DICT_FLAG_FOLD_FIX |
+ DICT_FLAG_UTF8_REQUEST |
+ DICT_FLAG_NO_PROXY |
+
Re: Accessing the sending user from a canonical(5) table
On 10/8/20 3:19 PM, Wietse Venema wrote:
Demi M. Obenour:
On 10/8/20 8:25 AM, Wietse Venema wrote:
Demi M. Obenour:
On 10/6/20 4:23 PM, Wietse Venema wrote:
If the feature is turned on then there should probably be a
default action for users not listed in the table (deny or allow).
Its not going to be pretty when only the numerical UID is avaialble
(a 1:1 mapping username->sender would not make sense).
What about defaulting to allow if local_sender_login_maps has its
default value, and deny otherwise? That keeps the current default
behavior, while still allowing administrators to lock it down.
The action (deny) for unmatched users should not depend on the
(non-empty) local_sender_login_maps value.
Should this be a configuration option?
It is not needed. If someone wants unmatched users to allow all,
just say so:
local_sender_login_maps = address map> static:*
This still enforces the allowed email addressesfor users that have
an entry.
If local_sender_login_maps is turned on, indexing with "#" plus the
UID as a string would do the job. It is not like users can arbitrarily
remove their login name from the password file.
Here is an updated diff which implements this idea. It also splits out
the FROM access check into its own function, and uses var_rcpt_delim
instead of hard-coding "+". $mail_owner is now allowed to send mail
as any user.
Wietse
Demi
diff -ur ../postfix-3.5.7-old/src/global/mail_params.h
../postfix-3.5.7/src/global/mail_params.h
--- ../postfix-3.5.7-old/src/global/mail_params.h 2020-10-05
18:52:49.51800 -0400
+++ ../postfix-3.5.7/src/global/mail_params.h 2020-10-05 18:10:32.64100
-0400
@@ -1667,6 +1667,10 @@
#define DEF_SMTPD_SASL_TYPEDEF_SERVER_SASL_TYPE
extern char *var_smtpd_sasl_type;
+#define VAR_LOCAL_SND_AUTH_MAPS "local_sender_login_maps"
+#define DEF_LOCAL_SND_AUTH_MAPS"static:*"
+extern char *var_local_snd_auth_maps;
+
#define VAR_SMTPD_SND_AUTH_MAPS"smtpd_sender_login_maps"
#define DEF_SMTPD_SND_AUTH_MAPS""
extern char *var_smtpd_snd_auth_maps;
diff -ur ../postfix-3.5.7-old/src/postdrop/postdrop.c
../postfix-3.5.7/src/postdrop/postdrop.c
--- ../postfix-3.5.7-old/src/postdrop/postdrop.c2020-10-05
18:52:49.51800 -0400
+++ ../postfix-3.5.7/src/postdrop/postdrop.c2020-10-09 12:53:43.24900
-0400
@@ -114,6 +114,7 @@
#include/* remove() */
#include
#include
+#include
#include
#include
#include
@@ -147,6 +148,10 @@
#include
#include
#include
+#include
+#include
+#include
+#include
/* Application-specific. */
@@ -167,9 +172,13 @@
* Local mail submission access list.
*/
char *var_submit_acl;
+char *var_local_snd_auth_maps;
+char *var_rcpt_delim;
static const CONFIG_STR_TABLE str_table[] = {
VAR_SUBMIT_ACL, DEF_SUBMIT_ACL, &var_submit_acl, 0, 0,
+VAR_LOCAL_SND_AUTH_MAPS, DEF_LOCAL_SND_AUTH_MAPS,
&var_local_snd_auth_maps, 0, 0,
+VAR_RCPT_DELIM, DEF_RCPT_DELIM, &var_rcpt_delim, 0, 0,
0,
};
@@ -220,6 +229,65 @@
postdrop_sig(0);
}
+/*
+ * postdrop_check_access - check if a user is authorized to send mail as
+ * the envelope sender
+ */
+static void postdrop_check_access(VSTRING *const buf, uid_t const uid) {
+MAPS*local_sender_login_maps;
+struct mypasswd *username;
+const char *owners;
+char *saved_owners, *cp, *name, *stripped, *stripped_copy,
username_buf[22];
+char *pw_name;
+int found = 0;
+
+/* Root and $mail_owner are always allowed to send as anyone. */
+if (uid == 0 || uid == var_owner_uid)
+ return;
+
+/*
+ * Get the username.
+ */
+if ((username = mypwuid(uid)) != NULL)
+ pw_name = username->pw_name;
+else {
+ int status = snprintf(username_buf, sizeof username_buf, "#%"PRIu64,
(uint64_t)uid);
+ if (status < 0 || status >= sizeof username_buf)
+ msg_fatal("snprintf");
+ pw_name = username_buf;
+}
+
+local_sender_login_maps = maps_create(VAR_SMTPD_SND_AUTH_MAPS,
+ var_local_snd_auth_maps,
+ DICT_FLAG_FOLD_FIX |
+ DICT_FLAG_UTF8_REQUEST |
+ DICT_FLAG_NO_PROXY |
+ DICT_FLAG_NO_UNAUTH);
+
+if (vstring_memchr(buf, '\0') != NULL)
+ msg_fatal("uid=%ld: NUL in FROM", (long) uid);
+if ((owners = maps_find(local_sender_login_maps, pw_name, 0)) == NULL)
+ msg_fatal("uid=%ld: not found in local_sender_login_maps", (long) uid);
+if (local_sender_login_maps->error != 0)
+ msg_fatal("map lookup succeeded, but error set");
+if (strcmp(owners, "*") == 0)
+ return;
+if ((stripped
Re: Accessing the sending user from a canonical(5) table
On 10/8/20 8:25 AM, Wietse Venema wrote: Demi M. Obenour: On 10/6/20 4:23 PM, Wietse Venema wrote: If the feature is turned on then there should probably be a default action for users not listed in the table (deny or allow). Its not going to be pretty when only the numerical UID is avaialble (a 1:1 mapping username->sender would not make sense). What about defaulting to allow if local_sender_login_maps has its default value, and deny otherwise? That keeps the current default behavior, while still allowing administrators to lock it down. The action (deny) for unmatched users should not depend on the (non-empty) local_sender_login_maps value. Should this be a configuration option? Alternatively, we could look up unknown UIDs as strings, perhaps prefixed with a character that is not valid in an email address. Other places in Postfix use "unknown" to mean something similar. We could also have local_sender_login_maps be empty by default, and have unmatched users be allowed by default if and only if it is empty. Wietse Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Why I prefer Postfix to OpenSMTPD for most uses
I used to consider OpenSMTPD to be highly secure, until CVE-2020-8794 and CVE-2020-7247 came out. Both allow an attacker to execute arbitrary shell commands as root. Even though both of these attacks have been fixed, I am still not sure if it is possible for a compromised unprivileged OpenSMTPD process to escalate privileges by similar means. There is a workaround (setting a specific "mda wrapper" in the configuration file), but it is off by default, and disables delivery to commands and files. It turns out that not only is Postfix not vulnerable to either attack, but it is still not vulnerable even if an attacker has a 0-day exploit in one of the unprivileged Postfix processes. Command injection via MDAs (CVE-2020-7247) would not be possible because Postfix does not use a shell for delivery by default, and even when it does use a shell, the sanitization done by the local service replaces all metacharacters with underscores. Command injection via envelope files (CVE-2020-8794) would not be possible either, because Postfix uses the "safe" (rather than "exact") model for delivery status management. This means that commands and files are not stored in the envelope file, but rather read from ~/.forward during delivery. Taken together, the above factors make me trust Postfix far more when it comes to security, especially when local deliveries are enabled. I don't need to worry that a future vulnerability in Postfix will potentially allow others to execute arbitrary code as my user, whereas OpenSMTPD needs special configuration before I can be anywhere near as confident. Postfix has other advantages, too. Its sendmail(1) works even if the mail system is stopped, whereas OpenSMTPD's does not. Postfix also supports other security features, such as DANE, which are lacking in OpenSMTPD. Finally, Postfix has far more flexible authentication and header processing. Wietse Venema, thank you for your years of hard work on Postfix. If any of the OpenSMTPD developers read this, I hope it provides some ideas for improvement. Sincerely, Demi M. Obenour OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 10/6/20 4:23 PM, Wietse Venema wrote: Demi M. Obenour: On 10/6/20 12:46 PM, Wietse Venema wrote: For me, 'not found' also includes the case that the user is not found in the passwd file. By "allow 'not found' users", do you mean that such users will automatically be granted access, or that they will still be looked up (perhaps by numeric UID) in local_sender_login_maps? Postfix sendmail looks up the username only if no sender was specified with -f, and terminates if the username cannot be found. That behavior should not change by default. That's fine. If the feature is turned on then there should probably be a default action for users not listed in the table (deny or allow). Its not going to be pretty when only the numerical UID is avaialble (a 1:1 mapping username->sender would not make sense). What about defaulting to allow if local_sender_login_maps has its default value, and deny otherwise? That keeps the current default behavior, while still allowing administrators to lock it down. In the unlikely event that the table lookup itself fails, I believe that postdrop should log an error and exit. That avoids accidental security holes due to temporary failures. Wietse Sincerely, Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 10/6/20 12:46 PM, Wietse Venema wrote: Demi M. Obenour: On 10/6/20 9:47 AM, Wietse Venema wrote: allow 'not found' users, similar to smtpd_sender_login_maps Would it be possible to make this configurable? The documentation seems to imply that reject_sender_login_mismatch considers ?not found? to be an error, while reject_known_sender_login_mismatch does not. On systems I administer, I would prefer for an unknown user to not be allowed to submit mail, but I understand if this cannot be the default. The patch I submitted blocks ?not found? users, but the default local_sender_login_maps (static:*) matches every user, so the default behavior is the same as now. For me, 'not found' also includes the case that the user is not found in the passwd file. By "allow 'not found' users", do you mean that such users will automatically be granted access, or that they will still be looked up (perhaps by numeric UID) in local_sender_login_maps? Wietse Demi P.S.: I noticed that UTF-8 characters are getting mangled somewhere. Is this intentional? OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 10/6/20 9:47 AM, Wietse Venema wrote: Demi M. Obenour: Patch (made against 3.5.7) attached. I lightly tested it locally and it seems to work, but there could very well be bugs. I am virtually certain that I violated the Postfix coding style somewhere, sorry. I can also send the patch inline if you prefer. I can read it. I'll try to massage the code later this week (instead of a dozen back-and-forth email messages about awkward details). Thank you so much! Should I look for it in a future unstable release? mail_addr_find -> maps_find "+" -> var_rcpt_delim Good catch! I knew I missed something there. allow 'not found' users, similar to smtpd_sender_login_maps Would it be possible to make this configurable? The documentation seems to imply that reject_sender_login_mismatch considers “not found” to be an error, while reject_known_sender_login_mismatch does not. On systems I administer, I would prefer for an unknown user to not be allowed to submit mail, but I understand if this cannot be the default. The patch I submitted blocks “not found” users, but the default local_sender_login_maps (static:*) matches every user, so the default behavior is the same as now. put the new code inside its own function, avoiding gotos Indeed that would be an improvement. Wietse Demi OpenPGP_0xB288B55FFF9C22C1.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 10/5/20 6:15 PM, Wietse Venema wrote:
Demi M. Obenour:
There was a recent vulnerability in OpenBSD due to libc malfunctioning
in a set-uid-root program under very low resource limits. I would
prefer to minimize the amount of third-party libraries that are used
by postdrop. That said, another option would be to error out if the
resource limits are below what we consider a reasonable minimum.
Another good reason for not using set-uid root programs...
Indeed.
I don't think it is practical for Postfix to have universal minimal
resource limts. You can add some custom OPENBSD code later.
That makes sense. The OpenBSD vulnerability has been fixed, and was
merely used as an example. No OpenBSD-specific code will be needed,
at least not for this purpose.
Surprise: Postfix has a strip_addr() function that can remove adress
extensions before enforcing the ACL.
Good to know! That proved critical.
Is the code in smtpd_check.c a good place to start?
Yes. It also helps you to become familiar with Postfix's
approach to parsing.
Indeed it was helpful. Thanks for the tip!
Patch (made against 3.5.7) attached. I lightly tested it locally and
it seems to work, but there could very well be bugs. I am virtually
certain that I violated the Postfix coding style somewhere, sorry.
I can also send the patch inline if you prefer.
For what it is worth, I found the Postfix source code to be very clean
and easy to read. Writing this patch probably took about four hours
of work, which is significantly less than I expected for a non-trivial
feature. Thank you for all the work you have put into Postfix!
Wietse
Thank you,
Demi
diff -ur postfix-3.5.7-old/src/global/mail_params.h postfix-3.5.7/src/global/mail_params.h
--- postfix-3.5.7-old/src/global/mail_params.h 2020-05-09 11:51:27.0 -0400
+++ postfix-3.5.7/src/global/mail_params.h 2020-10-05 18:10:32.64100 -0400
@@ -1667,6 +1667,10 @@
#define DEF_SMTPD_SASL_TYPE DEF_SERVER_SASL_TYPE
extern char *var_smtpd_sasl_type;
+#define VAR_LOCAL_SND_AUTH_MAPS "local_sender_login_maps"
+#define DEF_LOCAL_SND_AUTH_MAPS "static:*"
+extern char *var_local_snd_auth_maps;
+
#define VAR_SMTPD_SND_AUTH_MAPS "smtpd_sender_login_maps"
#define DEF_SMTPD_SND_AUTH_MAPS ""
extern char *var_smtpd_snd_auth_maps;
diff -ur postfix-3.5.7-old/src/postdrop/postdrop.c postfix-3.5.7/src/postdrop/postdrop.c
--- postfix-3.5.7-old/src/postdrop/postdrop.c 2019-10-13 11:32:18.0 -0400
+++ postfix-3.5.7/src/postdrop/postdrop.c 2020-10-05 18:48:02.07800 -0400
@@ -147,6 +147,10 @@
#include
#include
#include
+#include
+#include
+#include
+#include
/* Application-specific. */
@@ -167,9 +171,10 @@
* Local mail submission access list.
*/
char *var_submit_acl;
-
+char *var_local_snd_auth_maps;
static const CONFIG_STR_TABLE str_table[] = {
VAR_SUBMIT_ACL, DEF_SUBMIT_ACL, &var_submit_acl, 0, 0,
+VAR_LOCAL_SND_AUTH_MAPS, DEF_LOCAL_SND_AUTH_MAPS, &var_local_snd_auth_maps, 0, 0,
0,
};
@@ -231,6 +236,7 @@
int c;
VSTRING *buf;
int status;
+MAPS*local_sender_login_maps;
MAIL_STREAM *dst;
int rec_type;
static char *segment_info[] = {
@@ -239,6 +245,7 @@
char **expected;
uid_t uid = getuid();
ARGV *import_env;
+struct mypasswd *username;
const char *error_text;
char *attr_name;
char *attr_value;
@@ -316,16 +323,6 @@
get_mail_conf_str_table(str_table);
/*
- * Mail submission access control. Should this be in the user-land gate,
- * or in the daemon process?
- */
-mail_dict_init();
-if ((errstr = check_user_acl_byuid(VAR_SUBMIT_ACL, var_submit_acl,
- uid)) != 0)
- msg_fatal("User %s(%ld) is not allowed to submit mail",
- errstr, (long) uid);
-
-/*
* Stop run-away process accidents by limiting the queue file size. This
* is not a defense against DOS attack.
*/
@@ -369,11 +366,34 @@
/* End of initializations. */
/*
+ * Mail submission access control. Should this be in the user-land gate,
+ * or in the daemon process?
+ */
+mail_dict_init();
+if ((errstr = check_user_acl_byuid(VAR_SUBMIT_ACL, var_submit_acl,
+ uid)) != 0)
+ msg_fatal("User %s(%ld) is not allowed to submit mail",
+ errstr, (long) uid);
+
+local_sender_login_maps = maps_create(VAR_SMTPD_SND_AUTH_MAPS,
+ var_local_snd_auth_maps,
+ DICT_FLAG_FOLD_FIX |
+ DICT_FLAG_UTF8_REQUEST |
+ DICT_FLAG_NO_PROXY |
+ DICT_FLAG_NO_UNAUTH);
+
+/*
* Don't trust the caller's time information.
*/
GETTIMEOFDAY(&start);
/*
+ * Get the username.
+ */
+if ((username = mypwuid(uid)) == NULL)
+ msg_fatal("Cannot get username for uid %ld", (long) uid);
+
+/*
* Create queue file. mail_stream_file() ne
Accessing the sending user from a canonical(5) table
On 10/5/20 10:51 AM, Wietse Venema wrote: Demi M. Obenour: On 2020-10-04 19:55, Wietse Venema wrote: Demi M. Obenour: Checking application/pgp-signature: FAILURE -- Start of PGP signed section. On 2020-09-30 16:35, Wietse Venema wrote: Demi M. Obenour: - If a message arrives via the SMTPS or submission ports, I want to replace the address part of the user-supplied From: header with the envelope From: header. This allows me to use reject-sender-login-mismatch to prevent users from sending messages with forged From: addresses. There are two parts to this: 1) Locking down the envelope.from. With authenticated smtp submission, the envelope.from can be constrained by smtpd_sender_login_maps. With sendmail/postdrop submission the UNIX login name can be overidden with "sendmail -f". There is no code in Postfix to lock down "sendmail -f", and there is no 'plugin' interface that could do this, either. I don't like the idea of adding complex logic to the set-gid postdrop command to lock down "sendmail -f". Doing the lockdown in the pickup daemon would be more secure but has the problem that the 'reject' happens too late. I looked at the postdrop source code to see what locking down "sendmail -f" would entail. Checking that the current user can use `-f` seems to be just looking up the current username in an ACL, which postdrop already does for authorized_submit_users. Checking that -f was not passed looks to just be a string equality check, unless I am missing something. Of course, converting the same UID to a username three times is not a good idea performance-wise, but that can be fixed with some minor refactoring. Another option is to emit a good error message from sendmail, and then do the security check in pickup. If a user calls postdrop directly, the reject will happen late, but my understanding is that this isn't supported. Would you be interested in a patch that implemented either of these options? I think that the envelope.from lockdown should be enforced in pickup or before pickup but not both. If it is both then the code in the pickup daemon will be a NOOP. WHen code is usually a NOOP no-one will notice when they break it. If a sender_login_maps feature can be implemented in postdrop without giving an untrusted user control over the programn, then let's try that. I would be willing to try, but I suggest we only support ?simple? maps here. Postfix supports a wide variety of map types, and I would rather not expose that much attack surface in postdrop. Furthermore, in all of the use cases I can think of, a simple policy is fine. I have no idea what that means. Postfix has a separation between lookup mechanisms (hash, pcre) and table-driven mechanisms (access maps, smtp_sender_login_maps). There was a recent vulnerability in OpenBSD due to libc malfunctioning in a set-uid-root program under very low resource limits. I would prefer to minimize the amount of third-party libraries that are used by postdrop. That said, another option would be to error out if the resource limits are below what we consider a reasonable minimum. If you are thinking of skipping the Postfix dictionary API and crafting your own table lookups, then that is definiitely not going to happen. Indeed that would be a bad idea. What about allowing everyone to send mail as themselves, and having a list of users who can send mail as anyone? That is what Sendmail provides. If a delimiter is specified in the configuration, it would be honored. By default, a user can send mail as everyone, and that default behavior cannot be changed. So the question becomes: what is the behavior? I think it should behave like smtp_sender_login_maps, because there is no reason to invent new behavior. Agreed. This also lets us use common code for both ACLs. This means the search string is the UNIX username, and the lookup result is a comma-separated list of things (usernames and/or email addresses) that they may specify as the envelope sender address. Your 'identity' mapping then looks like this: /etc/postfix/main.cf: local_sender_login_maps = pcre:/etc/postfix/local_sender_logins /etc/postfix/local_sender_logins /(.+)/ $1, $1...@example.com I.e. a user can send mail only if the envelope sender equals their login name, or usern...@example.com. There will need to be a wildcard pattern that allows all, as would be needed by a content filter that re-injects mail using the Postfix sendmail command. That looks good. I do wish there was a way (other than allow all) to express that Alice can send mail as al...@example.com, alice+t...@example.com, alice+te...@example.com, etc. I might just not know of such a method, however, and in any case that is not related to the current project. Is the code in smtpd_check.c a good place to start? We just need to make sure tha
Re: Accessing the sending user from a canonical(5) table
On 2020-10-04 19:55, Wietse Venema wrote: > Demi M. Obenour: > > Checking application/pgp-signature: FAILURE > -- Start of PGP signed section. >> On 2020-09-30 16:35, Wietse Venema wrote: >>> Demi M. Obenour: >>>> - If a message arrives via the SMTPS or submission ports, I >>>> want to replace the address part of the user-supplied From: >>>> header with the envelope From: header. This allows me to use >>>> reject-sender-login-mismatch to prevent users from sending messages >>>> with forged From: addresses. >>> >>> There are two parts to this: >>> >>> 1) Locking down the envelope.from. >>> >>>With authenticated smtp submission, the envelope.from can be >>>constrained by smtpd_sender_login_maps. >>> >>>With sendmail/postdrop submission the UNIX login name can be >>>overidden with "sendmail -f". There is no code in Postfix to >>>lock down "sendmail -f", and there is no 'plugin' interface that >>>could do this, either. I don't like the idea of adding complex >>>logic to the set-gid postdrop command to lock down "sendmail >>>-f". Doing the lockdown in the pickup daemon would be more >>>secure but has the problem that the 'reject' happens too late. >> >> I looked at the postdrop source code to see what locking down "sendmail >> -f" would entail. Checking that the current user can use `-f` seems >> to be just looking up the current username in an ACL, which postdrop >> already does for authorized_submit_users. Checking that -f was not >> passed looks to just be a string equality check, unless I am missing >> something. Of course, converting the same UID to a username three >> times is not a good idea performance-wise, but that can be fixed with >> some minor refactoring. >> >> Another option is to emit a good error message from sendmail, and then >> do the security check in pickup. If a user calls postdrop directly, >> the reject will happen late, but my understanding is that this isn't >> supported. >> >> Would you be interested in a patch that implemented either of these >> options? > > I think that the envelope.from lockdown should be enforced in pickup > or before pickup but not both. If it is both then the code in the > pickup daemon will be a NOOP. WHen code is usually a NOOP no-one will > notice when they break it. > > If a sender_login_maps feature can be implemented in postdrop > without giving an untrusted user control over the programn, then > let's try that. I would be willing to try, but I suggest we only support “simple” maps here. Postfix supports a wide variety of map types, and I would rather not expose that much attack surface in postdrop. Furthermore, in all of the use cases I can think of, a simple policy is fine. What about allowing everyone to send mail as themselves, and having a list of users who can send mail as anyone? That is what Sendmail provides. If a delimiter is specified in the configuration, it would be honored. > Note that /usr/sbin/sendmail submission path has not been optimized > for performance, so adding another getpwuid() call should not be a > deal breaker. > >>> 2) Locking down the header.from. based on rge envelope.from. >>> >>> You need a way to restrict the values of header.from that may >>> be used with a given envelope.from. There is no such code >>> Postfix, but this can be done with a plugin such as a Milter. >> >> It looks like this can be implemented (without changes to Postfix >> itself) by using header_checks(5) to ignore the From: header. >> cleanup(8) will then insert its own From: header. >> >> Is this a good idea? It worked for me when I used sendmail(1). > > This will break email that legitmately overrides the envelope sender > address, such as mailing list managers. Indeed it would, which is why I was hoping to only specify it for mail submitted via the submission service, rather than mail coming on port 25. That said, for mail not generated locally, this might lose phrases in From: headers. OpenSMTPD can be configured to replace the From: header address with the envelope MAIL FROM address, and hopefully this would not be too hard to add to Postfix. > You could specify "pickup -o cleanup_service=local_cleanup" and > define a custome cleanup service with a custom header_checks action. Good idea, thanks! > Wietse Thank you, Demi signature.asc Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 2020-09-30 16:35, Wietse Venema wrote: > Demi M. Obenour: >> - If a message arrives via the SMTPS or submission ports, I >> want to replace the address part of the user-supplied From: >> header with the envelope From: header. This allows me to use >> reject-sender-login-mismatch to prevent users from sending messages >> with forged From: addresses. > > There are two parts to this: > > 1) Locking down the envelope.from. > >With authenticated smtp submission, the envelope.from can be >constrained by smtpd_sender_login_maps. > >With sendmail/postdrop submission the UNIX login name can be >overidden with "sendmail -f". There is no code in Postfix to >lock down "sendmail -f", and there is no 'plugin' interface that >could do this, either. I don't like the idea of adding complex >logic to the set-gid postdrop command to lock down "sendmail >-f". Doing the lockdown in the pickup daemon would be more >secure but has the problem that the 'reject' happens too late. I looked at the postdrop source code to see what locking down "sendmail -f" would entail. Checking that the current user can use `-f` seems to be just looking up the current username in an ACL, which postdrop already does for authorized_submit_users. Checking that -f was not passed looks to just be a string equality check, unless I am missing something. Of course, converting the same UID to a username three times is not a good idea performance-wise, but that can be fixed with some minor refactoring. Another option is to emit a good error message from sendmail, and then do the security check in pickup. If a user calls postdrop directly, the reject will happen late, but my understanding is that this isn't supported. Would you be interested in a patch that implemented either of these options? > 2) Locking down the header.from. based on rge envelope.from. > > You need a way to restrict the values of header.from that may > be used with a given envelope.from. There is no such code > Postfix, but this can be done with a plugin such as a Milter. It looks like this can be implemented (without changes to Postfix itself) by using header_checks(5) to ignore the From: header. cleanup(8) will then insert its own From: header. Is this a good idea? It worked for me when I used sendmail(1). Doing it on mails received via SMTP would not be a good idea, and I don’t see a non_smtp_header_checks, but there might be an alternative that I am missing. Thank you for your time and effort. Sincerely, Demi signature.asc Description: OpenPGP digital signature
Re: strange issue with postfix
On 2020-10-01 15:18, Ranjan Maitra wrote: > Thanks, very much. So when I hit "Send" on sylpheed, it goes on a tailspin, > and says: Connecting to SMTP server: localhost > > Looking at the /var/log/maillog as you suggested, I get: > > Oct 1 14:08:00 localhost postfix/smtpd[4142479]: fatal: in parameter > smtpd_relay_restrictions or smtpd_recipient_restrictions, specify at least > one working instance of: reject_unauth_destination, defer_unauth_destination, > reject, defer, defer_if_permit or check_relay_domains That is your problem. You haven’t told Postfix what restrictions it should impose on mail relaying, so it exits to avoid becoming an open relay. Demi signature.asc Description: OpenPGP digital signature
Re: Mailrelay: wait for downstream response
On 2020-10-01 08:19, luc...@dds.nl wrote: > Hello, > > I am managing a Postfix mail relay service in our internal network. The > relay itself is more permissive than the downstream SMTP server. So it > is possible, and indeed it regularly happens, that my relay accepts a > message which is subsequently bounced by the downstream server. There is > no way to inform the original sender what happened to the message after > the relay accepted it. > > Is it possible to delay closing the SMTP connection from the original > sender until the response from the downstream server is received, and > then respond accordingly in the original connection? > > I could imagine this would imply some complications in terms of > performance, and the question what happens if downstream is not > available. But in the way it would appear the advantage of telling the > original sender what happened to a message, outweighs the disadvantages. This seems to be a job for a reverse proxy, such as NGINX (which already supports SMTP). NGINX is event-driven, so it can handle thousands of connections without a significant performance hit. > Regards, > > Lucas Sincerely, Demi signature.asc Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 2020-09-30 20:32, Jaroslaw Rafa wrote: > Dnia 30.09.2020 o godz. 16:35:37 Wietse Venema pisze: >>With authenticated smtp submission, the envelope.from can be >>constrained by smtpd_sender_login_maps. >> >>With sendmail/postdrop submission the UNIX login name can be >>overidden with "sendmail -f". There is no code in Postfix to >>lock down "sendmail -f", and there is no 'plugin' interface that >>could do this, either. I don't like the idea of adding complex >>logic to the set-gid postdrop command to lock down "sendmail >>-f". Doing the lockdown in the pickup daemon would be more >>secure but has the problem that the 'reject' happens too late. > > Slightly off topic, but the original sendmail when "-f" parameter was used > added the following header to the sent message: > > X-Authentication-Warning: : set sender to > using -f > > was of course the user who was calling > /usr/sbin/sendmail. Is Postfix able to do similar thing? This would meet my requirements as well; my understanding is that blocking a message containing a certain header is trivial. Demi signature.asc Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 2020-09-30 12:18, @lbutlr wrote: > On 30 Sep 2020, at 10:04, Demi M. Obenour wrote: >> while www-data can only send mail as majordomo, > > That will simply brea mailing list. Look at the headers from this message, > for example. Your policy on the postfox.org server would screw up this list. My policy was meant as an example, not as one that should be put into production. Different sites will need different policies. Sorry for the confusion. Sincerely, Demi signature.asc Description: OpenPGP digital signature
Re: Accessing the sending user from a canonical(5) table
On 2020-09-30 10:08, Wietse Venema wrote: > Demi M. Obenour: > > Checking application/pgp-signature: FAILURE >> When a message is submitted using postdrop, Postfix is obviously aware >> of which user submitted it, as it includes the UID in the Received: >> header. Is it possible to use this information in a canonical(5) >> table, or is a milter required? >> >> Thank you, > > You mean the numerical UID that that is shown in a comment: > > Received: by mail.example.com (Postfix, from userid 1001) > id 4C1dKq2WvyzJrNw; Wed, 30 Sep 2020 10:04:31 -0400 (EDT) > > Postfix address rewriting is limited to headers that contain only > addresses: From:, To:, Cc:, Reply-To:, and the like. And address > rewriting never looks at the content of comments. > > For everything else Postfix can only replace entire headers (through > header checks or milter header delete/insert actions). > > Wietse Darn. I was hoping that I could get by without a milter, but now it is clear that a milter will be needed, at least with current Postfix. How difficult would it be to implement this natively in Postfix? More specifically: - If a message arrives via the SMTPS or submission ports, I want to replace the address part of the user-supplied From: header with the envelope From: header. This allows me to use reject-sender-login-mismatch to prevent users from sending messages with forged From: addresses. - I want to create a table that maps the local user to the addresses that they can send mail through Postfix as. As an example, I might want Alice to be able to send mail as anyone, while www-data can only send mail as majordomo, and everyone else can only send mail as themselves. Ideally, this should work both for mail submitted with postdrop(1), and for mail submitted via SMTP over AF_UNIX sockets. If this belongs in Postfix, then I would like to request it as a feature. However, a milter might be the best option, and that is also fine. I am no expert on mail servers. Sincerely, Demi signature.asc Description: OpenPGP digital signature
Accessing the sending user from a canonical(5) table
When a message is submitted using postdrop, Postfix is obviously aware of which user submitted it, as it includes the UID in the Received: header. Is it possible to use this information in a canonical(5) table, or is a milter required? Thank you, Demi signature.asc Description: OpenPGP digital signature
