Does Postfix cache resolv.conf?
I had a quite strange issue. About a week ago my bind9 broke down and I could not get it running again on the same machine. So moved it to another machine and changed the /etc/resolv.conf of my machines to try both IP. Apparently everything worked fine. Today I was puzzled that the corresponding bug-report to the Debian list was somehow missing. I resent it watching the postfix logs and found that potfix was missing the MX entry of my relay host and refused to send. Since the host itself actually does not have a MX entry, I was sidetracked assuming postfix was not smart enough to strip the host name from the domain. During this trouble shooting I had postfix reload its configuration a couple of times. After setting the name in [] postfix reported that the A entry was missing, which definitely was wrong. I restarted postfix and voilá it continued working like it did all the years before. Now I know that it is smart enough to strip the relay host name from the domain to lookup MX. ;) Apparently postfix missed the switching of nameservers and did not learn of the new DNS until restart. Is this a bug or a feature? Postfix Version: 2.5.5 (Current Debian stable) Regards, - lars.
Re: Does Postfix cache resolv.conf?
* Dr. Lars Hanke l...@lhanke.de: I had a quite strange issue. About a week ago my bind9 broke down and I could not get it running again on the same machine. So moved it to another machine and changed the /etc/resolv.conf of my machines to try both IP. Apparently everything worked fine. Today I was puzzled that the corresponding bug-report to the Debian list was somehow missing. I resent it watching the postfix logs and found that potfix was missing the MX entry of my relay host and refused to send. Since the host itself actually does not have a MX entry, I was sidetracked assuming postfix was not smart enough to strip the host name from the domain. During this trouble shooting I had postfix reload its configuration a couple of times. After setting the name in [] postfix reported that the A entry was missing, which definitely was wrong. I restarted postfix and voilá it continued working like it did all the years before. Now I know that it is smart enough to strip the relay host name from the domain to lookup MX. ;) Apparently postfix missed the switching of nameservers and did not learn of the new DNS until restart. Is this a bug or a feature? Postfix Version: 2.5.5 (Current Debian stable) The Debian packages of Postfix are running smtpd in a chroot by default. The files necessary for this are copied by the init script /etc/init.d/postfix - and amongst them is the resolv.conf you changed. I guess what happened is: 1. You didn't disable smtpd's chroot in master.cf. 2. You changed /etc/resolv.conf without copying it to the chroot. 3. You restarted Postfix using Debian's init script which copied the changed resolv.conf file to the chroot. Stefan
Anyone else had this issue with maildrop?
Hi,
maildrop seems to reject any efforts on my part, take a look at this:
Tried using maildrop with postfix as described in the second part of [1], but
it failed. strace showed me:
connect(3, {sa_family=AF_FILE, path=/var/lib/courier/authdaemon/socket...},
110) = -1 ENOENT (No such file
or directory).
Then I tried to copy another socket with name socket, but it
failed with:
connect(3, {sa_family=AF_FILE,
path=/var/lib/courier/authdaemon/socket...}, 110) = -1 ECONNREFUSED
(Connection refused).
I'm confused about this, what is wrong? Apparently sockets can't be created as
easy as pipes (excuse me, what commandline tool cares for that?).
Maybe someone in here had to solve this problem before?
[1] http://www.postfix.org/MAILDROP_README.html
signature.asc
Description: This is a digitally signed message part.
Re: Anyone else had this issue with maildrop?
Kārlis Repsons a écrit :
Hi,
maildrop seems to reject any efforts on my part, take a look at this:
Tried using maildrop with postfix as described in the second part of [1], but
it failed. strace showed me:
connect(3, {sa_family=AF_FILE,
path=/var/lib/courier/authdaemon/socket...},
110) = -1 ENOENT (No such file
or directory).
Then I tried to copy another socket with name socket, but it
failed with:
connect(3, {sa_family=AF_FILE,
path=/var/lib/courier/authdaemon/socket...}, 110) = -1 ECONNREFUSED
(Connection refused).
I'm confused about this, what is wrong? Apparently sockets can't be created
as
easy as pipes (excuse me, what commandline tool cares for that?).
Maybe someone in here had to solve this problem before?
sockets are communication end points. a server creates a socket where
clients can connect to and talk to the server...
Your maildrop needs to contact the courier authdaemon server. Make sure,
authdaemon is running and correctly configured. if you don't need it,
you'll need to rebuild maildrop without authlib support.
anyway, this is not postfix related. maildrop is part of the courier
suit. you'll get more help on the courier-maildrop mailing list.
[1] http://www.postfix.org/MAILDROP_README.html
Re: Limitations of smtpd_proxy_filter
Stefan Foerster: So, my question is: Are there any plans to further enhance Postfix's functionality as far as SMTP proxy filtering is concerned, and make that kind of load balancing available to pre-queue setups using smtpd_proxy_filter? For TRUE load balancing you need a stateful system that remembers which connections are going where. You could try the nginx reverse proxy; it speaks SMTP among many protocols, and is often used for software-based load balancing. For STATISTICAL load balancing you can get by with multiple IP addresses per proxy filter host name. However this requires new code in Postfix to randomize the address order. You may know that the Postfix smtpd_proxy client does not make its own SMTP connections. Instead it uses the same Postfix library functions for making a connection over TCP, UNIX-domain sockets, System V streams, etc. as all other Postfix programs (except the SMTP client, which does its own name-to-address translation). Changing this library means changing applications that call into this library (Postfix invokes the _connect() functions via function pointers, so a simple #define old(x,y,z) new(x,y,z,0) won't do the job. I have a bunch of deliverables in the second and third quarters of 2010, so I must get Postfix 2.7 out in the first quarter. This means I have few cycles left for Postfix 2.7. I would like to spend those on postscreen. Wietse
Re: Does Postfix cache resolv.conf?
On Sun, 10 Jan 2010 11:32:34 +0100 Dr. Lars Hanke l...@lhanke.de wrote: I had a quite strange issue. About a week ago my bind9 broke down and I could not get it running again on the same machine. So moved it to another machine and changed the /etc/resolv.conf of my machines to try both IP. Apparently everything worked fine. Today I was puzzled that the corresponding bug-report to the Debian list was somehow missing. I resent it watching the postfix logs and found that potfix was missing the MX entry of my relay host and refused to send. Since the host itself actually does not have a MX entry, I was sidetracked assuming postfix was not smart enough to strip the host name from the domain. During this trouble shooting I had postfix reload its configuration a couple of times. After setting the name in [] postfix reported that the A entry was missing, which definitely was wrong. I restarted postfix and voilá it continued working like it did all the years before. Now I know that it is smart enough to strip the relay host name from the domain to lookup MX. ;) Apparently postfix missed the switching of nameservers and did not learn of the new DNS until restart. Is this a bug or a feature? This is true of most services, not just postfix. They will read /etc/resolv.conf at startup and not again unless told to do so -- John
Re: Limitations of smtpd_proxy_filter
* Wietse Venema wie...@porcupine.org: For STATISTICAL load balancing you can get by with multiple IP addresses per proxy filter host name. However this requires new The main difference I see here is that delivery to a content_filter will try more than one server, giving us not only (statistical) load balancing, but also some kind of redundancy. code in Postfix to randomize the address order. You may know that the Postfix smtpd_proxy client does not make its own SMTP connections. Instead it uses the same Postfix library functions for making a connection over TCP, UNIX-domain sockets, System V streams, etc. as all other Postfix programs (except the SMTP client, which does its own name-to-address translation). Changing this library means changing applications that call into this library (Postfix invokes the _connect() functions via function pointers, so a simple #define old(x,y,z) new(x,y,z,0) won't do the job. I have a bunch of deliverables in the second and third quarters of 2010, so I must get Postfix 2.7 out in the first quarter. This means I have few cycles left for Postfix 2.7. I would like to spend those on postscreen. Understood. I can live with deploying load balancers. Thanks anyways! Stefan
Re: Limitations of smtpd_proxy_filter
Stefan Foerster: * Wietse Venema wie...@porcupine.org: For STATISTICAL load balancing you can get by with multiple IP addresses per proxy filter host name. However this requires new The main difference I see here is that delivery to a content_filter will try more than one server, giving us not only (statistical) load balancing, but also some kind of redundancy. That would work only when the smtpd_proxy client is modified to use the SMTP connection cache. Otherwise, a connection still has to time out before the next host is tried. (*) There is no such performance hit with true load balancers, which maintain state about which hosts are busy/dead/other. Wietse (*) It takes major surgery to make the smtpd_proxy client as smart as the Postfix SMTP client (address randomization, connection caching, bug workarounds, and more) Namely, to make the Postfix SMTP client usable as an smtpd_proxy client.
Re: Does Postfix cache resolv.conf?
Dr. Lars Hanke: Apparently postfix missed the switching of nameservers and did not learn of the new DNS until restart. Is this a bug or a feature? Like most programs, Postfix never reads /etc/resolv.conf. Instead, that file is read by the NSSWITCH system library functions, and the DNS system library functions. Postfix does not run on bare metal - instead it runs on an operating system that provides libraries and other runtime support. Wietse
OT: How do mail clients discover submission and imap servers
Hi List, This is not a real postfix related question… We offer e-mail services to our customers and some of our customers have a hard time configuring their mail client. I noticed that most mail clients try to 'find' the correct settings when an account is configured. So the mail client (at least the ones I've tried) only ask for a friendly username, mail address and password and then they try to find the servers. First I thought this was done by requesting the appropriate SRV records in DNS so I set them up for our domain (youngguns.nl) and tried to configure an account in Thunderbird with no success. So mail clients don't seem to use SRV records. Does anybody know what technique is behind the auto-discovery? thanks, Martijn
Re: OT: How do mail clients discover submission and imap servers
On Jan 10, 2010, at 9:24 PM, Eero Volotinen wrote: On 1/10/10 10:17 PM, Martijn de Munnik wrote: Hi List, This is not a real postfix related question… We offer e-mail services to our customers and some of our customers have a hard time configuring their mail client. I noticed that most mail clients try to 'find' the correct settings when an account is configured. So the mail client (at least the ones I've tried) only ask for a friendly username, mail address and password and then they try to find the servers. First I thought this was done by requesting the appropriate SRV records in DNS so I set them up for our domain (youngguns.nl) and tried to configure an account in Thunderbird with no success. So mail clients don't seem to use SRV records. Does anybody know what technique is behind the auto-discovery? Well, it looks like fuzzy logic: some dns lookups + common ports on mail, mx, smtp -prefixed hostnames? I was hoping that was not the answer ;) We try to let our customers use the submission port and imaps port. We have a SSL certificate for secure.youngguns.nl but it seems most mail clients are just trying mail.example.com on smtp en imap ports if the the mail address is custo...@example.com -- Eero
Re: OT: How do mail clients discover submission and imap servers
On 1/10/10 10:32 PM, Martijn de Munnik wrote: On Jan 10, 2010, at 9:24 PM, Eero Volotinen wrote: On 1/10/10 10:17 PM, Martijn de Munnik wrote: Hi List, This is not a real postfix related question… We offer e-mail services to our customers and some of our customers have a hard time configuring their mail client. I noticed that most mail clients try to 'find' the correct settings when an account is configured. So the mail client (at least the ones I've tried) only ask for a friendly username, mail address and password and then they try to find the servers. First I thought this was done by requesting the appropriate SRV records in DNS so I set them up for our domain (youngguns.nl) and tried to configure an account in Thunderbird with no success. So mail clients don't seem to use SRV records. Does anybody know what technique is behind the auto-discovery? Well, it looks like fuzzy logic: some dns lookups + common ports on mail, mx, smtp -prefixed hostnames? I was hoping that was not the answer ;) We try to let our customers use the submission port and imaps port. We have a SSL certificate for secure.youngguns.nl but it seems most mail clients are just trying mail.example.com on smtp en imap ports if the the mail address is custo...@example.com see: https://bugzilla.mozilla.org/show_bug.cgi?id=450710 and https://wiki.mozilla.org/Thunderbird:Autoconfiguration -- Eero
Re: Huge active queue and system idle, not delivering
Wietse, Please try the following, as asked half a week ago: postconf -e smtp_connection_cache_on_demand=no postfix reload and report if this makes a difference. Wietse I have tested this since yesterday night. I got some problems with Linux per user number of processes limit. I fixed it. I also increased some delivery concurrency figures, and now I can see up to 1300 processes delivering emails to the qmail servers. I had a few minutes shot today at a rate of 6300 emails per minute. I ran a full hour at 180,000 emails per hour. The outbound line was saturated. CPU is about 30% loaded, no Wait I/O, no swap, memory is large. I think I will reach about 600,000 emails per hour if I fix some timeout on the qmails (replace by postfix?). Maybe I could reach 1 million? The full architecture that I plan will include 2 to 3 clustered postfix relays and 50 2nd level qmails(or postfix) delivery servers, each with 3 to 5 IP addresses, and upgraded outbound internet connection. With your help, I better understand now the impact of timeout and concurrency parameters. In fact, delivery was blocked because postfix was trying to reuse connections, so was waiting each email to complete to send the next one. Also, because hundreds processes were created at start time to manage inbound messages, there were no slots to fork processes to deliver messages on the other hand. Same problem caused very slow DNS and EHLO, because no available slots to fork. Of course, if you want me to post my conf, I will with pleasure. Many thanks to you, to Victor and Stan. Patrick
Re: Huge active queue and system idle, not delivering
Patrick Chemla put forth on 1/10/2010 3:00 PM: Wietse, Please try the following, as asked half a week ago: postconf -e smtp_connection_cache_on_demand=no postfix reload and report if this makes a difference. Wietse I have tested this since yesterday night. I got some problems with Linux per user number of processes limit. I fixed it. I also increased some delivery concurrency figures, and now I can see up to 1300 processes delivering emails to the qmail servers. I had a few minutes shot today at a rate of 6300 emails per minute. I ran a full hour at 180,000 emails per hour. The outbound line was saturated. CPU is about 30% loaded, no Wait I/O, no swap, memory is large. I think I will reach about 600,000 emails per hour if I fix some timeout on the qmails (replace by postfix?). Maybe I could reach 1 million? The full architecture that I plan will include 2 to 3 clustered postfix relays and 50 2nd level qmails(or postfix) delivery servers, each with 3 to 5 IP addresses, and upgraded outbound internet connection. With your help, I better understand now the impact of timeout and concurrency parameters. In fact, delivery was blocked because postfix was trying to reuse connections, so was waiting each email to complete to send the next one. Also, because hundreds processes were created at start time to manage inbound messages, there were no slots to fork processes to deliver messages on the other hand. Same problem caused very slow DNS and EHLO, because no available slots to fork. Of course, if you want me to post my conf, I will with pleasure. Many thanks to you, to Victor and Stan. Patrick On a technical level I'm happy you got it working. Just please tell us you're not sending mass spam with this setup. -- Stan
Re: Huge active queue and system idle, not delivering
Patrick Chemla: Wietse, Please try the following, as asked half a week ago: postconf -e smtp_connection_cache_on_demand=no postfix reload and report if this makes a difference. Wietse I have tested this since yesterday night. I got some problems with Linux per user number of processes limit. I fixed it. I also increased some delivery concurrency figures, and now I can see up to 1300 processes delivering emails to the qmail servers. I had a few minutes shot today at a rate of 6300 emails per minute. I ran a full hour at 180,000 emails per hour. The outbound line was saturated. CPU is about 30% loaded, no Wait I/O, no swap, memory is large. I think I will reach about 600,000 emails per hour if I fix some timeout on the qmails (replace by postfix?). Maybe I could reach 1 million? OK, so you can turn back on that connection caching. Note that qmail creates and destroys two processes per SMTP session, so reusing a session is also a win from a CPU resource point of view. 1M/hour, or less than 300/s, should be possible (you mention the queue is on a solid-state disk). Barring brain damage such as synchronous syslogging by default on some Linux boxes, borked DNS, process/file/etc. resource limits, etc. Perhaps this is a good time to mention that SecurityFocus was ezmlm-qmail based, and that they switched to Postfix for outbound deliveries, because qmail simply could not keep up with the volume: inbound mail - qmail - ezmlm - multiple postfix MTAs - internet That was 2001 when I added QMQP support to Postfix, and this is still what they appear to be using now, if I must believe their own Received: message headers. Received: from lists2.securityfocus.com (lists2.securityfocus.com [205.206.231.20]) by outgoing2.securityfocus.com (Postfix) with QMQP id 8AC0814370A; Thu, 7 Jan 2010 14:11:35 -0700 (MST) My very first qmail/Postfix benchmarks showed that qmail was up to three times slower as a transit MTA, simply because qmail creates three queue files where Postfix creates one. Creating/deleting files involves more disk access operations than reading/writing files, and that hurts especially with small email messages. Wietse
Re: Limitations of smtpd_proxy_filter
Stefan Foerster: One of the greatest improvements in the 2.7 tree is the ability to defer transmission of received messages to a SMTP proxy until the message receiption completes (smtpd_proxy_options = speed_adjust). Can you be more specific about the benefits? Wietse
Re: Huge active queue and system idle, not delivering
Le 10/01/2010 23:58, Stan Hoeppner a écrit : On a technical level I'm happy you got it working. Just please tell us you're not sending mass spam with this setup. -- Stan I have to do it for a customer who send as he said, only opt-in mass emails. He has a big blacklisted email database where he keeps all unsubscribe messages. He said he has the right filters not to send unwanted emails. Thanks Patrick
Re: Huge active queue and system idle, not delivering
Le 11/01/2010 01:13, Wietse Venema a écrit : Patrick Chemla: Wietse, Please try the following, as asked half a week ago: postconf -e smtp_connection_cache_on_demand=no postfix reload and report if this makes a difference. Wietse I have tested this since yesterday night. I got some problems with Linux per user number of processes limit. I fixed it. I also increased some delivery concurrency figures, and now I can see up to 1300 processes delivering emails to the qmail servers. I had a few minutes shot today at a rate of 6300 emails per minute. I ran a full hour at 180,000 emails per hour. The outbound line was saturated. CPU is about 30% loaded, no Wait I/O, no swap, memory is large. I think I will reach about 600,000 emails per hour if I fix some timeout on the qmails (replace by postfix?). Maybe I could reach 1 million? OK, so you can turn back on that connection caching. Note that qmail creates and destroys two processes per SMTP session, so reusing a session is also a win from a CPU resource point of view. Wietse If I do so, will postfix open more than one connexion to each qmail for parallel deliveries? I am afraid that if we use connection caching this will create a single queue on each qmail. As far as I have available resources, I think prefer parallel deliveries. Patrick
Re: Huge active queue and system idle, not delivering
Patrick Chemla put forth on 1/11/2010 1:02 AM: Le 10/01/2010 23:58, Stan Hoeppner a écrit : On a technical level I'm happy you got it working. Just please tell us you're not sending mass spam with this setup. -- Stan I have to do it for a customer who send as he said, only opt-in mass emails. He has a big blacklisted email database where he keeps all unsubscribe messages. He said he has the right filters not to send unwanted emails. Sigh... This doesn't pass the sniff test. I fear we've helped enable the sending of mass UBE. Patrick would you mind providing the IP netblock(s) you will be sending these mass mailings from? Or provide them to me off list please? Thanks. -- Stan
Re: Huge active queue and system idle, not delivering
Le 11/01/2010 09:27, Stan Hoeppner a écrit : Patrick Chemla put forth on 1/11/2010 1:02 AM: Le 10/01/2010 23:58, Stan Hoeppner a écrit : On a technical level I'm happy you got it working. Just please tell us you're not sending mass spam with this setup. -- Stan I have to do it for a customer who send as he said, only opt-in mass emails. He has a big blacklisted email database where he keeps all unsubscribe messages. He said he has the right filters not to send unwanted emails. Sigh... This doesn't pass the sniff test. I fear we've helped enable the sending of mass UBE. Patrick would you mind providing the IP netblock(s) you will be sending these mass mailings from? Or provide them to me off list please? Thanks. -- Stan Don't be afraid Stan. They work only on french market, maybe also on french people who have a mailbox overseas. You have very very very low chance to be concerned. Patrick
reject_authenticated_sender_login_mismatch
As follows smtpd_recipient_restrictions = reject_invalid_hostname, reject_non_fqdn_sender, reject_non_fqdn_recipient, reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, check_recipient_access hash:/etc/postfix/access, permit_mynetworks, reject_authenticated_sender_login_mismatch mysql:/etc/postfix/mysql-sender-access.cf reject_unauth_destination mysql-sender-access.cf hosts = localhost user = USER password = PASS dbname = system query = SELECT username FROM mailboxes WHERE email='%s' AND active='1' The error returned is: RECEIVER ADDRESS (The server responded: 5.7.1 SENDER ADDRESS: Sender address rejected: not owned by user SASL USER) What;s going on? Running the query directly against the SQL database returns the desired result.
