Re: Mail stuck (Connection Timed-Out)
I was reading about Defferred queue full of dictionary attack bounces which I think might be an issue here. So i performed a qshape analysis and I got this: command: qshape deferred | head T 5 10 20 40 80 160 320 640 1280 1280+ TOTAL 583 0 0 0 0 0 1 8 47 25 502 adbaa.org 214 0 0 0 0 0 0 0 00 214 onramp.bz 191 0 0 0 0 0 1 6 26 10 148 unitedimagingpartners.com 62 0 0 0 0 0 0 0 7550 mmvacations.com 26 0 0 0 0 0 0 0 1025 fishwindowcleaning.com 12 0 0 0 0 0 0 0 31 8 warrensouth.com 5 0 0 0 0 0 0 0 10 4 ecodiscoverypark.com 5 0 0 0 0 0 0 1 00 4 pfg.com 4 0 0 0 0 0 0 0 12 1 Luckily the active and and incoming queues aren't showing any signs of backscatter. I'm going to be checking firewall tomorrow to see if there's an issue there. Thanks for your help. Gonzo Fernandez On Jan 30, 2012, at 9:09 PM, Noel Jones wrote: On 1/30/2012 10:30 PM, Noel Jones wrote: On 1/30/2012 6:46 PM, Gonzo Fernandez wrote: Thank you Noel. Our server sends out copies of email confirmations to our clients and if the client decides to make a large order they end up pushing our volume up and we end up getting blocked by their mail server. I seem to be getting connection timed out on a lot of the hosts. I even try to telnet to ip and port 25 but it keeps timing out. I used grep to search in /var/log/maillog and I got this. Any ideas? This isn't a postfix problem. If you can't telnet to any client port 25, then you have a connectivity problem; maybe your ISP is blocking that port, or some firewall has been misconfigured. Contact your networking team or your ISP. If you can't telnet to this one destination port 25, they're blocking you. You'll need to contact them to get this resolved. Good luck. -- Noel Jones If the destination can't handle the load you're sending it, you can slow postfix down. Details here: http://www.postfix.org/QSHAPE_README.html#active_congestion Of course, this won't help until they stop blocking you. -- Noel Jones
Re: Postfix Not Sending Emails. Timeout on Cleanup socket error.
Hi guys, So far no luck searching for 8A2993E3003B. :( The system is heavily used. From what I understand this machine relays all mail to our main mx server and we have a lot of email being relayed daily. The strange part is that it works all week up until late Saturday/Sunday in the AM hours. Here are all log files for mail. I have searched all log files for the specific string but no luck. Performing another search for all warning, error, fatal, panic on all previous log files has given me a bit more info. The following is certain errors I have noticed pop up more often than not: # egrep (warning|error|fatal|panic): /var/log/maillog* | more /var/log/maillog:Dec 11 05:09:19 batch-ca4-02 postfix/cleanup[31691]: warning: timeout on cleanup socket while reading input attribute name /var/log/maillog:Dec 11 05:31:27 batch-ca4-02 postfix/cleanup[31691]: warning: 8A2993E3003B: read timeout on cleanup socket /var/log/maillog:Dec 13 12:04:13 batch-ca4-02 postfix/postdrop[15153]: warning: uid=0: Illegal seek /var/log/maillog:Dec 13 04:04:13 batch-ca4-02 postfix/sendmail[15152]: fatal: root(0): queue file write error /var/log/maillog:Dec 14 12:03:07 batch-ca4-02 postfix/postdrop[12285]: warning: uid=0: Illegal seek /var/log/maillog:Dec 14 04:03:07 batch-ca4-02 postfix/sendmail[12280]: fatal: root(0): queue file write error /var/log/maillog:Dec 15 12:03:07 batch-ca4-02 postfix/postdrop[21744]: warning: uid=0: Illegal seek /var/log/maillog:Dec 15 04:03:07 batch-ca4-02 postfix/sendmail[21739]: fatal: root(0): queue file write error /var/log/maillog.1:Dec 4 05:18:29 batch-ca4-02 postfix/cleanup[6555]: warning: timeout on cleanup socket while reading input attribute name /var/log/maillog.1:Dec 4 05:45:13 batch-ca4-02 postfix/cleanup[6555]: warning: 0433D3E3003A: read timeout on cleanup socket /var/log/maillog:Dec 15 12:03:07 batch-ca4-02 postfix/postdrop[21744]: warning: uid=0: Illegal seek /var/log/maillog:Dec 15 04:03:07 batch-ca4-02 postfix/sendmail[21739]: fatal: root(0): queue file write error /var/log/maillog.1:Dec 5 12:03:47 batch-ca4-02 postfix/postdrop[4667]: warning: uid=0: Illegal seek /var/log/maillog.1:Dec 5 04:03:47 batch-ca4-02 postfix/sendmail[4660]: fatal: root(0): queue file write error (I tried searching for 0433D3E3003A as well but to no avail.) # ls -ld /var/log/maillog* -rw--- 1 root root 194889523 Dec 15 10:48 /var/log/maillog -rw--- 1 root root 370337428 Dec 11 04:04 /var/log/maillog.1 -rw--- 1 root root 644886321 Dec 4 04:03 /var/log/maillog.2 -rw--- 1 root root 250479845 Nov 27 04:03 /var/log/maillog.3 -rw--- 1 root root 11139266 Nov 20 04:02 /var/log/maillog.4 I also commented out fallback relay as it was active but left blank in master.cf. Not sure if that has anything to do with it but better safe than sorry. # When relaying mail as backup MX, disable fallback_relay to avoid MX loops relay unix - - n - - smtp # -o fallback_relay= I'm going to wait and see what happens this weekend and when it happens again I'll try to get more info for you. Thanks again for the help. Gonzo Fernandez Network Engineer On Dec 14, 2011, at 11:01 PM, Lorens Kockum wrote: On Wed, Dec 14, 2011 at 05:43:58PM -0800, Gonzo Fernandez wrote: I'm not sure I'm understanding the log file info you would like. I tried doing grep search for 8A2993E3003B on all log files under /var/log/* and only found the following line to show up: Dec 11 05:31:27 batch-ca4-02 postfix/cleanup[31691]: warning: 8A2993E3003B: read timeout on cleanup socket Can you please be more specific? Thanks for your patience and help. Your log line is dated 5 AM plus a little. Maybe your log file got rotated (that usually happens at 4 or 5 AM). The reason that your grep on all log files did not yield the result is probably that the rotated log files are zipped. zgrep (at least my version) behaves nicely when the files aren't zipped, so zgrep 8A2993E3003B /var/log/maillog* should yield the information Wietse was asking for. If not, an ls -ld /var/log/maillog* would be useful. Is this a very heavily used system?
Re: Postfix Not Sending Emails. Timeout on Cleanup socket error.
=500 from=x...@.com Dec 11 04:09:19 batch-ca4-02 postfix/cleanup[31691]: 7905B3E3003A: message-id=20111211120919.7905B3E3003A@batch-ca4-02 Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: trigger_server_accept_fifo: trigger arrived Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: master_notify: status 0 Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: request: 87 (W) Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: qmgr_scan_start: start incoming queue scan Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: master_notify: status 1 Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: qmgr_active_feed: queue incoming Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: qmgr_active_feed: incoming/7905B3E3003A Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: qmgr_message_alloc: active 7905B3E3003A Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: 7905B3E3003A: from=batc...@usaepay.com, size=1084, nrcpt=1 (queue active) Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: start sorted recipient list Dec 11 04:09:19 batch-ca4-02 postfix/qmgr[28082]: qmgr_message_sort: x...@.com Dec 11 04:09:39 batch-ca4-02 postfix/smtpd[31631]: rewrite stream disconnect Dec 11 04:09:39 batch-ca4-02 postfix/smtp[31689]: idle timeout -- exiting Dec 11 04:16:47 batch-ca4-02 postfix/smtpd[31631]: idle timeout -- exiting Dec 11 05:09:19 batch-ca4-02 postfix/cleanup[31691]: warning: timeout on cleanup socket while reading input attribute name Dec 11 05:30:09 batch-ca4-02 postfix/smtpd[5663]: dict_eval: const mail Dec 11 05:31:27 batch-ca4-02 postfix/cleanup[31691]: warning: 8A2993E3003B: read timeout on cleanup socket Dec 11 05:53:51 batch-ca4-02 postfix/smtpd[5663]: dict_eval: const ipv4 I know I originally stated that the email is from the pickup but now I'm not so sure. I'm seeing a lot of postfix/smtp verbose messages. I hope this helps. Gonzo Fernandez Network Engineer On Dec 15, 2011, at 2:07 PM, Lorens Kockum wrote: On Thu, Dec 15, 2011 at 11:49:35AM -0800, Gonzo Fernandez wrote: /var/log/maillog:Dec 14 04:03:07 batch-ca4-02 postfix/sendmail[12280]: fatal: root(0): queue file write error /var/log/maillog:Dec 15 12:03:07 batch-ca4-02 postfix/postdrop[21744]: warning: uid=0: Illegal seek In addition to what Wietse asked for in another mail just now, do look at the output of the commands mount dmesg That should show if there is a hardware problem. If it is not, then is it not possible that the 8A2993E3003B you are worrying about is a locally submitted mail from some system daemon (like logwatch) that is too big for your configured system limits? The times on the error messages make me suspect some kind of cron job. See: http://www.electrictoolbox.com/logwatch-postfix-sendmail-errors/ http://www.cyberciti.biz/tips/linux-unix-posttfix-mutt-illegal-seek-error.html 8A2993E3003B would then be a minor consequence of your main problem, which is mail freezing up over weekends. To diagnose that problem, it would be interesting to grep on a queue id that was correctly queued, like (from an earlier mail): Dec 12 10:08:50 batch-ca4-02 postfix/pickup[26935]: warning: 884643E30022: message has been queued for 1 days Since mails such as those have been correctly queued, there should be a wealth of information about when and how and why it's staying in the queue so long. HTH
Re: Postfix Not Sending Emails. Timeout on Cleanup socket error.
Hi Wietse, The UID 500 is from our batch email not spam. It constantly sends out emails to our clients on a regular basis. Also, I turned off verbose logging on all options. I originally turned it on to try and figure out why it was getting stuck on the weekends, but now it is off. I will see if that helps this weekend. Thank you. Gonzo Fernandez Network Engineer On Dec 15, 2011, at 5:35 PM, Wietse Venema wrote: Gonzo Fernandez: # egrep 884643E30022 /var/log/maillog Dec 12 10:08:50 batch-ca4-02 postfix/pickup[26935]: warning: 884643E30022: message has been queued for 1 days Dec 12 10:08:50 batch-ca4-02 postfix/pickup[26935]: 884643E30022: uid=500 from=x...@.com What user acccount has uid=500? Is this your web server? Perhaps you have an exploitable web application. This is very popular with spammers. To block local submissions from this user, /etc/postfix/main.cf: authorized_submit_users = !username, static:anyone Dec 12 10:08:50 batch-ca4-02 postfix/qmgr[26936]: qmgr_active_feed: incoming/884643E30022 Please turn off all your verbose logging in master.cf. It makes Postfix VERY VERY SLOW and makes mail overload problems much worse. On the other hand, if your email overload is caused by spam, please turn on verbose logging on everything, to slow down the delivery of spam. Wietse
Re: Postfix Not Sending Emails. Timeout on Cleanup socket error.
Hi Wietse, I apologize for the confusion. Here is the result of the following command. The mail arrived via pickup (local). # grep 8A2993E3003B /var/log/maillog Dec 11 05:31:27 batch-ca4-02 postfix/cleanup[31691]: warning: 8A2993E3003B: read timeout on cleanup socket Gonzo Fernandez Network Engineer On Dec 13, 2011, at 12:35 PM, Wietse Venema wrote: Gonzo Fernandez: Dec 11 05:09:19 batch-ca4-02 postfix/cleanup[31691]: warning: timeout on cleanup socket while reading input attribute name Dec 11 05:31:27 batch-ca4-02 postfix/cleanup[31691]: warning: 8A2993E3003B: read timeout on cleanup socket Did the email arrive via smtpd (network), or via pickup (local)? $ grep 8A2993E3003B /the/log/file Wietse
Re: Postfix Not Sending Emails. Timeout on Cleanup socket error.
I'm not sure I'm understanding the log file info you would like. I tried doing grep search for 8A2993E3003B on all log files under /var/log/* and only found the following line to show up: Dec 11 05:31:27 batch-ca4-02 postfix/cleanup[31691]: warning: 8A2993E3003B: read timeout on cleanup socket Can you please be more specific? Thanks for your patience and help. Gonzo Fernandez Network Engineer On Dec 14, 2011, at 4:12 PM, Wietse Venema wrote: Gonzo Fernandez: Hi Wietse, I apologize for the confusion. Here is the result of the following command. The mail arrived via pickup (local). PLEASE show the logfile records that I asked for. Wietse # grep 8A2993E3003B /var/log/maillog Dec 11 05:31:27 batch-ca4-02 postfix/cleanup[31691]: warning: 8A2993E3003B: read timeout on cleanup socket Gonzo Fernandez Network Engineer On Dec 13, 2011, at 12:35 PM, Wietse Venema wrote: Gonzo Fernandez: Dec 11 05:09:19 batch-ca4-02 postfix/cleanup[31691]: warning: timeout on cleanup socket while reading input attribute name Dec 11 05:31:27 batch-ca4-02 postfix/cleanup[31691]: warning: 8A2993E3003B: read timeout on cleanup socket Did the email arrive via smtpd (network), or via pickup (local)? $ grep 8A2993E3003B /the/log/file Wietse
Postfix Not Sending Emails. Timeout on Cleanup socket error.
Hi all, I'm not too familiar with postfix but our server seems to stop sending emails usually only on the weekends. The rest of the week it's fine but once it gets to about Saturday/Sunday morning it gets stuck. I've come in usually on Monday and type mailq only to find hundreds if not thousand or so emails stuck in the incoming queue. I have to restart the postfix service for all the emails to get sent. I was hoping someone a little more experienced than me can point me in the right direction and explain what's going on as I'm not that familiar with it yet. I'm sure I'm not the first to have this issue and I've searched online but have found nothing specific that could be causing this. This is what cat /var/log/maillog | grep warning shows: Dec 11 05:09:19 batch-ca4-02 postfix/cleanup[31691]: warning: timeout on cleanup socket while reading input attribute name Dec 11 05:31:27 batch-ca4-02 postfix/cleanup[31691]: warning: 8A2993E3003B: read timeout on cleanup socket Here is my postconf -n: alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases command_directory = /usr/sbin config_directory = /etc/postfix daemon_directory = /usr/libexec/postfix debug_peer_level = 2 html_directory = no in_flow_delay = 10s inet_interfaces = localhost mail_owner = postfix mailq_path = /usr/bin/mailq.postfix manpage_directory = /usr/share/man mydestination = $myhostname, localhost.$mydomain, localhost mydomain = usaepay.com myorigin = $mydomain newaliases_path = /usr/bin/newaliases.postfix queue_directory = /var/spool/postfix readme_directory = /usr/share/doc/postfix-2.3.3/README_FILES relayhost = 192.168.x.x sample_directory = /usr/share/doc/postfix-2.3.3/samples sendmail_path = /usr/sbin/sendmail.postfix setgid_group = postdrop unknown_local_recipient_reject_code = 550 Can anyone please help?? If I need to submit more info please let me know. Thank you! Gonzo Fernandez Network Engineer
Re: Postfix Not Sending Emails. Timeout on Cleanup socket error.
error Gonzo Fernandez Network Engineer On Dec 13, 2011, at 12:21 PM, Brian Evans - Postfix List wrote: egrep '(warning|error|fatal|panic):' /some/log/file | more
