fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem
Hi
After years of a smoothly running mail server, my logs are starting to
fill with this error message:
fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table
lookup problem
I haven't updated anything or changed any configs (in either mysql or
postfix) so I'd appreciate some hints as to how to track this down?
According to mysql postfix is behaving properly (as you'd expect).
+--+-+---+---+-+--+---+--+
| Id | User| Host | db| Command | Time | State | Info |
+--+-+---+---+-+--+---+--+
| 379 | postfix | localhost | Mail | Sleep | 25 | | NULL
|
So I'm not sure why it thinks there is a lock. I found one stub that
suggested this happened with the mysql.sock was not in the chroot
jail. But as I said, it's been running flawlessly for years, so I
don't forsee that as the issue.
Nevertheless, here's my postconf -n
access_map_reject_code = 550
alias_database = hash:/etc/postfix/aliases
alias_maps = $alias_database
allow_untrusted_routing = no
append_dot_mydomain = no
biff = no
body_checks_size_limit = 51200
bounce_size_limit = 5
broken_sasl_auth_clients = yes
command_directory = /usr/sbin
config_directory = /etc/postfix
content_filter = smtp-amavis:[127.0.0.1]:10024
daemon_directory = /usr/lib/postfix
debug_peer_level = 1
default_destination_concurrency_limit = 25
disable_vrfy_command = yes
fast_flush_domains = $relay_domains
header_checks = regexp:/etc/postfix/header_checks
header_size_limit = 102400
home_mailbox = Maildir/
inet_interfaces = all
invalid_hostname_reject_code = 501
local_destination_concurrency_limit = 2
local_recipient_maps = proxy:unix:passwd.byname $virtual_alias_maps $alias_maps
mail_spool_directory = /var/spool/mail
mailbox_command = procmail -a $EXTENSION
mailbox_size_limit = 512000
message_size_limit = 2048
mydestination =
mydomain = perlphreak.net
myhostname = mail.perphreak.net
mynetworks = 127.0.0.0/8
mynetworks_style = host
myorigin = $mydomain
recipient_delimiter = -
reject_code = 550
relay_domains = /etc/postfix/mxbackups
relay_domains_reject_code = 550
relayhost =
smtp_tls_CAfile = /etc/ssl/keys/perlphreak-ca.crt
smtp_tls_cert_file = /etc/ssl/keys/mail.perlphreak.net.crt
smtp_tls_key_file = /etc/ssl/private/mail.perlphreak.net.key
smtp_tls_loglevel = 1
smtp_tls_note_starttls_offer = yes
smtp_tls_security_level = may
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
smtp_tls_session_cache_timeout = 3600s
smtpd_banner = $myhostname ESMTP $mail_name ($mail_version) (Debian/GNU)
smtpd_data_restrictions = sleep 1, reject_unauth_pipelining, permit
smtpd_delay_reject = yes
smtpd_helo_required = yes
smtpd_recipient_limit = 250
smtpd_recipient_restrictions = reject_non_fqdn_sender,
reject_non_fqdn_recipient,permit_sasl_authenticated,
reject_sender_login_mismatch,
reject_authenticated_sender_login_mismatch, check_helo_access
hash:/etc/postfix/helo_checks,reject_invalid_helo_hostname,
reject_non_fqdn_helo_hostname, reject_unknown_helo_hostname,
reject_unknown_sender_domain, reject_unknown_recipient_domain,
permit_mynetworks, reject_unauth_destination,
reject_unlisted_recipient, check_recipient_access
mysql:/etc/postfix/Mail-Disabled.cf, check_helo_access
hash:/etc/postfix/helo_checks,check_recipient_access
hash:/etc/postfix/laxdomains,check_client_access
hash:/etc/postfix/ip_whitelist, check_sender_access
hash:/etc/postfix/backscatter
check_reverse_client_hostname_access pcre:/etc/postfix/fqrdns.pcre,
check_policy_service unix:private/policy-spf, check_policy_service
inet:127.0.0.1:10031, reject_rbl_client bl.spamcop.net,
reject_rbl_client zen.spamhaus.org, permit
smtpd_sasl_auth_enable = yes
smtpd_sasl_local_domain = perlphreak.net
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_tls_security_options = $smtpd_sasl_security_options
smtpd_sasl_type = dovecot
smtpd_timeout = 300s
smtpd_tls_CAfile = /etc/ssl/keys/perlphreak-ca.crt
smtpd_tls_auth_only = no
smtpd_tls_cert_file = /etc/ssl/keys/mail.perlphreak.net.crt
smtpd_tls_key_file = /etc/ssl/private/mail.perlphreak.net.key
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_security_level = may
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtpd_tls_session_cache_timeout = 3600s
strict_rfc821_envelopes = yes
tls_random_source = dev:/dev/urandom
unknown_address_reject_code = 554
unknown_client_reject_code = 554
unknown_hostname_reject_code = 554
unknown_local_recipient_reject_code = 554
virtual_alias_maps = hash:/etc/postfix/virtual_user_aliases,
proxy:mysql:/etc/postfix/mailalias.cf
virtual_gid_maps = static:115
virtual_mailbox_base = /var/spool/mail/virtual
virtual_mailbox_domains = proxy:mysql:/etc/postfix/maildomain.cf
virtual_mailbox_limit = 50
virtual_mailbox_limit_inbox = no
Re: fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem
Simon Brereton: Hi After years of a smoothly running mail server, my logs are starting to fill with this error message: fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem I would expcet to see some warning before an uninformative fatal message. BTW the (0,lock|fold_fix) are Postfix-internal flags. It does not mean that there is a problem locking mysql. the lock would be used with files such as Berkeley DB. Wietse
Re: fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem
On 5 March 2013 16:47, Wietse Venema wie...@porcupine.org wrote: Simon Brereton: Hi After years of a smoothly running mail server, my logs are starting to fill with this error message: fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem I would expcet to see some warning before an uninformative fatal message. BTW the (0,lock|fold_fix) are Postfix-internal flags. It does not mean that there is a problem locking mysql. the lock would be used with files such as Berkeley DB. Wietse Yes, I suppose there would be! I found two different types. Mar 5 08:45:12 mail postfix/smtpd[24830]: connect from unknown[117.6.222.107] Mar 5 08:45:14 mail postfix/proxymap[24831]: warning: mysql query failed: MySQL server has gone away Mar 5 08:45:14 mail postfix/trivial-rewrite[24833]: fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem Mar 5 08:45:15 mail postfix/master[4283]: warning: process /usr/lib/postfix/trivial-rewrite pid 24833 exit status 1 Mar 5 08:45:16 mail postfix/trivial-rewrite[24854]: fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem Mar 5 08:45:17 mail postfix/smtpd[24830]: warning: problem talking to service rewrite: Success Mar 5 08:45:17 mail postfix/master[4283]: warning: process /usr/lib/postfix/trivial-rewrite pid 24854 exit status 1 Mar 5 08:45:17 mail postfix/master[4283]: warning: /usr/lib/postfix/trivial-rewrite: bad command startup -- throttling and Mar 5 10:53:21 mail postfix/smtpd[29350]: connect from unknown[81.213.239.67] Mar 5 10:53:22 mail postfix/proxymap[26043]: warning: connect to mysql server localhost: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) Mar 5 10:53:22 mail postfix/trivial-rewrite[26046]: fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem Mar 5 10:53:23 mail postfix/master[4283]: warning: process /usr/lib/postfix/trivial-rewrite pid 26046 exit status 1 Mar 5 10:53:24 mail postfix/trivial-rewrite[29357]: fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem Mar 5 10:53:25 mail postfix/smtpd[29350]: warning: problem talking to service rewrite: Success Mar 5 10:53:25 mail postfix/master[4283]: warning: process /usr/lib/postfix/trivial-rewrite pid 29357 exit status 1 Mar 5 10:53:25 mail postfix/master[4283]: warning: /usr/lib/postfix/trivial-rewrite: bad command startup -- throttling mysqld.sock does exist.. mail:~# ls /var/run/mysqld/mysqld.sock srwxrwxrwx 1 mysql mysql 0 Mar 5 12:14 /var/run/mysqld/mysqld.sock I rebooted the box and until now I haven't had a repeat of the error. I suppose it's just possible that after ~500 days uptime a reset was needed, but I'm always wary when problems like this suddenly appear. There isn't a heavy load, and so I tend to regard symptoms like this as highly suspicious. As per my initial email, I don't think postfix caused this in anyway, but of course it felt the force of it. I wonder now I can increase the redundancy. Simon
Re: Accept and store locally any mail received
On Mon, Mar 04, 2013 at 01:35:42PM -0600, Noel Jones wrote: There's several ways to do this. I think the easiest is: # main.cf header_checks = regexp:/etc/postfix/header_checks in your header_checks file: /./ REDIRECT some...@example.com where someone is a valid local user, and example.com is listed in mydestination. This works perfectly. Thanks a lot Noel. -- Nicolas
Re: fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem
Simon Brereton: Mar 5 08:45:14 mail postfix/proxymap[24831]: warning: mysql query failed: MySQL server has gone away The mysql server closed the connection (or crashed). Mar 5 10:53:22 mail postfix/proxymap[26043]: warning: connect to mysql server localhost: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) The mysql did not accept connections (probably was not running). Wietse
Re: fatal: proxy:mysql:/etc/postfix/mailalias.cf(0,lock|fold_fix): table lookup problem
On 5 Mar 2013 19:07, Wietse Venema wie...@porcupine.org wrote: Simon Brereton: Mar 5 08:45:14 mail postfix/proxymap[24831]: warning: mysql query failed: MySQL server has gone away The mysql server closed the connection (or crashed). Mar 5 10:53:22 mail postfix/proxymap[26043]: warning: connect to mysql server localhost: Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) The mysql did not accept connections (probably was not running). Thanks Wietse - I'll continue looking for a cause elsewhere.. Simon
virtual_alias_maps differs between internal and externally hosted domains
Hi ,
I use virtual_alias_maps with a mysql server to manage aliases and
catch-all accounts.
For the past few years this approach worked like a charm, but after
upgrading to 2.10 I get two different behaviours.
When mailing from a domain to a domain both hosted on this server, the
virtual_alias_maps behave correctly.
When mailing from a domain on a different server to one hosted on my
server the alias lookup fails (the message gets through when the actual
user exists).
MySQL log when the email is received from any external mailserver:
130305 21:48:54 283 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='hazelhof.nl' LIMIT 1
284 Query SELECT 1 FROM `virtual_domains` WHERE
name='hazelhof.nl' LIMIT 1
297 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='i...@domain.nl' LIMIT 1
297 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='@domain.nl' LIMIT 1
282 Query SELECT '/var/mail/hazelhof.nl/alias' AS
`home`, 5000 AS `uid`, 5000 AS `gid`, CONCAT('*:bytes=', quota_kb) AS
`quota_rule` FROM `virtual_users` WHERE `email` = 'al...@hazelhof.nl'
LIMIT 1
which results in a reject, Mail.log: Mar 5 21:55:30 black-mail
postfix/smtpd[19444]: NOQUEUE: reject: RCPT from
domain.nl[2a00:d10:101::26:0]: 554 5.7.1 al...@hazelhof.nl: Recipient
address rejected: Unknown user; from=i...@domain.nl
to=al...@hazelhof.nl proto=ESMTP helo=domain.nl
MySQL log when mailing from an internal domain
130305 21:48:23 283 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='hazelhof.nl' LIMIT 1
284 Query SELECT 1 FROM `virtual_domains` WHERE
name='hazelhof.nl' LIMIT 1
307 Connect provider@localhost on provider
307 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='mich...@hazelhof.nl' LIMIT 1
307 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='@hazelhof.nl' LIMIT 1
308 Connect provider@localhost on provider
308 Query SELECT `email` FROM `virtual_users`
WHERE email='mich...@hazelhof.nl' LIMIT 1
307 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='al...@hazelhof.nl' LIMIT 1
307 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='@hazelhof.nl' LIMIT 1
283 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='hazelhof.nl' LIMIT 1
284 Query SELECT 1 FROM `virtual_domains` WHERE
name='hazelhof.nl' LIMIT 1
290 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='al...@hazelhof.nl' LIMIT 1
290 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='@hazelhof.nl' LIMIT 1
290 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='postmas...@black-mail.nl' LIMIT 1
290 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='postmaster' LIMIT 1
290 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='@black-mail.nl' LIMIT 1
283 Query SELECT `destination` FROM
`virtual_aliases` WHERE source='black-mail.nl' LIMIT 1
284 Query SELECT 1 FROM `virtual_domains` WHERE
name='black-mail.nl' LIMIT 1
309 Connect provider_spamass@localhost on
provider_spamassassin
309 Query set autocommit=1
309 Query SELECT preference, value FROM userpref
WHERE username = 'postmas...@black-mail.nl' OR username = '$GLOBAL' OR
username = CONCAT('%','black-mail.nl') ORDER BY username ASC
Which results in a correct relay to the postmaster account.
It appears to me that there are two different lookups being done, have I
missed something in the 2.10 changelog?
user = provider
password = x
hosts = 127.0.0.1
dbname = x
query = SELECT `destination` FROM `virtual_aliases` WHERE source='%s' LIMIT 1;
# | ID | domain_id| source| destination
# | int(11) | int(11)| varchar(100) | varchar(1000)
# | 1 | 1 | @hazelhof.nl | postmas...@black-mail.nl
Re: virtual_alias_maps differs between internal and externally hosted domains
Forgot to add the proper postfinger file. postfinger - postfix configuration on Tue Mar 5 23:20:05 CET 2013 version: 1.30 --System Parameters-- mail_version = 2.10.0 hostname = black-mail.nl uname = Linux black-mail.nl 3.2.0-4-amd64 #1 SMP Debian 3.2.35-2 x86_64 GNU/Linux --Packaging information-- looks like this postfix comes from deb package: postfix-2.10.0-1 --main.cf non-default parameters-- alias_maps = hash:/etc/aliases append_dot_mydomain = no biff = no clamsmtp_destination_recipient_limit = 1 delay_warning_time = 4h disable_vrfy_command = yes enable_long_queue_ids = yes extraslow_destination_concurrency_failed_cohort_limit = 15 extraslow_destination_concurrency_limit = 1 extraslow_destination_rate_delay = 2s extraslow_destination_recipient_limit = 1 html_directory = /usr/share/doc/postfix/html inet_interfaces = 127.0.0.1,93.186.177.127,[2a00:d10:101::27:] mailbox_size_limit = 0 message_size_limit = 26214400 mydestination = localhost mynetworks = 127.0.0.1 myorigin = /etc/mailname postscreen_dnsbl_action = enforce postscreen_dnsbl_sites = zen.spamhaus.org b.barracudacentral.org dnsbl-1.uceprotect.net bl.spamcop.net postscreen_greet_action = enforce recipient_delimiter = + slow_destination_concurrency_limit = 2 slow_destination_recipient_limit = 5 smtpd_data_restrictions = reject_unauth_pipelining, permit smtpd_helo_required = yes smtpd_recipient_restrictions = permit_sasl_authenticated, reject_unauth_destination, reject_invalid_hostname, reject_unauth_pipelining, reject_non_fqdn_sender, reject_non_fqdn_helo_hostname, reject_non_fqdn_recipient, reject_unknown_sender_domain, reject_unknown_recipient_domain, check_policy_service unix:private/policyspf, check_recipient_access pcre:/etc/postfix/recipient_checks.pcre, check_helo_access hash:/etc/postfix/helo_checks, check_policy_service unix:private/quota-status smtpd_relay_restrictions = smtpd_sasl_auth_enable = yes smtpd_sasl_path = private/auth smtpd_sasl_type = dovecot smtpd_sender_login_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf smtpd_sender_restrictions = reject_non_fqdn_sender, reject_unknown_sender_domain, reject_unlisted_sender, reject_authenticated_sender_login_mismatch smtpd_tls_CApath = /etc/apache2/ssl/black-mail.nl/ smtpd_tls_cert_file = /etc/apache2/ssl/black-mail.nl/ssl.crt smtpd_tls_key_file = /etc/apache2/ssl/black-mail.nl/ssl.key smtpd_tls_security_level = may smtpd_tls_session_cache_database = btree:/var/lib/postfix/smtpd_scache smtpd_use_tls = yes smtp_tls_session_cache_database = btree:/var/lib/postfix/smtp_scache spamassassin_destination_recipient_limit = 1 transport_maps = hash:/etc/postfix/transport virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-email2email.cf virtual_gid_maps = static:5000 virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf virtual_transport = clamsmtp:[127.0.0.1]:10025 virtual_uid_maps = static:5000 --master.cf-- smtpd pass - - - - - smtpd smtp inet n - n - 1 postscreen tlsproxy unix - - n - 0 tlsproxy dnsblog unix - - n - 0 dnsblog 616 inet n - n - - smtpd -o smtpd_etrn_restrictions=reject -o content_filter=dksign:[127.0.0.1]:10028 -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o receive_override_options=no_address_mappings -o smtpd_client_restrictions=permit_sasl_authenticated,reject submission inet n - - - - smtpd -o smtpd_etrn_restrictions=reject -o content_filter=dksign:[127.0.0.1]:10028 -o smtpd_enforce_tls=yes -o smtpd_sasl_auth_enable=yes -o receive_override_options=no_address_mappings -o smtpd_client_restrictions=permit_sasl_authenticated,reject pickupunix n - - 60 1 pickup -o content_filter=dksign:[127.0.0.1]:10028 cleanup unix n - - - 0 cleanup qmgr unix n - n 300 1 qmgr tlsmgrunix - - - 1000? 1 tlsmgr rewrite unix - - - - - trivial-rewrite bounceunix - - - - 0 bounce defer unix - - - - 0 bounce trace unix - - - - 0 bounce verifyunix - - - - 1 verify flush unix n - - 1000? 0 flush proxymap unix - - n - - proxymap smtp unix - - - - - smtp relay unix - - - - - smtp showq unix n - - - - showq error unix - - - - - error retry unix - - - - - error discard unix - - - - - discard local unix -
Re: virtual_alias_maps differs between internal and externally hosted domains
On Tue, Mar 05, 2013 at 11:19:22PM +0100, Michiel Hazelhof wrote: which results in a reject, Mail.log: Mar 5 21:55:30 black-mail postfix/smtpd[19444]: NOQUEUE: reject: RCPT from domain.nl[2a00:d10:101::26:0]: 554 5.7.1 al...@hazelhof.nl: Recipient address rejected: Unknown user; from=i...@domain.nl to=al...@hazelhof.nl proto=ESMTP helo=domain.nl Nothing in Postfix generates an Unknown user response of any kind (the string Unknown user does not appear in the Postfix source code). Even such a string were returned it would not be with a 5.7.1 DSN code, which indicates site policy not address validity. Perhaps you have a milter or policy service that generates this response. -- Viktor.
Re: ldap_table and insignificant spaces
On Fri, Mar 01, 2013 at 03:19:42PM +0100, Bastian Blank wrote: I found that one MTA bounced several mails. The mails where sent to test@example.com and accepted by Postfix. The backend LMTP then rejected the mails. This is what I found out: - RCPT TO: test@example.com - The ldap table gets the sanitized address: ` t...@example.com' (note the leading space is still there). - This is converted into a ldap query (mail= t...@example.com). - The ldap server sanitices the query to (mail=t...@example.com) as mandated by RFC 4717, 4.2.3; it removes the insignificant space. I see the insignificant space handling defined in 4518 (referenced from 4517). https://tools.ietf.org/html/rfc4518#section-2.6.1 It seems to suggest that exact string matches should take the form attr=SPACEvalueSPACE where any spaces inside the value are encoded as SPACESPACE. Is this backwards compatible with older LDAP servers that are not UTF-8 based? An easy way to achieve this would be: query_filter = (mail = %s ) if such spaces are not removed at a higher level by the LDAP library. Does this help? Not sure if something should be done about it. At least it is a surprising outcome for a simple question; while both parties works perfectly fine. Another thing that could help is if Postfix would use the external form of the address: test@example.com with the quotes as the query string. I seem to recall that this is already the case with lookup keys in virtual_alias_maps, but it may not be the case with other tables. Which Postfix mumble_maps parameter are you using with LDAP? Arguably, all lookup keys in tables should be in external (RFC-5322) form. The suggested doubling of internal spaces is far less important in practice that avoiding the loss of leading spaces. -- Viktor.
Re: ldap_table and insignificant spaces
Viktor Dukhovni: Arguably, all lookup keys in tables should be in external (RFC-5322) form. The suggested doubling of internal spaces is far less important in practice that avoiding the loss of leading spaces. Which external form? - RFC 5321 and 5322 have different rules. - Worse, The mappings are not one-to-one. That is, there are multiple correct implementations for quote_821_local() and quote_822_local(). Wietse
Re: ldap_table and insignificant spaces
On Tue, Mar 05, 2013 at 11:16:18PM -0500, Wietse Venema wrote: Viktor Dukhovni: Arguably, all lookup keys in tables should be in external (RFC-5322) form. The suggested doubling of internal spaces is far less important in practice that avoiding the loss of leading spaces. Which external form? - RFC 5321 and 5322 have different rules. I was a bit sloppy in my wording, I guess 5321 is the right one for processing envelope addresses, and perhaps 5322 for processing header addresses (with canonical and generic rewrites for example). This complicates recipient extension support, since to compute an address sans extension we'd need to dequote, lose the extension and then requote (using the right quoting style). This is not trivial. - Worse, The mappings are not one-to-one. That is, there are multiple correct implementations for quote_821_local() and quote_822_local(). So long as we pick something consistent and documented, users can likely adapt their table keys. In most cases the external form and the internal form are identical, but the exceptions are a pain. For this specific LDAP issue, if changing the query works that's the fastest path to success, and we can adjust the ldap_table(5) document with the latest best practice syntax. The more things change, the less they stay the same. :-) -- Viktor.
