Dear everyone,
I'm working on some Antispam-Proxy, using Postfix as MTA. Postfix is
2.6.2-RC1 on an Ubuntu 8.04 LTS base-system.
Preconditions:
* Postfix shall only accept mails addressed to valid (=existing)
recipients. To accomplish this, I'm using a regexp:/ map on
relay_recipient_maps (the specific file is called "usermaps").
* This usermaps file is automatically generated from an hourly cron-job,
fetching all valid email-addresses via LDAP (however, the Postfix
installation doesn't care about LDAP at all, this is autonomously done
by some perl script).
* The data gathered from LDAP is stuffed into a temporary file until
finished, and then "atomatically" copied over the original usermaps
file, before Postfix is triggered to reload.
Problem:
* At very irregular intervals, varying in time and quantity, Postfix
refuses to accept Mails because the recipient address is seemingly
unknown, altough that specific mail address (changes every time,
unpredictable) is correctly defined in the usermaps file. The
log-messages are like:
2010-01-26T15:10:29+01:00 hostmail postfix/smtpd[22884]: NOQUEUE:
reject: RCPT from smtp.citrix.com[66.165.176.89]: 550 5.1.1
<[email protected]>: Recipient address rejected: User unknown in
relay recipient table; from=<[email protected]>
to=<[email protected]> proto=ESMTP helo=<SMTP.CITRIX.COM>
* Assuming the hourly cron-job is executed 24 times a day, 1-4 times
Postfix logs the following message:
2010-01-26T08:57:25+01:00 hostmail postfix/smtpd[3398]: warning: regexp
map /etc/postfix/usermaps, line 2434: no closing regexp delimiter "/":
skipping this rule
The lines-number is always randomly changing, and I have made quite some
effort to make sure that the usermaps file is always complete,
syntactically correct and consistent. As you see, the logentry above is
timed "08:57:25" (the cron-job begins fetching addresses via LDAP always
at *:57).
Interestingly, my 'watch stat /etc/postfix/usermaps' shows this:
# Before the 08:57 cron-job touches usermaps
@Tue Jan 26 08:57:24 CET 2010
Access: 2010-01-26 07:57:24.000000000 +0100
Modify: 2010-01-26 07:57:22.000000000 +0100
Change: 2010-01-26 07:57:22.000000000 +0100
# After the 08:57 cron-job re-wrote usermaps, but Postfix hasn't read it
# yet
@Tue Jan 26 08:57:26 CET 2010
Access: 2010-01-26 08:57:25.000000000 +0100
Modify: 2010-01-26 08:57:25.000000000 +0100
Change: 2010-01-26 08:57:25.000000000 +0100
# After Postfix read the new usermaps after reloading
@Tue Jan 26 08:57:36 CET 2010
Access: 2010-01-26 08:57:35.000000000 +0100
Modify: 2010-01-26 08:57:25.000000000 +0100
Change: 2010-01-26 08:57:25.000000000 +0100
If you look at these times, the file is *read* by Postfix at 08:57:35,
but the log-line above claims the warning at 07:57:25. How can this be?
The 10 seconds delay is because of an intended sleep() between writing
the usermaps and reloading Postfix.
Moreover, when mails a rejected as described above, the *time* these
rejects happen do not seem to correlate with the regexp-warnings, nor do
the rejected recipient mail-addresses. It seems like everything happens
quite random here.
What I've already checked:
* Generation of usermaps file is OK and always succeeds. All addresses
are successfully fetched, the file is writen syntactically correct and
complete.
* I/O- and buffering-issues have been tested and shouldn't be the
problem (e.g. reloading Postfix while I/O buffer hasn't been flushed
yet).
* The basic Postfix configuration works perfectly and never made any
troubles. That usermaps issue seems to occur only then the usermaps is
getting large (>1k lines; in this specific case, it's about 10k lines
large).
The installation runs on a virtualized platform, using XEN. Postfinger
output is attached. I should also mention that, for various reasons,
it's not *easily* possible for me to simply upgrade the Postfix version.
Thank you very much,
Erik
postfinger - postfix configuration on Tue Jan 26 15:18:25 CET 2010
version: 1.30
Warning: postfinger output may show private configuration information,
such as ip addresses and/or domain names which you do not want to show
to the public. If this is the case it is your responsibility to modify
the output to hide this private information. [Remove this warning with
the --nowarn option.]
--System Parameters--
mail_version = 2.6.2-RC1
hostname = hostmail
uname = Linux hostmail 2.6.24-24-server #1 SMP Tue Jun 30 21:03:25 UTC 2009
i686 GNU/Linux
--Packaging information--
looks like this postfix comes from deb package: postfix-2.6.2~rc1-1
--main.cf non-default parameters--
alias_maps = hash:/etc/aliases
anvil_rate_time_unit = 30m
append_dot_mydomain = no
biff = no
bounce_queue_lifetime = 1h
broken_sasl_auth_clients = yes
content_filter = smtp-amavis:[127.0.0.1]:10024
header_checks = regexp:/etc/postfix/header_checks
local_recipient_maps = hash:/etc/postfix/local_rcpt_map
mailbox_size_limit = 0
mailbox_transport_maps = hash:/etc/postfix/mbox_transport
maximal_queue_lifetime = 6h
message_size_limit = 500000000
mydestination = localhost, $myhostname
myhostname = hostmail.XXXXXXXX.de
mynetworks = 127.0.0.0/8
queue_minfree = 1000000000
recipient_delimiter = +
relay_domains = hash:/etc/postfix/transport
relay_recipient_maps = regexp:/etc/postfix/usermaps
smtpd_banner = $myhostname ANTISPAM PROXY
smtpd_client_connection_rate_limit = 200
smtpd_client_restrictions = check_client_access
cidr:/etc/postfix/amavis_bypass_internal_warn, check_client_access
cidr:/etc/postfix/amavis_bypass_internal_filter, check_client_access
cidr:/etc/postfix/amavis_bypass_filter_smtpcrypt, check_client_access
cidr:/etc/postfix/amavis_bypass_filter, check_client_access
cidr:/etc/postfix/amavis_bypass_accept, check_client_access
cidr:/etc/postfix/amavis_bypass_internal_accept,
smtpd_data_restrictions = reject_unauth_pipelining,
smtpd_helo_required = yes
smtpd_recipient_restrictions = check_client_access
cidr:/etc/postfix/amavis_bypass_internal_accept, check_recipient_access
regexp:/etc/postfix/filter-quarantine.regexp, check_policy_service
inet:127.0.0.1:10040, permit_sasl_authenticated, permit_mynetworks,
reject_unauth_destination, reject_non_fqdn_sender,
reject_unknown_sender_domain, permit
smtpd_restriction_classes = rc_greylisting
smtpd_sasl_authenticated_header = yes
smtpd_sender_restrictions = permit_sasl_authenticated, check_sender_access
regexp:/etc/postfix/amavis_senderbypass_filter, permit_mynetworks, permit
smtpd_timeout = 60
transport_maps = hash:/etc/postfix/transport
virtual_gid_maps = static:114
virtual_mailbox_base = /var/quarantine
virtual_mailbox_limit = 1000000000
virtual_mailbox_maps = hash:/etc/postfix/virtual_mbox
virtual_uid_maps = static:106
--master.cf--
0.0.0.0:smtp inet n - - - 48 smtpd
pickup fifo n - - 60 1 pickup
-o content_filter=
cleanup unix n - - - 0 cleanup
qmgr fifo n - n 300 1 qmgr
tlsmgr unix - - - 1000? 1 tlsmgr
rewrite unix - - - - - trivial-rewrite
bounce unix - - - - 0 bounce
defer unix - - - - 0 bounce
trace unix - - - - 0 bounce
verify unix - - - - 1 verify
flush unix n - - 1000? 0 flush
proxymap unix - - n - - proxymap
smtp unix - - - - - smtp
relay unix - - - - - smtp
-o fallback_relay=
showq unix n - - - - showq
error unix - - - - - error
discard unix - - - - - discard
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - - - - lmtp
anvil unix - - - - 1 anvil
scache unix - - - - 1 scache
maildrop unix - n n - - pipe
flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
uucp unix - n n - - pipe
flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
ifmail unix - n n - - pipe
flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp unix - n n - - pipe
flags=Fq. user=bsmtp argv=/usr/lib/bsmtp/bsmtp -t$nexthop -f$sender $recipient
scalemail-backend unix - n n - 2 pipe
flags=R user=scalemail argv=/usr/lib/scalemail/bin/scalemail-store ${nexthop}
${user} ${extension}
mailman unix - n n - - pipe
flags=FR user=list argv=/usr/lib/mailman/bin/postfix-to-mailman.py
${nexthop} ${user}
smtp-amavis unix - - n - 16 smtp
-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes
-o disable_dns_lookups=yes
-o max_use=20
127.0.0.1:10025 inet n - n - - smtpd
-o content_filter=
-o smtpd_restriction_classes=
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=reject_unauth_pipelining
-o smtpd_end_of_data_restrictions=
-o mynetworks=127.0.0.0/8
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o smtpd_milters=
-o local_header_rewrite_clients=
-o local_recipient_maps=
-o relay_recipient_maps=
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
smtpcrypt unix - n n - - pipe
flags=Rq user=smtpcrypt argv=/usr/local/bin/smtpcrypt.pl sasl${sasl_method}
${client_address} ${sender} ${recipient}
retry unix - - - - - error
-- end of postfinger output --
