Exploring conditional local log and external firewall control. Best practices?
My recently installed Postfix works as I'd hoped; I moved it into full production as our corporate server yesterday. There's one annoyance, and I admit that's all it is, that I'd like to get rid of. *Noisy* pests. They irritate me. I'm interested in what others do in similar circumstance. My 'smtpd_recipient_restrictions' includes checks against DNSBLs, e.g. spamhaus. The typical log pattern for a successful block is 5-10 of these: May 2 08:13:26 liam postfix/smtpd[17563]: NOQUEUE: reject: RCPT from 206.pool85-50-110.dynamic.orange.es[85.50.110.206]: 554 5.7.1 Service unavailable; Client host [85.50.110.206] blocked using zen.spamhaus.org; http://www.spamhaus.org/query/bl?ip=85.50.110.206; from= to=<@..> proto=ESMTP helo= within 5 minutes, then another round or few every 4-12 hourse for a couple of days. I'll end up with 10-100 log entries effectively documenting the fact that each pest is a pest. Postfix does what it's supposed to, and blocks them. I'd like to do two things: (1) limit log entries for these items with a logical condition: If this connection rejection has been previously attempted and rejected more than Z times within the last YY minutes, then reject as usual, but do not bother to log for the next minutes. Just reject silently. (2) communicate with a firewall on another box to act according to similar logic: If a connection attempt has been made and rejected more than ZZ times within the last minutes, then add the offending IP to an IPTABLES firewall rule on another box I suspect (1) is doable in Postfix configutation, but I haven't noticed the right parameter yet. Is it 'in' Postfix? For (2) I've starting looking at Fail2Ban. Seems like it might work. Is there something more native to Postfix that's available? Or a better recommendation? -- Thanks, Karen
Re: Flexible formatting of Postfix log entries?
On Sat, Apr 28, 2012, at 12:19 PM, Noel Jones wrote: > While it would be possible to patch postfix to write logs > differently, the better choice is to investigate some of the > existing log parsers, such as pflogsumm or postfix-logwatch, and > possibly customize them. I'll take a look at both of those. > Note that once you modify postfix logging > none of the off-the-shelf tools will work anymore, which is a good > reason to modify existing tools rather than modifying the logging. > http://www.postfix.org/addon.html#logfile It will require more in-depth digging on my part but those tools appear to do after the fact log processing. Your argument for not patching what Postfix does is a good one and echoes my gut anyway. > It is also possible to log to an sql database for flexible custom > reports. This is done in the syslog daemon (such as rsyslog) rather > than in postfix. I'd considered that as well as looking at Splunk. Moving human-readable text logs into a DB grabs me as overkill and smacks too much of Redhat's "Let's change the way logs work", which I personally find distasteful, but worth some additional thought. > For watching the logs in realtime, look at multitail. It can filter > and colorize the display to highlight the parts important to you. I use multitail on occassion. Color/highlight is one option. My specific goal is just to improve my own ability to live-tail a functional log, and more efficiently ID problems. Right now I spend most of my energy trying to visually parse the logs -- which 5 log entries go together, what are the Message IDs/PIDs/etc. Columns work for me, colors don't. I'll take a look at those log analysis/parser tools again. -- Thanks, Karen
Re: Flexible formatting of Postfix log entries?
On Sat, Apr 28, 2012, at 11:35 AM, John Peach wrote:
> > Since it's Postfix doing the writing to the logs in the 1st place, is it
> > possible to config Postfix to (free)format those
>
> It's not postfix - it's syslog.
If you look in Postfix's source code
./src/util/msg_syslog.c
...
static int syslog_facility;
/* msg_syslog_print - log info to syslog daemon */
static void msg_syslog_print(int level, const char
*text)
{
static int log_level[] = {
LOG_INFO, LOG_WARNING, LOG_ERR, LOG_CRIT, LOG_CRIT,
};
static char *severity_name[] = {
"info", "warning", "error", "fatal", "panic",
};
if (level < 0 || level >= (int) (sizeof(log_level) /
sizeof(log_level[0])))
msg_panic("msg_syslog_print: invalid severity level:
%d", level);
if (level == MSG_INFO) {
--> syslog(syslog_facility | log_level[level], "%.*s",
(int) MSG_SYSLOG_RECLEN, text);
} else {
--> syslog(syslog_facility | log_level[level], "%s:
%.*s",
severity_name[level], (int)
MSG_SYSLOG_RECLEN, text);
}
}
...
that looks to me like Postfix sending pre-assembled string(s) info to
the sys-logger with a format specified.
I can change those strings in Postfix code, recompile, and the output
format that appears in the syslog changes.
Your comment suggests to me that I can instead change that format "in"
syslog.
If that's the case, can you provide an example of how to use syslog
directly to alter that format?
--
Thanks,
Karen
Flexible formatting of Postfix log entries?
I've been writing scripts for my loganalysis chores. A typical log entry for a mail transaction looks like, Apr 28 07:01:28 liam postfix/smtpd[17751]: connect from out.somewhere.com[99.99.99.99] Apr 28 07:01:29 liam postfix/smtpd[17751]: 447FC600E1: client=out.somewhere.com[99.99.99.99] Apr 28 07:01:29 liam postfix/qmgr[899]: 447FC600E1: from=, size=3219, nrcpt=1 (queue active) Apr 28 07:01:29 liam postfix/smtpd[17751]: disconnect from out.somewhere.com[99.99.99.99] Apr 28 07:01:33 liam postfix/qmgr[899]: 286E0601B5: from=, size=3904, nrcpt=1 (queue active) Apr 28 07:01:33 liam postfix/smtp[17758]: 447FC600E1: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=5, delays=0.79/0.02/0.02/4.2, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 286E0601B5) Apr 28 07:01:33 liam postfix/lmtp[17767]: 286E0601B5: to=, relay=mail.mydomain.com[192.168.1.100]:7025, delay=0.57, delays=0.38/0.02/0/0.17, dsn=2.1.5, status=sent (250 2.1.5 Delivery OK) Personally, I find that difficult to parse at a glance. Although it's certainly a matter of personal taste, I prefer a bit more columnar structre. Something like (using monospaced fonts), Apr 28 07:01:28 liam postfix/smtpd[17751]: connect from out.somewhere.com[99.99.99.99] Apr 28 07:01:29 liam postfix/smtpd[17751]: 447FC600E1: client=out.somewhere.com[99.99.99.99] Apr 28 07:01:29 liam postfix/qmgr[899]:447FC600E1: from=, size=3219, nrcpt=1 (queue active) Apr 28 07:01:29 liam postfix/smtpd[17751]: disconnect from out.somewhere.com[99.99.99.99] Apr 28 07:01:33 liam postfix/qmgr[899]:286E0601B5: from=, size=3904, nrcpt=1 (queue active) Apr 28 07:01:33 liam postfix/smtp[17758]: 447FC600E1: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:10024, delay=5, delays=0.79/0.02/0.02/4.2, dsn=2.0.0, status=sent (250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 286E0601B5) Apr 28 07:01:33 liam postfix/lmtp[17767]: 286E0601B5: to=, relay=mail.mydomain.com[192.168.1.100]:7025, delay=0.57, delays=0.38/0.02/0/0.17, dsn=2.1.5, status=sent (250 2.1.5 Delivery OK) for a start. I can write scripts to parse & format the logs after the fact. Doing so for live log tailing is a little more challenging, but still can be done. Since it's Postfix doing the writing to the logs in the 1st place, is it possible to config Postfix to (free)format those log entries? I suppose I can find the code in Postfix and patch, but that seems unwise. Ideally, a configuration -- somewhere. If not, do a
Re: Can I improve the efficiency of my dnsbl reject configuration?
On Fri, Apr 27, 2012, at 03:12 PM, Wietse Venema wrote: > For small sites, postscreen has an up-front blacklist that kicks > off clients before wasting resources on them. Although I was warned off postscreen in an earlier post being 'heavier' than the checks against locally cached DNS, your comment's consistent with my initial read about postscreen. Iiuc it, too, uses a standard postfix SMTP process, and probably does that local DNSBL IP caching as well. Sounds like the lightest solution, without additional clever patching or dameon writing, may well be postscreen 'out front'. I-- Thanks, Karen
Re: Can I improve the efficiency of my dnsbl reject configuration?
On Fri, Apr 27, 2012, at 02:04 PM, /dev/rob0 wrote: > But consider this: the TTL of a DNSBL listing is a feature. Sometimes > legitimate sites will be listed, for example, in the CBL. Once they > clean up the problem, do you still want to block them? That's not within the scope of my use case. The vast majority of these 'repeat connection culprits' are doing so within a timeframe that's vastly smaller than typical CBL delisting times. *Within* that scope, the process idle/retire parameters that were shared earlier are more than sufficient, given that I've learned that Postfix does its own DNSBL IP caching. -- Thanks, Karen
Re: Can I improve the efficiency of my dnsbl reject configuration?
On Fri, Apr 27, 2012, at 08:54 PM, Bron Gondwana wrote: > Just as an interesting point from a fairly large site (fastmail.fm) we > do something very like that. We run a standalone daemon, and we keep > a "bad list" of IPs who get dumped immediately without even a DNS lookup. > > One of our patches to postfix allows that, dropping the connection while > doing nothing more than a syslog of the IP address. That's interesting. Just our of curiosity, as I'm in the midst of reading about policy daemons, milters, before & after queue filtering, etc. At a high-level -- how did you implement this? Sounds like you're actually patching postfix code, and not handing off to a dameon/milter/etc early in the process. -- Thanks, Karen
Re: Can I improve the efficiency of my dnsbl reject configuration?
On Fri, Apr 27, 2012, at 02:20 PM, Wietse Venema wrote: > kar...@mailcan.com: > Each Postfix SMTP server process is reused. > > http://www.postfix.org/postconf.5.html#max_use > http://www.postfix.org/postconf.5.html#max_idle That answers my question. Both of the defaults seem to fit nicely enough for ~95% of what I see in my logs. -- Thanks, Karen
Re: Can I improve the efficiency of my dnsbl reject configuration?
On Fri, Apr 27, 2012, at 08:16 PM, Ansgar Wiechers wrote: > >>> reject_non_fqdn_recipient > For my personal mail server I use this rule, too. However, you need to > be aware that it might reject some legit mail (e.g. from mail servers > configured by stupid, but valid, customers), hence the rule may be > inappropriate in situations where you handle business mail or mail for > third parties. Erring on the side of caution, reading http://www.postfix.org/postconf.5.html#reject_non_fqdn_recipient reject_non_fqdn_recipient Reject the request when the RCPT TO address is not in fully-qualified domain form, as required by the RFC. Iiuc, that "RCPT TO" belongs to the *sender's* mail, right? I.e., if they've misconfigured their RCPT TO (Different than 'Reply to', afaict) to, say, what -- 'alice@mail' ? or something? Just looking for an example case of misconfiguration. If it's a likely/reasonable scenario - and from you comment there *is* a probability of "some" - I'll probably remove it and see what happens. -- Thanks, Karen
Re: Can I improve the efficiency of my dnsbl reject configuration?
On Fri, Apr 27, 2012, at 01:47 PM, Wietse Venema wrote: > > I'd still think that a local check by Postfix to an 'auto-expiring hash > > table' (unclear so far it that can be done) to which the 'bad' address > > Each Postfix SMTP server caches its own DNSBL lookup results. Those > results are not shared with other Postfix processes. > > The in-process cache saves time when one message has multiple > recipients, and the DNSBL check is done once per recipient in > smtpd_recipient_restrictions. Does a single "Postfix SMTP server" process instance remain resident/open across multiple connections? Or is a new one spawned for each connection, or perhaps after some delay, load or other trigger? If it's the former, and I understand this right, then the two somewhat-concurrent connections from the same spamhaus-banned IP, as referenced in my OP, Apr 26 11:13:07 liam postfix/smtpd[22946]: connect from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:13:08 liam postfix/smtpd[22946]: NOQUEUE: reject: RCPT from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23]: 554 5.7.1 Service unavailable; Client host [130.43.53.23] blocked using zen.spamhaus.org; http://www.spamhaus.org/query/bl?ip=130.43.53.23; ... Apr 26 11:13:32 liam postfix/smtpd[22946]: connect from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:13:32 liam postfix/smtpd[22946]: NOQUEUE: reject: RCPT from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23]: 554 5.7.1 Service unavailable; Client host [130.43.53.23] blocked using zen.spamhaus.org; http://www.spamhaus.org/query/bl?ip=130.43.53.23; ARE using the Postfix 'internal' DNSBL cache? How do I learn/observe that use of the cached value from logs? If it's the latter, I understand that results are not shared with other Postfix processes. -- Thanks, Karen
Re: Can I improve the efficiency of my dnsbl reject configuration?
On Fri, Apr 27, 2012, at 06:43 PM, Bastian Blank wrote: > On Fri, Apr 27, 2012 at 08:55:15AM -0700, kar...@mailcan.com wrote: > > smtpd_recipient_restrictions = > > check_recipient_access hash:/usr/local/etc/postfix/conf/bozos > > Remove or at least move _after_ reject_unauth_destination. This is prone for > open relay. 'bozos' holds a list of "To:" addresses that I discard. I'm not clear why/how that's "prone for open relay". I'll look into it. > > reject_non_fqdn_recipient > > Why? Because countless documentation examples suggest it, including in "The Postfix Book". > > permit_sasl_authenticated > > permit_mynetworks > > reject_unauth_destination > > reject_unlisted_recipient > > reject_non_fqdn_sender > > reject_unknown_sender_domain > > reject_rbl_client zen.spamhaus.org > > reject_rbl_client b.barracudacentral.org > > permit > > Remove, it does nothing anyway. Remove what, specifically? All of those? What "does nothing"? -- Thanks, Karen
Re: Can I improve the efficiency of my dnsbl reject configuration?
Please respond to the list as well, thanks. On Fri, Apr 27, 2012, at 05:38 PM, Jim Reid wrote: > Er, think about this. How will postscreen do those RBL checks? Clearly, as I said I'm still reading, I'm not sure. > It will do DNS lookups! Right. The 1st time. And if it *was* capable of storing temporary/expirable data to a local in-memory hash table, it could be very efficient. I'm asking questions and thinking about things, not debating. -- Thanks, Karen
Re: Can I improve the efficiency of my dnsbl reject configuration?
On Fri, Apr 27, 2012, at 05:32 PM, Jim Reid wrote: > This is beginning to smell very > much like something the DNS already provides for free. If that auto-expiry hash table functionality is not already build into Postfix (which would be kind of nice to have for other things to; may look into it cobbling it up anyway), then that's a good point. At the moment, the only difference being the 'network traffic' to/from the DNS server. Likely not much of a burden. I could address it if need be by putting a caching DNS slave on the same box as the Postfix server. Overkill, it sounds like. -- Thanks, Karen
Re: Can I improve the efficiency of my dnsbl reject configuration?
On Fri, Apr 27, 2012, at 05:23 PM, Jim Reid wrote: > The info will already be cached at your local DNS server. So you've snip. Nicely explained. > My advice is to leave this alone. It's already working at maximum > efficiency pretty much straight out of the box and there are no > meaningful postfix (or DNS) configuration tweaks which could make > things even faster. I'd still think that a local check by Postfix to an 'auto-expiring hash table' (unclear so far it that can be done) to which the 'bad' address was added would still be more efficient and less overall network load than a query to locally cached DNS. I understand it's a matter of degree and from you explanation I may be down to splitting-hairs. > > I'll first ask how to do this without postscreen. > > postscreen is not the answer anyway. This is likely to be far, far > more expensive than a DNS lookup. So don't do that. :-) I haven't used postscreen yet -- still reading about it. I thought the idea behind it was to be very-lightweight rejection as far forward in the transaction process as possible. At least its use of DNSBL checks. -- Thanks, Karen
Re: Can I improve the efficiency of my dnsbl reject configuration?
On Fri, Apr 27, 2012, at 06:09 PM, Dennis Guhl wrote: > The caching is done in your local resolver, not in postfix. Ok, I can check that and make sure that those results are being returned from my LAN DNS server's cache. Is there any way to prevent Postfix from making those repeated DNS checks, regardless of whether it's externally to Spamhaus' servers, or to a locally cached DNS result? If an IP is identified as 'bad' via a DNSBL check once, that should be sufficient. Any subsequent instances of that IP should also be considered 'bad', if only for some user-defined period of time. -- Thanks, Karen
Can I improve the efficiency of my dnsbl reject configuration?
I just installed a Postfix server and enabled DNSBL-based rejection with smtpd_recipient_restrictions = check_recipient_access hash:/usr/local/etc/postfix/conf/bozos, reject_non_fqdn_recipient, permit_sasl_authenticated, permit_mynetworks, reject_unauth_destination, reject_unlisted_recipient, reject_non_fqdn_sender, reject_unknown_sender_domain, reject_rbl_client zen.spamhaus.org reject_rbl_client b.barracudacentral.org, permit It looks like it's working. Spam is getting rejected. In every case though there are multiple connections made with multiple rejects. For example Apr 26 11:13:07 liam postfix/smtpd[22946]: connect from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:13:08 liam postfix/smtpd[22946]: NOQUEUE: reject: RCPT from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23]: 554 5.7.1 Service unavailable; Client host [130.43.53.23] blocked using zen.spamhaus.org; http://www.spamhaus.org/query/bl?ip=130.43.53.23; from= to= proto=ESMTP helo= Apr 26 11:13:08 liam postfix/smtpd[22946]: lost connection after DATA from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:13:08 liam postfix/smtpd[22946]: disconnect from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:13:32 liam postfix/smtpd[22946]: connect from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:13:32 liam postfix/smtpd[22946]: NOQUEUE: reject: RCPT from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23]: 554 5.7.1 Service unavailable; Client host [130.43.53.23] blocked using zen.spamhaus.org; http://www.spamhaus.org/query/bl?ip=130.43.53.23; from= to= proto=ESMTP helo= Apr 26 11:13:33 liam postfix/smtpd[22946]: lost connection after DATA from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:13:33 liam postfix/smtpd[22946]: disconnect from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:13:59 liam postfix/smtpd[22946]: connect from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:13:59 liam postfix/smtpd[23175]: connect from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:14:00 liam postfix/smtpd[22946]: NOQUEUE: reject: RCPT from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23]: 554 5.7.1 Service unavailable; Client host [130.43.53.23] blocked using zen.spamhaus.org; http://www.spamhaus.org/query/bl?ip=130.43.53.23; from= to= proto=ESMTP helo= Apr 26 11:14:00 liam postfix/smtpd[23175]: NOQUEUE: reject: RCPT from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23]: 554 5.7.1 Service unavailable; Client host [130.43.53.23] blocked using zen.spamhaus.org; http://www.spamhaus.org/query/bl?ip=130.43.53.23; from= to= proto=ESMTP helo= Apr 26 11:14:00 liam postfix/smtpd[22946]: lost connection after DATA from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:14:00 liam postfix/smtpd[22946]: disconnect from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:14:00 liam postfix/smtpd[23175]: lost connection after DATA from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] Apr 26 11:14:00 liam postfix/smtpd[23175]: disconnect from 130.43.53.23.dsl.dyn.forthnet.gr[130.43.53.23] In the end it's getting blocked, and that's what I want. But, if I understand how this works, every one of those rejects is a DNS check to spamhaus, and some postfix load on my server. Can I somehow configure to be more efficient about this? Maybe somehow cache the rejected IP for 15mins or something? I'll first ask how to do this without postscreen. -- Thanks, Karen
