Re: [Dovecot] Fail2Ban and the Dovecot log
On Sun, 2009-05-17 at 15:28 -0400, Lou Duchez wrote: > > Yeah. I don't know what I was thinking when I made it work like that. > > > I know what you were thinking: if dovecot is writing to a log such as > "mylogfile.log", and other utilities are also writing to > "mylogfile.log", it's good to know which lines are dovecot. I think it's a bit unlikely use case. Also seems like a bad idea to combine logs in that way. :) There is anyway still some kind of a prefix always, such as: May 17 17:19:52 dovecot: Info: Dovecot v1.2.rc3 starting up May 17 17:19:52 auth(default): Info: passwd-file /usr/local/etc/passwd.imap: Read 4 users > But I am satisfied with using syslog logging; it just should be recorded > somewhere that syslog is required for compatibility with Fail2Ban. I > tried to edit wiki.dovecot.org with this information, but was too > incompetent to figure out how to add a page. You can just write the page name to the URL and then click "Create this page" link. Anyway I added it: http://wiki.dovecot.org/HowTo/Fail2Ban signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Fail2Ban and the Dovecot log
Timo Sirainen wrote: > On Mon, 2009-05-11 at 14:48 -0700, Bill Landry wrote: >>> If you log via syslog, the timestamp will be at the beginning of line. >> >> Well, then that would explain it. Maybe it would be a good idea then to >> remove the "dovecot: " from the beginning of each log line when not >> using syslog for logging, since I'm pretty sure that anyone checking the >> "dovecot.log" would know that it was the "dovecot" log they were looking >> at (no need to remind then at the beginning of each and every line). ;-) > > Yeah. I don't know what I was thinking when I made it work like that. > v1.2 has more sensible logging now, but I don't want to break anyone's > setup by doing the change to v1.1. These patches probably apply to v1.1 > too though: > > http://hg.dovecot.org/dovecot-1.2/rev/0669bc561763 > http://hg.dovecot.org/dovecot-1.2/rev/194b80691980 Thanks Timo, your responsiveness to requests (no matter how mundane at times) is always very much appreciated! Bill
Re: [Dovecot] Fail2Ban and the Dovecot log
Yeah. I don't know what I was thinking when I made it work like that. I know what you were thinking: if dovecot is writing to a log such as "mylogfile.log", and other utilities are also writing to "mylogfile.log", it's good to know which lines are dovecot. But I am satisfied with using syslog logging; it just should be recorded somewhere that syslog is required for compatibility with Fail2Ban. I tried to edit wiki.dovecot.org with this information, but was too incompetent to figure out how to add a page. If I had to create a page with Fail2Ban instructions, it would look like: 1) Make sure that /etc/dovecot.conf does not have any “log_path” variable set. We need dovecot.conf to use the default system logging so the log is written in a format that fail2ban can work with. 2) Create the filter file /etc/fail2ban/filter.d/dovecot-pop3imap.conf: [Definition] failregex = (?: pop3-login|imap-login): (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed).*rip=(?P\S*),.* ignoreregex = 3) Add the following to /etc/fail2ban/jail.conf: [dovecot-pop3imap] enabled = true filter = dovecot-pop3imap action = iptables-multiport[name=dovecot-pop3imap, port="pop3,imap", protocol=tcp] logpath = /var/log/maillog maxretry = 20 findtime = 1200 bantime = 1200
Re: [Dovecot] Fail2Ban and the Dovecot log
On Mon, 2009-05-11 at 14:48 -0700, Bill Landry wrote: > > If you log via syslog, the timestamp will be at the beginning of line. > > Well, then that would explain it. Maybe it would be a good idea then to > remove the "dovecot: " from the beginning of each log line when not > using syslog for logging, since I'm pretty sure that anyone checking the > "dovecot.log" would know that it was the "dovecot" log they were looking > at (no need to remind then at the beginning of each and every line). ;-) Yeah. I don't know what I was thinking when I made it work like that. v1.2 has more sensible logging now, but I don't want to break anyone's setup by doing the change to v1.1. These patches probably apply to v1.1 too though: http://hg.dovecot.org/dovecot-1.2/rev/0669bc561763 http://hg.dovecot.org/dovecot-1.2/rev/194b80691980 signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Fail2Ban and the Dovecot log
Ed W wrote: Just when I think I've achieved ultimate pefection on this, someone comes along with a great idea. Thanks! ... action = iptables-multiport[name=smtppop3imap, port="smtp,pop3,imap", protocol=tcp] Can I suggest the name "mail" would summarise the stack of items above? Did you test this - is it correct? Totally works! Though the setting of "2" retries is a bit severe; "4" is probably more reasonable.
Re: [Dovecot] Fail2Ban and the Dovecot log
Just when I think I've achieved ultimate pefection on this, someone comes along with a great idea. Thanks! ... action = iptables-multiport[name=smtppop3imap, port="smtp,pop3,imap", protocol=tcp] Can I suggest the name "mail" would summarise the stack of items above? Did you test this - is it correct? Cheers Ed W
Re: [Dovecot] Fail2Ban and the Dovecot log
Ed W wrote: Lou Duchez wrote: So any failure at any of the three protocols (SMTP, POP3, IMAP) is considered a "strike" by all three, and they should all ban the same guys at the same time. This is as yet untested, but seems like it should be pretty sound. I think you only need one service and you can use the iptables-multi (or something similar) to block all the ports if you get a hit? Ed W !!! Just when I think I've achieved ultimate pefection on this, someone comes along with a great idea. Thanks! So I guess we take out the "sasl-iptables" part of jail.conf and replace it with: [smtppop3imap] enabled = true filter = smtppop3imap action = iptables-multiport[name=smtppop3imap, port="smtp,pop3,imap", protocol=tcp] logpath = /var/log/maillog ignoreip = 192.168.1.0/24 123.123.123.123/27 234.234.234.234 maxretry = 2 findtime = 1200 bantime = 1200 smtppop3imap.conf is as previously described: [Definition] failregex = : warning: [-._\w]+\[\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed (?: pop3-login|imap-login): (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed).*rip=(?P\S*),.* ignoreregex =
Re: [Dovecot] Fail2Ban and the Dovecot log
Lou Duchez wrote: Ed W wrote: Lou Duchez wrote: This arrangement is designed to trap POP3 and IMAP separately, and also to allow a high number of errors before temporarily "jailing" a user. This is to decrease the likelihood that a single user from a single IP will get all his coworkers (temporarily) banned over an honest mistake in configuration. I have noticed recent breaking attempts which appear to be a slow coordinated botnet using multiple IPs and trying a combination of SMTP + POP + IMAP (can't remember if it did both of the later or just POP?). As a result I tried to combine all three into a single test. Actually I did the wrong thing, but if you look through my previous posts you can see someone (Bill?) correct me and post the correct config for this I would recommend you be aware of this - in my case I was seeing less than a few attempts from a given IP in a 10 min period, but lots of what appeared to be coordinated attempts at the server level. (eg some servers were only trying a few logins per day, but across enough IP addresses this was a fairly rapidly filling the logs) Good luck Ed W Thanks for the heads-up! Okay then, perhaps the best solution is to make use of the "ignoreip" setting in jail.conf to protect known IP addresses, something like this: (snip) Or even smarter: create a single filter file called smtppop3imap.conf, and use that same filter for SMTP, POP3, and IMAP. Here's what the filter would look like: [Definition] failregex = : warning: [-._\w]+\[\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed (?: pop3-login|imap-login): (?:Authentication failure|Aborted login \(auth failed|Disconnected \(auth failed).*rip=(?P\S*),.* ignoreregex = The first regex will cover SMTP authentication errors generated by Postfix. The second regex is for Dovecot and authentication errors with POP3 and IMAP. Sorry to keep posting iterative improvements; every time I think I'm done, I come up with something better (and perhaps worth sharing).
Re: [Dovecot] Fail2Ban and the Dovecot log
Lou Duchez wrote: So any failure at any of the three protocols (SMTP, POP3, IMAP) is considered a "strike" by all three, and they should all ban the same guys at the same time. This is as yet untested, but seems like it should be pretty sound. I think you only need one service and you can use the iptables-multi (or something similar) to block all the ports if you get a hit? Ed W
Re: [Dovecot] Fail2Ban and the Dovecot log
Ed W wrote: Lou Duchez wrote: This arrangement is designed to trap POP3 and IMAP separately, and also to allow a high number of errors before temporarily "jailing" a user. This is to decrease the likelihood that a single user from a single IP will get all his coworkers (temporarily) banned over an honest mistake in configuration. I have noticed recent breaking attempts which appear to be a slow coordinated botnet using multiple IPs and trying a combination of SMTP + POP + IMAP (can't remember if it did both of the later or just POP?). As a result I tried to combine all three into a single test. Actually I did the wrong thing, but if you look through my previous posts you can see someone (Bill?) correct me and post the correct config for this I would recommend you be aware of this - in my case I was seeing less than a few attempts from a given IP in a 10 min period, but lots of what appeared to be coordinated attempts at the server level. (eg some servers were only trying a few logins per day, but across enough IP addresses this was a fairly rapidly filling the logs) Good luck Ed W Thanks for the heads-up! Okay then, perhaps the best solution is to make use of the "ignoreip" setting in jail.conf to protect known IP addresses, something like this: [sasl-iptables] enabled = true backend = polling filter = sasl action = iptables[name=sasl, port=smtp, protocol=tcp] logpath = /var/log/maillog ignoreip = 192.168.1.0/24 123.456.543.210/28 321.654.123.456 maxretry = 2 findtime=1200 bantime = 1200 [dovecot-pop3] enabled = true filter = dovecot-pop3 action = iptables[name=POP3, port=pop3, protocol=tcp] logpath = /var/log/maillog ignoreip = 192.168.1.0/24 123.456.543.210/28 321.654.123.456 maxretry = 2 findtime=1200 bantime = 1200 [dovecot-imap] enabled = true filter = dovecot-imap action = iptables[name=IMAP, port=imap, protocol=tcp] logpath = /var/log/maillog ignoreip = 192.168.1.0/24 123.456.543.210/28 321.654.123.456 maxretry = 2 findtime=1200 bantime = 1200 Note that SMTP, POP3, and IMAP are all looking at the same log file, they all have the same ban parameters (more aggressive than previously proposed), and they all ignore the same IP ranges (in this case a local subnet, a range of public IPs, and one additional public IP). Then in keeping with this, all three filter files (sasl.conf, dovecot-pop3.conf, and dovecot-imap.conf) would have identical configurations: [Definition] failregex = : warning: [-._\w]+\[\]: SASL (?:LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed (?: pop3-login: Authentication failure).*rip=(?P\S*),.* (?: pop3-login: Aborted login \(auth failed).*rip=(?P\S*),.* (?: pop3-login: Disconnected \(auth failed).*rip=(?P\S*),.* (?: imap-login: Authentication failure).*rip=(?P\S*),.* (?: imap-login: Aborted login \(auth failed).*rip=(?P\S*),.* (?: imap-login: Disconnected \(auth failed).*rip=(?P\S*),.* ignoreregex = So any failure at any of the three protocols (SMTP, POP3, IMAP) is considered a "strike" by all three, and they should all ban the same guys at the same time. This is as yet untested, but seems like it should be pretty sound.
Re: [Dovecot] Fail2Ban and the Dovecot log
Lou Duchez wrote: This arrangement is designed to trap POP3 and IMAP separately, and also to allow a high number of errors before temporarily "jailing" a user. This is to decrease the likelihood that a single user from a single IP will get all his coworkers (temporarily) banned over an honest mistake in configuration. I have noticed recent breaking attempts which appear to be a slow coordinated botnet using multiple IPs and trying a combination of SMTP + POP + IMAP (can't remember if it did both of the later or just POP?). As a result I tried to combine all three into a single test. Actually I did the wrong thing, but if you look through my previous posts you can see someone (Bill?) correct me and post the correct config for this I would recommend you be aware of this - in my case I was seeing less than a few attempts from a given IP in a 10 min period, but lots of what appeared to be coordinated attempts at the server level. (eg some servers were only trying a few logins per day, but across enough IP addresses this was a fairly rapidly filling the logs) Good luck Ed W
Re: [Dovecot] Fail2Ban and the Dovecot log
On Mon, 11 May 2009 15:56:45 -0400 Lou Duchez wrote: > Hi, > > Is there any way to disable the "dovecot: " at the beginning of each > line of the log? Fail2Ban responds poorly to it. I know there are a > number of sites with "failregex" strings for Fail2Ban and Dovecot, but > I've tried them all, and they don't work, at least with the latest > Fail2ban and the latest Dovecot. The Fail2Ban wiki is pretty clear > about why there will be a problem: > > "In order for a log line to match your failregex, it actually has to > match in two parts: the beginning of the line has to match a timestamp > pattern or regex, and the remainder of the line has to match your > failregex.". > > So in other words, Fail2Ban expects that each line of the log will start > with a timestamp. > > Thanks all! Dovecot rocks. > Well, this is not completely true... I have a working fail2ban config using the dovecot log file, not syslog, and it's working fine... I had to change the date format for the log file, but after doing that, the fail2ban works as it should... BTJ
Re: [Dovecot] Fail2Ban and the Dovecot log
Maybe there could be a page in the dovecot wiki about Fail2Ban? A definitive Dovecot / Fail2ban resource would be useful. (If nobody else creates one in a week, perhaps I will. But I have to perfect my Fail2banning first ...) I couldn't figure out how to add new pages to wiki.dovecot.org, but here is what I have come up with for an easy Fail2ban recipe: --- Configuring Fail2Ban with Dovecot 1) Make sure your dovecot "log_path" string is empty, in other words allow syslog to do your Dovecot logging (into the default mail log). This ensures that the log entries will be in a format Fail2ban can work with. 2) In your Fail2ban "jail.conf" file (most likely /etc/fail2ban/jail.conf), add entries like the following: [dovecot-pop3] enabled = true filter = dovecot-pop3 action = iptables[name=POP3, port=pop3, protocol=tcp] logpath = /var/log/maillog maxretry = 20 bantime = 1200 [dovecot-imap] enabled = true filter = dovecot-imap action = iptables[name=IMAP, port=imap, protocol=tcp] logpath = /var/log/maillog maxretry = 20 bantime = 1200 This arrangement is designed to trap POP3 and IMAP separately, and also to allow a high number of errors before temporarily "jailing" a user. This is to decrease the likelihood that a single user from a single IP will get all his coworkers (temporarily) banned over an honest mistake in configuration. 3) Create a jail called dovecot-pop3.conf (most likely as /etc/fail2ban/filter.d/dovecot-pop3.conf): [Definition] failregex = (?: pop3-login: Authentication failure).*rip=(?P\S*),.* (?: pop3-login: Aborted login).*rip=(?P\S*),.* (?: pop3-login: Disconnected).*rip=(?P\S*),.* ignoreregex = 4) Create a jail called dovecot-imap3.conf (most likely as /etc/fail2ban/filter.d/dovecot-imap.conf): [Definition] failregex = (?: imap-login: Authentication failure).*rip=(?P\S*),.* (?: imap-login: Aborted login).*rip=(?P\S*),.* (?: imap-login: Disconnected).*rip=(?P\S*),.* ignoreregex = 5) Restart Fail2ban.
Re: [Dovecot] Fail2Ban and the Dovecot log
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, 11 May 2009, Bill Landry wrote: Well, then that would explain it. Maybe it would be a good idea then to remove the "dovecot: " from the beginning of each log line when not using syslog for logging, since I'm pretty sure that anyone checking the "dovecot.log" would know that it was the "dovecot" log they were looking at (no need to remind then at the beginning of each and every line). ;-) Well, I use it to _detect_ I'm parsing the Dovecot logs ;-) Bye, - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iQEVAwUBSgkx5HWSIuGy1ktrAQKxtgf9EsOByCghSTNLm6bZpg/kddMj+aErCGT+ c7ARtutq58ap3t6aBiolizn/MIBmdYqYgSP25MomeyJGPxWMZbFx0B5Ra1G9sVZV Whh0cLTt6zLmBa9Dq5SCkmuf9njVpXEb0fzWGc4rTpunnlD9e69T5EfWmuUofsV8 1b6HgMUg+2BhGw+edJCYXWh0GHQixzxh/QnFD236wQ8jY7BHXl2niTuQPLcPHmvS NPe61QE4M5F2m4xCUVmg8ODmjm+cRCgVG+8SsCJe4ZHmcVhHZ48cyHkOstTGNHDf uEURelkMmd0Wb9OgDQOWAzcdRuz0u+cP3X7Pn64So/mhF9REMq4yLg== =2zlI -END PGP SIGNATURE-
Re: [Dovecot] Fail2Ban and the Dovecot log
Bill Landry wrote: Timo Sirainen wrote: On Mon, 2009-05-11 at 17:15 -0400, Lou Duchez wrote: Re: the "dovecot: " at the beginning of the line in the log. I should mention that other applications encounter a similar issue with Fail2Ban -- for example, if you're running Asterisk, you have to alter the log format such that the timestamp is at the beginning of the line: If you log via syslog, the timestamp will be at the beginning of line. Well, then that would explain it. Maybe it would be a good idea then to remove the "dovecot: " from the beginning of each log line when not using syslog for logging, since I'm pretty sure that anyone checking the "dovecot.log" would know that it was the "dovecot" log they were looking at (no need to remind then at the beginning of each and every line). ;-) Bill Maybe there could be a page in the dovecot wiki about Fail2Ban? A definitive Dovecot / Fail2ban resource would be useful. (If nobody else creates one in a week, perhaps I will. But I have to perfect my Fail2banning first ...) Thanks, guys, for helping me out!
Re: [Dovecot] Fail2Ban and the Dovecot log
Timo Sirainen wrote: > On Mon, 2009-05-11 at 17:15 -0400, Lou Duchez wrote: >> Re: the "dovecot: " at the beginning of the line in the log. I should >> mention that other applications encounter a similar issue with Fail2Ban >> -- for example, if you're running Asterisk, you have to alter the log >> format such that the timestamp is at the beginning of the line: > > If you log via syslog, the timestamp will be at the beginning of line. Well, then that would explain it. Maybe it would be a good idea then to remove the "dovecot: " from the beginning of each log line when not using syslog for logging, since I'm pretty sure that anyone checking the "dovecot.log" would know that it was the "dovecot" log they were looking at (no need to remind then at the beginning of each and every line). ;-) Bill
Re: [Dovecot] Fail2Ban and the Dovecot log
On Mon, 2009-05-11 at 17:15 -0400, Lou Duchez wrote: > Re: the "dovecot: " at the beginning of the line in the log. I should > mention that other applications encounter a similar issue with Fail2Ban > -- for example, if you're running Asterisk, you have to alter the log > format such that the timestamp is at the beginning of the line: If you log via syslog, the timestamp will be at the beginning of line. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Fail2Ban and the Dovecot log
Re: the "dovecot: " at the beginning of the line in the log. I should mention that other applications encounter a similar issue with Fail2Ban -- for example, if you're running Asterisk, you have to alter the log format such that the timestamp is at the beginning of the line: http://www.voip-info.org/wiki/view/Fail2Ban+(with+iptables)+And+Asterisk
Re: [Dovecot] Fail2Ban and the Dovecot log
Original-Nachricht > Datum: Mon, 11 May 2009 15:56:45 -0400 > Von: Lou Duchez > An: dovecot@dovecot.org > Betreff: [Dovecot] Fail2Ban and the Dovecot log > Hi, > Hello > Is there any way to disable the "dovecot: " at the beginning of each > line of the log? Fail2Ban responds poorly to it. I know there are a > number of sites with "failregex" strings for Fail2Ban and Dovecot, but > I've tried them all, and they don't work, at least with the latest > Fail2ban and the latest Dovecot. The Fail2Ban wiki is pretty clear > about why there will be a problem: > > "In order for a log line to match your failregex, it actually has to > match in two parts: the beginning of the line has to match a timestamp > pattern or regex, and the remainder of the line has to match your > failregex.". > > So in other words, Fail2Ban expects that each line of the log will start > with a timestamp. > Could you attach a example log and tell us what you would like to match in that log. > Thanks all! Dovecot rocks. -- Neu: GMX FreeDSL Komplettanschluss mit DSL 6.000 Flatrate + Telefonanschluss für nur 17,95 Euro/mtl.!* http://dslspecial.gmx.de/freedsl-surfflat/?ac=OM.AD.PD003K11308T4569a
Re: [Dovecot] Fail2Ban and the Dovecot log
Bill Landry wrote: > Lou Duchez wrote: > >> Is there any way to disable the "dovecot: " at the beginning of each >> line of the log? Fail2Ban responds poorly to it. I know there are a >> number of sites with "failregex" strings for Fail2Ban and Dovecot, but >> I've tried them all, and they don't work, at least with the latest >> Fail2ban and the latest Dovecot. The Fail2Ban wiki is pretty clear >> about why there will be a problem: >> >> "In order for a log line to match your failregex, it actually has to >> match in two parts: the beginning of the line has to match a timestamp >> pattern or regex, and the remainder of the line has to match your >> failregex.". >> >> So in other words, Fail2Ban expects that each line of the log will start >> with a timestamp. > > Hmmm, I'm using: > > dovecot --version > 1.2.rc3 > > rpm -q fail2ban > fail2ban-0.8.3-18.fc10.noarch > > and this seems to work just fine for me: > >failregex = auth.*passwd.*,\).*(unknown user|Password mismatch) > > in my /etc/fail2ban/filter.d/dovecot.conf. Oh, and you can test this with: fail2ban-regex /path/to/dovecot.log "auth.*passwd.*,\).*(unknown user|Password mismatch)" Adjust the path in the string above to point to your dovecot.log file. Bill
Re: [Dovecot] Fail2Ban and the Dovecot log
Lou Duchez wrote: > Is there any way to disable the "dovecot: " at the beginning of each > line of the log? Fail2Ban responds poorly to it. I know there are a > number of sites with "failregex" strings for Fail2Ban and Dovecot, but > I've tried them all, and they don't work, at least with the latest > Fail2ban and the latest Dovecot. The Fail2Ban wiki is pretty clear > about why there will be a problem: > > "In order for a log line to match your failregex, it actually has to > match in two parts: the beginning of the line has to match a timestamp > pattern or regex, and the remainder of the line has to match your > failregex.". > > So in other words, Fail2Ban expects that each line of the log will start > with a timestamp. Hmmm, I'm using: dovecot --version 1.2.rc3 rpm -q fail2ban fail2ban-0.8.3-18.fc10.noarch and this seems to work just fine for me: failregex = auth.*passwd.*,\).*(unknown user|Password mismatch) in my /etc/fail2ban/filter.d/dovecot.conf. Bill
[Dovecot] Fail2Ban and the Dovecot log
Hi, Is there any way to disable the "dovecot: " at the beginning of each line of the log? Fail2Ban responds poorly to it. I know there are a number of sites with "failregex" strings for Fail2Ban and Dovecot, but I've tried them all, and they don't work, at least with the latest Fail2ban and the latest Dovecot. The Fail2Ban wiki is pretty clear about why there will be a problem: "In order for a log line to match your failregex, it actually has to match in two parts: the beginning of the line has to match a timestamp pattern or regex, and the remainder of the line has to match your failregex.". So in other words, Fail2Ban expects that each line of the log will start with a timestamp. Thanks all! Dovecot rocks.
