Re: fail2ban vs. syslogd compression (to be solved soon)
This one time, at band camp, Maxim Kammerer said: > I have no clue what this patch looks like. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=440037 -- - | ,''`.Stephen Gran | | : :' :[EMAIL PROTECTED] | | `. `'Debian user, admin, and developer | |`- http://www.debian.org | - signature.asc Description: Digital signature
Re: fail2ban vs. syslogd compression (to be solved soon)
On Thu, 6 Sep 2007, Maxim Kammerer wrote: Just to conclude the thread: I wrote to the author, Cyril Jaquier. This was his answer: I have received a patch from Stephen Gram for this. I will review this as soon as possible and will commit this to 0.8 branch. So it should be available in the next 0.8 release. I have no clue what this patch looks like. Maxim -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED] I read in netbsd or freebsd (or maybe it was dragonfly) they have an option to turn it off, this will be great if it is implemented in syslogd for Linux! Justin. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: fail2ban vs. syslogd compression
On Thu, Aug 30, 2007 at 11:42:03AM +0200, Maxim Kammerer wrote: > 1) Clarify if it is really true that the message "last message repeated \d+ > times" does not always refer to the last message, as suggested in one post. > I thought that syslogd's raison d'etre was exactly to provide a unified > tracking system for log messages, so it really should know where it's The standard network protocol used to send log messages over the network is not reliable so as soon as you start processing logs on a remote host missing messages become a very real possibility. -- "You grabbed my hand and we fell into it, like a daydream - or a fever." signature.asc Description: Digital signature
Re: fail2ban vs. syslogd compression
On Fri, Aug 31, 2007 at 12:34:13PM +0100, G.W. Haywood wrote: > Most people on dynamic IPs don't have the same address for more than a > day! Yes, you'll be an innocent victim of the spammers, but normally > only if you try to send mail directly to our mailservers. In which > case we don't want it, thank you, because in that case your computer > has probably been compromised. (You wouldn't want to be making other > kinds of connections to our mailservers, would you? :) Your computer > should use your service provider's mailservers to send your mail to > our mailservers. If you run a mailserver it should be on a static IP > and it, along with your DNS data, should be properly configured. Most people on "dynamic" IPs assigned by DSL and cable networks have the same IP for months at a time. Sometimes years. I had the same IP address for three years, despite the MediaOne being bought by AT&T and then by Comcast. I run a mailserver for personal use. I don't trust mailservers outside of my control, and history seems to have proven me right. It has a CNAME through DynDNS. Would you like to guess how many domains won't accept email from me because of that dynamic IP? 26. At least, that's the number of domains that I specially route mail through a friend's box with a static IP. > Incidentally we also block _all_ connections (not just mail) from most > of Africa, Arab countries, Bangladesh, Canada, China, Denmark, Eastern > Europe, France, India, Israel, Italy, Portugal, Russia, South America, > Spain, Taiwan, Turkey... You don't do business with anyone in any of those countries? Hrm. I highly encourage you to evangelize your methods among my competitors. -dsr- -- Every time you give up a right, the terrorists win. http://tao.merseine.nu/~dsr/eula.html is hereby incorporated by reference. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: fail2ban vs. syslogd compression
Hi there, On Thu, 30 Aug 2007, Jack T Mudge III wrote: > On Wednesday 29 August 2007 03:56, G.W. Haywood wrote: > > Most offenders > > are blocked permanently, at the last count we're blocking about 27,750 > > ranges. ?Our scripts could handle the 'repeat' messages if they needed > > to, but they don't. ?The script kiddies don't get five tries, we block > > them after the first. :) > > Forgive me, but as I understand IP and the whole DHCP concept and whatnot, IP > addresses ARE reused after some time. I rarely have the same internet address > for more than a month -- and if I randomly ended up with one of your blocked > addresses, wouldn't I be an innocent victim? You're forgiven. :) Most people on dynamic IPs don't have the same address for more than a day! Yes, you'll be an innocent victim of the spammers, but normally only if you try to send mail directly to our mailservers. In which case we don't want it, thank you, because in that case your computer has probably been compromised. (You wouldn't want to be making other kinds of connections to our mailservers, would you? :) Your computer should use your service provider's mailservers to send your mail to our mailservers. If you run a mailserver it should be on a static IP and it, along with your DNS data, should be properly configured. One problem is that computers in these botnets are programmed to seem at least superficially to be real mailservers, which they aren't, and if we let them they'd fill our logs with so much garbage that the real information would be totally obliterated. Another problem is that we pay for the bandwidth, 95% of which would be consumed by criminals if we let them do it. > Given the dynamic nature of the internet in general, doesn't it make more > sense to block for, maybe 2 months, tops? No. Most dynamic ranges are huge blocks owned by the likes of NTL, Wanadoo, Verizon, Bellsouth, Covad, Roadrunner... There are 207 ISPs in our blacklist at present. One of the problems is that if you block a single dynamic IP, then a few minutes later that same compromised PC just comes back again trying from a different IP in the same ISP's blocks of dynamic addresses. So we block the whole lot as soon as we can. The ISPs could all _easily_ stop the huge botnets using their services sending spam email to millions of people every second. But they don't bother - some of them even ignore the police (*) when they're notified of fraudsters using their networks - so I and other overloaded admins like me have to deal with all this crap instead. > This isn't meant to downcast your job or anything, I'd just like to know the > reasoning behind permanent versus temporary blocks (I use temporary, and it's > always done well for me). I understand. The reason is experience. The fact is that any dynamic IP is eventually going to be a source of crap so we block every last one we can find. There are databases of dynamic IPs from the likes of SBL, we use them too but I'm afraid they're far from complete. Incidentally we also block _all_ connections (not just mail) from most of Africa, Arab countries, Bangladesh, Canada, China, Denmark, Eastern Europe, France, India, Israel, Italy, Portugal, Russia, South America, Spain, Taiwan, Turkey... > fail2ban blocks for 10 minutes; 10 minutes has thus far been enough to stop > all but the most determined script kiddies, who are then blocked again (and > again until they stop). Ten minutes is a little short in my experience, but yes the bulk of the problems is dealt with by a temporary block. Unfortunately there are hard-core cases which temporary blocks will not deal with, hence the permanent blocks. I have logs showing PCs which have been trying to send crap to us for many months from many different IPs. Sending mail to the the abuse department at Telstra, for example, is in my experience a complete waste of time. One of their customers has been trying to send mail to us every ten minutes since May. > Even using a 450mhz pentium II for my router/firewall, it's not even > a noticeable load on the system. The load on the system isn't the issue. It's the load on the system administrator. I actually look at my logs, and if they're so full of crap that I can't see the things I need to see, I may as well not bother. Then I might miss something important. A potential sale, maybe. -- 73, Ged. (*) I contact the police about serious fraud attempts. My experience is that the police are as frustrated with irresponsible ISPs as I am.
Re: fail2ban vs. syslogd compression
On Wednesday 29 August 2007 03:56, G.W. Haywood wrote: > Most offenders > are blocked permanently, at the last count we're blocking about 27,750 > ranges. Our scripts could handle the 'repeat' messages if they needed > to, but they don't. The script kiddies don't get five tries, we block > them after the first. :) Forgive me, but as I understand IP and the whole DHCP concept and whatnot, IP addresses ARE reused after some time. I rarely have the same internet address for more than a month -- and if I randomly ended up with one of your blocked addresses, wouldn't I be an innocent victim? Given the dynamic nature of the internet in general, doesn't it make more sense to block for, maybe 2 months, tops? This isn't meant to downcast your job or anything, I'd just like to know the reasoning behind permanent versus temporary blocks (I use temporary, and it's always done well for me). fail2ban blocks for 10 minutes; 10 minutes has thus far been enough to stop all but the most determined script kiddies, who are then blocked again (and again until they stop). Even using a 450mhz pentium II for my router/firewall, it's not even a noticeable load on the system. -- Sincerely, Jack [EMAIL PROTECTED] My GPG Public Key can be found at: https://www.theanythingbox.com/pgp.htm (top link is current) I appreciate signatures, but if you only know me online, please use the --lsign-key, not the --sign-key. I appreciate trust -- but too much makes it less valuable. pgpIp28y3ZZiN.pgp Description: PGP signature
Re: fail2ban vs. syslogd compression
Ok, thanx to everybody for the advice. I am no step closer to a solution however. I see different routes: 1) Clarify if it is really true that the message "last message repeated \d+ times" does not always refer to the last message, as suggested in one post. I thought that syslogd's raison d'etre was exactly to provide a unified tracking system for log messages, so it really should know where it's messages came from and should take great pains in keeping its output sound. Otherwise, that would be a serious bug, wouldn't it? If the messages are reliable, which I tend to assume, then the obvious patch to fail2ban should work. Unfortunately I can't read greek, so I don't know if more detailed problems are mentioned in the referred to post from greek lug. 2) The other idea is to keep seperate temporary logs, with anti-syslog-compression. it really raises the effort needed to maintain the system (thus makes it likely to break). When I find some time, I'll get in touch with the fail2ban-developers. I am back on the list once I head something useful. Thanks again. Maxim -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: fail2ban vs. syslogd compression
Hi there, On Tue, 28 Aug 2007, Maxim Kammerer wrote: > I believe this belongs to the security-mailing list. Agreed. :) > ... pop3-cracking attempts ... stupid ... There's a lot of it about. They'll try ftp, irc, ssh and http as well. In fact they'll try anything that offers them a connection. There's a moral in there somewhere. > ... fail2ban didn't respond. ... 'last message repeated 4 times', > which is not helpful at all to fail2ban. You might call that a bug in fail2ban, but I think the authors are aware of the issue and it's not necessarily very easy to deal with. http://lists.hellug.gr/pipermail/linux-greek-users/2007-April/068169.html You need to be careful that the 'last message repeated' really was the last message you saw from the log stream. Messages from multiple log relays could easily confuse the logging process(es), so it's best to deal with it upstream. > However, I consider it a realworld scenario that a cracker/script > kiddy would hit the server in a short time. It does happen. :( > I then sought to disable this kind of log compression I've seen 'Last message repeated 3785 times'. =:0 > So I ended up with not knowing what to do and turned to the debian > security list. you people have any idea, or what are you doing? On our public servers we use syslog-ng, partly because it doesn't do message aggregation but mostly because it's more easily configured than the usual syslogd. We rolled our own log message analysis tools. Connections (+attempts) are piped via syslog-ng through Perl scripts. Including a generous amounts of commenting and whitespace there are about 1700 lines of Perl, so it's quite a bit of work but it saves an enormous amount of time trawling logs and stops much skulduggery. (M-x ispell-word... :) This is mostly about stopping spam, and blocking on IP stops over 90% of spam attempts, but attempts to hack our Webservers is also an issue. As you say, they're stupid. Most attacks are attempts to compromise Microsoft software - we don't run any - and the bulk of the rest are either attempts to exploit known vulnerabilities in PHP, Cacti and other stuff that we also don't run, or attempts to use our servers as free proxies. The scripts log to a database, maintain hashes of the connections with some timeouts, implement port knocking for ssh access, and other stuff like that. We block an IP (using iptables & ipsets) at the slightest provocation and we never block less than a /24 range. Most offenders are blocked permanently, at the last count we're blocking about 27,750 ranges. Our scripts could handle the 'repeat' messages if they needed to, but they don't. The script kiddies don't get five tries, we block them after the first. :) -- 73, Ged. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: fail2ban vs. syslogd compression
In article <[EMAIL PROTECTED]> you wrote: >> Wouldn't a better option be to teach fail2ban how to parse the "last >> message repeated".. messages? > > Maxim or Dann: When you find out how to do that, please post it to the list > for archiving / information-sharing purposes. I can tell you the obvious: rember last and current line. If current line!="last message repeated" then store it as last line and read next line as current otherwise increment counter of the entry pointed to in last line by the number of lines skiped and read next line as current. *g* Sorry no coding today :) Gruss Bernd -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: fail2ban vs. syslogd compression
On Tuesday 28 August 2007 12:24, dann frazier wrote: > On Tue, Aug 28, 2007 at 12:43:10PM +0200, Maxim Kammerer wrote: > > > > I then sought to disable this kind of log compression, but it is not > > stated in the man pages how to do that. > > > > So I ended up with not knowing what to do and turned to the debian > > security list. you people have any idea, or what are you doing? > > Wouldn't a better option be to teach fail2ban how to parse the "last > message repeated".. messages? Maxim or Dann: When you find out how to do that, please post it to the list for archiving / information-sharing purposes. Thanks. -- -- System Administrator - Cedar Creek Software http://www.cedarcreeksoftware.com -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: fail2ban vs. syslogd compression
On Tue, Aug 28, 2007 at 12:43:10PM +0200, Maxim Kammerer wrote: > Hello everybody, > > I believe this belongs to the security-mailing list. I recently took a > server online and it was immediately hit by pop3-cracking attempts. Well, > they were quite stupid, since they were attempting once for each name taken > from a 'frequent names list', so I guess somebody was looking for > non-password protected accounts. However, being annoyed, I wanted to tweak > fail2ban, which I am already using for ssh, to pop3 and imap, too. No > problem, standard debian /etc/fail2ban/jail.conf issue has the relevant > sections, so I went ahead. > > But then I ran a test, and fail2ban didn't respond. The reason was that I > hit the server 5 times (my fail2ban max-retry) in quite a short time, so > instead of logging 'pop3: login failed ' 5 times to mail.log, it > logged the message once and afterwards issued 'last message repeated 4 > times', which is not helpful at all to fail2ban. However, I consider it a > realworld scenario that a cracker/script kiddy would hit the server in a > short time. > > I then sought to disable this kind of log compression, but it is not stated > in the man pages how to do that. While the freebsd syslogd seems to have > such a commandline switch (-c -c ), the syslogd shipped with debian doesn't > have it, and syslogd-ng seems to not have it, either. > > So I ended up with not knowing what to do and turned to the debian security > list. you people have any idea, or what are you doing? Wouldn't a better option be to teach fail2ban how to parse the "last message repeated".. messages? -- dann frazier -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
