Re: Botnet FPs from Webmail Senders
Andy Dorman wrote on Wed, 28 Jan 2009 11:03:01 -0600: We were just hoping someone else in this group would have run into this situation and have a suggestion, given how ubiquitous webmail is these days. If not, we will keep pounding at it. Well, the easiest and fastest solution is to go to the Horde code and change the few bits in it, so that it resembles the authentication line that gets added to the typical SMTP AUTH transaction. And submit it as a feature request to the Horde developers. Or remove that line completely and/or change to an informational header, like: X-Submit-Host: (rbn1s-216-180-93-118.adsl.hiwaay.net [216.180.93.118]) Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
Re: Botnet FPs from Webmail Senders
On 28-Jan-2009, at 10:03, Andy Dorman wrote: Received: by beatrice.ironicdesign.com (Postfix, from userid 112) id E92BC148C16A; Fri, 23 Jan 2009 16:03:43 -0600 (CST) Received: from rbn1s-216-180-93-118.adsl.hiwaay.net (rbn1s-216-180-93-118.adsl.hiwaay.net [216.180.93.118]) by mail.homefreemail.com (Horde Framework) with HTTP; Fri, 23 Jan 2009 16:03:43 -0600 Message-ID: 20090123160343.74244lrgx7cvq...@mail.homefreemail.com I think the REAL source of your problems is not your dynamic IP address, but rather mail.homefreemai.com. and that second Received header. Notice how it's missing the 'from ' section of the header an just starts of 'by beatrice'? What happens if you fix that? (here's the headers from a webamil session on my own server) Received: from mail.covisp.net (ns2.covisp.net [75.148.117.93]) by mx.google.com with ESMTP id o30si17650155ugd. 35.2009.01.29.04.51.18; Thu, 29 Jan 2009 04:51:19 -0800 (PST) Received: from webmail.covisp.net (localhost [127.0.0.1]) by mail.covisp.net (Postfix) with ESMTP id F12A7118BCD4 for gkr...@gmail.com; Thu, 29 Jan 2009 05:51:12 -0700 (MST) Received: from 67.164.162.51 (SquirrelMail authenticated user kreme) by webmail.covisp.net with HTTP; Thu, 29 Jan 2009 07:51:13 -0500 (EST) I don't hit botnet... -- BILL: I can't get behind the Gods, who are more vengeful, angry, an dangerous if you don't believe in them! HENRY: Why can't all these God just get along? I mean, they're omni- potent and omnipresent, what's the problem?
Re: Botnet FPs from Webmail Senders
LuKreme wrote on Thu, 29 Jan 2009 05:57:09 -0700: Received: from 67.164.162.51 (SquirrelMail authenticated user kreme) by webmail.covisp.net with HTTP; Thu, 29 Jan 2009 07:51:13 -0500 (EST) I don't hit botnet... but that might be because of the authenticated Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com
Re: Botnet FPs from Webmail Senders
On Thu, 29 Jan 2009, Kai Schaetzl wrote:
Andy Dorman wrote on Wed, 28 Jan 2009 11:03:01 -0600:
We were just hoping someone else in this group would have run into this
situation and have a suggestion, given how ubiquitous webmail is these
days. If
not, we will keep pounding at it.
Well, the easiest and fastest solution is to go to the Horde code and change
the
few bits in it, so that it resembles the authentication line that gets added
to the
typical SMTP AUTH transaction. And submit it as a feature request to the Horde
developers. Or remove that line completely and/or change to an informational
header, like:
X-Submit-Host: (rbn1s-216-180-93-118.adsl.hiwaay.net [216.180.93.118])
Kai
We ran into the exact same issue. As Kai says munging the Horde code to
add SMTP AUTH transaction headers is simple, for us the patch was:
[r...@server31 html]# diff -C 6 horde/lib/Horde/MIME/Headers.php-orig
horde/lib/Horde/MIME/Headers.php
*** horde/lib/Horde/MIME/Headers.php-orig 2006-03-28 07:22:01.0
-0600
--- horde/lib/Horde/MIME/Headers.php2007-09-24 14:14:25.0 -0500
***
*** 294,305
--- 294,314
$received .= $_SERVER['REMOTE_IDENT'] . '@' . $remote . ' ';
} elseif ($remote != $_SERVER['REMOTE_ADDR']) {
$received .= $remote . ' ';
}
$received .= '[' . $remote_addr . ']) ';
+ /* Add 'authenticated' part to header to prevent spam blocking clients */
+
+ $curent_user = Auth::getAuth();
+ if (!empty($curent_user)) {
+ $received .= '(authenticated user='. $curent_user . ' bits=0) ';
+ } else {
+ $received .= '(authenticated bits=0) ';
+ }
+
if (!empty($GLOBALS['conf']['server']['name'])) {
$server_name = $GLOBALS['conf']['server']['name'];
} elseif (!empty($_SERVER['SERVER_NAME'])) {
$server_name = $_SERVER['SERVER_NAME'];
} elseif (!empty($_SERVER['HTTP_HOST'])) {
$server_name = $_SERVER['HTTP_HOST'];
--
Dave Funk University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{
Re: Botnet FPs from Webmail Senders
Is that your own webmail server (run by you, or run by a service you use/trust/contract-with/etc.)? Because, a webmail transaction shouldn't go to the generic internet anymore than a desktop client should, IMO ... so they're in the same boat as everyone else. That host should go to their local MTA, and then to you (if they aren't YOUR webmail servers). If they are a combined Webmail and MTA host ... then they're just as badly named as any other MTA with an IP address in the name. Plus, HTTP transactions are no more nor less subject to manipulation than SMTP transactions. If they ARE your webmail servers, then you should use existing options about trusted networks, Botnet's exceptions for skipping or exempting certain IP's, etc. It's the same as if they were your own desktop clients with dhcp based hostnames. So, it would seem to me that the current tools already handle it, and the specific nature of the request isn't any different from other cases. Does anyone think that's way off base? The only thing I can think of is that (trying to remember what I wrote many years ago), Botnet has an exemption for authenticated submissions. But, Botnet doesn't detect that natively, IIRC, it trusts Spam Assassin to determine that. That feature in Botnet just honors that Spam Assassin said it was authenticated. So, if what you're saying is HTTP relays should be treated like SMTP-AUTH relays, then I would say I think that's a Spam Assassin issue, not a Botnet limitation. If SA is properly recognizing it, but Botnet isn't honoring it, then let me know, and I'll see how I can add it to the Botnet authentication exemption (assuming there's a way to tell the difference between Webmail and some cgi/php script, as I don't want to automatically exempt the latter, I think). On Wed, Jan 28, 2009 at 05:14, Andy Dorman ador...@ironicdesign.com wrote: We have found Botnet to be very helpful in trapping spam generated by misbehaving home computers and office workstations. So we want to avoid making changes that could reduce it's effectiveness. However, we are having a problem with it also catching legitimate email sent from webmail hosts. The untrusted relay that is triggering the test looks like this example. Received: from rbn1s-216-180-93-118.adsl.hiwaay.net (rbn1s-216-180-93-118.adsl.hiwaay.net [216.180.93.118]) by mail.homefreemail.com (Horde Framework) with HTTP; Fri, 23 Jan 2009 16:03:43 -0600 And we can see what happens when we run in debug mode. ... [538] dbg: Botnet: starting [538] dbg: Botnet: no trusted relays [538] dbg: Botnet: Skipped ip 192.168.0.5 [538] dbg: Botnet: get_relay good RDNS [538] dbg: Botnet: IP is '216.180.93.118' [538] dbg: Botnet: RDNS is 'rbn1s-216-180-93-118.adsl.hiwaay.net' [538] dbg: Botnet: HELO is 'rbn1s-216-180-93-118.adsl.hiwaay.net' [538] dbg: Botnet: sender 'andydor...@comehome.net' [538] dbg: Botnet: hit (client,ipinhostname,clientwords) [538] dbg: rules: ran eval rule BOTNET == got hit (1) What we would like Botnet to do is recognize that this is an HTTP transaction, not smtp, and hence not hit on it. We were first wondering if anyone else has encountered this issue and possibly there is a fix we are not aware of? Otherwise, we would be willing to give a shot at a patch to handle this issue. Just did not want to re-invent the wheel. Thanks, -- Andy Dorman Ironic Design, Inc.
Re: Botnet FPs from Webmail Senders
John Rudd wrote: Is that your own webmail server (run by you, or run by a service you use/trust/contract-with/etc.)? Because, a webmail transaction shouldn't go to the generic internet anymore than a desktop client should, IMO ... so they're in the same boat as everyone else. John, good points. Here is more info and thank you for your work on Botnet and your insights. First to answer your question, we are dealing with several sets of webmail servers. One set is owned and controlled by us and the other by the ISP that provides our co-lo service. The problem we are trying to solve is when someone (that will be authenticated) from a dynamic IP sends email through these webmail servers. Unfortunately the header does not indicate the user was authenticated as it does when doing authenticated smtp. So SA misses that. And we certainly see your point about foolish web CGI/PHP pages that can send emails with no authentication. I had forgotten that people still pull that stunt. In the hope of clarification, below is the full header from the example I already used. I logged into the webmail server (beatrice) from my home workstation with a dynamic DSL connection and sent a test email to ador...@ironicdesign (on william). The first untrusted relays are skipped. In the case below it is skipped because of the local IP. In the case of the ISP webmail servers, they are skipped because we added them to be skipped in Botnet.cf so botnet emails forwarded through them could be caught. = full example header = Return-Path: andydor...@comehome.net Received: from william.ironicdesign.com ([unix socket] by william.ironicdesign.com (Cyrus v2.2.13-Debian-2.2.13-14+b3) with LMTPA; Fri, 23 Jan 2009 16:03:44 -0600 X-Sieve: CMU Sieve 2.2 Received: from beatrice.ironicdesign.com (beatrice.ironicdesign.com [192.168.0.5]) by william.ironicdesign.com (Postfix) with ESMTP id F3ACE460057 for ador...@ironicdesign.com; Fri, 23 Jan 2009 16:03:43 -0600 (CST) Received: by beatrice.ironicdesign.com (Postfix, from userid 112) id E92BC148C16A; Fri, 23 Jan 2009 16:03:43 -0600 (CST) Received: from rbn1s-216-180-93-118.adsl.hiwaay.net (rbn1s-216-180-93-118.adsl.hiwaay.net [216.180.93.118]) by mail.homefreemail.com (Horde Framework) with HTTP; Fri, 23 Jan 2009 16:03:43 -0600 Message-ID: 20090123160343.74244lrgx7cvq...@mail.homefreemail.com Date: Fri, 23 Jan 2009 16:03:43 -0600 From: Andy Dorman andydor...@comehome.net To: ador...@ironicdesign.com Subject: Test ... = end full example header = The case above got caught by the IP embedded in the host name, rbn1s-216-180-93-118.adsl.hiwaay.net. Our ISP is pretty good about setting fwd/rev for their dynamic IPs. The other case that triggered Botnet that we would like to try to address is the case of sending webmail using an ATT (or any ISP) DSL connection with NO fwd/rev DNS. If they ARE your webmail servers, then you should use existing options about trusted networks, Botnet's exceptions for skipping or exempting certain IP's, etc. It's the same as if they were your own desktop clients with dhcp based hostnames. Well we are already skipping these servers so forwarded Botnet mails will be caught. And we dare not grant a blanket exemption for that same reason. ;-) So, it would seem to me that the current tools already handle it, and the specific nature of the request isn't any different from other cases. Does anyone think that's way off base? I think the only reason the current tools do not appear to handle it is because we have configured ourselves in a corner with skipping the webmail server IPs so we can catch the forwarded Botnet emails. For webmail, that pushes the Botnet tests down to the HTTP transaction from those dynamic IPs. The only thing I can think of is that (trying to remember what I wrote many years ago), Botnet has an exemption for authenticated submissions. But, Botnet doesn't detect that natively, IIRC, it trusts Spam Assassin to determine that. That feature in Botnet just honors that Spam Assassin said it was authenticated. So, if what you're saying is HTTP relays should be treated like SMTP-AUTH relays, then I would say I think that's a Spam Assassin issue, not a Botnet limitation. If SA is properly recognizing it, but Botnet isn't honoring it, then let me know, and I'll see how I can add it to the Botnet authentication exemption (assuming there's a way to tell the difference between Webmail and some cgi/php script, as I don't want to automatically exempt the latter, I think). You are correct. We would like these webmail HTTP relays to be treated as authenticated. But at the same time we do NOT want to exempt any old cgi/php script. That way lies chaos. We were just hoping someone else in this group would have run into this situation and have a suggestion, given how ubiquitous webmail is these days. If not, we will keep pounding at it. Thanks for your time. -- Andy Dorman
