RE: [Mimedefang] Many many MX records
> From: David F. Skoll > Sent: Tuesday, January 13, 2004 2:03 PM > To: MIMEDefang, > Subject: Re: [Mimedefang] Many many MX records > > > On Tue, 13 Jan 2004 [EMAIL PROTECTED] wrote: > > > This led me to wonder what would happen if I registered a bunch of high > > valued MX records, ie: > > > Domain.com IN MX 10mailfilter.domain.com > > > Domain.com IN MX 100 bogusaddress.domain.com > > . > > . > > Domain.com IN MX 200 bogusaddress.domain.com > > I wouldn't recommend doing this. DNS uses UDP, and it's inadvisable to > have so many MX records that they won't fit in one decent-sized > UDP packet. > It will cause a lot of load on your DNS server, and I bet will trigger > weird bugs in a lot of resolver libraries out there. > > It would also hurt legitimate senders, especially ones that aren't smart > enough to discard duplicate MX records. How about Domain.com IN MX 100 spamtrap.domain.com And set up spamtrap as a real or virtual server, which might safely assume that mail directed at is either spam, or mail from broken MTA that doesn't deserve the privilege of sending mail to domain.com. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Many many MX records
> -Original Message- > From: David F. Skoll > Sent: Wednesday, January 14, 2004 5:37 AM > To: [EMAIL PROTECTED] > Subject: RE: [Mimedefang] Many many MX records > > > On Tue, 13 Jan 2004, Gary Funck wrote: > > > How about > > Domain.com IN MX 100 spamtrap.domain.com > > But what if all your other MX records are down or unreachable? You'll > blacklist a whole bunch of innocent mail. > My thinking was that this would be virtual host, resident on the same physical system. Thus, if the main MX isn't working, due to hardware failure neither is the backup. Then, as I thought about this some more, I realized that the main MX could be broken due to software or operator failure, and that would be bad. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] fade-to-black grey listing?
Was watching the Spam Conf. webcast at http://spamconference.org/ and there was by Bill Yerazunis entitled "Beyond 99.9% accuracy", and one idea that he mentioned using information such as evidence of dictionary attacks, or messages sent to a mail "land mine" (honey pot) as way of improving on the accuracy of spam determingation. Although not discussed directly, it seemed like this idea could be combined with grey listing. The idea would be to hold off acceptance from an IP/user that hadn't been previously recorded/validated long enough to give time for additional mail to come in via either a honeypot, or to determine that a dictionary attack is underway. Once it was determined that the IP is being used to send spam, it could be black listed, so that a subsequent attempt would be denied. A couple of complications with this proposal: (1) since spammers are increasingly using zombies running on unsuspecting DSL/dialup users' PC's, it is quite possible for the spammer to fan out over several of these intermediate PC's to make it difficult to single out one IP as an offender. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] MIMEdefang on CPAN? or in RPM form?
Hello, I'm about ready to install MIMEdefang, and was wondering if MIMEdefang is available on CPAN, or available in RPM form? I've been trying to move towards always installing perl modules/distributions via RPM in order to better track what is installed. For CPAN, I'm using 'Ovid' to build RPM's, but of course MIMEdefang has to be on CPAN first, for that to work. I ran a quick scan on CPAN for MIMEdefang, but didn't find it there ... ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] MIMEdefang on CPAN? or in RPM form?
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Kenneth > Porter > Sent: Saturday, December 25, 2004 9:05 PM > To: mimedefang@lists.roaringpenguin.com > Subject: Re: [Mimedefang] MIMEdefang on CPAN? or in RPM form? > > > --On Saturday, December 25, 2004 8:51 PM -0500 "David F. Skoll" > <[EMAIL PROTECTED]> wrote: > > > It's not on CPAN, because it's not really modular enough. You can build > > an RPM like this: > > > > rpmbuild -tb mimedefang-2.49.tar.gz > > Note that you should not generally package RPM's as root. There's > a little > setup required before issuing the above command as a mortal. Perhaps we > could set up a FAQ entry about what's needed? > Hmm, first I've heard that there might be probs. building rpm's as root. What are they? > > It will also want some of the Perl module dependencies installed as > > RPMs, so you might need: > > > > rpmbuild --nodeps -tb mimedefang-2.49.tar.gz > > The other approach is to roll your own RPM's for the other CPAN > requirements. Get the RPM::Specfile module from CPAN and build > it. Then use > the cpanflute2 script within it to convert other CPAN modules into RPM's. > (Maybe such SRPM's can be included on the module requirements page on the > MD site?) I've been experimenting with a Ovid, also available via CPAN: http://search.cpan.org/~gyepi/Ovid-0.04/ovid It does a pretty good job, but has a few problems bundling distributions such as MailTools. Request for enhancement in existing MIMEdefang rpm spec file -- Currently, it states: Requires: perl-Digest-SHA1 perl-MIME-tools perl-IO-stringy perl-MailTools a more consistent capitalization standard would be: Requires: perl-Digest-SHA1 perl-MIME-Tools perl-IO-Stringy perl-MailTools this may impact the other tarballs available on the MIMEdefang download page also. The correction of capitalization is important for those of us building RPM's from CPAN distribution tar files, in an automated way. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] MIMEdefang on CPAN? or in RPM form?
> -Original Message- > From: Kevin A. McGrail > Sent: Monday, December 27, 2004 8:19 AM > > Personally, I'm also starting to agree that some elitism is necessary and > making installation too easy is a bad thing. Having a few hurdles to make > people install MD sorts out the weaker system admins from the herd so we > don't have to listen to "what's perl"? Yep, removing the root filesystem while attempting an rpmbuild would definitely classify as a 'hurtle'. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] two minor doc. errors
At the end of sa-mimedefang.cf: # Add your own customised scores for some tests below. The default scores are # read from the installed "spamassassin.cf" file, but you can override them # here. To see the list of tests and their default scores, go to # http://spamassassin.taint.org/tests.html . That URL should read: # http://spamassassin.apache.org/tests_3_0_x.html on the assumption that a 2.x.x SA is no longer relevant. And in the README file 5. INSTALLATION --- There's an excellent MIMEDefang-HOWTO contributed by Mickey Hill at http://www.rudolphtire.com/mimedefang-howto/. It explains everything in this README in much greater detail. [] The URL is no longer operational. This one works: http://www.mickeyhill.com/mimedefang-howto/ ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] MIMEdefang on CPAN? or in RPM form?
> -Original Message- > From: Kris Deugau > Sent: Wednesday, December 29, 2004 8:52 AM > > Gary Funck wrote: > > Hmm, first I've heard that there might be probs. building rpm's as > > root. What are they? > > The biggest one is that the build/install part of building the package > will accidentally overwrite real system files. This is a Very Bad > Thing, and can *really* screw things up if you're building on the system > you're going to install on; you may end up overwriting something you > really didn't want to overwrite. > I've since created and rpmadmin account and am building packages under that personna. As they say, works for me. > > Request for enhancement in existing MIMEdefang rpm spec file -- > > > > Currently, it states: > > > > Requires: perl-Digest-SHA1 perl-MIME-tools perl-IO-stringy > > perl-MailTools > > > > a more consistent capitalization standard would be: > > > > Requires: perl-Digest-SHA1 perl-MIME-Tools perl-IO-Stringy > > perl-MailTools > > This would be nice, but it would also be incorrect. > > Those Perl modules are named as-is, for whatever reason it was, and > changing the capitalization will Break Things. I don't recall any > particular problems building any of those modules myself. Researching this further, it seems that my rpm building tool of choice, Ovid, chose a different capitalization convention than (perhaps) the tool used to create the RPM's that require the presence of the modules installed by the RPM. I modified the .spec file to supply the expected resource and rebuilt. No problema. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] two minor doc. errors
> From: Mickey Hill > Sent: Thursday, December 30, 2004 5:37 AM [...] > > No, although the location has changed, the original URL is still valid > and works perfectly well, returning a 301 with the new location. OK. "no longer operational" was not technically correct. As the author, do you prefer that new mimedefang users indirect through rudolphtire.com, or that they go directly to mickeyhill.com? ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] simple method to set up a test recipient?
I've nearly got MDF installed, but before unleashing it on the users at large, I'd like to try it out, and tune it up. To do this, I'd like to set up a mailtest user, and only for that user invoke MDF and its actions. Is there something simple I can do in /etc/aliases, or in sendmail.mc to make that happen? ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] features?
After installing MDF, I ran mimedefang.pl -features and obtained the attached report. A few of the missing features caused me some concern: Archive::Zip : no File::Scan: no HTMLCleaner : no Virus:CLAMAV : no Virus:CLAMD : no Virus:FileScan: no Anomy::HTMLCleaner: missing Archive::Zip : missing File::Scan: missing I had previously installed clamAV, but for some reason it isn't finding this feature. Do I need to install File::Scan and File::Scan::ClamAV to bring clamav in as a feature? Also, I assume that Archive::Zip is needed so that MDF can scan/manipulate Zip files and I should inatll that module as well? What are the various merits of clamav versus the clamd demon? Is Anomy::HTMLCleaner recommended? HTML::Parser : yes HTML::TokeParser : yes Net::DNS : yes Path:CONFDIR : yes (/etc/mail) Path:QUARANTINEDIR: yes (/var/spool/MD-Quarantine) Path:SENDMAIL : yes (/usr/sbin/sendmail) Path:SPOOLDIR : yes (/var/spool/MIMEDefang) SpamAssassin : yes Unix::Syslog : yes Archive::Zip : no File::Scan: no HTMLCleaner : no Virus:AVP : no Virus:AVP5: no Virus:BDC : no Virus:CLAMAV : no Virus:CLAMD : no Virus:CSAV: no Virus:FPROT : no Virus:FPROTD : no Virus:FSAV: no Virus:FileScan: no Virus:HBEDV : no Virus:NAI : no Virus:NVCC: no Virus:OpenAV : no Virus:SOPHIE : no Virus:SOPHOS : no Virus:SymantecCSS : no Virus:TREND : no Virus:TROPHIE : no Virus:VEXIRA : no Anomy::HTMLCleaner: missing Archive::Zip : missing Digest::SHA1 : Version 2.10 File::Scan: missing HTML::Parser : Version 3.35 HTML::TokeParser : Version 2.28 IO::Socket: Version 1.28 IO::Stringy : Version 2.109 MIME::Base64 : Version 3.05 MIME::Tools : Version 5.415 MIME::Words : Version 5.415 Mail::Mailer : Version 1.65 Mail::SpamAssassin: Version 3.02 Net::DNS : Version 0.48 Unix::Syslog : Version 0.99 [EMAIL PROTECTED] ~]$ which clamav clamav: Command not found. [EMAIL PROTECTED] ~]$ ls /usr/bin/cl* /usr/bin/clamav-config /usr/bin/clamscan /usr/bin/clamdscan /usr/bin/clear ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] features?
> From: Gary Funck > Sent: Thursday, December 30, 2004 2:46 PM > > After installing MDF, I ran mimedefang.pl -features > > A few of the missing features caused me some concern: > [...] > Virus:CLAMAV : no > Virus:CLAMD : no Although clamav was installed, mimedefang.pl apparently didn't know about it. After investigating this issue, I discovered that the problem is due to the way the .spec file inside the MIMEdefang tarfile distribution configures MIMEdefang. Here is the relevant invocation: + ./configure --prefix=/usr --mandir=/usr/share/man --sysconfdir=/etc --with-spooldir=/var/spool/MIMEDefang --with-quarantinedir=/var/spool/MD-Qua rantine --with-user=defang --disable-anti-virus Doing some searching, apparently this issue has been discussed before: http://lists.roaringpenguin.com/pipermail/mimedefang/2004-February/020343.ht ml The trick is to build the rpm with antivirus checking enabled. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] how to build SA-compatible encapsulated spam message?
Presently, MIMEdefang can be configured to add the SpamAssassin report
as a separate text attachment to the orignal message. The code that
does this is in mimedefang-filter at approximately line 317:
if ($hits >= $req) {
action_change_header("X-Spam-Score", "$hits ($score)
$names");
md_graphdefang_log('spam', $hits, $RelayAddr);
# If you find the SA report useful, add it, I guess...
action_add_part($entity, "text/plain", "-suggest",
"$report\n",
"SpamAssassinReport.txt", "inline");
Although this code performs its intended purpose, it creates a mail message
that is incompatible with the SA -d (remove markup) switch. Spamassassin
creates a message of the form:
header:
Content-Type: multipart/mixed; boundary="--=_41CC5623.58847487"
body:
This is a multi-part message in MIME format.
=_41CC5623.58847487
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
[spamassassin report]
=_41CC5623.58847487
Content-Type: message/rfc822; x-spam-type=original
Content-Description: original message before SpamAssassin
Content-Disposition: attachment
Content-Transfer-Encoding: 8bit
[... original unaltered message]
=_41CC5623.58847487--
===
This message format is useful, because (for example) spam messages
can be fed back into SA for unlearning if they were false positives,
and for building a clean mbox file that contains only the original
spam messages. SA doesn't understand the mail message format
created by MIMEdefang's default action.
What actions are required to have MIMEdefang build a spam message
containing the report and the original message, as outlined above?
thanks - Gary
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] how to build SA-compatible encapsulated spam message?
> From: James Ebright > Sent: Monday, January 03, 2005 7:08 AM > > I attached a snippet from my filter code... gleaned from KAM a bit over a > year ago.. it works well and I think will do the job. Jim, thanks. Looks like that'll do the trick. Might make a nice addition to the 'contirb' directory, or as an option on how MDF reports spam. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] feed MimeDefanged mail to sa-learn?
(cross-posted from the SA list. I'd developed this as an interim method for normalizing the spam message formats. - Gary) -Original Message- From: Gary Funck [mailto:[EMAIL PROTECTED] Sent: Monday, January 03, 2005 1:52 AM Attached, is a Perl script, mdf2sa.pl, which converts spam messages that have been marked up by MIMEDefang, into a form that is similar to the message format used by SA. Also attached is a simple procmail script, mdf2sa.rc, which drives the perl script, and can be invoked via formail as follows: formail -s procmail -m mdf2sa.rc < spam.mbox > new_spam.mbox (the procmail script writes to a logfile named x.log). The conversion could be made upon delivery to a spam mailbox as follows: :0 fbhw * ^X-Spam-Score: * ^X-Scanned-By: MIMEDefang * B ?? ^Content-Type: text/plain; name=\"SpamAssassinReport\.txt\" | perl $HOME/mdf2sa.pl :0: * ^X-Spam-Score:.*\(\*\*\*\*\* spam-mail disclaimer: the Perl script is only lightly tested. I'd appreciate hearing any suggestions to improve the Perl coding style, or bug fixes/improvements. PS: the code that removes a couple of Received headers when reconstructing the message header is implmentation and needs to be tuned for your installation. mdf2sa.pl Description: Binary data mdf2sa.rc Description: Binary data ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] RE: How got arcor.de into blocklist?
> From: Thomas Arend > Sent: Monday, January 03, 2005 7:55 AM > > I have received a FalsePositive because arcor.de got into trhe blocklists. > > Has someone informations why arcor.de got into the blocklist? > You probably need to go to the places where the URL's are blacklisted, and requrest that the blacklisting be removed. For example, http://urlblacklist.com/?sec=search&URL=arcor.de&action=Search ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] RE: How got arcor.de into blocklist?
That reply was meant for the SA forum. Sorry about that. - Gary ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] how to build SA-compatible encapsulated spam message?
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of James > Ebright > Sent: Monday, January 03, 2005 7:08 AM > To: mimedefang@lists.roaringpenguin.com > Subject: Re: [Mimedefang] how to build SA-compatible encapsulated spam > message? > > > I attached a snippet from my filter code... gleaned from KAM a bit over a > year ago.. it works well and I think will do the job. Follow-up, I had to change the attachment attributes a bit to make them liine up with the way Spamassassin was defining them in its "safe report". Without this change, Outlook would display only a blank page, because the SA report was tagged as having a FileName. Here's the gist of the change: $container = MIME::Entity->build(Type => 'message/rfc822', Description => 'Original message before MIMEDefang', Disposition => 'Attachment', Data => [ "" ]); [...] $report2 = MIME::Entity->build(Type => 'text/plain', Description => 'SpamAssassin Warning', Data => ["$report\n"], Disposition => "inline", Encoding => "-suggest"); ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] restart or reload after changin SA rules?
I use RulesDuJour (http://www.exit0.us/index.php?pagename=RulesDuJour) and am trying to now integrating them into MIMEDefang. This script downloads the latest custom-developed SA rules and auto-updates them into a config directory. Once it is done it want to run '/etc/init.d/spamd restart'. With MDF, we don't run spamd, but run the MDF multiplexor instead. Q: after changing the SA rule set in one/more config dirs, is it necessary to run /etc/init.d/mimedefang restart, or will "reload" work just as well? ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] restart or reload after changin SA rules?
> -Original Message- > From: James Ebright > Sent: Tuesday, January 04, 2005 8:34 AM > > I use reload in my script for the few configs I use from there. > OK. thanks Jim (and David). We're running Fedora Core 3, with Perl 5.8, so I'll assume that it is competent enough to handle reload. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] temp failing - got code?
Would like to implement temp failing/grey listing along the following lines: - If haven't seen a given incoming IP address in a while, then temp fail, for say about 1 hour. If the sender doesn't just go away by then, then socre the mail per usual and let it through. To implement this, we'd need a persistent data base (perhaps a local DNS sever like Tiny DNS?), I'd suppose. My thinking is that if we hold off on accepting incoming mail from new IP addresses, it gives time for the various distributed spam registry databases to register the spam, its URL's and so on, thus making the 'network checks' more effective. That's assuming of course that the sending IP even bothers to retry. Have you done something like this, and have code you could offer as a starting point? thanks. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] tagging mail sent to aliases
I notice a lot of spam gets sent to one of a number of mail aliases, which we've added to /etc/aliases to catch alternate spellings of our e-mail addresses. Is there a method within sendmail to notice when it has mode an alias substituion, and to then have MdF add a header tag noting this happened? If it can do that, then it'd be easy to write a custom rule that dings the usage of alternate spellings. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] temp failing - got code?
> -Original Message- > From: Rob MacGregor > Sent: Wednesday, January 05, 2005 10:54 AM > > On Wed, 5 Jan 2005 09:38:30 -0800, Gary Funck <[EMAIL PROTECTED]> wrote: > > > > Would like to implement temp failing/grey listing along the > following lines: > > > > - If haven't seen a given incoming IP address in a while, then > temp fail, > > for > > say about 1 hour. If the sender doesn't just go away by > then, then socre > > the mail per usual and let it through. > > This sounds similar to greylisting - have a look and see if that does > what you're after (I don't know much about greylisting). Thanks. I looked quickly in the MdF distro, but didn't see anything discussing greylisting. Is there an implementation already generally available? ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] tagging mail sent to aliases
> -Original Message- > From: James bright > Sent: Thursday, January 06, 2005 6:58 AM > To: mimedefang@lists.roaringpenguin.com > Subject: Re: [Mimedefang] tagging mail sent to aliases > > > Unless you delay checks. 1. we delay checks 2. I've seen code that adds custom headers for SA to score them, by rewriting INPUTMSG in its entirety. I don't recall if it also updated HEADERS, or if it wrote HEADERS first and then made a call to recreate INPUTMSG. Maybe not the most efficient thing to do, but given the other processing that goes on in spam detectin, probably not a big additional impact either. 3. On our system, I believe that /etc/aliases might be a hash database, and therefore it seems that it might be pretty easy to lookup recipients in the alias hash directly? Though clearly invoking sendmail to do this is safer, and the result could be cached internally and the cache flushed if /etc/aliases time stamp is updated. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] tagging mail sent to aliases
> From: James Ebright
> Sent: Friday, January 07, 2005 7:55 AM
> To: mimedefang@lists.roaringpenguin.com
> Subject: RE: [Mimedefang] tagging mail sent to aliases
>
>
> I would think checking the /etc/aliases yourself would be less memory
> intensive than another sendmail child. You can probably do this
> very early on
> as well if need be like in filter_recipient (remember to pass the
> multiplexer
> the -t flag to enable this filter in mimedefang-filter)
[...]
Thanks Jim. Perl question, what's the pros/cons between
this sort of dbopen:
if(dbmopen(%ALIAS, "/etc/aliases.db", undef)) {
and one that uses the "tie" interface?
use DB_File;
tie(%ALIAS, "DB_File", "/etc/aliases.db");
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] MySQL Greylisting: grey_old records
> From: Todd Aiken > Sent: Monday, January 10, 2005 2:53 PM [...] > > Greetings all. After having some problems with database corruption, I > modified my greylisting to use a MySQL database according to the > following web page set up by John Kirkland: > > http://www.bl.org/~jpk/md-greylist/ Todd, I notice that you now use (SQLite^H^H^H^H^H^H) MySQL. Are you saying that the SQLite implementation had database consistency problems? ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] MySQL Greylisting: grey_old records
> From: Todd Aiken > Sent: Monday, January 10, 2005 2:53 PM [...] > I'm assuming that this is supposed to happen when a whitelisted entry is > found in the database that has been there and not accessed for longer > than the time period defined by gdb_white (currently set to 30 days), but > I only implimented this setup earlier this morning with a brand new > database! Does anybody have any ideas as to why this is happening on my > system already? Or am I misunderstanding grey_old? You mention above that $gdb_white is set to 30 days, but the originaal implementation you refer to at http://www.bl.org/~jpk/md-greylist/mimedefang-filter-greylist has these config. constants: $gdb_black = 30*$minute; $gdb_grey = 5*$hour; $gdb_white = 7*$day; And $gdb_white is 7 days (or 7*24*60*60 seconds). Is it possible that you set $gdb_white to 30 seconds, or 30 mins instead of 30 days? ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] early experiences with grey listing
With help from the MDf list members, and a few trips through the MIMEDefang archives, I was able to implement a form of grey listing. Mainly, I used the implementation here: http://www.bl.org/~jpk/md-greylist/ There are different ways to configure this implementation. I went with the default where it keeps a (sender address, relay class C ip, recipient address) triple and stalls the sender whenever the triple hasn't been seen before or is still in the initial blackout period. I bumped the blackout period to 1 hour with the thought it would allow some time for the spammers' URL's to make it into the various SURBL databases. I found that I get a little nervous when mail comes in from a client, and I know we're going to stall it for an hour. I just hope the senders' MTA will try again in a timely fashion. So far so good. However, if I'm waiting for a website to send me my password info., I really would like the reply sooner rather than later. I'm thinking that for our set up, the more appropriate thing to do might be to try and verify that the sender's domain reverse maps to the class C sending IP address. If it does then let it through without delay. After all, SA is still available to scan the message, and most spam uses zombies and/or address spoofs. The current implementation already resets the greylist info. for relay ip/sender's who send mail that is calculated to be spam. Thus, the logic should check first to see if the sender is registered as being in the blackout period, then check to see if the sender's domain maps to the sending relay ip. I'm also thinking it would be worthwhile whitelisting any recipients of mail originated from the local/trusted network. Thus, if a user initiates a contact, there is some assurance that he'll see the reply without delay. The other thing that I've noticed is a lot of spam sends to local mail addresses that don't exist. With greylisting enabled, these errant messages are first tempfail'ed, and only if/when they come back after the black out period, is the recipient id checked, and the sender is then bounced with "unkown user". Not necessarily good, if a sender mistypes a local email address, and would like to know about it soonder rather than later. I don't know this authentication can be implemented in the MIMEDefang filter or not, with some reasonable level of effort, but it seems like it might be a good thing to do. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] how to remove milter messages from the log?
(disclosure: I'm a syslog novice) I'm finding the following sorts of log entries to be not useful/needed: Jan 12 16:55:37 intrepid sendmail[10286]: j0D0tWUD010286: Milter change: header Content-Type: from text/plain; charset=us-ascii to text/plain; charset=us-ascii Jan 12 16:55:37 intrepid sendmail[10286]: j0D0tWUD010286: Milter change: header MIME-Version: from 1.0 to 1.0 Jan 12 16:55:37 intrepid sendmail[10286]: j0D0tWUD010286: Milter change: header Content-Transfer-Encoding: from 7bit to 7bit Jan 12 16:55:37 intrepid sendmail[10286]: j0D0tWUD010286: Milter message: body replaced Jan 12 16:55:37 intrepid sendmail[10286]: j0D0tWUD010286: Milter add: header: X-Scanned-By: MIMEDefang 2.49 on 192.195.190.1 Is there a simple way to keep those sorts of milter-related log entries from being genearated? Is this the only log-related control in sendmail? dnl # dnl # default logging level is 9, you might want to set it higher to dnl # debug the configuration dnl # dnl define(`confLOG_LEVEL', `9')dnl would moving it lower eliminate the milter messages? ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] GTUBE test fails - MD + Sendmail + libmilter
> From: Matthew Lee > Sent: Wednesday, January 12, 2005 9:45 AM [...] > > However when using the GTUBE test > http://spamassassin.apache.org/gtube/ > No report is written and the message is not tagged. 1) first make sure that Spamassassin is working. % wget 'http://spamassassin.apache.org/gtube/gtube.txt' % spamassassin -L -x -t < gtube.txt should give you a report like this: Content analysis details: (999.5 points, 5.0 required) pts rule name description -- -- 1.1 L_t_NObody Addressed to "Undisclosed Recipients" or equivalen 0.8 L_T_COMBINED Destination email address suggests this is spam -2.4 ALL_TRUSTEDDid not pass through any untrusted hosts 1000 GTUBE BODY: Generic Test for Unsolicited Bulk Email That'll confirm that SA is working. Then, verify that MDF knows about it: % mimedefang.pl -features and look for: SpamAssassin : yes So far so good. Then, try sending it to yourself: % sendmail -oi $USER < gtube.txt You may need to do this from an external site, or cut/paste the body into an e-mail, and send that from an external site. It depends upon how MDF and sendmail are configured. If this doesn't work, look in the maillog for clues. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] early experiences with grey listing
> From: Paul Murphy > Sent: Thursday, January 13, 2005 2:06 AM [...] > > The default in most implementations is one minute - the point of > greylisting is that most spam mailers have several million addresses > to send to, so even if they bother to check the return codes, most > take no action on them, and skip to the next address. [...] (background: we're discussing an implementation based upon http://www.bl.org/~jpk/md-greylist/, which contains elements of http://lists.roaringpenguin.com/pipermail/mimedefang/2004-February/020126.ht ml and http://whatever.frukt.org/mimedefang-filter.shtml.) Paul, I understand that is the recommended way of doing things, and given the risks of missing real non-spam mail, there are some good reasons for not waiting an hour. I chose this longer interval to give time for the various SURBL's (see http://www.surbl.org) to register any offending URL's that might be contained in the message. Thus, it amounts to a two pronged defense: (1) chasing off the spammers who don't retry, and (2) increasing the reliability of the SURBL checks when/if the spam comes in. This seems to be working well. However, I will point out that only two spam messages made it into my spam folder last night, where there might've usually been 10 to 30 (we also implement IP black lists which eliminates about 2/3's of the incoming spam right off the bat). It seems, 90% of the spammers didn't retry. So, perhaps, given the downside, it is best to shorten the initial grey list period, and see how that works. I'm now thinking there may be a hybrid strategy, where we shorten the initial blackout period to 1 min., and then if the sender does retry, we place it in a holding tank for 1 hour. This would ensure ultimate delivery at least, and give time for the SURBL's to populate. The cost would be a delay in delivery of some mail. I don't know, however, how this might be implemented within MIMEDefang without tieing up a thread for each delayed mail processing event. Is there a method for locally requeinng for re-delivery 1 hour later? > [...] > Not necessarily true - I've had issues with spam from domains > which appear to be > legitimate, and in large quantities. Even though SA scores them > between 8 and > 25, and we bounce anything over 10, their persistence made it a > nuisance, and I > ended up blocking their IP addresses at the firewall level. > Yeah, we haven't gone with blocking at the router level, but would add such spam sources to the access.db reject list. Net, net - it looks as if a fairly comprehensive grey list implementation is required to make sure that it is both effective, and that it doesn't kill off legitimate mail. It seems to be risky to run with only an 80/20 type of implementation. > > It is already available using md_check_against_smtp_server() in > your filter if > that is supported (and note that Exchange 2000 is broken, so > doesn't work), Yeah some of the other implementations try to validate the incoming sender, using md_check_against_smtp_server(), and that may be a good idea. In a related area, what I'm seeing in the current implementation is that it doesn't seem to handle these sorts of call backs well when they arrive on an incoming connection. It seems that mailers on the other side don't appreciate having their probe delayed, even though they may be using it to implement greylisting on their side. However, it is not a good idea to whitelist From <> outright, because it is a favorite spammer sending address 99% of the time. > you can also use LDAP to do real-time queries against AD or any other > LDPA-compatible directory system. Alternatively, some people > harvest valid > addresses from their systems into a local DB file daily, and > check that from MD. In general, it is difficult to do everything that sendmail might do to validate a user. It is configuration dependent, and some greylist implementations I've looked at, actually go to a fair amount of work to make virtual host and virtual user substitions, etc. Some harvest the mail logs, and note the [EMAIL PROTECTED]'s that were actually delivered and populate a database with that info, although I'm not sure how that info. is put into use. We have delay_checks on, fyi. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] early experiences with grey listing
> From: David F. Skoll > Sent: Thursday, January 13, 2005 8:38 AM [...] > > You'd have to spool the mail into some other queue that gets processed > every so often, and set a MinQueueAge of an hour on that queue. It > seems way too tricky to me. > I agree. Another question: How does CanIT (or other robust grey listing implementations) handle messages with no sender (ie, From <>) address? How do you distinguish between a call back to validate an address/mailer, and a spammer getting ready to jam its message through? ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] early experiences with grey listing
> From: Gary Funck > Sent: Thursday, January 13, 2005 8:11 AM [...] > only two spam messages made it into my spam folder > last night, where there might've usually been 10 to 30 (we also implement > IP black lists which eliminates about 2/3's of the incoming spam right off > the bat). It seems, 90% of the spammers didn't retry. follow-up. It looks like the only spam messages that did retry were addressed to a mailing list, which as part of its implementation re-delivers. The mailing list is bug-glibc@gnu.org, btw, which unfortunately is 95% spam, with useful technical messages sprinkled in every other week/so. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] MD/SA and Outlook
> Although I've certainly had cases, using Outlook 2003, where Outlook's > idea of where the headers ended was different from MD's. The general > thing these items seem to have had in common is a blank line embedded in > the headers (which might be something for the original poster to look > for a bit more closely...) I've seen Outlook 2000 get confused not by a blank line in the header, but they the arrangement of the SA preview, and the attached original e-mail message. I wrote it off to an interaction between the code that is adding the attachments, and the action_rebuild() call at the end of filter_end() (on the 'spam found' path), and the subsequent action_rebuild() here (which we have eabled): # Deal with malformed MIME. # Some viruses produce malformed MIME messages that are misinterpreted # by mail clients. They also might slip under the radar of MIMEDefang. # If you are worried about this, you should canonicalize all # e-mail by uncommenting the action_rebuild() line. This will # force _all_ messages to be reconstructed as valid MIME. It will # increase the load on your server, and might break messages produced # by marginal software. Your call. action_rebuild(); Regarding the blank line in the header, I wondered it if might be some sort of CRLF issue, where, let's say a new header is added that has only an LF, but the rest of the header is delimited with CRLF's or vice verse? The RFC's, I believe, say it shouldn't matter, but I could see where Outlook might occassionallly get confused, because PC's are so CRLF centric. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] MD/SA and Outlook
> -Original Message-
> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED] Behalf Of Todd
> Aiken
> Sent: Thursday, January 20, 2005 7:58 AM
> To: mimedefang@lists.roaringpenguin.com
> Subject: Re: [Mimedefang] MD/SA and Outlook
>
>
> On 20 Jan 2005 at 10:50, David F. Skoll wrote:
>
> > On Thu, 20 Jan 2005, Todd Aiken wrote:
> >
> > > Absolutely true, and the line:
> >
> > > action_change_header('X-Spam-Report', $fixed_report);
> >
> > > in the filter_end section of my mimedefang-filter does that.
> >
> > Ah. So you did modify the example filter. :-)
>
> Oh, I thought that was part of the default example file! I must have
> pulled it from another example file, as I don't remember adding it
> myself.
Did you really want to put the full report into the headers, and not
something
a bit more abbreviated such as the more/less original rendition?
action_change_header("X-Spam-Score", "$hits ($score) $names");
The code that KAM/JKC contributed, is what we use:
# Spam checks if SpamAssassin is installed
if ($Features{"SpamAssassin"}) {
if (-s "./INPUTMSG" < 100*1024) {
# Only scan messages smaller than 100kB. Larger messages
# are extremely unlikely to be spam, and SpamAssassin is
# dreadfully slow on very large messages.
my($hits, $req, $names, $report) = spam_assassin_check();
my($score);
if ($hits < 40) {
$score = "*" x int($hits);
} else {
$score = "*" x 40;
}
# We add a header which looks like this:
# X-Spam-Score: 6.8 (**) NAME_OF_TEST,NAME_OF_TEST
# The number of asterisks in parens is the integer part
# of the spam score clamped to a maximum of 40.
# MUA filters can easily be written to trigger on a
# minimum number of asterisks...
if ($hits >= $req) {
action_change_header("X-Spam-Score", "$hits ($score)
$names");
md_graphdefang_log('spam', $hits, $RelayAddr);
# BEGIN REPORTSAFE - KAM 10-29-2003
# Implement Report_Safe -- Thanks to James W. Curtis for the
original starting code!
my ($container, $parser, $original, $report2);
$container = MIME::Entity->build(Type => 'message/rfc822',
Description => 'Original message before MIMEDefang',
Disposition => 'Attachment',
Data => [ "" ]);
$parser = new MIME::Parser;
open(IN, '< INPUTMSG');
$original = $parser->parse(\*IN);
close(IN);
$original->head()->replace('X-Relay-Addr', $RelayAddr);
$RelayHostname ||= 'N/A';
$original->head()->replace('X-Relay-Host', $RelayHostname);
$original->head()->replace('X-Relay-Time',
scalar(localtime));
# Add the original message to the container
$container->add_part($original);
$report2 = MIME::Entity->build(Type => 'text/plain',
Description => 'SpamAssassin Warning',
Data => ["$report\n"],
Disposition => "inline",
Encoding => "-suggest");
$entity->parts([$report2]);
$entity->head()->mime_attr('content-type' =>
'multipart/mixed');
$entity->head()->mime_attr('content-type.boundary' =>
'=_' . scalar(time) . "-$$-nikc");
$entity->add_part($container);
action_rebuild();
# END REPORTSAFE
} else {
# pass the hits through on non-spam as well
action_change_header("X-Spam-Score", "$hits ($score)
$names");
}
}
}
Note that we add the "report safe" attachment if it is determined that the
mail is spam, and we add the spam score, along with the $names of the rules
that hit, even it is determined not to be spam.
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Berkley DB
> We're on Fedora Core 3, compiling Sendmail from source as per the > Mimedefang HOWTO. We're getting an error indicating that Berkley DB is > not installed, yet it is. We linked the libdb.so file to the > libdb-4.2.so and now it's trying to compile in Berkley DB. However, it > keeps erroring out saying in cannot find the DB header file. We > download the Berkley DB source from Fedora, but we're not sure where the > source files should go to make sendmail to see them. According to the following: $ ls /usr/include/db.h /usr/include/db.h $ rpm -qf /usr/include/db.h db4-devel-4.2.52-6 You need to have the db4-devel RPM installed. You might be able to obtain it by running rpm --rebuild on your source RPM, or simply obtain this specific RPM and install it. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Perl help: quarantine and bounce criteria
> sub action_discard_bounce ($) {
> my($message) = @_;
> # don't quarantine if all recipients are @sewingwitch.com
> my $non_sewingwitch = grep !/[EMAIL PROTECTED]/i, @Recipients;
> # check for only recipient being
> # hostmaster or info (almost certain spam so don't quarantine)
> if ( $non_sewingwitch &&
> ( (scalar @Recipients != 1) ||
>($Recipients[0] !~
> /^(info|hostmaster)[EMAIL PROTECTED]/) )) {
Above, you first check for $non_sewingwitch, which presumably means that
the mail isn't destined for sewingwitch, but then you _and_ this condition
with the following check for info or hostmaster as the only recipient.
Did you mean || instead of && ?
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] sending relay has no MX record?
Am seeing some spam where the sender's From_ address's domain doesn't have an MX record. Was considering noting that fact in the header as an extra X- field, and then letting SA score it negatively. Has anyone tried that sort of thing? Can you offer some prototype code that does something like that? ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] updated Logwatch for blacklist/greylist messages
In our configuration, where we implement blacklists using DNSBL's and greylists in MdF, we see tons of messages as follows: blacklists: [EMAIL PROTECTED] (553 5.0.0 CBL Proxy/Trojan): 1 Time(s) [EMAIL PROTECTED] (553 5.0.0 CBL Proxy/Trojan): 1 Time(s) [EMAIL PROTECTED] (553 5.0.0 CBL Proxy/Trojan): 1 Time(s) [EMAIL PROTECTED] (553 5.0.0 SBL http://www.spamhaus.org/SBL/sbl.lasso?query=SBL17483): 1 Time(s) greylists: Milter: to=<[EMAIL PROTECTED]>, reject=451 4.3.0 greylisted for 1 minutes and 0 seconds.: 1 Time(s) Milter: to=<[EMAIL PROTECTED]>, reject=451 4.3.0 greylisted for 58 seconds.: 1 Time(s) Milter: to=<[EMAIL PROTECTED]>, reject=451 4.3.0 greylisted for 1 minutes and 0 seconds.: 1 Time(s) Has anyone taken the time to implement a new, improved Logwatch filter that would consolidate message entries like those above? Actually, I'd be interested in any logwatch improvements you've made. thanks - Gary ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] More patent stupidity
Checkpoint's firewall, and Trusted Information Systems' Gauntlet likely did the job: http://groups-beta.google.com/group/muc.lists.firewalls/browse_thread/thread/398284916e5f4019/fe79931c4d44d1e3?q=gauntlet+virus+scan +1997&rnum=1#fe79931c4d44d1e3 short url: http://tinyurl.com/exaoo That's a July 28, 1997 posting, so may not be quite early enough, but gives the drift on what people were thinking. A 1996 posting about TIS being acquired by McAfee and folded into Network Associates, http://groups-beta.google.com/group/list.cypherpunks/browse_thread/thread/ab2df364e580c485/c4676e032bbf2ea5?q=gauntlet+virus+scan+19 96&rnum=11#c4676e032bbf2ea5 short url: http://tinyurl.com/8ay5g quoting: "With the completion of this transaction, Network Associates will be the largest security software company in the industry. Network Associates, which was formed late last year by the merger of McAfee Associates and Network General, is now positioned to offer a complete suite of security technology. Their product line includes enterprise encryption, authentication, intrusion detection, anti-virus and firewall protection. Network Associates plans to combine all of these technologies and market them as a suite of security programs, similar to the way Microsoft Corp., has bundled collections of personnel computer software." --- This 1996 posting of the "virus FAQ" is interesting and shows this to be about the turning point in thinking that virus scanning at the inbound firewall might/might not be a good idea: http://groups-beta.google.com/group/alt.comp.virus/browse_thread/thread/e7efc26077b126be/95603592d2f04a49?q=firewall+virus+scan+1995 &rnum=4&hl=en#95603592d2f04a49 short url: http://tinyurl.com/77flm Quoting: What about firewalls? - What indeed? There is scope for more discussion here, but firewalls don't generally screen computer viruses, and, arguably, never will very effectively. Reassembling a packet stream to scan for search strings on the fly introduces unacceptable overheads. See comp.security, or, if you don't mind your mail by the ton, the firewalls mailing-lists. MIMESweeper, by Integralis, has been described as 'a sort of firewall for viruses', which it isn't, by most useful definitions of the term firewall. It's a Windows package which, I think, shunts incoming material for inspection offline, which may be a viable approach. In general, make sure your mail attachments, WWW downloads etc. can't be automatically executed and use a good scanner. Books: Firewalls and Internet Security (Cheswick, Bellovin) - Addison-Wesley Building Internet Firewalls (Chapman, Zwicky) - O'Reilly ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] interaction between sendmail 'access' check and MdF's milter checks
Hello, We're receiving daily "log watch" logs from a misconfigured system, apparently hanging off a big ISP. That system ironically seems to have the same name our our domain and due to misconfiguration the mail comes in looking more/less like it came from our system, except of course, the Received line at our mail demarcation point shows that the mail was sent from the errant system. We currently implement blacklisting via sendmail's "access.db" file, so I addeed the culprit's IP address there. I've verified that access.db was rebuilt and that the IP address is listed there: $ makemap -u hash access.db | grep 1.2.3.4 1.2.3.4 REJECT I've obfuscated the IP address to protect the guilty. I know the access list in general is working properly because blacklisted IP rejections are listed in the mail log file. However, mail from 1.2.3.4 is still coming in. Since the from line lists [EMAIL PROTECTED] (where example.com is our domain), perhaps that is causing confusion. I've verified that root isn't listed as a SPAMFRIEND. On our system we've enabled FEATURE(`delay_checks', `friend'). So, what I don't get, is why the IP address isn't getting bounced with the access.db check. Do milter checks override the access.db mechanism? Here's what the obfuscated mail log entries look like: Jun 9 17:05:05 example mimedefang.pl[11128]: RELAY: <1.2.3.4> Jun 9 17:05:05 example mimedefang.pl[11128]: MDLOG,j5A055tX013534,grey_white,0,1. 2.3.4,[EMAIL PROTECTED],[EMAIL PROTECTED],? Jun 9 17:05:05 example sendmail[13534]: j5A055tX013534: from=<[EMAIL PROTECTED]>, size=12162, class=0, nrcpts=1, msgid=<[EMAIL PROTECTED]>, pr oto=ESMTP, daemon=MTA, relay=c-1.2.3.4.xxx.yyy.example.net [1.2.3.4] above, example.com is our site, and example.net is the ISP that hosts the errant system. Any ideas what might be going? ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] interaction between sendmail 'access' check andMdF's milter checks
> -Original Message-
> From: Mike Atkinson
>
> Gary Funck wrote:
>
> > $ makemap -u hash access.db | grep 1.2.3.4
> > 1.2.3.4 REJECT
>
> Entries in the access map should be tagged, try:
>
> Connect:1.2.3.4 REJECT
Maybe that's it, but I've got a gazillion IP addresses listed without
a tage and they're honored by sendmail.
Just noticed this in /usr/share/sendmail-cf/README:
Entries in the access map should be tagged according to their type.
Three tags are available:
Connect:connection information (${client_addr}, ${client_name})
From: envelope sender
To: envelope recipient
Notice: untagged entries are deprecated.
If the required item is looked up in a map, it will be tried first
with the corresponding tag in front, then (as fallback to enable
backward compatibility) without any tag, unless the specific feature
requires a tag.
[...]
-
Although my entry isn't in good form, I think it should work.
I've got bunch of other entries that aren't tagged that appear
to work fine.
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] interaction between sendmail 'access' check andMdF'smilter checks
> Nope, each "phase" of the conversation passes through ALL milters (though most > milters only actually do something with the data phase) in addition to > sendmail. Since delay checks changes the order sendmail calls these parts, it > also changes the order the corresponding milter parts will be called (we run > run delay_checks here as well. [...] Kelson, Matt, Jim - thanks for your replies. It does seem that my filter_sender action is interceding ahead of the access.db check. Good idea on using the Perl access.db routine and deferring to filter_recipient aa well. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] interaction between sendmail 'access' checkandMdF'smilter checks
Quick follow-up ... the problem I was experiencing is that the errant IP address was white listed in the "grey list". That is, it had previously been greylisted and then accepted for delivery, because at the time the IP address wasn't listed as a reject in access.db. The fix is to delete the IP address from the greylist database, so that it will first be tempfailed, then checked (and rejected) in access.db. All this might be better if the greylisting logic first consulted access.db directly. However, philosophically, greylist advocates like the idea of first tempfailing, and then outright rejecting an errant IP address if the sender retries. The ideal situation, probably is that the nightly job that "ages" the greylist (deleting old entries), also consults access.db and removes any whitelisted addresses that would be rejected by the access.db check. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] building Mdf rpm - why are antivirus checks disabled by default?
Hello, I recently upgraded to mimedefang 2.52, installing
it by first building the rpm, using the supplied
mimedefang.spec file. All went well, except I
notice that the build seemed to miss the fact
that CLAMAV is installed. Here's the diff between
the previous 2.49 install and the present 2.52
install:
105,106c130,131
< $Features{'Virus:CLAMAV'} = ('/usr/bin/clamscan' ne '/bin/false' ?
'/usr/bin/clamscan' : 0);
< $Features{'Virus:CLAMD'}= ('/usr/sbin/clamd' ne '/bin/false' ?
'/usr/sbin/clamd' : 0);
---
> $Features{'Virus:CLAMAV'} = ('/bin/false' ne '/bin/false' ? '/bin/false' :
> 0);
> $Features{'Virus:CLAMD'}= ('/bin/false' ne '/bin/false' ? '/bin/false' :
> 0);
When I look at the rpmbuild step, I see the following:
*** Virus scanner detection results:
All virus-scanner detection disabled by --disable-anti-virus
Could not find any recognized virus scanner... do not use
any of the contains_virus functions in your filter.
Found Mail::SpamAssassin. You may use spam_assassin_* functions
Did not find Anomy::HTMLCleaner. Do not use anomy_clean_html()
Found HTML::Parser. You may use append_html_boilerplate()
Note: SpamAssassin, File::Scan, HTML::Parser and Anomy::HTMLCleaner are
detected at run-time, so if you install or remove any of those modules, you
do not need to re-run ./configure and make a new mimedefang.pl.
And when I look inside the RPM spec, in the changelog
it says:
#
# Revision 1.22 2002/10/25 14:01:51 dfs
# Build RPM with --disable-anti-virus
Here's the operational code:
%define with_antivirus 0
%{?_with_antivirus: %{expand: %%define with_antivirus 1}}
%{?_without_antivirus: %{expand: %%define with_antivirus 0}}
[...]
%if %{with_antivirus}
--with-user=%{user}
%else
--with-user=%{user} \
--disable-anti-virus
%endif
---
So, clearly I can change the default, or explicitly
supply --with-antivirus when building the RPM, but
I was curious as to why this option was disabled
by default in the RPM spec.?
thanks, - Gary
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Mdf 2.52 suggested-minimum-filter-for-windows-clients?
As noted previously, I installed Mdf via RPM built from
the mimedefang.spec.
While resolving .rpmsave conflicts, I noticed the
following diffs between mimedefang-filter.rpmsave
(my previous version) and mimedefang-filter, the
new version:
$ diff mimedefang-filter{.rpmsave,}
14c14
< # $Id: mimedefang-filter,v 1.2 2005/04/09 21:43:05 root Exp $
---
> # $Id: suggested-minimum-filter-for-windows-clients,v 1.82 2005/02/14
> 16:56:24 dfs Exp $
84a85,87
> # Detect and load Perl modules
> detect_and_load_perl_modules();
>
The addition of detect_and_load_perl_modules() seems
just fine, but the differences in the file $Id
looks strnage since it refers to "windows clients".
This is install is on x86 Linux running Fedora FC3.
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Wiki: installing mimedefang and clamav
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Oliver > Schulze L. > Sent: Tuesday, December 06, 2005 1:37 PM > To: mimedefang@lists.roaringpenguin.com > Subject: [Mimedefang] Wiki: installing mimedefang and clamav > > > Hi, > I just updated this page in the Wiki: > "Instructions for configuring clamav with mimedefang" > http://www.mimedefang.org/kwiki/index.cgi?ConfigureClamAV > Hey, I read your page just yesterday. Wanted to mention something we ran into here (Fedora 3 with updates, clamav installed via yum and rpm's). Everything worked well until we rebooted yesterday, and then clam wouldn't start, which in turn caused MdF to hold off incoming mail with temp fails. Analysis indicated that clamav couldn't write to its log files. After investigating this further, we determined that the user-id definition in /etc/logrotate.d/clamd was listed as clamav instead of defang. After changing the user id, and chowning the files under /var/log/clamav, and restarting clamd and and MdF things got better. Since all this stuff was working before ... it seemed to me that a recent update (via yum) had brought in a logrotate definition for clamav that we didn't have before. Either that, or one we had before got overwritten. Didn't investigate. I've never quite understood why clamd and MdF can't run as separate users and still get along. There'd be fewer config. gotchas if clamav could run with its own user-id. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: Sober (Was Re: [Mimedefang] code 421 and filter_sender)
> -Original Message- > From: Paul Whittney > Sent: Tuesday, December 06, 2005 5:35 PM [...] > > However, for some sites that deal with a small number of domains that > accept email, the first thought is to "block all that could be the virus", > and then move to the next task of the day (or hour ;-). I've actually had > good responses with checking the IP addresses that are sending to some > of our domains, doing a whois on the IP, and calling/emailing the tech > contact listed. Remember, the reason the emails are knocking on your > server's door is that an infected machine has your users email address > somewhere on their system (okay, thats a bit too simple, as it could be > going through cached/saved files looking for emails, but still..). > Actually, more often than not, it is a "zombie" either controlled remotely or acting as a relay/proxy, so my e-mail address isn't sitting on the computer the mail originated from, but further upstream. Interesting idea, though to dig out the originating IP address, and see if it is anywhere in my extensive mail archive, trying to backtrack it to the likely owner. > Do it nicely, and not by saying "hey, you're infected, stop it!". Offer > logs, if needed. What do you get out of it? Less infected emails! Isn't > that the point? Deal with the problem, not the symptom. Its like Dshield > for emails ;-P > We're getting plenty of e-mails auto-returned to our postmaster saying they won't deliver the virus that postmaster at our site sent them. Which is really a waste of time since our postmaster didn't send it. If they honored our SPF record they'd know that because the incoming address to their system is from some dsl or dial-up account half way across the world, and not our site. Better, they could look at that received IP, dig up its MX, and send the bounce back to a postmaster who might actually be able to do something about it. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] disclamer only for out going mails.
> -Original Message- > From: bablu bablu > > Problem is disclamer is geeting attached with outgoing > as well as incoming mails. > Is there any way we can only enable disclamer for > outgoing mails. Among other things, you'll need this: http://www.mail-archive.com/mimedefang@lists.roaringpenguin.com/msg06231.htm l Spend some time with the list archives, and look up things like "boiler_plate", "confidentiality statement", "append_text_boilerplate" and so on. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Greylisting
I used John Kirkland's implementation, using MYSQL, with great success (on an FC3 system): http://www.bl.org/~jpk/md-greylist/ ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Greylisting
something worth mentioning: greylisting is highly effective, but it takes some getting used to. Like when you go to a web site and register a new account, and wait, wait, wait for the reply to come in with your account confirmation e-mail. Or when you forget your password and wait for the mail to come back with the password in it. You learn little tricks like waiting one minute and then clicking again so the second message goes through immediately. Also, e-mails can come in out of order if a new sender sends several mails, sometimes the second is accepted immediately, while the first message comes in later when the sender retries. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] disclamer only for out going mails.
> From: bablu bablu
> Sent: Friday, December 16, 2005 2:32 AM
>
> I am just making a genuine effort to get some help.
Apart from the fact that there are no great supporters on this
list of adding disclaimers, it is also true that there is
no direct implementation in the mimedefang filter
as it comes out of the box.
You mentioned "I am not much of script writer". Unfortunately,
that's a bit of problem if you're working with MIMEDefang -- it is
set up to let you "roll your own" mail filters, and that requires
some proficiency in Perl. Remember, that when you work on a MIMEDefang
filter, you're possibly working with a company's mail system at a
important level. It is oh-so-easy to drop mail, send back unwarranted
bounces, and so on. In short, MdF may not be the tool of choice for
you, if you can't bridge the gap to working with Perl scripts. Even if
someone gave you a template for how to add disclaimers, you'll still
need to change/add some code to detect whether a message is outgoing
or not, and you'll need to add your own disclaimer text, which may require
some knowledge of Perl's quoting rules, or how to write a Perl code
sequence that reads the disclaimer text in from a file.
Personally, I think it would be good if a page/two were added to the Wiki
that demonstrates a working implementation of how to add a boilerplate
to outgoing mail. The question comes up a lot, and even though the
idea of adding disclaimers demonstrates a certain degree of Corporate
Cluelessness, http://goldmark.org/jeff/stupid-disclaimers/
it is an often-requested feature.
As I mentioned in my previous e-mail, first you'll have to determine
whether the mail is outgoing or not,
http://tinyurl.com/e4lx6
That particular piece of code disables all MIMEDefang processing of
all outgoing mails, which is _not_ what you want. However, it shows
you how to check for outgoing mail. You'll need that code sequence
in your boilerplate adding code (see below), but you'll be adding it to
filter_end(). You'll need to capture the Sender's IP address in
filter_sender(). As I understand things, filter_sender() can only
communicate with filter_end() via a file in the $CWD directory (or via
an external database, which is more complicated).
Next, spend some time with "man mimedefang-filter", note the following:
append_text_boilerplate($entity, $boilerplate, $all)
This action should only be called from filter_end. It
appends the text "\n$boilerplate\n" to the first text/plain
part (if $all is 0) or to all text/plain parts (if $all is
1).
append_html_boilerplate($entity, $boilerplate, $all)
This action should only be called from filter_end. It adds
the text "\n$boilerplate\n" to the first text/html part (if
$all is 0) or to all text/html parts (if $all is 1). This
function tries to be smart about inserting the boilerplate;
it uses HTML::Parser to detect closing tags and inserts the
boilerplate before the tag if there is one, or before
thetag if there is no . If there is no
or tag, it appends the boilerplate to the end
of the part.
Do not use append_html_boilerplate unless you have installed
the HTML::Parser Perl module.
and then turn to the MIMEDefang How-To:
http://www.mickeyhill.com/mimedefang-howto/#s9
Scan down the page, to section 9.8 ("filter_end"), and note this part,
Another common use of filter_end is the insertion of boilerplate text into a
message. The append_text_boilerplate and append_html_boilerplate functions
append text to the first text/plain or text/html part found in the message.
These functions would be added inside the existing filter_end:
append_text_boilerplate($entity, "All information contained in " .
"this email is confidential and may be used by the intended " .
"recipient only.", 0);
append_html_boilerplate($entity, "All information contained in " .
"this email is confidential and may be used by the intended " .
"recipient only.", 0);
Your boilerplate adding code will need to be added to filter_end,
as described.
Others here, please correct me if the following is incorrect,
or if there is an easier way, but here are the steps that based
upon my understanding need to be done to accomplish the task:
1. assert MX_SENDER_CHECK in /etc/sysconfig/mimedefang:
MX_SENDER_CHECK=yes
2. add checking code in fllter_sender() to write the sender's
IP address and e-mail address to a file called sender_id.txt
under the $CWD directory.
3. in filter_end() read the Sender's IP address from the
sender_id.txt file and check it against the IP addresses
of your local machines, using the technique mentioned
earlier. If it is determined to be outgoing, then add
the boilerplate.
4. Restart MimeDefang and test. It would be best to
test this on a test
RE: [Mimedefang] dictionary attacks looking for a valid user
> From: David F. Skoll > Sent: Thursday, December 15, 2005 1:53 PM > > Unfortunately, MIMEDefang only sees exactly what was in the > RCPT TO: command. It doesn't know the results of virtusertable > changes. > > (Though it occurs to me that it can see the mailer, so if you > map invalid addresses to something magical in virtusertable, and > have that magical thing select the "error" mailer, then MIMEDefang > might see it... have to test.) David, Can the "socket map" feature be put to work here? SOCKET MAPS If you have Sendmail 8.13 or later, and have compiled it with the SOCKETMAP option, then you can use a special map type that communi- cates over a socket with another program (rather than looking up a key in a Berkeley database, for example.) mimedefang-multiplexor implements the Sendmail SOCKETMAP protocol if you supply the -N option. In that case, you can define a function called filter_map to implement map lookups. filter_map takes two arguments: $mapname is the name of the Sendmail map (as given in the K sendmail configuration directive), and $key is the key to be looked up. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] disclamer only for out going mails.
> From: Joseph Brennan > Sent: Friday, December 16, 2005 12:51 PM > To: mimedefang@lists.roaringpenguin.com > > Anyone attempting this is going to need to think through precisely > which messages need the special treatment. > > Gary's example focuses on mail coming from IP addresses owned by the > company. This seems to exclude travelling employees using the company > smtp server, and to include visitors using the company's wire. It > also seems to include company employees sending mail to each other. > Does that get the disclaimer? (I don't know.) > Good point. > I was going to suggest that what you need to do instead is concentrate > on where the mail is going. If the recipient is not local, then you > need the disclaimer. However milter sees only the envelope recipient > as given, not the resolved address. Aliases, lists, and forwarding > could route apparently local recipients to other systems. > > So maybe milter is the wrong place. In sendmail, the mailer is where > you really know where a message is going. So you could define a > mailer for all domains other than your own, and with it, send all > such mail to an outbound gateway host. The gateway would just > disclaim everything it gets. This seems like it may be a useful approach for other situations as well. For example, right now disable MdF processing of mail originating from our internal IP addresses. One could argue that might at least scan for viruses, but we don't do that. In general though it would be useful and better to determine "outgoing" without explicit checks for IP addresses. Can you give an example of how this might be configured so that the "outgoing" gateway runs on the same box, and how MdF checks for the mailer ID? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] disclamer only for out going mails.
BTW, the sendmail FAQ had this to say: http://www.sendmail.org/faq/section3.html#3.35 Q3.35 -- How do I add a footer/signature to all (outgoing) e-mail messages? Date: October 9, 2000 Updated: August 1, 2001 Updated: May 16, 2002 Updated: February 22, 2003 This is quite complicated. At first sight it might be simple: just "cat" some text (taken from a file or whatever) to the end of an e-mail message passing through sendmail. However, there is a big problem: what about structured e-mail messages, i.e., MIME messages? These can be arbitrarily complex and just "cat"ting a footer to the end of the body can break the MIME structure. (A MIME aware MUA will just not show such a footer, so it's pretty useless in any case.) But signed messages (think: PGP) will break. Another problem is the character set used by the mime part to which the disclaimer is added needs to match the actual character set of the disclaimer itself. Hence, there is no easy solution to this problem! If you know enough about MIME and some C programming, then take a look at sendmail 8.11 (or later) and libmilter/README. It now offers the functionality to achieve this goal. Some open source milters are capable of adding footers, e.g. MIMEDefang. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] disclamer only for out going mails.
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of Jan > Pieter Cornet > Sent: Saturday, December 17, 2005 7:12 AM > To: mimedefang@lists.roaringpenguin.com > Subject: Re: [Mimedefang] disclamer only for out going mails. > > > On Fri, Dec 16, 2005 at 11:46:35AM -0800, Gary Funck wrote: > > filter_end(). You'll need to capture the Sender's IP address in > > filter_sender(). As I understand things, filter_sender() can only > > communicate with filter_end() via a file in the $CWD directory (or via > > an external database, which is more complicated). > > Err, no! This is overly complicated. > > There should probably be a page on the wiki that says something like: > _ALL_ information available in filter_relay, filter_sender, and > filter_recipient is _also_ available in filter_begin, filter, > filter_multipart and filter_end through various global variables > (See GLOBAL VARIABLES SET BY MIMEDEFANG.PL in man mimedefang-filter). > > In this case, through $RelayAddr. The documentation doesn't make this immediately clear: $RelayHostname The host name of the relay. This is the name of the host which is attempting to send e-mail to your host. May be "undef" if the host name could not be determined. This variable is avail- able in filter_relay, filter_sender and filter_recipient. $RelayAddr The IP address of the sending relay (as a string consisting of four dot-separated decimal numbers.) One potential use of $RelayAddr is to limit mailing to certain lists to people within your organization. This variable is available in fil- ter_relay, filter_sender and filter_recipient. [...] However, further up it says: GLOBAL VARIABLES SET BY MIMEDEFANG.PL The following global variables are set by mimedefang.pl and are avail- able for use in your filter. All of these variables are always avail- able to filter_begin, filter, filter_multipart and filter_end. In addition, some of them are available in filter_relay, filter_sender or filter_recipient. If this is the case, it will be noted below. So ... it seems to be accurate enough, but if one turns to just the variable description itself ... it seems to be more specific, and perhaps overriding the general statement. Not sure how to fix that. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] disclamer only for out going mails.
> From: Jan Pieter Cornet
> Sent: Saturday, December 17, 2005 7:12 AM
> To: mimedefang@lists.roaringpenguin.com
>
> In this case, through $RelayAddr.
OK, thanks. Here is the ammended method:
In filter_end() obtain the Sender's IP address from the
$RelayAddr global variable, and check it against the IP addresses
of your local machines, using the technique mentioned
earlier. If it is determined to be outgoing, then add
the boilerplate.
4. Restart MimeDefang and test. It would be best to
test this on a test server.
And here, is the candidate code.
First, add this subroutine somewhere above filter_end.
sub is_ip_local ($) {
use Socket;
my $ip = shift @_;
# List networks that are to be classified as local ip's
my %local_subnets = (
'127.0.0.0','255.0.0.0', # loopback
'192.168.0.0', '255.255.0.0', # internal net
'199.199.199.0','255.255.255.0'# Your Class C address here
);
my $addr = inet_aton $ip;
while (my ($network_string, $mask_string) = each %local_subnets) {
my $network = inet_aton $network_string;
my $mask = inet_aton $mask_string;
if (($addr & $mask) eq $network) {
return 1;
}
}
return 0;
}
Modify or delete the line above for your Class C address and mask,
as required. Add other IP ranges and masks as required.
Then towards the end of filter_end add the following:
if (is_local_ip ($RelayAddr)) {
append_text_boilerplate($entity,
"text disclaimer", 0);
apend_html_boilerplate($entity,
"HTML disclaimer", 0);
action_rebuild();
}
The caveats that Joseph Brennan pointed out apply. This approach
won't necessarily catch employees who access the mail server from
a remote network, and it will add disclaimers to mail sent from
the internal network by visitors and contractors. It will also
add the disclaimer to mail sent between employees within the
network, and it will re-add the disclaimier even it is already
present in the e-mail body. Basically, a lot more work is needed
to cover those additional cases.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Greylist with shared data
> > I notice that a number of people have implemented John Kirkland's MySQL > greylist implementation from http://www.bl.org/~jpk/md-greylist, but his > website appears to be no longer available?? Try again. Seems to be working just now. - Gary ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Pre-Emptive Greylist entries
> From: Roland Pope > Sent: Monday, January 09, 2006 5:50 PM > > One idea I had was to try and create a whitelist entry in the > database for > emails sent from within my network to customers, to try and reduce delays > for initial replies from said customers. > > Ie. When one of my users sends an email from [EMAIL PROTECTED] to > [EMAIL PROTECTED], I lookup custnet.com in DNS, get the IP's of the > highest priority MX's and create a whitelist entry so that it > decreases the > chance that a reply from [EMAIL PROTECTED] to [EMAIL PROTECTED] gets > delayed by the greylist code. Others have mentioned possible problems with this approach. I'll add one more: viruses. If one of the client PC's on your network gets hit with a virus then it may try to send mail to every address in the user's address book; presumably many of those addresses will be for clients. You might catch many of those virus initiated e-mails via a virus scanner. However, many installations don't scan outgoing messages for viruses (we don't), so they won't be caught in this fashion. Q: As a matter of best practice should we be scanning outgoing messages for viruses, and rejecting them? A couple of ideas on the incoming side: 1. RDNS the sending relay, and match the domain name against a list of domain names of customers, which might be found by saving outbound domains as you describe above (with the caveats as described also). 2. Something I've toyed with: _if_ the sending relay supports SPF and the SPF validates - accept the mail unconditiionally and don't greylist it. You might need to run another milter for validation, or adapt some Perl code to the task. There may be other validation methods (HABEAS, etc) that work here. 3. Match the sender's mail address domain to the RDNS domain. If they match, let the mail through without tempfailing (this is weaker than SPF, so must be done after the SPF check). To catch some of the cases where a spammer hijack's a legitimate user's PC, and sends mail as that user (haven't seen this, except for viruses, but it seems possible), perhaps it is a good idea to _always_ tempfail messages with many recipient addresses? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Pre-Emptive Greylist entries
> > > 2. Something I've toyed with: _if_ the sending relay supports SPF and > >the SPF validates - accept the mail unconditiionally and > > don't greylist it. > > It is rapidly becoming the case that SPF validation is a > higher-than-average > sign of SPAM, since the spammers have more of an incentive to get it right > than the rest of the world. Don't rely on it, and certainly don't bypass > greylisting because of it. Effectively, you are trusting someone > else's DNS > records when determining whether your policy can be bypassed. Paul, I appreciate what you're saying above. But isn't it likely that if spammers are going to the trouble to add SPF records that they're also going to the trouble to retry after a tempfail, and thus defeat greylisting? The case that I was trying to check for was the case where the spammer sends through a proxy or a zombie and otherwise impersonates the sender. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Pre-Emptive Greylist entries
> From: David F. Skoll > Sent: Tuesday, January 10, 2006 11:02 AM > > [EMAIL PROTECTED] wrote: > > > That is the sole purpose of SPF, to force domains that send spam to > > admit ownership of the sending servers. > > spammer.com.1d IN TXT "v=spf1 +all" > > I own the world! :-) > > (Yes, I know SPF implementations can treat such a record with suspicion, > but in principle, it's a valid record.) OK. It is funny, but my suggestion was serious. When I suggested that the SPF record be checked, I was suggesting that it be checked using a reasonably conservative intepretation of SPF. And, I agree with Matthews, that if the spammers are willing to validate their presence via SPF, I'm willing to bypass greylisting and blacklist them a short time later. Keep in mind that greylisting is only a heuristic first defense. If I have the time, I'll give my suggestions regarding the use of SPF and RDNS a shot, and report back on the results. My hunch is that they'll offer decent improvements, especially in handling first time senders. Better, perhaps I'll process the message logs and give some feedback on how this approach might fare. - Gary ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] 40K+ emails a day and choking update
> From: David F. Skoll > Sent: Tuesday, January 10, 2006 5:35 PM > > Ah. That screams network problems. DNS latencies can kill you, > especially if you're using SURBL lookups inside SpamAssassin. > High DNS latency causes slave processes to build up. A caching DNS server running on the same box can help? - Gary ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] limiting SA's RBL timeouts
As a follow-up to the discussion regarding the fact that sendmail and spamassassin perform a lot of DNS lookups, one thing we do here to speed up SA a bit is to tell it to limit the time it waits for a response from a spam block list lookup. We place the following in our local.cf file in SA's rules directory: # limit RBL checking to max. 5 secs instead of default of 30 rbl_timeout 5 How much this improves things, I don't know. It was suggested on the SA mailing list a while ago. Is there a similar timeout value for sendmail lookups? - Gary ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Pre-Emptive Greylist entries
> > If I have the time, I'll give my suggestions regarding the use > of SPF and RDNS a shot, and report back on the results. My hunch > is that they'll offer decent improvements, especially in handling > first time senders. Better, perhaps I'll process the message logs > and give some feedback on how this approach might fare. > I ran some tests using our mail logs as input data. After reviewing the results and the process that I used, I think I can get better and more useful results by changing the analysis program, but that will be more work and will have to wait for another day. My program tracked 'grey_new' entries, where we record the first time we see a (sender_from_address, recipient_address, helo_ip/24) triple. I also extracted the helo_address (eg. [EMAIL PROTECTED]) from a related log entry. Given this triple, I can look it up in the SQL data base to see whether we ultimately accepted mail from that tempfail, or whether it never returned after being tempfailed. At the moment, we don't recycle the triples in our database, and it is rather large (about 350,000 entries). I've wanted to run some programs on the data to come up with a blacklist of particular offensive ISP's, so have not recycled old entries. Given the sender/recipient/helo_address triples above, I noted whether the helo_address appeared to be forged (based upon sendmail's determination), and using Mail::SPF::Query I noted whether this sender_address/helo_address pair registered as as a 'pass', or anything else. With this data in hand, I looked at messages that either (1) received a 'pass' from SPF, or (2) were not forged and the sender's from address matched the domain part of helo address. Of a total of 38215 greylist new entries (first time, tempfail), 1631 met the SPF/sender address criteria. Of those 1631 entries 682 entries were ultimately accepted for delivery, thus there was no harm in white listing them early using the heuristic. 931 would have been 'false positives' in that we would have accepted them early using the heuristic, when in fact they never retried after the tempfail in the old scheme. The heuristic would've accepted 4.2% newly seen sender/recipient triples, with a roughly 60/40 split of 'false positives' to messages that would ultimately have been white listed anyway. Note that of the 'false positives' not all of them were necessarily spammers. Some of them might have been legitimate senders using poorly configured software. In any event, this technique at worst adds only 2.5% more entries which are delivered and which must subsequently be processed using the access list and content filters. I hand inspected a few entries accepted by the technique above, for early bypass of the greylist mechanism. The heuristic did a good job of letting through legitimate first timers, which of course is the point of going to all the trouble to make these extra checks. Overall, I'd say this heuristic using SPF and simple analysis of the sender address and helo address has promise in improving the system's ability to let legitimate first time senders through immediately. Perhaps by also validating the helo address as a valid mx for the sender address, or noting that it is in the same /24 as the sender, the heuristic can be improved further. - Gary ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] limiting SA's RBL timeouts
> -Original Message- > From: Joseph Brennan > Sent: Thursday, January 12, 2006 6:41 AM > > You could have Mimedefang do those lookups instead, early in the > process. If you will reject for being in certain RBLs then you can > dispose of those messages without running the SA stuff. > > Also, I think Spamassassin looks up all the IPs in the headers, > while Mimedefang can test just the IP relaying the mail. In my > opinion this is more accurate as well as more efficient. On the latter point first, I don't think SA looks at all IP's in the headers. Using the trusted_networks configuration parameters, you can tell it whhich Received lines it can trust. It will then check the incoming IP's on those lines, but I don't recall whether it does RBL checking on those lines by default. SA also does RBL checking on IP addresses derived from URL's in the message body, which is something that would be quite a bit of work for MdF. We do use RBL's via sendmail's access list, using a program called spfilter to build up those access lists from known user-selectable RBL's sources. It works well. - Gary ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] validating 'possibly forged' helo IP's?
I need a program that will convert a HELO IP address into a
FQDN with some confidence. I've prototyped one, below.
Is it doing the right thing? Couple of questions:
1. Is it okay to use the first (and only the first) PTR record?
2. Is it okay to use the (default) recursive search?
3. Is it okay to use the first "A" record to validate
the name returned by rDNS?
4. Does this seem like a reasonable way to validate a HELO IP
address and convert it into a useable HELO address?
thanks, - Gary
dns.pl --
#!/usr/bin/perl -w
use strict;
use Net::DNS;
my $res = Net::DNS::Resolver->new;# DNS resolver object
my $ip = shift;
if ($ip =~ /(\d+\.){3}\d+/) {
my $dns_query = $res->search($ip);
my $domain_name;
my $rdns;
if ($dns_query) {
for my $rr ($dns_query->answer) {
next unless $rr->type eq "PTR";
$rdns = $rr->ptrdname;
last;
}
if (defined($rdns)) {
$dns_query = $res->search($rdns);
if ($dns_query) {
for my $rr ($dns_query->answer) {
next unless $rr->type eq "A";
$domain_name = $rr->name;
last;
}
if (defined($domain_name)) {
print "domain: $domain_name\n";
} else {
print STDERR "no A record for $rdns\n";
exit 2;
}
} else {
print STDERR "cannot resolve rDNS $rdns\n";
exit 2;
}
} else {
print STDERR "no PTR record for $ip\n";
exit 2;
}
} else {
print STDERR "ip: $ip not found\n";
exit 2;
}
} else {
print STDERR "not an IP address: $ip\n";
exit 2;
}
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] validating 'possibly forged' helo IP's?
> 4. I don't know what end goal you are trying to achieve but using reverse > records for any type of sercurity or blocking has pretty high > false positive > rates. ISPs in my experience don't even really care about setting reverse > DNS up. KAM, thanks. I'm looking to munge my greylist log entries looking for further improvements on the 'accept without delay' heuristic. A greylist log entry is of the form: MDLOG,k08G82sc010344,grey_new,60,202.223.97.20,[EMAIL PROTECTED],[EMAIL PROTECTED] ntrepid.com,? The perl script says the following: ./dns.pl 202.223.97.20 domain: pdf6114.tokynt01.ap.so-net.ne.jp where the fields are queue id, record type, timeout, helo IP, sender address, recipient address, and subject. Subject is always '?' on new greylist entries. I'd like to validate the HELO IP in some fashion without having to schlep further into the log to see if sendmail complains with "possibly forged", because the heuristic when implemented won't have the luxury of knowing what sendmail thinks down the line. Of course what is missing in the log entry above is the claimed HELO name. Given that I could try and resolve that to an IP and then compare that IP to relay IP, which would be a more reliable check. Perhaps a better heuristic for me to try is to take the sender's domain, convert that to an IP address, and then check to see if the relay address is an MX for the sender, or in the same /24 as the sender, or has a 'pass' in the SPF records. Keep in mind that the only bad thing that happens if the heuristic fails is that the message is not tempfailed. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] validating 'possibly forged' helo IP's?
> ISPs in my experience don't even really care about setting reverse > DNS up. Isn't it good practice for mail exchanges to have a PTR record? I can understand why ISP's don't go to the trouble to rDNS every IP in the network, but it would seem to be a good idea to support rDNS on their outwardly visible servers. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] a case for supporting EXPN/VRFY on an internal server
One of the difficulties we run into with MdF in general and greylisting in particular is that recipient address verification (via the access database) is delayed via delay_checks. So, basically we tempfail messages with invalid recipient addresses that we should reject outright at the HELO phase, and these messages will have to be bounced later if the sender retries. Further, if we're front-ending other servers, we may not even know whether the user address is valid. But ... if the internal servers (including a dummy server on the relay box for access_db checks) supported VRFY, then MdF could check the addresses early on by first consulting the internal server(s). Make sense? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Change Required Score
> > I want to decrease the value of "required_score" or "required_hits". I > want to decrease it from 5 to 4.3. I edit the file > /etc/mail/spamassassin/local.cf and put this line: An alternative to tweaking the threshold is to look at the messages that are getting through and either upward adjusting the rule scores for the rules that hit, or adding new rules to hit on features in the marginal SPAM messages. Also, perhaps install Rules du Jour: http://www.exit0.us/index.php?pagename=RulesDuJour ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] This one got a chuckle
> From: Philip Prindeville > Sent: Friday, January 13, 2006 12:44 AM > > Which reminds me... I need to start looking into how to add > filters for non-latin1 email content... If you're using Spamassassin, you can tell it which locales and languages you expect, http://spamassassin.apache.org/full/3.0.x/dist/doc/Mail_SpamAssassin_Conf.ht ml For example, this should catch most European (Latin) dialects: ok_languages en fr it es de ok_locales en We place this sort of local configuration info. in local.cf along with other driectives like trusted_networks, whitelists, and local overrides of rule scores. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Mimedefang errors: What might be the cause?
> From: Lisa Casey > Sent: Saturday, January 14, 2006 8:16 AM > > I'm running Mimedefang/Spamassassin on a Redhat server with Sendmail. Which versions? What sort of hardware (cpu type, speed, memory size) How many average messages/day? > This > has all been running fine for a couple of years now. Suddenly, > this morning > my customers were unable to send e-mail. Looking at > /var/log/maillog, I saw > lots of entries such as this: > > Jan 14 02:22:18 Raydeus-Dee mimedefang-multiplexor[1014]: No free slaves > Jan 14 02:22:18 Raydeus-Dee mimedefang-multiplexor[1014]: No free slaves Unless you have a very active night shift, 2AM is not a typical time to expect a mail overload. Couple of things to check: 1. disk space: try 'df -h' and note if any file systems are full or near full. Especially the one that /tmp lives on. 2. check your version of SA. Versions 3.0.0 through 3.0.3 are subject to Denial of Service attacks. 3. Try running, 'md-mx-ctrl rawstatus', and check 'man md-mx-ctrl' for an explanation of the results. With some work, you could write a cron script that tails /var/log/maillog and looks for the 'no free slaves' and have it run some combination of 'top -b | head -20', 'md-mx-ctrl rawstatus' and any other commands that can give you some info. on what might be going on. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Mimedefang errors: What might be the cause?
> Try running, 'md-mx-ctrl rawstatus' 'md-mx-ctrl load' is also useful, and human readable. The various other commands described in 'man md-mx-ctrl' may also help provide some insight into what is going on. For example, 'md-mx-ctrl slaveinfo 0' will tell you which pid is assigned to slave 0. Then you can run 'strace -p' to follow the system calls it makes and try to understand where it may be running aground. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] validating 'possibly forged' helo IP's?
Speaking of rDNS, check out this log entry (user name and sub-domain, obfuscated as 'fred'). Jan 10 09:09:02 intrepid sendmail[31995]: k0AH8pZE031992: to=<[EMAIL PROTECTED]>, ctladdr=<[EMAIL PROTECTED]> (1001/1001), delay=00:00:06, xdelay=00:00:03, mailer=esmtp, pri=151951, relay=mailhost.cotse.com. [216.112.42.58], dsn=4.0.0, stat=Deferred: 451 4.7.1 No RDNS: Sender IP address is not resolving: http://mail.cotse.net/cgi-bin/whitelist-request-form.cgi?h=nrdns The web page stats: Cotse.Net Whitelist Request Form You have been directed to this page either because your machine issued an attack on our server and was automatically blocked, because it has no rdns, or because it's rdns looks like an end user machine and not a mail server (due to massive numbers of spam zombied end user machines we've been forced to employ some common pattern matching). Fill out the form below to get automatically whitelisted. We're not on any RBL's, and haven't had any other complaints, so I assume they're being overly cuatious. Interesting approach though. PS: Although our name server handles the rDNS properly, however http://dnsstuff.com tells me that our upstream ISP-based NS does not. Perhaps that is what this site was complaining about. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] poor performence from SA
> -Original Message- > From: [EMAIL PROTECTED] > Sent: Wednesday, January 11, 2006 6:51 AM > > I'm fed up with SA ! > Spam gets through no matter what i do :-( > and ham is blocked (well not all ham, but even one is > sometimes too much) Unfortunately, this is the nature of the beast. "False positives", where ham is classified as spam, are the worst, but do occur. Similarly some spam gets through. You can improve the situation by taking various steps such as: (1) raising the spam threshold, (2) implementing manual whitelists and blacklists, (3) using various black list name servers, (4) Adding custom rules such as those at http://www.rulesemporium.com/index.html, and (5) greylisting. Personally, I don't like tweaking the overall spam threshold as mentioned in (1), above, but prefer to decrease the score on certain rules that seem to trigger false positives, and to increase the score for rules that seem good at detecting spam. I've also strayed away from using Bayes, because I found it difficult to maintain and manage and have had decent luck using the other techniques. One thing that can be done when you initially set up Spamassassin is to make sure you have trusted_networks set correctly and then add a bunch of whitelist_from_rcvd rules for hosts that send promotional literature that you want to accept, so it won't get scored as spam. You can also ask your boss, the CEO, and executive staff to go through their inboxes and give you a list of from addresses of customers and friends that they want to accept mail from and add those to your SA whitelist rules. This is a bit ad hoc, but increases the chances that SA performs well (at least at first. ) and ensures that much of the important mail will make it through. There might be better, more elaborate, mechanisms that can be applied, but for a small system, this sort of "personal touch" approach is manageable. Whatever changes you make, do them incrementally. Make sure they're having the desired effect before moving on to the next one. Also, _always_ remember to restart the various demons involved (ie mimedefang and/or sendmail), after making changes. As others have mentioned, if you want something quicker, turnkey, with support try CanIT: http://www.roaringpenguin.com/ > My boss got MAD because he was expacting a mail from some > client...so i checked > the logs...mail.log ofcourse, i saw the usual from=...Milter > add: header: > X-Scanned-By: MIMEDefang 2.54 on x.x.x.x , to= stat=Sent > BUT the mail was gone!! nowhere to be found!! > not in the mailbox (/var/spool/mail/Xbox) not in spamdrop nowhere!! You'll need to provide us with the mail log entries if you want help on that. Suffice it to say it is highly unlikely that the mail was dropped. It is more likely that a follow-on delivery/filtering program such as procmail refiled the mail somwewhere you're not looking. It is also possible that Mimedefang quarrantined the mail for some reason. By looking at all the log messages you should be able to clear that up. > i going crazy! > so i whitelisted the origin domain and it worked...i started getting the > emails... > What am i doing wrong???!!!??? It isn't bad to whitelist important domains, but try doing it using trusted_networks and whitelist_from_rcvd, to avoid spoofing. What is bad is not first understanding the cause of the problem (lost mail) before shooting from the hip to "fix" it. When some other domain comes in, and has its mail "lost", you're still at square one on that one. > > Details (i know you want them...): > OS: debian serge 3.1a > Sendmail 8.13.4 + mimedefang 2.54 + SA 3.0.3 + clamav > > What else? did i forget anything? If you want help, you'd need to provide: 1. all log entries for the problem mail (hint: grep on the mail queue id) and submit them here. 2. provide your mimedefang-filter either as an attachment or via URL. 3. provide the output of 'mimedefang.pl -features' as an attachment or via URL. 4. provide your sendmail.mc file either as an attachment of via URL. 5. you need to understand if you're using procmail for mail delivery and if it has any default or custom filters in place. If you are using procmail, consider turning on logging (LOGABSTRACT=on) for each user, at least for now, but keep in mind the logfiles will keep growing, so you'll need a method to trim them back, if you keep logging turned on. 6. you should disable spamd if it is enabled 7. you should understand if you have other 'milters' (such as spamass-milter) installed and enabled that may be interacting with Mimedefang and disable them. > From: [EMAIL PROTECTED] > Sent: Thursday, January 12, 2006 1:55 AM > > I have upgraded to SA 3.1 but i get strange actions... I would _not_ have upgraded SA until I understood what problems I expected it to fix. You've just introduced new variables. BTW, as you upgrade to newer versions of SA (and Mimedefang) you increase the need to make sure that you have th
RE: [Mimedefang] Mimedefang + SA go crazy !
> From: [EMAIL PROTECTED] > Sent: Sunday, January 15, 2006 12:22 AM > To: mimedefang > Subject: [Mimedefang] Mimedefang + SA go crazy ! > > > hi, > > Now that i upgraded to SA 3.1 I have new problems. > Today (sunday) SA started consuming all cpu resorces!! > It opens threads/proccesses and use up to 98% cpu WTF??? > Sometimes only one instance of SA will use 98% cpu?!?!? > SA is called from mimedefang-filter (i don't run spamd). > When you type 'ps -u defang', do you actually see instances of spamassassin, or are you talking about instances of mimedefang.pl that you happen to know are calling SA internally? If you see actual instances of spamassassin, spamd, or spamc running, then you have a configuration error that you need to chase down. As far as being cpu bound ... SA is a cpu hungry program. So, when SA runs (via mimedefang) it will take all the cpu that it can get, at least for a second/two. Thus, while it is scanning a mail, it can peg the cpu usage meter. Make sure you check your mimedefang filter -- it should not be calling SA on large messages (say messages greater than 100K). > I don't have to say that this is cuasing big problems for me and > the company... > I also noticed (using top) that the spamassassin proccess are invoked by > 'regular' users of the mailserver , eh?? (the local delivery mailbox > users...non of them has shell access...) They may not have shell access, but procmail may be invoked via sendmail as a local delivery agent, and the global (or per-user) procmailrc file may be (incorrectly) invoking SA directly. Take a look at your sendmail.mc file to see if procmail is configured as a local delivery agent. If you've got stuff like this in sendmail.mc, then you are likely invoking procmail: define(`PROCMAIL_MAILER_PATH',`/usr/bin/procmail')dnl FEATURE(local_procmail,`',`procmail -t -Y -a $h -d $u')dnl MAILER(procmail)dnl In that case, you should take a look to see if there is a global /etc/procmailrc file, or per-user $USER/.procmailrc file. You should also check each user home directory for a .forward file to see if (and how) it is invoking procmail (or some other mail filing program). > also...when cpu is not used by 'user' or 'sys' then the 'wait' > state has the > ~90% instead of 'idle' why is that so? Don't know, but maybe it is waiting for network I/O or DNS lookups. Try typing 'md-mx-ctrl load' when the system is busy, for some additional info. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] MIMEDefang 2.55-BETA-2 is available
This is really an academic question, but would it be possible to scan/compile the mimedefang filter before including and adjust the prototype declaration and use accordingly? (This more a Perl question than a suggestion.) - Gary ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Mimedefang errors: What might be the cause?
> From:David F. Skoll > Sent: Monday, January 16, 2006 8:58 AM > > > Kris Deugau wrote: > > > define(`confQUEUE_LA', `2')dnl > > define(`confREFUSE_LA', `7')dnl > > Bad settings. > > Having REFUSE_LA higher than QUEUE_LA is a surefire way to kill > your server. Some handy tuning/debugging tips: http://www.brandonhutchinson.com/Miscellaneous_Sendmail_notes.html Quoting: "rejecting connections on daemon MTA" By default, Sendmail will reject MTA/MSA connections when the system load is 12 * number of processors, and operate in queue-only mode when system load is 8 * number of processors. egrep "QueueLA|RefuseLA" /etc/mail/sendmail.cf #O QueueLA=8 #O RefuseLA=12 >From cf/README: confQUEUE_LAQueueLA [varies] Load average at which queue-only function kicks in. Default values is (8 * numproc) where numproc is the number of processors online (if that can be determined). confREFUSE_LA RefuseLA[varies] Load average at which incoming SMTP connections are refused. Default values is (12 * numproc) where numproc is the number of processors online (if that can be determined). For dedicated mail servers, the value of RefuseLA should be greater than the value of QueueLA. Mail should not be accepted faster than it can be processed, as mentioned in sendmail/TUNING: It is important not to accept mail faster than it can be delivered otherwise the system will be overwhelmed. Hence RefuseLA should be lower than QueueLA, the number of daemon children should probably be lower than the number of queue runnners (MaxChildren vs. MaxQueueChildren). Also, read this comp.mail.sendmail post http://groups.google.com/group/comp.mail.sendmail/msg/41d65caec2a4edb3?hl=en &lr=&ie=UTF-8 On a dedicated mail server, you want to set QueueLA *higher* than RefuseLA - the other way around will cause your queue to explode when the load is high (assuming you reach QueueLA of course), you accept lots of new messages but can't get rid of them as the load average hovers around RefuseLA - the many messages in the queue drive up disk I/O and thereby the load average. The default of QueueLA lower than RefuseLA only makes sense for a server that does other things besides mail, where you want to reduce the impact of the mail load on the other functions by queuing instead of delivering when the load is high. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] 40K+ emails a day and choking
> From: Michael Lang > Sent: Tuesday, January 17, 2006 7:50 AM [...] > > at my last company we did with 4 machines, 3Mil/Day Messages without any > problem. The machines where HP DL360 2G Ram 1 CPU ;) With Mimedefang and SA? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Adding filter_helo support
I've kind of lost the thread here what is the recommended use for filter_helo? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Adding filter_helo support
> > Philip wanted an opportunity from MIMEDefang to change how Sendmail > reacts to the HELO/EHLO command. Right now, you can't; you have to wait > for MAIL FROM: to do anything based on the HELO argument. > OK, and what about the question raised as to how incoming mailers might react if, for example, tempfailed at HELO time, and related questions? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] BIG problems with mimedefang
> From: Lisa Casey > Sent: Wednesday, January 18, 2006 12:02 PM > To: mimedefang@lists.roaringpenguin.com > Subject: [Mimedefang] BIG problems with mimedefang > > > Hi, > > I'm running Redhat 7.2 This computer functions as a Radius > server (cistron > radius 1.6.7) and as a mail server (sendmail 8.12.6) which also runs > MIMEDefang 2.48 and SpamAssassin version 3.0.1 running on Perl version > 5.8.5. [...] > 11859 defang 10 0 30572 20M 1852 D 6.3 4.0 0:10 mimedefang.pl > 11657 defang 10 0 30172 9252 1820 D 3.0 1.8 0:07 mimedefang.pl > 11652 defang 9 0 29184 8400 1832 D 2.9 1.6 0:07 mimedefang.pl Try running strace on one of those busy slaves to see what sort of system calls it is making. Also, you might be able to ascertain what the errant process is doing by running the debugger on the errant processes (can the perl debugger be pointed to a particular process?), by looking at the various artifacts under /proc/. Perhaps others here can offer more extensive debugging tips. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] BIG problems with mimedefang
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf Of David > F. Skoll > Sent: Wednesday, January 18, 2006 2:03 PM > To: mimedefang@lists.roaringpenguin.com > Subject: Re: [Mimedefang] BIG problems with mimedefang > > > Gary Funck wrote: > > > Lisa Casey wrote: > > >> 11859 defang 10 0 30572 20M 1852 D 6.3 4.0 0:10 mimedefang.pl > >> 11657 defang 10 0 30172 9252 1820 D 3.0 1.8 0:07 mimedefang.pl > >> 11652 defang 9 0 29184 8400 1832 D 2.9 1.6 0:07 mimedefang.pl > > > Try running strace on one of those busy slaves to see what > > sort of system calls it is making. > > They're all in "D" state -- waiting for disk I/O. > > I bet /var/spool/MIMEDefang is not on a ramdisk. > Might be a good guess, but why did this suddenly start becoming a problem? Maybe 'strace' (combined with info. from 'lsof') would offers some insight into which files are being accessed? Any other ways to debug running Perl processes? Can the mimedefang "master" somehow start the slaves so they can later be debugged, or is there a related technique? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] BIG problems with mimedefang
David F. Skoll wrote: > Sent: Wednesday, January 18, 2006 5:37 PM > > If that's the case, SpamAssassin has a seriously broken BerkeleyDB Bayes > implementation. > If I recall correctly, 2/3 years ago, there was a particular version of the Berkeley DB implementation that was bugging, esp. with respect to locking (or lack thereof). It might've been in the Perl DB wrapper. Don't recall exactly -- but the docs. pointed you to a particular Berkeley DB version and Perl wrapper version. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] BIG problems with mimedefang
David F. Skoll wrote: > > Could you be thinking of this bug? > > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=116192 > > That wasn't a locking bug. It was a weird bug whereby Berkeley DB > would, for no reason at all, sleep for one second whenever it needed > to allocate memory! > > It's still present in Fedora Core 1, I believe. Yup, that might be the one. Here's the SA side of it: http://issues.apache.org/SpamAssassin/show_bug.cgi?id=3048 - Gary ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] BIG problems with mimedefang
David F. Skoll wrote: > https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=116192 > > That wasn't a locking bug. It was a weird bug whereby Berkeley DB > would, for no reason at all, sleep for one second whenever it needed > to allocate memory! > > It's still present in Fedora Core 1, I believe. So, is there a possibility that this bug might be manifesting itself in Lisa's case on a memory constrained RH 7.2 system? I guess if 'rpm -q db4' comes up with db4-4.1.25-8, she should consider upgrading, and should recheck the Perl module versions that MdF 2.48 depends upon? Lisa wrote: > Jan 14 10:40:23 Raydeus-Dee mimedefang-multiplexor[1211]: started; > minSlaves=2, maxSlaves=10, maxRequests=500, > maxIdleTime=300, busyTimeout=600, clientTimeout=10 > > and I think the maxSlaves looks like it is low to me. > > By the way, I'm using RedHat 7.2, mimedefang version 2.48, > and SpamAssassin version 3.0.1 running on Perl version 5.8.5 > (I'm basically using MimeDefang to run Spamassassin). ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Puttting MIMEDefang spool on tmpfs
> From: Jason Gurtz > Sent: Monday, January 23, 2006 11:30 AM > > My knee-jerk reaction is that perhaps the kernel doesn't have tmpfs > support compiled in or available/correctly configured as a kernel module. My gut reaction is that it would be better to find out _why_ MdF is red-llned, first. I hope moving to a RAM fs will help, but given the relatively small memory size on Lisa's system, dedicating a chunck to tempfs will only increase memory pressure for MdF and the other things that run on that system. If Lisa goes that route, it may be equally important to add more memory also. But somehow, I think the problem may lie elsewhere. Recall that in the first example Lisa gave that the system was getting bogged down at the wee hours of the AM. Is there a cronjob that runs then? Does it do something like age out old Bayes database entries or something else that might lock a critical resource, for example? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Question for the HOWTO page
> From: Philip Prindeville > Sent: Tuesday, January 24, 2006 5:09 PM > > I was wondering if we could update the HOWTO pages to describe > installing Mimedefang and Spamassassin both on a system, so that > the former is run, then the latter, or incoming email. > > I'd like to be able to reject mail that fails certain tests, like > containing > Hebrew, Cyrillic, and Han character sets (for instance)... rather than > accepting it and marking it as spam. Since you can coax SA to tag e-mails that have unacceptable languages and locales ... why not just run SA from MdF directly, and then look at the result (the tags) returned by SA? I don't know if the SA protocol will give you those tags directly, but it wouldn't be difficult pulling them from the headers. For example, X-Spam-Score: 11.565 (***) CHARSET_FARAWAY_HEADER,FORGED_HOTMAIL_RCVD,FORGED_RCVD_HELO,SPF_HELO_SOFTF AIL, SPF_SOFTFAIL,UNWANTED_LANGUAGE_BODY,URIBL_JP_SURBL ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Adding support for learning our addresses
> From: David Nelson > Sent: Saturday, January 28, 2006 9:13 AM > > I subscribe to ip2location.com, which provides geolocation services by > IP address. The info is downloaded on nightly from their web servers > and put into a database. I check the IP addresses contained in the > message against the database and if it's from a foreign country, I block > it. We use spfilter to build a large sendmail access list file: http://sourceforge.net/docman/display_doc.php?docid=14634&group_id=49927 We run it nightly to update the access list, and then rebuild access.db. Spfilter can produce the lists in many different formats. Here's the block lists currently available: http://spfilter.openrbl.org/code/xml-view.php We run with SPAM_SAFE,COUNTRY,CBL as well as our own local overrides. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Adding support for learning our addresses
> From: Philip Prindeville > Sent: Saturday, January 28, 2006 10:43 AM > > Cool. Too bad no one has written an XML way of retrieving it and > parsing it out. > They have. See BOGO below: http://spfilter.openrbl.org/code/xml-view.php BOGO INTERVAL="7" TYPE="cidr/3" MAZSIZE="2" OPTION="notext" home http://www.cymru.com/Documents/bogon-list.html url http://mirror.bliab.com/bogo/BOGO.cidr.aggreg.gz url http://www.cymru.com/Documents/bogon-bn-agg.txt url http://www.cymru.com/Documents/bogon-bn-nonagg.txt tag BOGON ROUTE append - http://openrbl.org/whois?i= ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Pre-Emptive Greylist entries
Following up on this dusty old thread ... one idea that I'd mentioned was perhaps using SPF as a way of validating IP relay addresses for whom their mail should not be delayed via greylisting. I noticed this in a mail header today: X-Greylist: Sender is SPF-compliant, not delayed by milter-greylist-1.7.5 (linuxbox.org [127.0.0.1]); Mon, 30 Jan 2006 16:04:18 -0600 (CST) It looks like the greylist milter developer people implemented some form of SPF checking. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] spams slipping by, because they bigger than the SA size cutoff
I've had a couple of spams drop in my inbox recently, and at first, I couldn't see how they made it past SA. I looked at the headers, and to my surprise, the message hadn't been scanned by Spamassassin(!). Why? How? I looked further, and noticed that one message was 800K bytes, and the other 140K. The first had an attached .wmv file (hopefully not one of _those_ .wmv files, but I didn't click on it to find out). Both messages avoided being scanned by SA because they were larger than the 100K limit we currently impose via MdF. What to do? I can bump the size limit, or have no limit at all. I could consider building a temporary copy of the message with non text and/or html attachments removed, and feed that to SA, although that sounds a bit complicated and computationally expensive. BTW, a couple of years ago, I experiment with simpiy peeling off the first N bytes off the front of large messages and tacking on the last N bytes (where N = approx. 50K). This actually worked rather well, modulo the occassional malformed MIME content. I could do that, but it is a bit of hack, and could result in having some valid ham messages mis-filed as spam, partly because the MIME parts are garbled. Thoughts? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] spams slipping by, because they bigger than the SA size cutoff
> From: Stephen J. Smoogen > Sent: Wednesday, February 01, 2006 4:22 PM > > Well depending on how patched your system is.. and what application > you are using for email you do not have to click on the wmv file. Just > having some clients process the email can cause problems (according to > one write up about WMV). I would recommend that you put wmv on the > extensions block list and your problem is solved. Good idea for .wmv files, but still doesn't address the correct action for spam files that exceed the cutoff. > > I would also recommend a grey-list or other mechanism. > Yup, we do that, and it works well. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] spams slipping by, because they bigger than the SA size cutoff
> From: Jan Pieter Cornet > Sent: Thursday, February 02, 2006 12:58 AM [...] > > A .wmv file is a windows media video file, and has nothing to > do with the .wmf exploits that were recently in the news. > You're right - I wasn't paying attention. In fact it is likely we already filter out .wmf files. I shouldn't have mentioned it, anyway, because it isn't central to the topic of scanning large messages. > > Both messages avoided being scanned by SA because they were > > larger than the 100K limit we currently impose via MdF. > > > > What to do? I can bump the size limit, or have no limit at all. > > I could consider building a temporary copy of the message > > with non text and/or html attachments removed, and feed > > that to SA, although that sounds a bit complicated and > > computationally expensive. > > It's a _LOT_ less computationally expensive than letting SA handle > the binary attachments. Note that SA can use binary attachments > in some rules (various HTML_IMAGE_* rules, and MIME encoding rules), > so if you remove them, only remove "big" ones. Are folks really doing this? I don't recall MdF having a lot of mechanism for generating a temp. copy of the message, removing large parts, reassemblying, and then scanning the temp. copy. I'm sure that by calling the correct MIME Tools primitives this could be done, it just seems like it would take a fair amount of logic. It might be tempting to simply quarrantine attachments bigger than 50K (which is a pretty puny attachment) or so, but I don't think those quarrantined attachments are visible until the incoming message is reassembled by MdF just before it is accepted and delivered. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] MIMEDefang and mailman
> I imagine this is because MIMEDefang can't infer what username to > use - is there a way I can tell it? Given that MdF's "slaves" include Spamassassin directly, they'll each have to run as the user of choice. You'll probably need a set up where you run two copies of MdF and its helpers. You'd need to set up a separate mail queue for the mailman traffic. Maybe this could be done by making your mailing list id's into virtual users, and redirect them to a virtual host, which is served by a separate sendmail instantiation, or perhaps simply as a separate queue? Not sure of the mechanics there. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Free Tool Ferrets Out Mail Server Problems
http://www.emailbattles.com/archive/battles/email_aaddbfghhe_ch/ Free Tool Ferrets Out Mail Server Problems Posted on 04/05/2006 @ 15:11:25 in Email. Trouble receiving mail? Installing a new mail server? Need to make sure all your email servers are accessible? Experienced network managers have long plodded through DNS queries, making sure that MX records matched A records which matched IP addresses. Then they checked SMTP ports to make sure the servers were open for business... This can be a fairly time-consuming and error-prone process, especially if you or your email vendor, sport 11 mail servers, like IBM... Or two MX records that point to 15 hosts, like 3com.com... Or five MX records pointing to 18 IP addresses, some of which are only for outbound email, like sun.com. [...] http://www.emailbattles.com/archive/battles/email_aaddhghiad_ih/ Why Yahoo Can't Deliver Email Posted on 04/12/2006 @ 16:55:14 in Email. Have you ever wondered why senders complain so much about Yahoo Mail's poor delivery? So did Email Battles. Curious, we pointed our hot new diagnostic toy, Mail Server Profiler, at Yahoo's mail servers, to see if we could find any hints. First, Mail Server Profiler mapped and analyzed the Yahoo Mail setup. It found 16 host IP addresses behind four mail server MX records, and identified half of them as closed, ie, not accepting any email messages. [...] Next, we took measurements every two minutes for half an hour. That's 15 separate readings of each of 16 IP addresses, for a total of 240 readings. The results were surprising. During that period, Yahoo mail servers were willing to accept mail just 55% of the time (133 "open" readings). Average availability for MX record groups over the period ranged from as low as 25% to a high of just 75%. [...] ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Image validator/OCR SA plugin
> From: Martin Blapp > Sent: Friday, April 14, 2006 9:43 AM > > This is just a little advertisement for my plugin which is now > in a usable state and works very well. > > > http://antispam.imp.ch/patches/patch-ocrtext > Martin, this is a Very Good Thing, and has been needed for quite some time. Thanks. Have you thought about posting this to the Spamassassin User's List? I know that a lot of users there would really appreciate this add in. http://mail-archives.apache.org/mod_mbox/spamassassin-users/ - Gary ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Image validator/OCR SA plugin
Martin wrote: > But problably the spammers > will soon change their tricks to different images which are more > difficult to read :-( > > http://antispam.imp.ch/patches/patch-ocrtext On this topic, Nick FitzGerald mentioned this article, http://www.jgc.org/blog/2006/01/do-spammers-fear-ocr.html Sunday, January 15, 2006 Do spammers fear OCR? Nick FitzGerald recently sent me two sample spams that seem to indicate that some spammers fear that using images in place of words isn't enough. They've started to obscure their messages to prevent optical character recognition. [...] ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: AW: [Mimedefang] Image validator/OCR SA plugin
> -Original Message- > From: Martin Blapp > Sent: Monday, April 17, 2006 8:00 AM > > Spamassassin version is 3.1.0, looks like I'll have to upgrade to 3.1.1 > > to get this to work? > > Seems so, yes. I'll correct the manual. Has this package/plugin been updated yet with the various fixes suggested to date? I had a little trouble tracking some of the suggested fixes. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Image validator/OCR SA plugin
Late follow-up to this thread ... was wondering if there might be something to be gained by running the OCR scan from mimdefang? The idea would be to run the scan, and if sufficient text results (I'd hesitate to suggest that a quick spelling scan would be run on the result, but that is a possibility) that this text is written by MdF into a new text attachment. The message is then reformulated and passed to Spamassassin. The advantage of this approach is that SA (and rules du jour) already have rules for catching things like pharma and stock scam e-mail, so the normal scoring should catch these things. Also this approach would work on versions of SA prior to 3.1.1. There is a design decision as to whether the OCR'd text attachment should remain in the message and then be delivered to the user, or whether it would only be kept if SA scores the message as spam. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
