[Mimedefang] ADMIN: Roaring Penguin no longer accepts mail from gmail.com
Hi David, +1 from my side. I had very long discussions about just the same issue two years ago with google, and all they said is, that they have different blocks of google IP-adresses where they route the outgooing good, and the outgoing bad mails. And of course they hide the originating IP. Unfortunatly their routing was and still is bad, so only blocking both IP-ranges helps. A very very bad design. Thats also a reason why we (I'm one of the admins there) set google.com at @dnswl.org to level NONE. http://www.dnswl.org/search.pl?s=google.com. You can skip graylisting for them, but not any spam checks. IMHO we have to do something against this. Since they record the google account in the mails, how about a dns block list of abused google accounts ;-) ? Just another blacklist, but one which would be effective ... -- Martin Hello, Roaring Penguin software no longer accepts mail from gmail.com addresses; our reasoning is here: http://www.roaringpenguin.com/whynogmail I see 46 gmail.com users subscribed to MIMEDefang. Sorry for the inconvenience, but if you want to post to MIMEDefang, you'll need to do so from a non-GMail account. Regards, David. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Sendmail logging bug ?
Hi all, hi David,
Is it only me or has Sendmail 8.14.3 a bug in store-and-forward logging ?
If a recipient is graylisted, I get this error with vanilla sendmail:
Jul 18 19:32:06 vm1 sendmail[23161]: n6IHW3ae023145: to=t...@xxx.ch,
delay=00:00:02, xdelay=00:00:02, mailer=esmtp, pri=30020, relay=mx.xxx.ch.
[1.1.1.1], dsn=4.3.0, stat=Deferred: Name server: mx.xxx.ch.: host name
lookup failure
But infact, this connection has been graylisted. After removing the part
below,
I get the real masked error.
Jul 18 20:38:21 vm1 sendmail[86970]: n6IIcIFe086960: to=t...@xxx.ch,
delay=00:00:02, xdelay=00:00:02, mailer=esmtp, pri=30020, relay=mx.xxx.ch.
[1.1.1.1], dsn=4.3.0, stat=Deferred: 451 4.3.0 Graylisted for 400 - 600
seconds, please try again later.
--- sendmail/deliver.c 2009-07-18 20:17:19.0 +0200
+++ sendmail/deliver.c 2009-07-18 20:16:14.0 +0200
@@ -4017,6 +4017,11 @@
(void) sm_strlcpy(bp, exmsg + 1, SPACELEFT(buf, bp));
bp += strlen(bp);
+#if NAMED_BIND
+ if (h_errno == TRY_AGAIN)
+ statmsg = sm_errstring(h_errno + E_DNSBASE);
+ else
+#endif /* NAMED_BIND */
{
if (errnum != 0)
statmsg = sm_errstring(errnum);
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Mimedefang and sendmail queueing
Hi, You have multiple options: 1.) Read the ESMTP SIZE, then use some kind of adress rewriting to select the queuegroup as described here: http://newsgroups.derkeiler.com/Archive/Comp/comp.mail.sendmail/2006-05/msg00303.html 2.) Use adress rewriting to modify the sendmail routing, reroute then to a local mailer which has several queuegroups and does normal routing. This has the advantage that you don't need to read the SIZE ESMTP argument, which doesn't need to be true at all. The sender could lie about the size. -- Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] [PATCH] filter_data implementation
Hi all,
And here the patch again, this time with examples and parts
of the manpage.
--
Martin
--- mimedefang.c2009-05-24 07:40:40.0 +0200
+++ mimedefang.c2009-05-24 06:27:35.0 +0200
@@ -233,6 +233,9 @@
/* Do recipient check? */
static int doRecipientCheck = 0;
+/* Do precontent check */
+static int doPreContentCheck = 0;
+
/* Keep directories around if multiplexor fails? */
static int keepFailedDirectories = 0;
@@ -978,12 +981,67 @@
*%RETURNS:
* Standard milter reply code
*%DESCRIPTION:
-* Does a post-DATA callback
+* Does a post-DATA callback before any content is submitted
***/
#ifdef MILTER_BUILDLIB_HAS_DATA
static sfsistat mf_data(SMFICTX *ctx)
{
-return SMFIS_CONTINUE;
+struct privdata *data = DATA;
+char ans[SMALLBUF];
+sfsistat retcode = SMFIS_CONTINUE;
+int i;
+
+DEBUG_ENTER(mf_data);
+if (!data) {
+ syslog(LOG_WARNING, postdata: Unable to obtain private data from milter
context);
+ DEBUG_EXIT(mf_data, SMFIS_TEMPFAIL);
+ return SMFIS_TEMPFAIL;
+}
+
+/* Post data check if enabled */
+if (doPreContentCheck) {
+ int n;
+
+ n = MXDataOK(MultiplexorSocketName, ans, data-sender, data-hostip,
+ data-hostname, data-firstRecip, data-heloArg,
+ data-dir, data-qid);
+
+ if (n == MD_REJECT) {
+ /* Reject this mail with all recipients */
+ set_dsn(ctx, ans, 5);
+
+ DEBUG_EXIT(mf_data, SMFIS_REJECT);
+ return SMFIS_REJECT;
+ }
+ if (n = MD_TEMPFAIL) {
+ /* Tempfail this mail with all recipients */
+ set_dsn(ctx, ans, 4);
+
+ DEBUG_EXIT(mf_data, SMFIS_TEMPFAIL);
+ return SMFIS_TEMPFAIL;
+ }
+ if (n == MD_ACCEPT_AND_NO_MORE_FILTERING) {
+ /* Called in case we don't need content filtering */
+ set_dsn(ctx, ans, 2);
+ cleanup(ctx);
+ DEBUG_EXIT(mf_data, SMFIS_ACCEPT);
+ return SMFIS_ACCEPT;
+ }
+ if (n == MD_DISCARD) {
+ set_dsn(ctx, ans, 2);
+
+ cleanup(ctx);
+ DEBUG_EXIT(mf_data, SMFIS_DISCARD);
+ return SMFIS_DISCARD;
+ }
+ if (n == MD_CONTINUE) {
+ /* Called only in case we need to delay */
+ set_dsn(ctx, ans, 2);
+ return SMFIS_CONTINUE;
+ }
+}
+DEBUG_EXIT(mf_data, SMFIS_CONTINUE);
+return retcode;
}
#endif
@@ -2092,6 +2150,7 @@
fprintf(stderr, -r-- Do relay check before
processing body\n);
fprintf(stderr, -s-- Do sender check before
processing body\n);
fprintf(stderr, -t-- Do recipient checks before
processing body\n);
+fprintf(stderr, -A-- Do pre content check for
processing body\n);
fprintf(stderr, -q-- Allow new connections to be
queued by multiplexor\n);
fprintf(stderr, -P file -- Write process-ID of daemon to
specified file\n);
fprintf(stderr, -T-- Log filter times to syslog\n);
@@ -2189,7 +2248,7 @@
}
/* Process command line options */
-while ((c = getopt(argc, argv,
NCDHL:MP:R:S:TU:Xa:b:cdhkm:p:qrstvx:z:)) != -1) {
+while ((c = getopt(argc, argv,
ANCDHL:MP:R:S:TU:Xa:b:cdhkm:p:qrstvx:z:)) != -1) {
switch (c) {
case 'N':
#ifdef MILTER_BUILDLIB_HAS_NEGOTIATE
@@ -2346,6 +2405,9 @@
case 't':
doRecipientCheck = 1;
break;
+ case 'A':
+ doPreContentCheck = 1;
+ break;
case 'h':
usage();
break;
--- mimedefang.h2009-05-24 07:40:40.0 +0200
+++ mimedefang.h2009-05-24 06:26:27.0 +0200
@@ -40,6 +40,10 @@
char const *dir, char const *qid,
char const *rcpt_mailer, char const *rcpt_host,
char const *rcpt_addr);
+extern int MXDataOK(char const *sockname, char *msg,
+char const *sender, char const *ip, char const *name,
+char const *firstRecip, char const *helo,
+char const *dir, char const *qid);
extern int safeWriteHeader(int fd, char *str);
extern void split_on_space(char *buf, char **first, char **rest);
--- mimedefang.pl.in2009-05-24 07:41:08.0 +0200
+++ mimedefang.pl.in2009-05-24 07:10:42.0 +0200
@@ -5645,6 +5645,21 @@
chdir($Features{'Path:SPOOLDIR'});
next;
}
+ if ($_ =~ /^dataok
(\S*)\s+(\S*)\s+(\S*)\s+(\S*)\s+(\S*)\s+(\S*)\s+(\S*)/)
{
+ $sender = percent_decode($1);
+ $ip = percent_decode($2);
+ $name = percent_decode($3);
+ $firstRecip = percent_decode($4);
+ $helo = percent_decode($5);
+ $CWD = percent_decode($6);
+
Re: [Mimedefang] Adding headers during filter_sender() and 2.68 Beta 1 issue.
Hi, I am quite aware that the milter interface accepts header changes only during the EOM phase. However, my hack still registers the change (changes are deferred until EOM). I certainly did not expect the code to read the RESULTS file after filter_sender() returns and try to set them there, otherwise, I would have written a hack that tries to do that (which is impossible as the milter interface would reject it). Is this also true for smfi_addrcpt() and smfi_delrcpt() ? If it is allowed at this stage, I'm missing the support to delete recipients in the post data stage (mf_data) and any support for mf_data() in mimedefang.pl. This could be very useful if you still like to be able to allow SMTP probes for SRS-modified recipient adresses in the per-recipient check, but delete any bad recipient adresses (with failing or missing BATV/SRS tags) in the post data stage, without having to save the whole message data to disk. Valid SMTP-probe ... MAIL From: 250 2.1.0 ... Sender ok RCPT To:ad...@mydomain.ch 250 2.1.5 ad...@mydomain.ch... Recipient ok RSET 250 2.0.0 Reset state QUIT 221 2.0.0 mydomain.ch closing connection -- Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Adding headers during filter sender () and 2.68 Beta 1 issue.
Hi, 4) To allow these functions in filter_recipient() may cause the addition to occur for EACH recipient. That appears inappropriate. In contrast, adding a TRACE header indicating some sort of forward looking status per recipient may be appropriate (although no RFC or standard I'm aware of requires such at this time). In mf_data() all recipients are already collected. mf_data() is the callback that happens immediatly after the data command is issued, but before any content is submitted to the filter. IMHO filter_begin() should be connected to mf_data, and not the the stage later. This would safe a lot of IO. However, I don't favor SRS. Cooperating forwarding arrangements should recognize the valid forwarder (whitelist or SMTP AUTH) and bypass (only) the SPF check, making SRS unnecessary. SPF only works at the front-end receiving MTA. If one cannot trust one's forwarders, maybe that relationship shouldn't exist. Therefore, I don't see any need. SRS, ok, that may not be needed. But BATV is definitly a good thing to have, but maybe in an adaptive way if there are too many bounces in some timeframe. -- Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Repost Update: ufs filesystem problems again!?
Hi, Any ideas? Any advice? Did I mention I'm desperate? Anyone need someone to hang out with when they come to Vegas?:) Is it possible that you're calling something external in your mimedefang-filter config ? I had the same/similar results calling an external c binary when trying to start it in the background ... Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] [PATCH] Have real load sharing between milters
Hi, Does it still do this if, instead of round robin on the A records, you use round robin on the MX records? Yes I guess that's the same problem here and would need fixing too. Maybe it's the best to fix this bug in the sm_gethostbyname sendmail function. THis would fix then all those resolving issues. Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] [PATCH] Have real load sharing between milters
Hi all, We use several mimedefang milter servers in parallel. Sendmail makes it easy, and bind helps us with round robin host milter defines. But after one of the milters have been shut down, we found out that DNS round robin with bind sucks. I'll explain why it sucks. Let's say we have three milters, then bind answers us only three possible results: 1) milter.172800 IN A 10.0.0.1 milter.172800 IN A 10.0.0.2 milter.172800 IN A 10.0.0.3 2) milter.172800 IN A 10.0.0.2 milter.172800 IN A 10.0.0.3 milter.172800 IN A 10.0.0.1 3) milter.172800 IN A 10.0.0.3 milter.172800 IN A 10.0.0.1 milter.172800 IN A 10.0.0.2 Do you see whats wrong ? The first entry is always randomized, but the following entries are not, they are sorted ascending. So let's imagine milter 10.0.0.3 has crashed. Then all requests to 157.161.9.21 are beeing redirected to filter 10.0.0.1 since sendmail just takes the next ip following 10.0.0.3. Filter 10.0.0.1 ends then with 2/3 of all connections, 10.0.0.2 with 1/3. This is IMHO unusable and may lead to tempfails and long delay on filtered emails. There are also DNS implementations which behave totally static, like one famous from Microsoft. The order it uses is always the same. Of course, there are also different implementations which behave more random-friendly and would make that patch unneccessary. But the patch is written for bind users in the first place :-) Anyway, I've made a patch to sendmail to fix this problem independent from DNS: http://antispam.imp.ch/patches/patch-sendmail-milterrandom If you find any bugs just tell me :-) Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MIMEDefang 2.57 is Released
Hi David, I just found why queueing never really worked here on my side. We used a modified 'examples/init-script.in' as startup script and in this script, ALLOW_NEW_CONNECTIONS_TO_QUEUE is not defined in the header. If ALLOW_NEW_CONNECTIONS_TO_QUEUE=yes is not set, queueing only works up to 5-10 queued messages, afterwords the filter is tempfailing. I thought long time that this is some mimedefang bug and I never gave it significant importance. But since the spam amount is constantly growing, queueing is more and more important for us. So can you please add a comment to examples/init-script.in and maybe set ALLOW_NEW_CONNECTIONS_TO_QUEUE=no in the header too ? This way new users can change that value and don't need to stick with the sourcecode to find out what ALLOW_NEW_CONNECTIONS_TO_QUEUE really means :-) Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Re: Pyzor stopped working since update to sa 3.1.3? (James E. Pratt)
Hi, There's a bug in Pyzor such that it crashes when trying to examin base64-encoded, multipart/mixed, or missing boundary messages. See https://sourceforge.net/tracker/index.php?func=detailaid=665950group_id=5atid=458242 Fixed long time ago with my patches: http://antispam.imp.ch/08-opensource.html?lng=0 Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Pyzor stopped working since update to sa 3.1.3?
Hi, Hi. ever since I updated a test relay to SA 3.1.3 from 3.1.2, pyzor (0.40) has stopped(?) working ... Thats because you now need to add the pyzor and dcc path to your local.conf if it's not in the local path ! dcc_path/usr/bin/dccproc pyzor_path /usr/bin/pyzor After you fix that, pyzor will work again. Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] [PATCH] Fix for hardcoded clamscan in mimedefang.pl
Hi David,
It seems that this last patch has been improperly done: I got various
failures like:
Jun 2 01:10:35 filter1 mimedefang.pl[56787]: k51N9qlg076697:
run_virus_scanner: Unable to execute clamscan -r --unzip --unrar --mbox
--stdout --disable-summary --infected ./Work 21: No such file or directory
Jun 2 01:10:35 filter1 mimedefang.pl[56787]: k51N9qlg076697: Clamd returned
error: Zip module failure
--- mimedefang-2.56/mimedefang.pl.in.orig Fri Jun 2 16:55:01 2006
+++ mimedefang-2.56/mimedefang.pl.in Fri Jun 2 16:55:48 2006
@@ -4603,7 +4603,7 @@
# This is despicable, but it might work
if ($err_detail =~ /zip module failure/i) {
my ($code, $category, $action) =
- run_virus_scanner(clamscan -r --unzip --mbox --stdout --disable-summary
--infected $CWD/Work 21);
+ run_virus_scanner($Features{'Virus:CLAMAV'} . -r --unzip --mbox --stdout
--disable-summary --infected $CWD/Work 21);
if ($action ne 'proceed') {
return (wantarray ? ($code, $category, $action) :
$code);
}
@@ -4699,7 +4699,7 @@
# This is despicable, but it might work
if ($err_detail =~ /zip module failure/i) {
my ($code, $category, $action) =
- run_virus_scanner(clamscan -r --unzip --mbox --stdout
--disable-summary --infected $CWD/Work 21);
+ run_virus_scanner($Features{'Virus:CLAMAV'} . -r --unzip --mbox
--stdout --disable-summary --infected $CWD/Work 21);
if ($action ne 'proceed') {
return (wantarray ? ($code, $category, $action) :
$code);
}
Thank you for fixing.
Martin
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Perl module update crashes MIMEDefang
Hi, Do you use FreeBSD ? Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Another silly idea
Hi, regional. How are you collecting the data? Is it only systems that have The data are updated in realtime per ns-update from about 8 different ISPs. sent to your server(s)? Do you age systems out of the RBL after three days? Yes, each night there is a script running which removes the old entries. Do you think it blocks much spam (the UCE type), or just repeat virus senders? The wormlist serves more as an page for the local providers to see which customers are infected by a worm/virus. Each provider can then warn his customer and also close the customers account if nothing happens. But the spamlist can be used as blacklist of course. It lists spamsenders from all over the world - mostly USA. Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Another silly idea
Hi, Do you mean something like: http://antispam.imp.ch/03-wormlist.html?lng=1 Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Multiplexor dies, leaving its slaves clueless behind...
Hi, To get a mimedefang corefile you may have to execute: # sysctl kern.sugid_coredump=1 Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Another silly idea
Hi, Is anyone using this list with success? Yes, some ISPs in europe do. As you can see, the listed IPs belong to europe ISPs mostly. The problem is that the data sources aren't mixed with global players. Virus and worm statistics seem always to be local based - in contrary to spam statistics as you can see. Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Image validator/OCR SA plugin
Hi, be something to be gained by running the OCR scan from mimdefang? The idea would be to run the scan, and if sufficient text results (I'd hesitate to suggest that a quick spelling scan would be run on the result, but that is a possibility) that this text is written by MdF into a new text attachment. The message is then reformulated and passed to Spamassassin. The advantage of this approach is that SA (and rules du jour) already have rules for catching things like pharma and stock scam e-mail, so the normal scoring should catch these Hmm, the SA and rules du jour stock and obfu rules suck ;-) Beside that, I also match some words which are 100% legitimate. And the OCR words are often truncated so one must match those too. things. Also this approach would work on versions of SA prior to 3.1.1. There is a design decision as to whether the OCR'd text attachment should remain in the message and then be delivered to the user, or whether it would only be kept if SA scores the message as spam. If you add the OCR'd text attachment to the message you'll have to resend the whole message. Not a good idea IMHO. Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Image blocking idea
Hi, On a different note concerning images, what about an email filter logging the possibility of the images containing hidden data (i.e. Steganography test). I already log possible text (I count alphanummeric chars in the ocr output) +header SPAMPIC_ALPHA_1 OCR-Output =~ /OCRTEXT: more than alpha1 chars found/ +describe SPAMPIC_ALPHA_1 Image contains many alphanumeric chars +score SPAMPIC_ALPHA_1 0.500 + +header SPAMPIC_ALPHA_2 OCR-Output =~ /OCRTEXT: more than alpha2 chars found/ +describe SPAMPIC_ALPHA_2 Image contains many alphanumeric chars +score SPAMPIC_ALPHA_2 1.000 + +header SPAMPIC_ALPHA_3 OCR-Output =~ /OCRTEXT: more than alpha3 chars found/ +describe SPAMPIC_ALPHA_3 Image contains many alphanumeric chars +score SPAMPIC_ALPHA_3 1.500 You could now do a statistic analytic to see if the chars match any language specific char occurance to see if its really text. Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Image blocking idea
Hi, I should have been clearer. I don't even believe they aren't even modifying the image. Indications are that they are simply truncating or adding random bits to the file because even a slightly corrupted image is still displayed. We already do 'Image checksumming' here with a modified pyzor (patches are available at http://antispam.imp.ch. The problem is that you first have to catch one of those graphics and in the last time they change often. Even with 250 spamtraps we let passing a lot of those mails passing the filter. I've not seen any instant changing pics up to now, but I guess the spammers will soon do this. The results I get from the ocr plugin are a lot better ! Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: AW: [Mimedefang] Image validator/OCR SA plugin
Hi, Spamassassin version is 3.1.0, looks like I'll have to upgrade to 3.1.1 to get this to work? Seems so, yes. I'll correct the manual. Thanks, Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Image validator/OCR SA plugin
Hi all, This is just a little advertisement for my plugin which is now in a usable state and works very well. Anyone interested should keep an eye on it - it really helps with the image only spam we get today. But problably the spammers will soon change their tricks to different images which are more difficult to read :-( http://antispam.imp.ch/patches/patch-ocrtext Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Mimedefang sometime doesn't timeout Mail::SpamAssassin::Timeout parts in SA plugins
Hi, I suspect SA itself has the same problem but since I did only some tests the problem never happened to me. This Plugin works in Spamassassin, but in mimedefang sometimes some of the timeouts get ignored. Does it work if you disable the embedded Perl feature? The same problems. Timeouts in SpamAssassin are (to the best of my recollection) implemented with SIGALRM, which by itself is evidence of scariness, but I don't think MIMEDefang should affect SA. I'm not sure if this is true on FreeBSD, but maybe try implementing your 100-second wait with something like this: select(undef, undef, undef, 100); Yes the example was a bad one since I'm working in my plugin completly with pipes so I should have made one with pipes too. I've changed SIGTERM now to be SIGKILL, let's see it that helps. Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Image validator/OCR SA plugin
Interesting... What's the performance like with this? How many messages do you scan per day with it? It is rather fast. On a Pentium IV 3Ghz I can scan a average jpg/gif picture in 0,2 - 0,3 seconds. I've limited the scantime to 5 seconds per image, and I allow only three images to be scanned per mail. Of course this is user configurable. The greps here are just up to now, not a full day. grep hits= /var/log/maillog | wc -l 78050 grep X-Spam-Status: Yes /var/log/maillog | wc -l 48400 grep hits=.*SPAMPIC /var/log/maillog | wc -l 9572 grep X-Spam-Status: Yes.*hits=.*SPAMPIC /var/log/maillog | wc -l 9558 grep X-Spam-Status: Yes.*hits=.*SPAMPIC /var/log/maillog | grep HTML_IMAGE_ONLY | wc -l 9528 # grep HTML_IMAGE_ONLY /var/log/maillog | wc -l 35834 This means 60% of all mails we get are SPAM. More than 10% of the SPAM are some gif and jpg pictures advertizing for stocks and meds. But almost 45% of all mails match HTML_IMAGE_ONLY, so it's unusable at all. I even use lower scores for those rules now - which gives me less FPS: score HTML_IMAGE_ONLY_041.400 score HTML_IMAGE_ONLY_081.300 score HTML_IMAGE_ONLY_121.200 score HTML_IMAGE_ONLY_161.100 score HTML_IMAGE_ONLY_200.950 score HTML_IMAGE_ONLY_240.900 score HTML_IMAGE_ONLY_280.700 score HTML_IMAGE_ONLY_320.400 Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Image validator/OCR SA plugin
# grep HTML_IMAGE_ONLY /var/log/maillog | wc -l 35834 This is wrong. It should have been # grep HTML_IMAGE_ONLY.*hits= /var/log/maillog | wc -l 17917 But almost 45% of all mails match HTML_IMAGE_ONLY, so it's unusable at all. I even use lower scores for those rules now - which gives me less FPS: 22% is still a lot ... Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Mimedefang sometime doesn't timeout Mail::SpamAssassin::Timeout parts in SA plugins
Hi David,
This Plugin works in Spamassassin, but in mimedefang sometimes some of the
timeouts get ignored. This explains also the hanging helper utilities, pyzor,
dcc, ocr etc I've experienced in the past.
The strange thing is that it only happens sometimes and the helper utils
always use a lot of CPU in those cases.
Any idea how to solve this problem ? Maybe its still a SA problem and
only happens sometimes (if the server is busy) ?
[38115] dbg: plugin: loading bugtest from /plugins/test.pm
[38115] dbg: plugin: registered bugtest=HASH(0x8dd058c)
[38115] dbg: plugin: bugtest=HASH(0x8dd058c) implements 'parsed_metadata'
[38115] dbg: bugtest: Set timeout to 10 seconds
[38115] dbg: bugtest: Before sleep of 100 seconds
[38115] dbg: bugtest: After timeout of 10 seconds
I suspect that some signal problems between Mdefang and SA. Maybe Mdefang
ignores some signals and SA doesn't receive anything back. Btw, I'm using
embedded perl 5.8.8.
---
package bugtest;
use strict;
use Mail::SpamAssassin;
use Mail::SpamAssassin::Plugin;
use warnings;
use bytes;
our @ISA = qw(Mail::SpamAssassin::Plugin);
sub dbg { Mail::SpamAssassin::dbg (@_); }
sub new {
my ($class, $mailsa, $server) = @_;
$class = ref($class) || $class;
my $self = $class-SUPER::new($mailsa);
bless ($self, $class);
return $self;
}
sub parsed_metadata {
my ($self, $opts) = @_;
my $permsgstatus = $opts-{permsgstatus};
$permsgstatus-enter_helper_run_mode();
dbg(bugtest: Set timeout to 10 seconds);
my $timer = Mail::SpamAssassin::Timeout-new({ secs = 10 });
my $err = $timer-run_and_catch(sub {
# Or do something that loads this slave a lot !
dbg(bugtest: Before sleep of 100 seconds);
system(sleep 100);
dbg(bugtest: After sleep of 100 seconds);
});
$permsgstatus-leave_helper_run_mode();
dbg(bugtest: After timeout of 10 seconds);
}
1;
---
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] FAQ: FreeBSD and Mimedefang and embedded perl (something for the manpages)
Hi, What's the best way to avoid a circular use XYZ; because I have that issue a lot? It doesn't hurt normally. But embedded perl doesn't seem to like it. I don't know how to check it. Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] FAQ: FreeBSD and Mimedefang and embedded perl (something for the manpages)
Hi Everybody, Maybe you still remember my postings some time ago about embedded perl and mimedefang in embedded perl mode. I had problems with the filter stability and so on. In the meantime I've resolved this issue. The problem is the way perl (the one from the FreeBSD ports tree) is compiled. The ports system does build perl with the system malloc(3), not the internal perl malloc. The reason for this it seems is that perl malloc has (had ?) problems with threaded perl on FreeBSD. But FreeBSD malloc(3) is very picky about errors, also double freeing some vars. If you have circular dependencies defined in your filter, you can be almost sure that mimedefang multiplexer will crash. To fix this issue with embedded perl just recompile perl with WITH_PERL_MALLOC defined in /etc/make.conf and be sure you recompile also all your perl modules. p5-Digest-SHA1 for examble will return false checksums if you don't. Or you carefully check if you include some module twice. There are still some cases then where your mimedefang multiplexer can crash, but it is better than nothing. Modifing the FreeBSD malloc flags doesn't help at all, the crashes where still the same. Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Modify $0 or use setproctitle in the slaves.
Hi all, I'd like to change $0 in the mimedefang filter to display in ps the subroutine/place where mimedefang is working in. After I've changed it a bit, the proctitle had changed but the slaves didn't do any filtering and where just sitting there. It the proctitle cmdline used elsewhere, does the multiplexer uses it iternally ? I'm using embeded perl mode. Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Mimedefang 2.56 and SA 3.1.1 - Idle slaves
Hi David, Have you already tried Mimedefang together with SA 3.1.1 ? I've experienced some problems with slaves getting idle and not processing anymore anything after some timeout in pyzor, razor, dcc has occured. The Changelog of SA 3.1.1. reads: Bug 4696: consolidated fixes for timeout bugs r384584 | sidney | 2006-03-09 19:15:44 + So they have changed this behaviour again. http://issues.apache.org/SpamAssassin/show_bug.cgi?id=4696 Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Mimedefang 2.56 and SA 3.1.1 - Idle slaves
Hi, package Mail::SpamAssassin::Dns; 1; package Mail::SpamAssassin::PerMsgStatus; I see. What is the impact of this ? Return value always true ? Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Milter: rejecting commands ?
Hi all,
After defineing sub filter_relay() and restarting mimedefang
filtering did not work at all. Instead of processing the mails
sendmail just gets if MX_RELAY_CHECK is set to yes in the startup
file.
Feb 2 21:26:40 mx2 sm-mta[12574]: k12KQenP012574: Milter: connect:
host=mail22.bluewin.ch, addr=195.186.19.66, rejecting commands
Feb 2 21:26:42 mx2 sm-mta[12599]: k12KQft7012599: Milter: connect:
host=[220.77.126.245], addr=220.77.126.245, rejecting commands
Feb 2 21:26:42 mx2 sm-mta[12603]: k12KQgMP012603: Milter: connect:
host=nat0.netburg.pl, addr=62.233.224.243, rejecting commands
Feb 2 21:26:44 mx2 sm-mta[12673]: k12KQiwR012673: Milter: connect:
host=p54BE3907.dip.t-dialin.net, addr=84.190.57.7, rejecting commands
Feb 2 21:26:45 mx2 sm-mta[12709]: k12KQjSf012709: Milter: connect:
host=wirelessdhcp-220-174.ideaone.net, addr=69.178.220.174, rejecting commands
Feb 2 21:26:46 mx2 sm-mta[12739]: k12KQk5s012739: Milter: connect:
host=sproxy.google.com, addr=64.233.170.130, rejecting commands
This is mimedefang 2.53. Any ideas ?
Since the check is only for 10.0.0.1 true, it should work for all
other adresses.
sub filter_relay {
my ($ip, $name) = @_;
if($ip eq 10.0.0.1) {
return ('REJECT', Too many spam-mails from $ip, $name is
blacklisted.);
}
return ('CONTINUE', ok);
}
Martin
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Milter: rejecting commands ?
Hi David, Are you seeing any stderr output from the slaves? (Assuming you're passing -l to the multiplexor. If not, please pass -l to the multiplexor!) My error, I've not pasted everything from my filter_relay and there was some error in it. The message rejecting command seems to be the warning sendmail logs if the mail has been rejected. I'm dooing now something like this test: # # If the last spam has been received ~20 minutes ago (depending how many spams we got), # allow the sender to send mails again. # my $dbh = $DBH_SLAVE; my $query = select ((unknown+spam+1)*1000)/((ham+1)*(UNIX_TIMESTAMP(now()) - UNIX_TIMESTAMP(timestamp))) as rating, from mailstats where octa=$octa and octb=$octb and octc=$octc and octd=$octd and (unknown+spam+1)/(ham+1) = 10 and ((spam 0 and unknown 0) or (unknown 30)) and ((unknown+spam+1)*1000)/((ham+1)*(UNIX_TIMESTAMP(now()) - UNIX_TIMESTAMP(timestamp))) = 10; my $th = $dbh-prepare($query); $th-execute; and then I test for the spam/ham ratio. Works very well now. Thanks anyway David for your time ! Martin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Problems with SpamAssassin 3.1 RC1and MIMEDefang
Hi, I don't know why the SA developers are even monkeying with the SIGCHLD handler in the Perl module; you'd have to ask them. It seems like a bad idea to me. Its used for executing external programms: pyzor and dcc. Martin ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Problems with SpamAssassin 3.1 RC1and MIMEDefang
Hi, I think I have a workaround; I'll release a beta soon. In the meantime, I believe that turning off the embedded interpreter will make it work properly. What exactly will you change ? In the meantime I'm going to make a SA 310 patch for the pyzor and dcc plugins. Can I do beta tests for you ? Disable embedded perl is no option here as the mem usage is too high. Martin ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MIMEDefang 2.53-BETA-1 is released
Hi David, Yes, this version fixes the Problem definitly. Thanks ! Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Problems with SpamAssassin 3.1 RC1and MIMEDefang
David, Please download SA3.1 Pre 1 and try yourself. I've setup a test installation and the problem is reproducable 100%. No slaves are killed anymore after a reload. Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Problems with SpamAssassin 3.1 RC1and MIMEDefang
I can confirm this isse. This is with the DCC/Pyzor patch applied. Aug 26 10:25:33 filter2 mimedefang-multiplexor[78519]: Slave 4 (pid 50099) taking way too long to exit; sending SIGKILL Aug 26 10:25:33 filter2 mimedefang-multiplexor[78519]: Slave 1 (pid 50068) taking way too long to exit; sending SIGKILL Aug 26 10:25:33 filter2 mimedefang-multiplexor[78519]: Slave 29 (pid 49767) taking way too long to exit; sending SIGKILL Aug 26 10:25:33 filter2 mimedefang-multiplexor[78519]: Slave 14 (pid 49853) taking way too long to exit; sending SIGKILL Aug 26 10:25:33 filter2 mimedefang-multiplexor[78519]: Slave 25 (pid 49817) taking way too long to exit; sending SIGKILL Aug 26 10:25:33 filter2 mimedefang-multiplexor[78519]: Slave 27 (pid 53445) taking way too long to exit; sending SIGKILL Aug 26 10:25:35 filter2 mimedefang-multiplexor[78519]: Slave 7 (pid 49869) taking way too long to exit; sending SIGKILL Aug 26 10:30:49 filter2 mimedefang-multiplexor[78519]: Slave 12 (pid 54146) taking too long to exit; sending SIGTERM Aug 26 10:30:51 filter2 mimedefang[78531]: mfconnect: No free slaves Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] mimedefang-2.52 and razor-agents-2.xx
Please upgrade to razor_agents 2.64 and add this fix (they managed to break spamassassin again with some debug output) Of course I mean razor-agents-2.74 ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] problems with check_against_smtp_server and a cyrus backend
David, In mimedefang filter you write for GLOBALS: In filter_begin Available to filter_begin, filter and filter_end In filter Available to filter and filter_end [...] In one of ~10 cases I see all globals vanished in filter_end(), they are lost for some reason, everything is empty. I tempfail such mails and 15 minutes later they are prozessed again, this time correctly. The strange thing is, sometime it doesn't happen for 2-3 days, then 10 times the day. Do you have an idea why this happens or where the problem could be ? Is there a way to have a safeguard in mimedefang which 'fix' those cases ? Martin ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] German spam (related to new worm Sober.q)
Hi, For those who don't follow the SpamAssassin list there are some new spam messages in German (racist ones apparently) Some extra rules are available on : http://mailscanner.prolocation.net/german.cf And here the rules for SA3: http://antispam.imp.ch/rules/sober_p.cf Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] [PATCH] clamav does not filter all zipfiles/rarfiles
Hi, Thanks; I'll look it over. But doesn't Clam 0.85 fix this? Nope. The problem is zlib which doesn't support method nine files bigger than 32k. ClamAV thinks it can decompress zip method 9 files, but can't. Look at the posts in the clamav mailing lists. :-) Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] [PATCH] clamav does not filter all zipfiles/rarfiles
Hi, Test it yourself: http://antispam.imp.ch/eicardeflate64.zip I haven't seen any of the input/output errors in /var/log/clamd.log since upgrading to 0.85. It was definitely a problem, however, in 0.84. ClamAV scans the file ... Clamav-Output: /tmp/phpd30b9a: Input/Output error ERROR Clamav DID NOT identify your sample as malicious content If you really think your sample is a virus or any other harmful thing clamav should detect please go to ... this is 0.85. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] [PATCH] clamav does not filter all zipfiles/rarfiles
Hi David,
there is currently a discussion in the clamav mailinglist about the
Input/Output error many users get with clamd. I've done a workaround
which works fairly well.
Can you include something (not tested, I've done a different patch) like this:
@@ -3714,7 +3714,7 @@
# Run clamscan
my($code, $category, $action) =
- run_virus_scanner($Features{'Virus:CLAMAV'} . --mbox --stdout --disable-summary
--infected $path 21);
+ run_virus_scanner($Features{'Virus:CLAMAV'} . --unzip --unrar --mbox --stdout
--disable-summary --infected $path 21);
if ($action ne 'proceed') {
return (wantarray ? ($code, $category, $action) : $code);
}
@@ -3738,7 +3738,7 @@
# Run clamscan
my($code, $category, $action) =
- run_virus_scanner($Features{'Virus:CLAMAV'} . -r --mbox --stdout
--disable-summary --infected ./Work 21);
+ run_virus_scanner($Features{'Virus:CLAMAV'} . -r --unzip --unrar --mbox --stdout
--disable-summary --infected ./Work 21);
if ($action ne 'proceed') {
return (wantarray ? ($code, $category, $action) : $code);
}
@@ -4455,8 +4455,21 @@
$VirusName = $1;
return (wantarray ? (1, 'virus', 'quarantine') : 1);
} elsif ($output =~ /^(.+) ERROR$/) {
- md_syslog('err', $MsgID: Clamd returned error: $1);
- return (wantarray ? (999, 'swerr', 'tempfail') : 1);
+ if ($1 =~ /(?:RAR module failure|Input\/Output error|Zip module
failure)/) {
+ entity_contains_virus_clamav($entity);
+ if ($output =~ /: (.+) FOUND/) {
+ $VirusScannerMessages .= clamscan found the $1 virus.\n;
+ $VirusName = $1;
+ return (wantarray ? (1, 'virus', 'quarantine') : 1);
+ } elsif ($output =~ /^(.+) ERROR$/) {
+ md_syslog('err', $MsgID: Clamscan returned error: $1);
+ return (wantarray ? (999, 'swerr', 'tempfail') : 1);
+ }
+ } else {
+ md_syslog('err', $MsgID: Clamd returned error: $1);
+ return (wantarray ? (999, 'swerr', 'tempfail') : 1);
+ }
+ }
}
And the same for message_contains_virus_clamd(). Of course Mimedefang then needs
dependencies to unzip and unrar. Maybe you could make that available dependent
if unzip and unrar are available ...
Else you'll get messages like this and some mails will never reach their
recipient ...
May 12 12:24:13 mx1 mimedefang.pl[7423]: j4CAO3Za051140: Clamd returned error:
/var/spool/MIMEDefang/mdefang-j4CAO3Za051140/Work/msg-7423-14.zip: Input/Output
error
May 12 19:58:10 mx1 mimedefang.pl[91187]: j4CHw8Ip013621: Clamd returned error:
/var/spool/MIMEDefang/mdefang-j4CHw8Ip013621/Work/msg-91187-38.zip: Zip module failure
Martin
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] [PATCH] clamav does not filter all zipfiles/rarfiles
Ok, I think this patch is more correct:
--- mimedefang.pl.in.orig Fri May 13 01:37:52 2005
+++ mimedefang.pl.inFri May 13 01:37:01 2005
@@ -3714,7 +3714,7 @@
# Run clamscan
my($code, $category, $action) =
- run_virus_scanner($Features{'Virus:CLAMAV'} . --mbox --stdout --disable-summary
--infected $path 21);
+ run_virus_scanner($Features{'Virus:CLAMAV'} . --unzip --unrar --mbox --stdout
--disable-summary --infected $path 21);
if ($action ne 'proceed') {
return (wantarray ? ($code, $category, $action) : $code);
}
@@ -3738,7 +3738,7 @@
# Run clamscan
my($code, $category, $action) =
- run_virus_scanner($Features{'Virus:CLAMAV'} . -r --mbox --stdout
--disable-summary --infected ./Work 21);
+ run_virus_scanner($Features{'Virus:CLAMAV'} . -r --unzip --unrar --mbox --stdout
--disable-summary --infected ./Work 21);
if ($action ne 'proceed') {
return (wantarray ? ($code, $category, $action) : $code);
}
@@ -4455,8 +4455,17 @@
$VirusName = $1;
return (wantarray ? (1, 'virus', 'quarantine') : 1);
} elsif ($output =~ /^(.+) ERROR$/) {
- md_syslog('err', $MsgID: Clamd returned error: $1);
- return (wantarray ? (999, 'swerr', 'tempfail') : 1);
+if ($Features{'Virus:CLAMAV'} $1 =~ /(?:RAR module
failure|Input\/Output error|Zip module failure)/) {
+ my($code, $category, $action) =
run_virus_scanner($Features{'Virus:CLAMAV'} .
+--unzip --unrar --mbox --stdout --disable-summary --infected $path
21);
+ if ($action ne 'proceed') {
+ return (wantarray ? ($code, $category, $action) :
$code);
+ }
+ return (wantarray ? interpret_clamav_code($code) : $code);
+ } else {
+ md_syslog('err', $MsgID: Clamd returned error: $1);
+ return (wantarray ? (999, 'swerr', 'tempfail') : 1);
+ }
}
return (wantarray ? (0, 'ok', 'ok') : 0);
}
@@ -4539,11 +4548,19 @@
$VirusName = $1;
return (wantarray ? (1, 'virus', 'quarantine') : 1);
} elsif ($output =~ /^(.+) ERROR$/) {
- md_syslog('err', $MsgID: Clamd returned error: $1);
- return (wantarray ? (999, 'swerr', 'tempfail') : 1);
- }
-}
-else {
+ if ($Features{'Virus:CLAMAV'} $1 =~ /(?:RAR module
failure|Input\/Output error|Zip module failure)/) {
+ my($code, $category, $action) =
+ run_virus_scanner($Features{'Virus:CLAMAV'} . -r --unzip --unrar --mbox
--stdout --disable-summary --infected ./Work 21);
+ if ($action ne 'proceed') {
+ return (wantarray ? ($code, $category, $action) :
$code);
+ }
+ return (wantarray ? interpret_clamav_code($code) : $code);
+ } else {
+ md_syslog('err', $MsgID: Clamd returned error: $1);
+ return (wantarray ? (999, 'swerr', 'tempfail') : 1);
+ }
+}
+} else {
# Could not connect to daemon
md_syslog('err', $MsgID: Could not connect to clamd daemon at
$clamd_sock);
return (wantarray ? (999, 'cannot-execute', 'tempfail') : 999);
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Tiny Text
Hi,
our Bayes data! It seems that the ASCII artists don't
always change the strings they use for their art, so things
like rvwndsho and xpoebbcr started to become statistically
significant in our Bayes data.
We use a specific ruleset against those 'ASCII artists', the
rawbody __SMALL_FONT/font-size:[\s\t ]{1,3}(?:1|2)(?:px|pt|;)/i
rule is part of them. We also look for different gaps between chars
rawbody __GAP_2_CHAR/[a-z][ ]{5}[a-z]/i
rawbody __GAP_3_CHAR/[a-z][ ]{6}[a-z]/i
rawbody __GAP_4_CHAR/[a-z][ ]{7}[a-z]/i
rawbody __GAP_5_CHAR/[a-z][ ]{8}[a-z]/i
rawbody __GAP_6_CHAR/[a-z][ ]{9}[a-z]/i
rawbody __GAP_7_CHAR/[a-z][ ]{10}[a-z]/i
rawbody __GAP_8_CHAR/[a-z][ ]{11}[a-z]/i
rawbody __GAP_9_CHAR/[a-z][ ]{12}[a-z]/i
rawbody __GAP_10_CHAR /[a-z][ ]{13}[a-z]/i
But just using this rules would produce too many false positives, so we
have developed our own ruleset (and I have tried to avoid false positives,
but I am still very happy to get bugfixes)
http://antispam.imp.ch/rules/asciispam.cf
Maybe it is useful for you.
Martin
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Tiny Text
http://antispam.imp.ch/rules/asciispam.cf The mails matching are all ASCII art mails: May 10 16:55:21 mx2 sm-mta[20827]: j4AEsmeP020827: Milter add: header: X-Spam-Status: Yes, hits=60.425 required=5 scantime=10.0960 seconds tests=ASCII_GAPS_5,BAYES_50,\n\tDATE_IN_FUTURE_12_24,HTML_MESSAGE,MIME_HTML_ONLY,\n\tRAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RBL_COMBO_B_3, \n\tRBL_COMBO_D_2b,RBL_COMBO_E_2b,RBL_COMBO_G_2b,\n\tRBL_COMBO_H_1,RCVD_IN_CBL_SPAM,RCVD_IN_SBL,\n\tRCVD_IN_XBL,SARE_SPOOF_COM2OTH,URIBL_S BL,\n\tURIBL_SC_SWINOG May 10 17:11:12 mx2 sm-mta[41248]: j4AFAUiH041248: Milter add: header: X-Spam-Status: Yes, hits=34.189 required=7 scantime=11.1506 seconds tests=ASCII_GAPS_4,BAYES_50,\n\tDATE_IN_FUTURE_03_06,HTML_MESSAGE,MIME_HTML_ONLY,\n\tRAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RBL_COMBO_D_1, \n\tRBL_COMBO_E_1,RBL_COMBO_G_1,RBL_COMBO_H_1,\n\tRCVD_IN_CBL_SPAM,RCVD_IN_IMP_SPAM,URIBL_SBL May 10 18:06:03 mx2 sm-mta[14056]: j4AG5oxK014056: Milter add: header: X-Spam-Status: Yes, hits=46.196 required=10 scantime=4.5805 seconds tests=ASCII_GAPS_4,BAYES_60,\n\tDATE_IN_FUTURE_06_12,HTML_MESSAGE,MIME_HTML_ONLY,\n\tRAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RBL_COMBO_B_2, \n\tRBL_COMBO_D_1,RBL_COMBO_E_1,RBL_COMBO_G_1,\n\tRBL_COMBO_H_1,RCVD_IN_BL_SPAMCOP_NET,\n\tRCVD_IN_CBL_SPAM,RCVD_IN_SBL,RCVD_IN_XBL,URIBL_ SBL May 10 18:33:08 mx2 sm-mta[46878]: j4AGWxbu046878: Milter add: header: X-Spam-Status: Yes, hits=15.514 required=5 scantime=6.9332 seconds tests=ASCII_GAPS_3,BAYES_60,HTML_MESSAGE,\n\tMAILTO_SUBJ_REMOVE,MAILTO_TO_REMOVE,MIME_QP_LONG_LINE,\n\tRCVD_ILLEGAL_IP,SEXUALLY_EXP,SUBJECT_SEXUAL May 10 18:58:44 mx2 sm-mta[76877]: j4AGvt1F076877: Milter add: header: X-Spam-Status: Yes, hits=49.002 required=7 scantime=7.4389 seconds tests=ASCII_GAPS_5,BAYES_50,\n\tDATE_IN_FUTURE_12_24,HTML_MESSAGE,MIME_HTML_ONLY,\n\tRAZOR2_CF_RANGE_51_100,RAZOR2_CHECK,RBL_COMBO_B_2,\ n\tRBL_COMBO_D_2b,RBL_COMBO_E_2b,RBL_COMBO_H_1,\n\tRCVD_IN_SORBS_DUL,SARE_SPOOF_COM2OTH,UNIQUE_WORDS,\n\tURIBL_SBL,URIBL_SC_SWINOG ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Installing on FreeBSD
Hi, === Generating temporary packing list [: based: unexpected operator -e: not found *** Error code 127 Stop in /usr/ports/mail/mimedefang. *** Error code 1 Stop in /usr/ports/mail/mimedefang. I can see the perl modules (including spamassassin) when I do a pkg_info -a now. But mimedefang isn't in there (probably because the make install failed). I do have /var/spool/MIMEDefang and /var/spool/MD-Quarantine and /usr/local/etc/mimedefang/mimedefang-filter so mimedefang is at least partially installed. I doubt it's installed enough to be actually working though since the make install failed and it does not show up in pkg_info -a. Any ideas on how to fix the make install for mimedefang? This is getting complicated ... Have you ever updated a freebsd system ? Looks like your port collection is not up to date. Install the package cvsup (the package, not the port). To do that type 'sysinstall', go to 'Configure', 'Packages'. If you have installed from cd, enter that cd. Under 'devel' you'll find a tool called 'cvsup-without-gui'. Install this one. After that, upgrade your system to FreeBSD 4.11, please read the handbook how to do that. The upgrade the ports collection with cvsup too. After that go to '/usr/ports/sysutils/portupgrade', do a 'make install'. Now you can just do a portupgrade mimedefang, and everything needed should be updated now. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Installing on FreeBSD
Hi Lisa,
Makefile, line 53: Malformed conditional (${PERL_LEVEL} 500601)
the port system expects that perl is installed also
from the ports system
either do:
cd /usr/ports/lang/perl5.8
make install
make clean
or edit /etc/make.conf and add (adjust the perl version)
PERL_VER=5.8.1
PERL_VERSION=5.8.1
PERL_ARCH=mach
and after that do:
cd /usr/ports/mail/mimedefang/
make install
make clean
Hope that helps.
Martin
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] [PATCH] SpamAssassin should use MIMEdefang $CWD instead of /tmp for DCC/PYZOR checks
Hi,
I noticed that spamassassin does create files in /tmp if used within
Mimedefang. And I really think those files don't belong there, since
I've made a ramdisk for the Mimedefang spool dir.
First I thought this could be achieved with setting $TMP or $TEMPDIR in ENV,
but this didn't help, unfortunately.
The change to spamassassin is small, but works wonderful:
Instead of:
/tmp/spamassassin.38827.iALQHS.tmp
we have now:
/var/spool/MIMEDefang/mdefang-j3TAmZJe021409/spamassassin.38827.iALQHS.tmp
--- lib/Mail/SpamAssassin/Util.pm Tue Mar 29 10:29:02 2005
+++ lib/Mail/SpamAssassin/Util.pm Fri Apr 29 12:44:04 2005
@@ -42,6 +42,7 @@
use Sys::Hostname (); # don't import hostname() into this namespace!
use Fcntl;
use POSIX (); # don't import anything unless we ask explicitly!
+use Cwd;
###
@@ -738,9 +750,7 @@
# thanks to http://www2.picante.com:81/~gtaylor/autobuse/ for this
# code.
sub secure_tmpfile {
- my $tmpdir = Mail::SpamAssassin::Util::untaint_file_path(
- File::Spec-tmpdir()
- );
+ my $tmpdir = Mail::SpamAssassin::Util::untaint_file_path(getcwd);
if (!$tmpdir) {
die Cannot find a temporary directory! set TMP or TMPDIR in env;
}
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] [PATCH] SpamAssassin should use MIMEdefang $CWD instead of /tmp for DCC/PYZOR checks
Hi, According to the File::Spec man page, setting TMPDIR should work. :-) Hmm, I set TMP=$SPOOLDIR; export TMP; TMPDIR=$SPOOLDIR; export TMPDIR; in the mimedefang startup script. That didn't work. Do you have an idea why ? Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Global variables in filter_sender() and filter_recipient() are not kept ?
Hi, PS: Oh, you want the ANSWER? :-) Read the mimedefang-filter man page, especially the MAINTAINING STATE section. RFTM MAINTAINING STATE section did help :-) Thanks ! Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Open filedescriptors warning on a busy server
Hi David,
Each slave is single-threaded, so that shouldn't happen. I honestly
don't know what it could be.
Is there a way under FreeBSD to see a process's open file descriptors?
(In Linux, it's under /proc/PID/fd) Maybe that would give us a clue?
There is lsof - and I was only able to see some fd's open on /dev/null
and the usual stuff.
1 of 100 mimedefang reloads kills the multiplexer here, could this be
related to the open FD's ?
Also a nice error we see is that the bayes yournal of spamassassin does
sometimes belong to 'root' ! instead of the mimedefang user after a reload
which makes bayes updates failing then.
We use now a bit modified startup skript to check and prevent this. And
we also check the validity of mimedefang-filter and spamassassin conf
at the beginning to prevent shooting yourself in your feet :-)
Feel free to add parts of this to the default mimedefang skript.
Martin
#!/bin/sh
#
# Generic start/stop script for MIMEDefang. Should work on most
# flavors of UNIX.
cd /tmp
PREFIX=%%PREFIX%%
LANG=C
export LANG=C
LC_ALL=C
export LC_ALL
RETVAL=0
prog='mimedefang'
SPOOLDIR='/var/spool/MIMEDefang'
PID=$SPOOLDIR/$prog.pid
MXPID=$SPOOLDIR/$prog-multiplexor.pid
_MXPIDNO=`cat $MXPID`;
# Is the program executable? We search in /usr/bin and /usr/local/bin.
if [ -x /$PREFIX/bin/$prog ] ; then
PROGDIR=/$PREFIX/bin
elif [ -x /usr/bin/$prog ] ; then
PROGDIR=/usr/bin
elif [ -x /usr/local/bin/$prog ] ; then
PROGDIR=/usr/local/bin
else
exit 0
fi
MX_EMBED_PERL=yes
MX_RECIPIENT_CHECK=yes
SOCKET=%%SOMESOCKET%%
MX_MIN_SLAVE_DELAY=0
MX_USER=mimedefang
SYSLOG_FACILITY=mail
MX_LOG=yes
MX_REQUESTS=100
MX_MINIMUM=20
MX_MAXIMUM=30
MX_IDLE=300
MX_BUSY=600
MX_BACKLOG=200
MX_QUEUE_SIZE=128
MX_QUEUE_TIMEOUT=60
MX_MAX_RSS=10
MX_MAX_AS=12
# Source configuration
if [ -f /$PREFIX/etc/mimedefang/$prog.conf ] ; then
. /$PREFIX/etc/mimedefang/$prog.conf
fi
# Make sure required vars are set
SOCKET=${SOCKET:=$SPOOLDIR/$prog.sock}
MX_SOCKET=${MX_SOCKET:=$SPOOLDIR/$prog-multiplexor.sock}
start_it() {
if test -r $PID ; then
if kill -0 `cat $PID` /dev/null 21 ; then
echo mimedefang (`cat $PID`) seems to be running.
return 1
fi
fi
if test -r $MXPID ; then
if kill -0 `cat $MXPID` /dev/null 21 ; then
echo mimedefang-multiplexor (`cat $MXPID`) seems to be running.
return 1
fi
fi
printf %-60s Starting $prog-multiplexor:
rm -f $MX_SOCKET /dev/null 21
if [ $MX_EMBED_PERL = yes ] ; then
EMBEDFLAG=-E
else
EMBEDFLAG=
fi
$PROGDIR/$prog-multiplexor -p $MXPID \
$EMBEDFLAG \
`[ -n $FILTER ] echo -f $FILTER` \
`[ -n $SYSLOG_FACILITY ] echo -S $SYSLOG_FACILITY` \
`[ -n $SUBFILTER ] echo -F $SUBFILTER` \
`[ -n $MX_MINIMUM ] echo -m $MX_MINIMUM` \
`[ -n $MX_MAXIMUM ] echo -x $MX_MAXIMUM` \
`[ -n $MX_LOG_SLAVE_STATUS_INTERVAL ] echo -L
$MX_LOG_SLAVE_STATUS_INTERVAL` \
`[ -n $MX_USER ] echo -U $MX_USER` \
`[ -n $MX_IDLE ] echo -i $MX_IDLE` \
`[ -n $MX_BACKLOG ] echo -I $MX_BACKLOG` \
`[ -n $MX_BUSY ] echo -b $MX_BUSY` \
`[ -n $MX_REQUESTS ] echo -r $MX_REQUESTS` \
`[ -n $MX_SLAVE_DELAY ] echo -w $MX_SLAVE_DELAY` \
`[ -n $MX_MIN_SLAVE_DELAY ] echo -W $MX_MIN_SLAVE_DELAY` \
`[ -n $MX_MAX_RSS ] echo -R $MX_MAX_RSS` \
`[ -n $MX_MAX_AS ] echo -M $MX_MAX_AS` \
`[ $MX_LOG = yes ] echo -l` \
`[ $MX_STATS = yes ] echo -t /var/log/mimedefang/stats` \
`[ $MX_STATS = yes -a $MX_FLUSH_STATS = yes ] echo -u` \
`[ $MX_STATS_SYSLOG = yes ] echo -T` \
`[ -n $MX_QUEUE_SIZE ] echo -q $MX_QUEUE_SIZE` \
`[ -n $MX_QUEUE_TIMEOUT ] echo -Q $MX_QUEUE_TIMEOUT` \
`[ -n $MX_NOTIFIER ] echo -O $MX_NOTIFIER` \
-s $MX_SOCKET
RETVAL=$?
if [ $RETVAL = 0 ] ; then
echo [ OK ]
else
echo [FAILED]
return 1
fi
# Start mimedefang
printf %-60s Starting $prog:
$PROGDIR/$prog -P $PID \
-m $MX_SOCKET \
`[ -n $MX_USER ] echo -U $MX_USER` \
`[ -n $SYSLOG_FACILITY ] echo -S $SYSLOG_FACILITY` \
`[ $MX_RELAY_CHECK = yes ] echo -r` \
`[ $MX_SENDER_CHECK = yes ] echo -s` \
`[ $MX_RECIPIENT_CHECK = yes ] echo -t` \
`[ $KEEP_FAILED_DIRECTORIES = yes ] echo -k` \
`[ $ALLOW_NEW_CONNECTIONS_TO_QUEUE = yes ] echo -q` \
`[ $MD_EXTRA != ] echo $MD_EXTRA` \
-p $SOCKET
RETVAL=$?
if [ $RETVAL = 0 ] ; then
echo [ OK ]
else
echo [FAILED]
kill `cat $MXPID`
return 1
fi
return 0
}
stop_it() {
# Stop daemon
printf %-60s Shutting down $prog:
if test -f $PID ; then
kill `cat $PID`
RETVAL=$?
else
RETVAL=1
fi
if [ $RETVAL = 0 ] ; then
echo [ OK ]
else
echo [FAILED]
fi
rm -f
[Mimedefang] PING - PONG support for mimedefang socket
Hi David,
What about this little nice patch :-) ?
@@ -4987,6 +4999,10 @@
# Change to spool dir -- ignore error
chdir($Features{'Path:SPOOLDIR'});
+ if ($_ =~ /^ping$/i) {
+ print_and_flush(PONG\n);
+ next;
+ }
if ($_ =~ /^scan (.*)$/) {
$workdir = $1;
do_scan($workdir);
Martin
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Open filedescriptors waring on a busy server
Hi David, I'm a but clueless, but the more load I see on our server, the following well known warning is ussued: WARNING: Something in your Perl filter appears to have opened a file descriptor outside of any function. With embedded Perl, you should move any code that opens a file descriptor into filter_initialize. On some systems, the C library may open a descriptor, but you should verify your filter just in case If I start mimedefang before sendmail everthing goes ok, and I don't see the warning. But if sendmail is already running at this time, mimedefang immediatly logs this error after startup. Of course we use filter_cleanup and filter_initialize for database connections in mimedefang-filter so I'm a bit clueless. Could it be that some of the main code is executed too slowly because the filter is spending to much time acception already arriving connections ? Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] SURBL lookups no longer happening after upgrade to 2.48
Hi, LOCAL_RULES_DIR after all the regular config items in the hash. After modifying mimedefang.pl (see attached diff/patch for mimedefang.pl.in) to do the same, I find that SURBL lookups work. So it wasn't the presence of that argument/key but rather it place in the hash that caused SURBL to not work. That makes no sense whatsoever; a hash is unordered! So it shouldn't matter where you put the key. (I'm not saying that it didn't fix the problem, but I am saying that it makes no sense!) Here we have the same problem. SURBL lookups stopped working after upgrading to 2.49. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] SURBL lookups no longer happening after upgrade to 2.48
Hi, It's the same file as is used when I call SA directly, and the SURBL lookups work fine there. Other RBL lookups work fine. Same here. I had to cut and paste all the SURBL lookups into the local-sa.cf file to get them working again. SPAMHAUS and other RBL still work in both situations. Only SURBL stopped working. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Header only tests for big mails in spamassassin
Hi all, We get more and more spams which are big enough to be skipped in spamassassin. I'm thinking now about adding another spamassassin check but only with the header as argument, so bad relays etc. would still be catched and the mails will be marked as SPAM. What do you think about this idea ? Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] After some IDLE time, mimedefang-multiplexer crashes silently
Hi, Anybody has seen something similar ? The Milter does run just wonderful if there is always mail arriving. After ~3 minutes idle time, the multiplexer itself crashes. OS is FreeBSD 5.3R. #0 0x282217ef in skip (p=0x6 Address 0x6 out of bounds) at /usr/src/lib/libc/gen/getttyent.c:183 Looks like it was an automated reload from crontab which got broken again by installing another (not yet identified) perl module. Many perl modules seem to cause problems with embeded perl mode which is really a pain. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] SURBL lookups no longer happening after upgrade to 2.48
Hi,
Not directly related to discussion.
I guess that header was added by MIMEDefang? How do you fetch original
SpamAssassin headers into MIMEDefang? I'd rather have SpamAssassin
style headers appended (X-Spam-Status, X-Spam-Report, and so on) than
X-Spam-Score from example mimedefang-filter.
I generate them. I think it would be nice to have something like that in
the example filter but it's not me deciding that. Anyway, here are parts
of our filter ...
#
# keep track of start time
#
my ($done, $start, $TIMEVAL_T);
if (! $skip_checks ) {
require 'sys/syscall.ph';
$TIMEVAL_T = LL;
$done = $start = pack($TIMEVAL_T, ());
syscall(SYS_gettimeofday, $start, 0) != -1
or die gettimeofday: $!;
}
[... Call Spamassassin ]
#
# Fix broken formatting done by spamassassin rules.
#
my $fixed_report = ;
if ($hits = $report_req) {
$fixed_report = $report;
$fixed_report =~ s/\n+\z//g;# fixes for multiline header
$fixed_report =~ s/\n[\t ]{0,}\n/\n/g; # removes empty lines
$fixed_report =~ s/\n/\n\t/g; # to stop sendmail complaining
}
#
# Use the excellent wrapping function of Text::Wrap.
#
my $firstpart;
my $secondpart;
$Text::Wrap::columns = 60;
$Text::Wrap::huge = 'wrap';
$Text::Wrap::break = '(?=[\s,])';
if ($names =~ /([0-9A-Z_,]{0,40},)(.*)/ ) {
$firstpart = $1 . \n\t;
$secondpart = Text::Wrap::wrap('',\t,$2);
$names = $firstpart . $secondpart;
} else {
$names = Text::Wrap::wrap('',\t,$names);
}
}
#
# Get the final scan time
#
my ($seconds, $scantime);
syscall( SYS_gettimeofday, $done, 0) != -1
or die gettimeofday: $!;
my @start = unpack($TIMEVAL_T, $start);
my @done = unpack($TIMEVAL_T, $done);
# fix microseconds
for ($done[1], $start[1]) {
$_ /= 1_000_000
}
$scantime = sprintf %.4f, ($done[0] + $done[1] ) - ($start[0] + $start[1]
);
$seconds = \ . $scantime . seconds\;
}
action_add_header(X-Spam-Report, $fixed_report);
action_add_header(X-Spam-Status, No, hits=$hits scantime=$seconds tests=$names);
Martin
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] SIG11's with Mimedefang 2.48
Hi David, Have you seen this error too on older versions ? I just got it once. Of course I didn't deleted any mails. Nov 2 16:22:51 mx3 mimedefang[56969]: iA2FMXpa037602: Could not open MIMEDefang 2.48 on 192.168.0.1/COMMANDS: No such file or directory Nov 2 16:22:51 mx3 mimedefang[56969]: lstat(MIMEDefang 2.48 on 192.168.0.1) failed: No such file or directory Nov 2 16:22:51 mx3 mimedefang[56969]: iA2FMXpa037602: failed to clean up MIMEDefang 2.48 on 192.168.0.1: No such file or directory [...] Nov 2 16:22:51 mx3 kernel: pid 56969 (mimedefang), uid 1001: exited on signal 11 Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] SIG11's with Mimedefang 2.48
Have you seen this error too on older versions ? I just got it once. Of course I didn't deleted any mails. Nov 2 16:22:51 mx3 mimedefang[56969]: iA2FMXpa037602: Could not open MIMEDefang 2.48 on 192.168.0.1/COMMANDS: No such file or directory Nov 2 16:22:51 mx3 mimedefang[56969]: lstat(MIMEDefang 2.48 on 192.168.0.1) failed: No such file or directory Nov 2 16:22:51 mx3 mimedefang[56969]: iA2FMXpa037602: failed to clean up MIMEDefang 2.48 on 192.168.0.1: No such file or directory Just got a second one followed by another coredump. Even if a dir is not there anymore, this is not a good sign when the multiplexer does crash completly. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] SIG11's with Mimedefang 2.48
Hi, Just got a second one followed by another coredump. Even if a dir is not there anymore, this is not a good sign when the multiplexer does crash completly. Just to say it. The filter is properly set up, with filter_initialize() and filter_cleanup() and of course no open sockets which are closed. The failure does happen 1-2 per hour. Next time I'll have a coredump. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] SIG11's with Mimedefang 2.48
Hi, It looks like the scan directory is being overwritten by MIMEDefang 2.48 on 192.168.0.1, which makes no sense whatsoever.. Could it be a FreeBSD-specific issue? A bug in pthreads? Argl. After I analyzed a coredump I found it to be a double free(), caused by a local patch. Sorry for the noise. I just found that I still used the SMALLBUF local patch and haven't adapted it to the new version. You know, without this patch I still get coredumps on FreeBSD on some specific mails because the stack limit gets hit. It would be a good thing to add the (now fixed patch) to the mainstream so I don't have to mess around with it each time you update mimedefang. And of course, Mimedefang on FreeBSD is only half usable because some mails can't be proceeded at all (if long headers are added for example). The code that generates MIMEDefang 2.48 on 192.168.0.1 is around line 1431 of mimedefang.c, and I see no way that it can overwrite data-dir (which is what appears to be happening.) Anyone else seeing this? Sorry for the noise and thank you for pointing to the place in mimedefang.c Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] SIG11's with Mimedefang 2.48
Hi, I'd be curious to hear what other issues you've encountered on FreeBSD. We've been ticking along here just fine literally for years, happily processing more than a million messages a day. Other than occasional delays in getting this or that patch added to a required port, we've been pretty happy. Either there are problems occurring that are slipping below my radar that will be revealed, or else I and others can help you with some of your outstanding FreeBSD issues. Either way, the community benefits. I add an X-Spam-Report: line to our mails, line length is 60 chars, but of course the report itself as whole is very long. We do this because we don't wan't to modify the mailbody, which does cost too much IO becuause we have to resend the mails. Another patch I made was tcpwrappers support which works quite fine. I'll forward you the patches offlist. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] SURBL lookups no longer happening after upgrade to 2.48
Hi, I'm unable to duplicate this. Anyone else? Please include OS and SpamAssassin version. Works still here with SpamAssassin 3.01 and Mimedefang 2.48 ... Nov 2 16:02:12 mx1 sm-mta[13819]: iA2F1oSl013819: Milter add: header: X-Spam-Status: Yes, hits=49.893 required=5 scantime=13.5556 seconds tests=BAYES_99,DOMAIN_RATIO,HTML_90_100, HTML_FONT_BIG,HTML_IMAGE_ONLY_08,HTML_MESSAGE,\n\tHTML_TITLE_EMPTY,MIME_HTML_ONLY, MSGID_SPAM_CAPS,RBL_COMBO_A_2,RBL_COMBO_B_2,RBL_COMBO_C_2,RBL_COMBO_F_3, RCVD_HELO_IP_MISMATCH,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_DSBL,RCVD_IN_SORBS_WEB, RCVD_IN_SWINOG_SPAM,RCVD_IN_XBL,RCVD_NUMERIC_HELO,URIBL_OB_SURBL,URIBL_SBL,URIBL_WS_SURBL Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Memory problems
Hi,
Depending how many rules you use you'll need this
spamassassin patch for some mails which have many
newlines ...
--- lib/Mail/SpamAssassin/Message.pmThu Sep 9 20:29:19 2004
+++ lib/Mail/SpamAssassin/Message.pmThu Sep 16 12:50:54 2004
@@ -197,10 +197,12 @@
# will get modified below
$self-{'pristine_body'} = join('', @message);
- # CRLF - LF
- for ( @message ) {
-s/\r\n/\n/;
- }
+ # Remove repeated empty lines and convert CRLF to LF
+ # This saves us a lot of mem.
+ my $tmpmsg = join('', $self-{'pristine_body'});
+ $tmpmsg =~ s/\r\n/\n/gs;
+ $tmpmsg =~ s/\n{100,}/\n/gs;
+ @message = split ( /^/m, $tmpmsg );
# If the message does need to get parsed, save off a copy of the body
# in a format we can easily parse later so we don't have to rip from
--
Martin
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Running multiple Mimedefangs on a single box?
Hi, For now I'm thinking about multiple Mimedefang instances (i.e. multiplexors, probably one per MTA or per group of MTAs) running same binaries and filters, but using smaller FD subsets and listening on different sockets. And all this on the same box and OS. I could wrap them in FreeBSD jails as well, but it seems more complicated. I'd prefer having just one set of binaries, because it makes the things much easier to manage. That's one of the reasons I don't have it installed on each MTA. So, would Mimedefang support such a configuration, and does anybody use something like that? Yes we have such a configuration. We ave two different Mimedefang's running on the same box. And of course you don't need jails. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MIMEDefang, SpamAssassin and URIDNSBLs
Hi, Why the heck would I get one score when called from MIMEDefang, and another when done 'by hand'? Sounds like you have two or more spamassassin config files. The one from spamassassin is residing in /usr/local/etc/mail/local.cf, the other is the mimedefang one in /docsis/etc/mimedefang/sa-mimedefang.cf. One of them has network tests disabled. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] MD in Embedded mode in linux with p5-Archive-Zip or p5-Net-DNS 0.44
Hi Linuxers, If you use those perl modules together with mimedefang, can you please test if MD reload works properly (no errors and aborts in embedded mode). I've experienced here aborts of all mimedefang-multiplexors because those modules seem to contain circular references somewhere which leads to freeing some buffers twice (fatal double free). If this happens with linux too, we should add a version check in mimedefang if the user wants to use embedded mode. Thank you for testing ! Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MIMEDefang + spamd
Replying to myself. If I look at spamds version of preload_modules_with_tmp_homedir() is is really bigger than just spam_assassin_init(). This may explain some delay. There seem to be only path and tmp handling in preload_modules_with_tmp_homedir. And spamd calls spam_assassin_init(0,1) and we just do spam_assassin_init(1). IMHO mimedefang needs to be faster calling the SA API directly since there are less context switches to do. fork() needs time, so if you call spamc you loose time. Imagine some high load situations with a lot of IO. There each fork gets really expensive and slow. I cannot beleave that calling spamc and asking spamd is faster than useing the precompiled SA API directly. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Embedded perl problems with 2.45, again ... (reload broken)
Hi all,
I just tried 2.45 out and found out that MD 2.45 is not able to reload
embedded perl slaves anymore. I've tracked it down to:
+(eval 'use Net::DNS; $Features{Net::DNS} = 1;')
+or $Features{Net::DNS} = 0;
This module seems to specify a var which is used somewhere else
(namespace conflict or something like this.)
Any ideas how to fix this ?
Martin
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Embedded perl problems with 2.45, again ... (reload broken)
+(eval 'use Net::DNS; $Features{Net::DNS} = 1;')
+or $Features{Net::DNS} = 0;
After looking at Net::DNS I suspect that a circular reference is the problem
here.
I dunno how to solve it.
Martin
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Solved: Embedded perl problems with 2.45, again ... (reload broken)
+(eval 'use Net::DNS; $Features{Net::DNS} = 1;')
+or $Features{Net::DNS} = 0;
An upgrade to Net::DNS 0.47 solved to problem. Net::DNS 0.44 does indeed break
embedded perl.
David, please use Net::DNS only if version is equal or higher than 0.47.
Martin
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] again - Overlong line in RESULTS file
David, I'm debugging the problem with a segfault and a 16k buffer on FreeBSD right now. Raising the buffer to 10k worked fine, but 16 seems to have problems. Notice for your FreeBSD users: If you wanna have coredumps, you have to enable sugid_coredump first. Else Mimedefang does not dump core ! sysctl kern.sugid_coredump=1 Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] again - Overlong line in RESULTS file
Hi, I'm debugging the problem with a segfault and a 16k buffer on FreeBSD right now. Raising the buffer to 10k worked fine, but 16 seems to have problems. (gdb) bt #0 0x28102da3 in getdiskbyname (name=0x5d Address 0x5d out of bounds) at /usr/src/lib/libc/gen/disklabel.c:114 #1 0x0805057c in ?? () #2 0x0005 in ?? () #3 0xbfbfea40 in ?? () #4 0x in ?? () #5 0xbfbfe9c0 in ?? () #6 0xbfbfe9b8 in ?? () Uhm. Something very bad happens here: Stack corruption and it looks like a stack overflow. The stack limit in libc_r or libpthreads in FreeBSD is 65536 and it is likely that we hit it here. So I experimentally raised the stack size for threaded apps in /usr/src/lib/libpthread/thread/thr_private.h to 128k: #define THR_STACK_DEFAULT 131072 recompiled the pthread lib and now it works. That means it is not save here to place SMALLBUF on the stack. I'll try to convert it to malloc(). Agreed David ? Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] again - Overlong line in RESULTS file
Hi, Disagreed :-). There should be no reason to have such a long line in the RESULTS file; I would rather the Perl filter be a bit more sensible about what it does. If you read the code, it's not as simple as just allocating a buffer in eom(); there may be other places that assume each line in RESULTS will fit in a SMALLBUF-sized buffer. As said, it even happens if I disable adding reports. These must be some special spam-mails which have lines that long ... Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] again - Overlong line in RESULTS file
hi all, Since yesterday we get this on a few spammails. I supect that too many things hit our spamfilter. The report field of the spamcheck seems to be responsable here. Sep 20 17:52:38 mx2 mimedefang[34009]: i8KFqWbE007558: Overlong line in RESULTS file - 8695 chars (max 8191) Sep 20 17:57:58 mx2 mimedefang[34009]: i8KFvqlS013123: Overlong line in RESULTS file - 8695 chars (max 8191) Sep 20 18:03:43 mx2 mimedefang[50276]: i8KG3bNC018738: Overlong line in RESULTS file - 8695 chars (max 8191) Sep 20 18:15:37 mx2 mimedefang[50276]: i8KGFVv7030414: Overlong line in RESULTS file - 8695 chars (max 8191) Sep 20 18:22:38 mx2 mimedefang[50276]: i8KGMXVw037219: Overlong line in RESULTS file - 8695 chars (max 8191) Is it dangerous to raise the length of the line limit ? Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] BUG: @Recipients contains sometimes lowercase versions of emailaddrs without
Hi David,
I have it happen that @Recipients contains already a lowercase version of
the recipient email-address. This leads to problems on the milter side.
smfi_delrcpt() should always be called on the original case-insensitive
version, else the address does not match and is still there.
It looks for me like it works if the recipient adress contains braces. If
they are missing, we fail.
if ($spambox ne ) {
my $recipient;
if ($debug) {
md_syslog('err', $QueueID: DEBUG: Spambox $spambox has been found,
redirect mail);
}
foreach $recipient (@Recipients) {
delete_recipient($recipient);
action_add_header(X-Original-Recipient, $recipient);
}
}
add_recipient($spambox);
}
The failed mail today was (note that the recipient has no braces).
Sep 17 16:33:26 mx1 sm-mta[50931]: i8HEXLs6050931: Milter delete: rcpt [EMAIL
PROTECTED]
Sep 17 16:33:26 mx1 sm-mta[50931]: i8HEXLs6050931: Milter add: header:
X-Original-Recipient: [EMAIL PROTECTED]
Sep 17 16:33:26 mx1 sm-mta[50931]: i8HEXLs6050931: Milter add: rcpt: [EMAIL PROTECTED]
Sep 17 16:33:26 mx1 sm-mta[50931]: i8HEXLs6050931: Milter add: header: X-Scanned-By:
MIMEDefang 2.44
Sep 17 16:33:26 mx1 sm-mta[50978]: i8HEXLs6050931: [EMAIL PROTECTED],[EMAIL
PROTECTED], delay=00:00:05,
xdelay=00:00:00, mailer=relay, pri=31867, relay=mail.test.ch. [XXX.XXX.XXX.XXX],
dsn=2.0.0,
stat=Sent ( [EMAIL PROTECTED] Queued mail for delivery)
My test worked, ([EMAIL PROTECTED] is still uppercase and I used braces.
Sep 17 20:41:28 mail sm-mta[70162]: i8HIfMW1070162: Milter delete: rcpt [EMAIL
PROTECTED]
Sep 17 20:41:28 mail sm-mta[70162]: i8HIfMW1070162: Milter add: header:
X-Original-Recipient: [EMAIL PROTECTED]
Sep 17 20:41:28 mail sm-mta[70162]: i8HIfMW1070162: Milter add: rcpt: [EMAIL PROTECTED]
Sep 17 20:41:28 mail sm-mta[70162]: i8HIfMW1070162: Milter add: header: X-Scanned-By:
MIMEDefang 2.44
Sep 17 20:41:31 mail sm-mta[70175]: i8HIfMW1070162: [EMAIL PROTECTED], delay=00:00:08,
xdelay=00:00:03, mailer=esmtp, pri=30265, relay=xxx.test.ch.
[XXX.XXX.XXX.XXX], dsn=2.0.0, stat=Sent (i8HIfSUM048927 Message accepted for delivery)
Martin
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Re: BUG: @Recipients contains sometimes lowercase versions of emailaddrs without
Thanks a lot but I already use Sendmail 8.12.11, so this must be new bug. Could you make me a favor and test if it works in sendmail Sendmail 8.13.1. I don't have one running here currently. Martin Replying to myself: See http://www.sendmail.org/8.12.11.html Perhaps this fixes it: When a milter invokes smfi_delrcpt() compare the supplied recipient address also against the printable addresses of the current list to deal with rewritten addresses. Based on patch from Sean Hanson of The Asylum. -- David. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] [PATCH] Spamassassin fix for mails with masses of newlines
Hi all,
The patch below reduces the mem usage of SA with a 200K mail
to almost 1/3 of it's previous usage. Instead of 210MB it only uses
here 80MB. You will only see an effect if a mail has many newlines.
Martin
--- lib/Mail/SpamAssassin/Message.pmThu Sep 9 20:29:19 2004
+++ lib/Mail/SpamAssassin/Message.pmThu Sep 16 12:50:54 2004
@@ -197,10 +197,14 @@
# will get modified below
$self-{'pristine_body'} = join('', @message);
- # CRLF - LF
- for ( @message ) {
-s/\r\n/\n/;
- }
+ # Remove repeated empty lines and convert CRLF to LF
+ # This saves us a lot of mem.
+ my $tmpmsg = join('', $self-{'pristine_body'});
+ $tmpmsg =~ s/\r\n/\n/gs;
+ $tmpmsg =~ s/\n{100,}/\n/gs;
+ @message = split ( /^/m, $tmpmsg );
+
+ $self-{'pristine_body'} = join('', @message);
# If the message does need to get parsed, save off a copy of the body
# in a format we can easily parse later so we don't have to rip from
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] [PATCH] Spamassassin fix for mails with masses of newlines
Hi all,
Previous patch had 2 uncessary lines. This one is more correct:
--- lib/Mail/SpamAssassin/Message.pmThu Sep 9 20:29:19 2004
+++ lib/Mail/SpamAssassin/Message.pmThu Sep 16 12:50:54 2004
@@ -197,10 +197,12 @@
# will get modified below
$self-{'pristine_body'} = join('', @message);
- # CRLF - LF
- for ( @message ) {
-s/\r\n/\n/;
- }
+ # Remove repeated empty lines and convert CRLF to LF
+ # This saves us a lot of mem.
+ my $tmpmsg = join('', $self-{'pristine_body'});
+ $tmpmsg =~ s/\r\n/\n/gs;
+ $tmpmsg =~ s/\n{100,}/\n/gs;
+ @message = split ( /^/m, $tmpmsg );
# If the message does need to get parsed, save off a copy of the body
# in a format we can easily parse later so we don't have to rip from
Martin
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Catching the porn spams
Hi, Maybe we need to think a little outside the box. Porn spam's objective is to get you to go to their website, subscribe etc. Now maybe we need to search the body for web links then match them against a blacklist. There are many ways to fight this porn spam. 1. Install Spamikaze http://spamikaze.nl.linux.org, make a RBL and use it. The delay for the bulding the rbl should be 1 minute. 2. You can record the URL's from mails from spamikaze too, make your own URI-BL. 3. Use Razor, Pyzor and DCC. 4. Use Spamcop, Sorbs and other Blacklists. Now to the interesting part. Combine these tests with your own tests. You'll see, you really get better matches then ! Martin # # Combine some blacklists and RBL's. Very effective # meta__RELAY_RBL_1 (RCVD_IN_NJABL_RELAY + RCVD_IN_NJABL_CGI + RCVD_IN_NJABL_PROXY + RCVD_IN_SORBS_HTTP + RCVD_IN_SORBS_MISC + RCVD_IN_SORBS_SMTP + RCVD_IN_SORBS_SOCKS + RCVD_IN_SORBS_WEB + RCVD_IN_SORBS_ZOMBIE + RCVD_IN_XBL + RCVD_IN_SBL + RCVD_IN_DSBL + RCVD_IN_BL_SPAMCOP_NET + RCVD_IN_NJABL_SPAM + RCVD_IN_SWINOG == 1) meta__RELAY_RBL_2 (RCVD_IN_NJABL_RELAY + RCVD_IN_NJABL_CGI + RCVD_IN_NJABL_PROXY + RCVD_IN_SORBS_HTTP + RCVD_IN_SORBS_MISC + RCVD_IN_SORBS_SMTP + RCVD_IN_SORBS_SOCKS + RCVD_IN_SORBS_WEB + RCVD_IN_SORBS_ZOMBIE + RCVD_IN_XBL + RCVD_IN_SBL + RCVD_IN_DSBL + RCVD_IN_BL_SPAMCOP_NET + RCVD_IN_NJABL_SPAM + RCVD_IN_SWINOG == 2) meta__RELAY_RBL_3 (RCVD_IN_NJABL_RELAY + RCVD_IN_NJABL_CGI + RCVD_IN_NJABL_PROXY + RCVD_IN_SORBS_HTTP + RCVD_IN_SORBS_MISC + RCVD_IN_SORBS_SMTP + RCVD_IN_SORBS_SOCKS + RCVD_IN_SORBS_WEB + RCVD_IN_SORBS_ZOMBIE + RCVD_IN_XBL + RCVD_IN_SBL + RCVD_IN_DSBL + RCVD_IN_BL_SPAMCOP_NET + RCVD_IN_NJABL_SPAM + RCVD_IN_SWINOG = 3) meta__SPAMHAUS_ALLRBL (URIBL_SBL + RCVD_IN_XBL + RCVD_IN_SBL = 1) meta__SPAMHAUS_RBL (RCVD_IN_XBL + RCVD_IN_SBL = 1) meta__SURBL_RBL (URIBL_AB_SURBL + URIBL_OB_SURBL + URIBL_WS_SURBL = 1) meta__URI_RBL_SINGLE(URIBL_SBL + URIBL_AB_SURBL + URIBL_OB_SURBL + URIBL_WS_SURBL + URIBL_SC_SWINOG == 1) meta__URI_RBL_MULTI (URIBL_SBL + URIBL_AB_SURBL + URIBL_OB_SURBL + URIBL_WS_SURBL + URIBL_SC_SWINOG = 2) meta__ONE_DIGEST_TRUE (DCC_CHECK + RAZOR2_CHECK + PYZOR_CHECK == 1) meta__DIGEST_TRUE (DCC_CHECK + RAZOR2_CHECK + PYZOR_CHECK = 1) meta__RBL_COMBO_MATCH ((RBL_COMBO_A_3 || RBL_COMBO_A_4 || RBL_COMBO_A_5 || RBL_COMBO_B_2 || RBL_COMBO_B_3 || RBL_COMBO_C_1 || RBL_COMBO_C_2 || RBL_COMBO_C_3 || RBL_COMBO_D_1 || RBL_COMBO_D_2a || RBL_COMBO_D_2b || RBL_COMBO_D_3 || __RELAY_RBL_2 || __RELAY_RBL_3) == 1) # # Combine at least two positive network tests. # metaRBL_COMBO_A_2 (__SURBL_RBL + DIGEST_MULTIPLE + URIBL_SC_SWINOG + SPF_FAIL + __RELAY_RBL_1 + __RELAY_RBL_2 + __RELAY_RBL_3 == 2) metaRBL_COMBO_A_3 (__SURBL_RBL + DIGEST_MULTIPLE + URIBL_SC_SWINOG + SPF_FAIL + __RELAY_RBL_1 + __RELAY_RBL_2 + __RELAY_RBL_3 == 3) metaRBL_COMBO_A_4 (__SURBL_RBL + DIGEST_MULTIPLE + URIBL_SC_SWINOG + SPF_FAIL + __RELAY_RBL_1 + __RELAY_RBL_2 + __RELAY_RBL_3 == 4) metaRBL_COMBO_A_5 (__SURBL_RBL + DIGEST_MULTIPLE + URIBL_SC_SWINOG + SPF_FAIL + __RELAY_RBL_1 + __RELAY_RBL_2 + __RELAY_RBL_3 = 5) describeRBL_COMBO_A_2 Blacklist Combo A (2) describeRBL_COMBO_A_3 Blacklist Combo A (3) describeRBL_COMBO_A_4 Blacklist Combo A (4) describeRBL_COMBO_A_5 Blacklist Combo A (5+) score RBL_COMBO_A_22.000 score RBL_COMBO_A_35.000 score
[Mimedefang] Inoffizial MIME-tools-5.411a-RP-Patched-04 available (includes data corruption fix)
Hi all, I just put up a new patched version of MIME-tools online: http://people.freebsd.org/~mbr/distfiles/MIME-tools-5.411a-RP-Patched-04.tar.gz Changes: - Backport from 6.002: Parser memory leak fixed. The closure-based task/benchmark mechanisms created for 5.5 were leaking references like crazy. That's been stopped. - Backport from 6.002: Correct 7_bit, 7-bit, and 7 bit to 7bit; likewise for 8bit. - MIME::Body produces incorrect quoted-printable encoding if the body is non-text data which can lead to data corruption. You'll need MIME::QuotedPrint 3.03 installed, because encode_qp() takes now three arguments. I just upgraded to FreeBSD port of MIME::Base64 to version 3.03, so portsupgrade will work there. This version is the only one which passes all tests from http://www.testvirus.org in combination with clamd and scan_entity(). In MIME-tools-5.411a-RP-Patched-03 I added BinHex-encoding support for application/mac-binhex40 and application/mac-binhex mimetypes. 6.002 has unfortunatly some bugs and does not work reliable so I'll continue to offer packages until David does fix this issue ;-) Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Inoffizial MIME-tools-5.411a-RP-Patched-04 available (includes data corruption fix)
task/benchmark mechanisms created for 5.5 were leaking references I just double checked this again. The benchmark mechanisms got already added in version 5.4. It seems that perl 5.6.1 is fine, older perl versions seem to leak a lot of ram there. Anyway, I've removed the whole benchmark code as it use almost useless. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] New spam kills mimedefang/spamassassin with newlines
Hi, I found out that if one adds 20'000 newlines to a mail, spamassassin uses a very high amount of ram (~200MB) or even more if you have more newlines. This is too much for mimedefang, so the same mail gets killed and killed over and again. Spamassassin seems to read all lines into it's memory and allocates some structures for each line (even the empty ones). I wonder if there is a way to circumvent this in mimedefang, or should it be done in spamassassin ? Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] New spam kills mimedefang/spamassassin with newlines
Hi, Have you reported this to the SA folks? Yes, PR 3712 Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] [PATCH] MIME-tools-6.200_02
Hi, I ported the MaxParts patch to MIME-tools-6.200. There were some minor parts to adjust. The other needed patches seem to be already there. I'm doing a new installation of MD and was trying to figure out which MIME-tools to install. I found the original author's page here: Here's the changelog for the current bleeding-edge release: http://search.cpan.org/~eryq/MIME-tools-6.200_01/lib/MIME/Tools/changes.pod You should not use this version because some of the de-mime extracts don't work. You can see this with the antivirus tests posted here earlier. If you like to have a modern version you should use: http://people.freebsd.org/~mbr/distfiles/MIME-tools-5.411a-RP-Patched-03.tar.gz I've merged all necessary patches into it, including adjusted binhex support. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Error from mimedefang-multiplexor error
Hi, Try to install http://people.freebsd.org/~mbr/distfiles/MIME-tools-5.411a-RP-Patched-03.tar.gz Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MimeDefang vs clamav
Hi, What difference does clamdscan see when invoked from clamav-milter and when invoked via mimedefang ? First, you should not use clamscan, you should use the direct clamd socket connection to speed up scanning. And I can tell you that mimedefang is much more stable on FreeBSD than the clamav-milter, because clamav-milter had severe bugs (no locking for getnamebyhost(), hosts_access()) etc ... In the developer version they might be fixed. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Sire MimeDefang 2.44: Undefined subroutine main::do_main_loop
Hi, Aug 2 12:50:50 gateway2 mimedefang-multiplexor[15598]: Reap: Idle slave 0 (pid 15621) exited normally with status 9 (SLAVE DIED UNEXPECTEDLY) Aug 2 12:50:50 gateway2 mimedefang-multiplexor[15598]: Slave 0 resource usage: req=0, scans=0, user=0.000, sys=0.000, nswap=0, majflt=2, minflt=61, maxrss=0, bi=0, bo=0 Aug 2 12:50:53 gateway2 mimedefang-multiplexor[15598]: Starting slave 0 (pid 15622) (1 running): Bringing slaves up to minSlaves (2) Aug 2 12:50:53 gateway2 mimedefang-multiplexor[15598]: Slave 0 stderr: Undefined subroutine main::do_main_loop called at /usr/local/bin/mimedefang.pl line 60. All you need to do is calling 'mimedefang.pl -test' to see where your bug is. Or call mimedefang.pl on a prepared Mimedefang message with INPUT/HEADERS file. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Do not use razor2.61 ! (was Mimedefang segfaults/crashes)
Hi all, I've tracked it down to razor ... Razor 2.40 is fine, Razor 2.61 crashes on some mail because it normalizes emails from html first and there is a bug in there. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Do not use razor2.61 ! (was Mimedefang segfaults/crashes)
And here is a workaround:
Razor 2.40 is fine, Razor 2.61 crashes on some mail
because it normalizes emails from html first and
there is a bug in there.
and this patch (not 100% correct) seems to fix it. It just takes one
byte away from the raw mailinput and terminates it properly with '\0'.
There must be an 'off-by-one' somewere, but I haven't been able to
track it down.
The scanoutput on 1800 mails was exactly the same on both tests.
Martin
--- Razor2-Preproc-deHTMLxs/_deHTMLxs.c Thu Jun 17 00:44:35 2004
+++ Razor2-Preproc-deHTMLxs/_deHTMLxs.c Sun Aug 1 09:48:54 2004
@@ -177,13 +177,17 @@
break;
case '':
- *t++ = html_tagxlat(s);
+ if (t != NULL)
+ *t++ = html_tagxlat(s);
break;
default:
valid:
- if (!tag)
- *t++ = c;
+ if (!tag) {
+ if (t != NULL ) {
+ *t++ = c;
+ }
+ }
break;
}
--- Razor2-Preproc-deHTMLxs/deHTMLxs.xs Sun Aug 1 13:55:03 2004
+++ Razor2-Preproc-deHTMLxs/deHTMLxs.xs Sun Aug 1 13:52:28 2004
@@ -191,6 +191,7 @@
text = SvRV(scalarref);
raw = SvPV(text,size);
+ *(raw + size - 1) = '\0';
if ( (cleaned = malloc(size+1))
(res = html_strip(raw, cleaned)) // html_strip will memset cleaned
to 0
) {
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] MIME-tools-5.411a with BinHex support
Hi, I just patched MIME-tools-5.411a to support BinHex-Encoding. You'll need the Convert::BinHex installed. Some FreeBSD fixes are also part of this distfile. Download URL: http://people.freebsd.org/~mbr/distfiles/MIME-tools-5.411a-RP-Patched-03.tar.gz This fixes the Virustest #5, #8 if scan_entity() is used. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MIME-tools-5.411a with BinHex support
Hi, Can you please expand (or show an example) of scan_entity() ? Sorry, shouldn't have made the (). The real function is entity_contains_virus() ... Clamd does now detect all BinHex encoded attachments. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Please keep 'deprecated' entity_contains_virus()
Hi, I strongly disagree to remove this function. We currently do the following: We block all emails containing worms, and substitute all mimeparts containing viruses. If entity_contains_virus() will be removed, we cannot do that anymore. Everybody who has the same problem should raise his/her hands ! Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Mimedefang segfaults/crashes on some mails
Hi, We have problems with mimedefang crashing on always the same mails. How can I fix that ? (I use perl 5.8.5 and Mimedefang 2.44 SpamAssassin 3PR3 on FreeBSD 5.2.1). Jul 31 10:15:30 mx2 kernel: pid 69123 (mimedefang-multiple), uid 1001: exited on signal 11 mx1# grep i6V7g7QU030905 /var/log/maillog Jul 31 10:15:30 mx1 sm-mta[30905]: i6V7g7QU030905: from=[EMAIL PROTECTED], size=64010, class=0, nrcpts=1, msgid=[EMAIL PROTECTED], proto=ESMTP, daemon=MTA, relay=some.server [XX.XXX.XXX.XX] Jul 31 10:15:32 mx1 sm-mta[30905]: i6V7g7QU030905: Milter: data, reject=451 4.7.1 Please try again later Jul 31 10:15:32 mx1 sm-mta[30905]: i6V7g7QU030905: to=[EMAIL PROTECTED], delay=00:00:04, pri=94010, stat=Please try again later Jul 31 10:15:30 mx2 mimedefang-multiplexor[752]: Slave 16 died prematurely -- check your filter rules Jul 31 10:15:30 mx2 mimedefang[762]: Error from multiplexor: ERR No response from slave Jul 31 10:15:30 mx2 mimedefang-multiplexor[752]: Reap: Idle slave 16 (pid 69123) exited due to signal 11 (SLAVE DIED UNEXPECTEDLY) Martin Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED] -- ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: finger -l [EMAIL PROTECTED] PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E -- ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] [PATCH] MIME-tools-6.200_02
Hi,
I ported the MaxParts patch to MIME-tools-6.200. There were some
minor parts to adjust. The other needed patches seem to be already there.
There are two parts where I'm not sure what to do. The FreeBSD
ports system has them, the MIMEdefang patched version on the
mainpage not:
sub process_header {
@@ -612,6 +614,10 @@
foreach (@headlines) { s/[\r\n]+\Z/\n/ } ### fold
### How did we do?
+if ($hdr_rdr-eos_type eq 'DELIM') {
+ $self-whine(bogus part, without CRLF before body);
+ return;
+}
sub process_part {
@@ -983,7 +989,17 @@
### Parse and add the header:
my $head = $self-process_header($in, $rdr);
-$ent-head($head);
+if (not defined $head) {
+ $self-debug(bogus empty part);
+ $head = $self-interface('HEAD_CLASS')-new;
+ $head-mime_type('text/plain; charset=US-ASCII');
+ $ent-head($head);
+ $ent-bodyhandle($self-new_body_for($head));
+ $ent-bodyhandle-open(w)-close;
+ $self-results-level(-1);
+ return $ent;
+}
+$ent-head($head);
Do you know if they are needed David ?
Martin
Martin Blapp, [EMAIL PROTECTED] [EMAIL PROTECTED]
--
ImproWare AG, UNIXSP ISP, Zurlindenstrasse 29, 4133 Pratteln, CH
Phone: +41 61 826 93 00 Fax: +41 61 826 93 01
PGP: finger -l [EMAIL PROTECTED]
PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E
--
--- lib/MIME/Parser.pm.orig Sat Jul 31 12:03:36 2004
+++ lib/MIME/Parser.pm Sat Jul 31 13:07:38 2004
@@ -294,6 +294,7 @@
$self-{MP_TmpToCore} = 0;
$self-{MP_IgnoreErrors}= 1;
$self-{MP_UseInnerFiles} = 0;
+$self-{MP_MaxParts} = -1;
@@ -358,6 +359,7 @@
### Re-init the filer:
$self-{MP_Filer}-purgeable([]); ### too late now, kids!
$self-{MP_Filer}-init_parse();
+$self-{MP_NumParts} = 0;
### Clear the TO-DO list:
$self-{MP_ToDo} = [];
@@ -749,6 +751,8 @@
#
# IInstance method.
# Process and return the next header.
+# Return undef if, instead of a header, the encapsulation boundary is found.
+# Fatal exception on failure.
# The PARAMHASH can contain:
#
#In = required: the input filehandle
@@ -917,6 +921,7 @@
Reader = $part_rdr,
Retype = $retype,
PartNum = $partno);
+ return undef unless defined($part);
$ent-add_part($part);
### ...and look at how we finished up:
@@ -1132,6 +1137,7 @@
### Parse the message:
my $msg = $self-process_part(In=$in, Reader=$rdr);
+return undef unless defined($msg);
### How to handle nested messages?
if ($self-extract_nested_messages eq $EXTRACT_REPLACE) {
@@ -1172,6 +1178,14 @@
my $retype = $p{Retype};
my $partnum = $p{PartNum} || 1;
+if ($self-{MP_MaxParts} 0) {
+ $self-{MP_NumParts}++;
+ if ($self-{MP_NumParts} $self-{MP_MaxParts}) {
+ # Return UNDEF if msg too complex
+ return undef;
+ }
+}
+
### Start logging:
#$self-logger-push_prefix(part $partnum);
@@ -1183,6 +1197,7 @@
my $head = $self-process_header(In = $in,
Reader = $rdr,
NoBody = \$no_body);
+
$ent-head($head);
### Tweak the content-type based on context from our parent...
@@ -1204,13 +1219,13 @@
### Classify... how should we parse it?
if($classify eq $CLASS_MULTIPART) {
- $self-process_multipart( In=$in, Reader=$rdr, Entity=$ent);
+ return undef unless defined($self-process_multipart( In=$in,
Reader=$rdr, Entity=$ent));
}
elsif ($classify eq $CLASS_MESSAGE) {
- $self-process_message(In=$in, Reader=$rdr, Entity=$ent);
+ return undef unless defined($self-process_message(In=$in,
Reader=$rdr, Entity=$ent));
}
elsif ($classify eq $CLASS_SINGLEPART) {
- $self-process_singlepart( In=$in, Reader=$rdr, Entity=$ent);
+ return undef unless defined($self-process_singlepart( In=$in,
Reader=$rdr, Entity=$ent));
}
else {
internal_error unknown classification '$classify';
@@ -1374,7 +1389,6 @@
=back
Returns the parsed MIME::Entity on success.
-Throws exception on failure.
=cut
@@ -1412,8 +1426,8 @@
or as Iany blessed object conforming to the IO:: interface
(which minimally implements getline() and read()).
-Returns the parsed MIME::Entity on success.
-Throws exception on failure.
+Throws exception on failure. If the message contained too many
+parts (as set by Imax_parts), returns undef.
=cut
@@ -1946,6 +1960,32 @@
sub last_head {
usage_warning deprecated: use \$parser-results-top_head\n;
shift-results-top_head
Re: [Mimedefang] Mimedefang segfaults/crashes on some mails
Hi, Wild guess: it's your virus scanner, that is integrated in the perl process using external libs, trying to extract a .rar file which is failing spectacularly. Nope. In the meantime I've isolated the 2 mails. I'll do now testings with them. Martin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
