Re: [Mimedefang] DKIM body hash won't verify.
On 2/22/19 10:48 AM, Dianne Skoll wrote: Hi, I just put DKIM on my mail system, but gmail rejects with "body hash did not verify", so something is wrong with how I'm hashing. Does this code look reasonable? [snip] The code looks fine. Do services other than Gmail report a valid DKIM signature? I've observed occasional DKIM verification failures by Gmail that I've yet to be able to explain, when all other verifiers work correctly. Regards, Dianne. Thanks for the feedback Dianne, it turns out that single part mime messages were being reduced to simple mime causing different headers to be signed. When I changed the code to sign the message in INPUTMSG instead of the entity that is passed filter_wrapup, everything started working fine. Thanks again. schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] delete_recipient not removing the recipient from @Recipients
MD Users,
Any reason why I wouldn't want to do this:
sub delete_recipient ($) {
my($recip) = @_;
write_result_line(S, $recip);
+@Recipients = grep { $_ ne $recip } @Recipients;
return 1;
}
I have a filter that removes recipients from the message when they don't
want spam and others do, but when graphdefang is called it shows mail_in
for every user because it iterates though @Recipients. This causes my
stats to be all jumbled up as well as makes the logs misleading.
Thanks,
schu
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Sendmail converting add_recipient addresses to lower case.
List, I'm trying to do some filtering in MD by removing the recipient address and replacing it with a user+det...@domain.com address so that the message will get routed to the correct sub-mailbox using the cyrus lmtp mda. The problem I'm running into is that Cyrus sub-mailboxes are case sensitive but whatever I put in the add_recipient() function is lowered before it gets to the mda. I put my milter on an inet socket so I can do some sniffing, and I can confirm that MD is passing the new address to sendmail with the mixed case in tact, but I can't seem to find where sendmail is lowering the address or how to fix it. Anyone on this list run into this problem and found a solution? It would be much appreciated if you can share how you got around it. Thanks, schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Sendmail converting add_recipient addresses to lower case.
On 06/14/2010 11:38 AM, Joseph Brennan wrote: Matthew Schumacher mat...@aptalaska.net wrote: I'm trying to do some filtering in MD by removing the recipient address and replacing it with a user+det...@domain.com address so that the message will get routed to the correct sub-mailbox using the cyrus lmtp mda. Lowercasing happens in the Cyrus mailer definition, in sendmail.cf. You can add u to the F= equate, to preserve case, but then you have to take care of the opposite problem, namely random uppercasing of users, so that mail to a...@example.com does not bounce because the user is abc not Abc. Joseph Brennan Columbia University Information Technology Right I forgot that Thanks Joseph. schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] rewrite_mail in SA.
List, I need to get some info that spamassassin returns when calling check() so I omit the normal mimedefang spam_assassin_check() and call spam_assassin_status() directly: #my($hits, $req, $names, $report) = spam_assassin_check(); my $status = spam_assassin_status(); my $hits = $status-get_score(); my $req = $status-get_required_hits(); my $names = $status-get_names_of_tests_hit(); my $report = $status-get_report(); my $rewritten_mail = $status-rewrite_mail(); # do stuff $status-finish(); Everything works fine except for rewrite_mail(). When that is called I get this in the logs: mimedefang-multiplexor[2482]: Slave 1 stderr: Use of uninitialized value in split at /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 825. mimedefang-multiplexor[2482]: Slave 1 stderr: Use of uninitialized value in pattern match (m//) at /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 832. mimedefang-multiplexor[2482]: Slave 1 stderr: Use of uninitialized value in concatenation (.) or string at /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 907. mimedefang-multiplexor[2482]: Slave 1 stderr: Use of uninitialized value in string ne at /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 638. mimedefang-multiplexor[2482]: Slave 1 stderr: Use of uninitialized value in substitution iterator at /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 639. last message repeated 5 times mimedefang-multiplexor[2482]: Slave 1 stderr: Use of uninitialized value in concatenation (.) or string at /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PerMsgStatus.pm line 910. Funny thing is though, $rewritten_mail now has the message header in it. Ultimately what I want to do is get at some spam assassin headers and pass them though like this: action_change_header(X-Spam-Relay-Country, $status-get( X-Spam-Relay-Country ) ); Does anyone know why rewrite_mail() acts like this? Shouldn't it rewrite the entire message and allow me to fetch SA headers with $status-get()? Thanks, schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] OT: Blocking Port 25
Ben Kamen wrote: David F. Skoll wrote: I suspect Ben meant: What's the consensus on blocking outbound port 25 for home users? (DSL/Cable-Modem/Dialup) Yea, I meant to control port 25 egress from nets like Cable/DSL/Dialup users... We have 2 rules in our pppoe servers, one that allows port 25 outbound to the ISP's mail server, and one that allows port 25 outbound to the Internet. In our customer provisioning tool we have a checkbox that allows you to set which rule is bound to the virtual interface at login. This allows us to pick and choose who we block. By default we block port 25, but if the customer calls and has a good reason and we believe the can be trusted to use port 25 then we will omit them from port 25 filtering. If they don't have a good reason we will walk them though a workaround such as using our smtp server for relay or sending a request to their admin to allow them to relay using smtps/smtpauth on port 465. This solution pretty much stopped spam from originating from our network, but yet accommodates people like David since having a static IP and managing a mail system is a good enough reason to get me to omit the filtering. schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] IMAP
Rob MacGregor wrote: On 1/24/07, Jan-Pieter Cornet [EMAIL PROTECTED] wrote: Users actually need to _do_ something to switch to imap, and rethink the way they store and view email. And last I heard Eudora and Outlook Express still didn't support imap anyway... Outlook Express has had IMAP support since at least v4. Yea, and it's terrible. schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Questions about stream_by_recipient and problems it creates.
David F. Skoll wrote: Our commercial CanIt-PRO product uses a third solution: 3) If all recipients have the same settings and rules, reject the spam in filter_end. Otherwise, remail copies. If an e-mail originating from 127.0.0.1 is rejected as spam, we call action_discard instead of action_bounce. Three years ago, I would have said this was a bad solution because it silently discarded e-mail. Today, unfortunately, I am of the opinion that it's the only viable way. Regards, David. N!! Not silently discard I don't know if I can bring myself to do that, occasionally we have problems with false positives, so silently dropping mail would seriously aggravate the issue. People around here depend on the bounce messages. Perhaps a better solution for me would be to to make filter_end accept if any one user accepts it, then for the users that reject it, call delete_recipient then email the sender an error message from mimedefang sourced from a black hole account. This would prevent me from processing the message more than once and would also allow me much more control over the bounce message where I would only quote the subject to reduce bandwidth and render bounce back spam useless due to the missing payload. Thoughts? schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Questions about stream_by_recipient and problems it creates.
Jan-Pieter Cornet wrote: A fourth option is to introduce a spam folder, where you file suspected spams. We already use this currently. Then if you cannot reject an email for one user (because another user does want to receive the email), flag the message to be delivered to the spam folder. This doesn't work because most of the people using the mail system use POP3. I would love for people to ditch pop3, but due to poor support for IMAP in widely-used-but-marginal (as David puts it) clients make IMAP pretty lame. Or combine this with option 3 by david, and introduce another threshhold score above which you can junk messages instead of delivering or rejecting them. Oh, and a fifth option is to detect this possibility at filter_recipient time, and tempfail recipients that have a filter profile that isn't on par with the filter of the first recipient. Optionally only for hosts that have some sort of flag set. This is an interesting idea, temp-failing recipients with dissimilar spam settings, but due to lame email clients, it sounds like it would be a lot of trouble. Anyway thanks for the thoughts, they have me thinking about other options. schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Questions about stream_by_recipient and problems it creates.
David F. Skoll wrote: Matthew Schumacher wrote: N!! Not silently discard Yes, silently discard. But wait! CanIt, out of the box, *NEVER EVER* rejects or discards an e-mail unless a human being tells it to. Now, you can tune the settings so that it will in fact reject or discard e-mail without intervention, but unless you've been insanely aggressive in your settings, even rejected/discarded e-mails create an incident in the database so you can see what happened, as well as look over the first 8kB of the mail to see what you might have missed. Perhaps a better solution for me would be to to make filter_end accept if any one user accepts it, then for the users that reject it, call delete_recipient then email the sender an error message from mimedefang sourced from a black hole account. Sending the sender an error message will very quickly get you blacklisted as a source of backscatter. That's the unfortunate reality. Isn't spam wonderful :| Here is a thought, what about declaring which users accepted and which rejected in the rejection message. So if recipient A wants everything tagged and recipient B wants spam rejected, then we reject the message with Recipient B thinks this is spam (message was delivered to other recipients) then in filter_end we queue the message for recipient A. This way the error message always declares what is going on, the sender knows what happened, recipient A always gets their email, there is never silently discarded email, and we are not sending backscatter to the Internet. Do you think this will work? schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] use Net::LDAP and file descriptor error.
List,
This code:
sub filter_initialize {
use Net::LDAP;
}
Causes me to get this error:
mimedefang-multiplexor[612]: WARNING: Something in your Perl filter
appears to have opened a file descriptor outside of any function. With
embedded Perl, you should move any code that opens a file descriptor
into filter_initialize. On some systems, the C library may opena
descriptor, but you should verify your filter just in case.
Since I'm putting it in filter_initialize and my filters seem to work
fine I'm thinking that I can safely ignore it, but I want to double
check with the list.
Anyone have thoughts on why this wouldn't work?
schu
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] use Net::LDAP and file descriptor error.
Dave O'Neill wrote: It's safe to ignore. That warning's getting removed in a future version of MIMEDefang anyway. The reason you're getting it is because Net::LDAP::Constant is a shocking abuse of Perl. That module opens its own source, via the DATA handle, so that it can parse its own POD documentation to retrieve the constant values, error names, and error text. It's a neat hack, because it removes duplication of the constant values and error texts, but unless you close(DATA) after using it, it will leave a dangling open filehandle. Cheers, Dave That is interesting. thanks for the heads up. schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MIMEDefang 2.57 is Released
David F. Skoll wrote: * mimedefang-multiplexor: New scheduling algorithm tries to keep commands sticky. For example, when looking for a slave to run recipok, we prefer to use a slave that recently ran recipok. NOTE!!! If your filter incorrectly retains state from earlier callbacks into filter_begin, this scheduling change WILL expose the bugs in your filter. David, Thanks for giving us mimedefang, we all owe you one!!! I'm trying to come up with a way to test this new algorithm before I upgrade production systems. One thought I had was to call exit at the end of each filter and test. Theoretically, the multiplexor would replace the slave so it shouldn't affect operation (other than performance which is not a concern while testing), and each new slave would case loss of state. I think this would show me any problems with my filter without trying to determine whether it works because my filter is correct or I got lucky and reused a slave which is far more likely on my test system. Does this sound reasonable to you? Also, I want to double check that I have a correct understanding of which vars will be set when a slaved is called: Vars created in filter_initialize(). Global vars such as $Sender or @Recipients. Vars that are passed to the function. Does that look right to you? I have been careful not depend on anything other than the vars above, so I think I'll be alright. Thanks, schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MIMEDefang 2.57 is Released
David F. Skoll wrote: Matthew Schumacher wrote: I'm trying to come up with a way to test this new algorithm before I upgrade production systems. One thought I had was to call exit at the end of each filter and test. You don't want to do that. The multiplexor will complain loudly, and mail will tempfail. Your best bet is to set up a test system with 2.57. The change should not make much difference to most people; only those with buggy filters that would mysteriously fail occasionally will notice that they now fail almost every time. :-) Right, I understand, which is why I want to make the the test system absolutely fail every time instead of almost every time. However, the only way I can think of to kill the state every time regardless of which slave is called is by killing the slave. Perhaps I'm being a little paranoid here, but having a test system that explicitly kills the state after every call is a great way to insure filters will work as intended. Vars that are passed to the function. Stuff like this: my($entity, $fname, $ext, $type) = @_; Let me stress that this change does *NOT* alter the documented behavior of MIMEDefang. It may, however, expose bugs in incorrect filters that rely on the old undocumented behavior of attempting to reuse the same slave as much as possible. While I was careful to not depend on state, there could be something that slipped though the cracks so I'm wanting a real black and white method of testing this to make sure all is well before upgrading. I don't have any reason to believe it won't work fine and I can't spot anything in my filter, but a very ridged test system would ease my paranoia. Thanks again, schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MIMEDefang 2.57 is Released
David F. Skoll wrote: Hmm... I suppose you could set MX_REQUESTS=1. That would make each slave quit after processing one request. Doh!!! I knew I was missing something simple. Thanks for pointing out the obvious. schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Hacking in authenticated local submissions
Philip Prindeville wrote:
I'm using port 465 and SSL for local submissions, and I'd like
to tweak either Mdf or SA (or both) so that if I forward (for
instance) a copy of a spam to someone outside, that I in
turn don't get my message rejected because it looks like spam.
Now, why isn't trusted_networks taking effect? Because I
might be connecting via the public Internet (and using IMAPS
and SMTPS to converse with my mail server).
Do we want to change the line:
if ($Features{SpamAssassin}) {
in mimedefang-filter, for instance, to skip this check if $auth_authen
is valid?
-Philip
Philip,
Go read this page: http://sial.org/howto/mimedefang/macro-pass/ . It
describes how to setup sendmail to pass macros to mimedefang. Use it to
pass a macro that tells MD whether the user authenticated or not. If
this user authenticated, and you trust them, then just omit them from
spam filtering altogether.
schu
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Another silly idea
Ben Kamen wrote: Don't tell us that - we know. The problem is the ISP's want the dumbest people they can find to maximize their investment of their company. That means 99% of the folks of the engineering staff typically can't even SPELL D-N-S, let alone use a program like 'vi'. Holy stereo typing Batman!!! While I agree that many (even most) ISPs don't have qualified people on staff, other ISPs have very talented people working for them. One of my co-workers handles cisco routers for our company and is very good at what he does. He was an electronics tech for many years, and has his ham radio license. Not only could he break down packets to the frame and even binary level, I have no doubt he could even explain how the +5/-5 volts on the wire works. Just recently the telephone guys where having problems with some equipment not ringing after a voip conversion, so he broke out his scope, and discovered the voip equipment wasn't generating a proper ring. The vendors tech support sure wasn't expecting the following phone call. I would be willing to bet that that was the first time ring problems had been reported at the frequency level. Anyway, some ISPs have good people on staff, I think the bigger problem is management limiting the authority of people that really know how to get the job done. My $.02 schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MySQL Greylisting - server has gone away
Paul Murphy wrote: Note that this only happens on very low volume servers or where you have many more slaves than you actually need - if each of your slaves can be guaranteed to process one connection every 8 hours, you'll never see this! In my case, I have a small system at home using 5 slaves and which receives about 120 connections per day, almost all of which are between 0700 and 2300, so some of the slaves will be idle for longer than 8 hours overnight. You may consider cutting back to two slaves to save system memory. You certainly don't need 5 slaves with this mail load. schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Mimedefang acting strange after upgrade to kernel 2.6.
So I finally upgraded to kernel 2.6.15.6 and now I am noticing strange
things. Anyone have an explanation for any of it?
1. I no longer see mimedefang.pl in the ps list. It only shows
/usr/bin/mimedefang-multiplexor and /usr/bin/mimedefang processes. In
the log it says there are slaves:
Starting slave 8 (pid 15420) (10 running): Bringing slaves up to
minSlaves (10)
But that process is a /usr/bin/mimedefang-multiplexor process:
defang 15420 2350 0 08:42 ?00:00:00
/usr/bin/mimedefang-multiplexor -p
/var/spool/MIMEDefang/mimedefang-multiplexor.pid -E -m 10 -x 50 -U
defang -b 90 -l -T -s /var/spool/MIMEDefang/mimedefang-multiplexor.sock
2. Every now and then I get this error:
mimedefang-multiplexor[3459]: Slave 51 stderr: dns: sendto() failed: at
/usr/lib/perl5/site_perl/5.8.6/Mail/SpamAssassin/DnsResolver.pm line 274.
I have looked though the SA code a little and from what I can tell, SA
is having a hard time sending a DNS packet. Anyone else see this?
Also, it's interesting that the error is coming from
mimedefang-multiplexor. Did the multiplexor and filters become one
process along the line and I missed it?
3. I am now getting this error:
Mar 14 07:25:50 larry mimedefang-multiplexor[3459]: Slave 6 stderr: DBI
connect('database=;host=localhost','',...) failed: Too many
connections at /etc/mail/mimedefang-filter line 542
I connect to a mysql database in filter_initialize so that I have a dbi
connection I can query as I process my filters, but suddenly I have too
many connections. This is really strange because the max slaves is set
to 60, and the default setting on mysql is to allow 100 connections.
Unless mimdefang is calling filter_initialize more than once per slave I
don't know how this could be possible, and mimedefang is the only
process using mysql on this host.
Anyway, if anyone has thoughts or ideas they would be greatly appreciated.
schu
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Accept mail before processing
[EMAIL PROTECTED] wrote:
That way, you only do the nasty hack for people connecting with
MUAs, and use normal real-time processing for everyone else.
I am thinking about doing something similar for users with MUA's, as
during peak times sendmail/MD can take 5-10 seconds to send an email.
I was thinking about setting up another sendmail instance on another
IP that just accepts mail from our MUA's, and then relays it through
the sendmail/MD instance.
I know I would loose functionality like immediately rejecting bad
recipients, a feature some MUA users like. But assumed a bad rcpt
bounce would be returned to the original sender, right?
You could set up another sendmail instance on a nonstandard port on the
*same* machine. That way you can get bad RCPT checking without the
MIMEDefang overhead.
Or you could require SMTP AUTH and run sendmail on 587, then configure
MIMEDefang to skip SpamAssassin checks on authenticated mail... or run them
on a no-network-tests config, which should be much faster.
This is how I do it in a corp environment, I turn off relaying by IP and
force users to authenticate to smtp before I will let them relay. Then
I require TLS before SMTPAUTH [define(`confAUTH_OPTIONS', `A,p,y')] just
to make sure that the password is never passed in the clear.
Once the user authenticates I can see it in the filter because I import
a sendmail macro (look at docs over at
http://sial.org/howto/mimedefang/macro-pass/ ). Now, I don't omit users
from virus scanning because I don't want a virus leaving, but I do omit
them from spamassassin which is the majority of the processing time.
Another interesting hack (that means use with caution) is this rule:
# If user is authenticated or coming from localhost then stop filtering
if ((exists $SendmailMacros{md_tag} and $SendmailMacros{md_tag} eq
'authenticated') or ($ip eq 127.0.0.1)) {
$authenticated = TRUE;
md_syslog('warning', user authenticated);
} else {
$authenticated = FALSE;
md_syslog('warning', user is NOT authenticated);
}
# if the user is sending from domain.com then they must authenticate
(or be localhost)
if( ($sender =~ /domain.com/i) ($authenticated ne TRUE) ){
return ('REJECT', 'You must authenticate to send mail from an
domain.com account.');
}
Basically, since we know that all of our users will authenticate, when
we see mail from domain.com that doesn't come from an authenticated user
we know it is spoofed so we reject it. While this is great because it
limits inter domain spoofing to authenticated users, and viruses from
[EMAIL PROTECTED] simply break, it also breaks things like ebay where
they send you mail from yourself.
Anyway, I think this is the best solution because you should treat
trusted users differently than public users. I don't mangle trusted
users email, and I haven't had a call from someone complaining about
messages being marked as spam from other staff members in a long time.
schu
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] FreeBSD 5.4, MD 2.54, and Quarantine Directories
David Nelson wrote:
I don't know if this is the proper forum for this, but I'd like to make
a feature request for MD.
I ran into a problem quarantining messages on a FreeBSD 5.4 box.
Specifically, I hit the upper limit for the number of directories a
single directory can hold (32767). I store about 2 weeks of quarantined
mail in case anything needs to be released and I average over 3000
quarantined messages daily.
The solution was to create 256 directories under MD-Quarantine (00-FF
inclusive), have get_quarantine_dir() randomly generate a number,
convert it to hex, and append it to $Features{'Path:QUARANTINEDIR'}.
I don't know if there's a more ''elegant'' way of doing it, but it does
take care of the problem.
I load my quarantine into a database which does a much better job of
storing this information since I simply run queries to cleanup or fetch
the information.
A simple perl script that wakes up and loads the information works, but
running the db queries inside your filter would be even cleaner.
schu
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Anyone noticing...
Richard Laager wrote: On Tue, 2006-01-17 at 17:30 +, Paul Murphy wrote: For more background, search the mailing list archives for Blocking spam senders using IPTables?. Before I spend a lot of time searching... Did you post the script, or just notes on the idea? Thanks, Richard I did something similar to this to reduce the spam (and load). Whenever I get a message I put the remote smtp server IP address in a sql table along with a timestamp, SA score, and counter. This lets me know what the SA average is for any host connecting. Now that I know what the averages are, I query this information during the filter_recipient function and bounce messages where we have at least 5 messages to average on, and where that average is higher than the spam threshold that is read out of our ldap directory for that user. So lets say that a user has the spam sensitivity set to high (5) and we have a zombie connect and pass 5 messages that average 7 points. On the 6th message the zombie will pass the mail from and rcpt to, then mimedefang finds that the average is higher than the users threshold and bounces the message. To keep the database cleaned up I delete all records that have not been updated for 24 hours every hour. This change basically limits zombies to 5 messages before they are rejected, which saves the system from running SA on messages that are almost certainly spam. I don't have exact numbers but the mail load on the system dropped significantly. I need to update my graphdefang config so that I know the exact statistics, but haven't got to it yet. The only drawback that I have found to this solution is users that forward all of their mail to our mail system. After the change several called and complained that the forwarding broke because the forwarded mail was mostly spam which caused the remote mta to get rejected. I told the users that the forwarded mail is mostly spam and that the provider forwarding the mail was indeed relaying spam. Every user had their issues resolved by either asking us to remove spam filtering from their account or asking the forwarding provider to filter messages for spam before they forward. The reason why I did it this way instead of iptables was because we need to be able to adjust the spam filtering on a per user bases. This is why we don't make any decisions on the message until after the rcpt to and ldap lookup. I also wanted to make sure that the support address always works because all of our bounce messages say to email support for help. This has worked very well for us since someone will run into a problem, then email support, which has spam filtering disabled (100 point threshold). When the helpdesk gets the message they simply look at the header which tells them exactly why the message was rejected and allows them to walk the user though how to fix it. If we had gone with iptables we wouldn't be able to leave our abuse, postmaster, and support addresses open, and users would be rejected without an error message explaining exactly what happened. Since rejected email only costs us one ldap and one sql lookup we will live with that since those things are really cheap compared to mimedefang and SA. Hope this helps someone. schu ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MX - 127.0.0.1
David F. Skoll wrote: I'm arguing that it's wrong and bad. I agree!!! Whenever I see mail loops back to me (MX problem?) errors I can't help but think how clueless the domain administrator is for sending that domains email to an unknown host that can be different on every network. If they find email being sent to the 2nd level domain A record due to a missing mx record then all they need to do is create a mx record, point it at a mail host, then tell the mail host to reply with a 5xx permanent error. Sorry for the rant, but this is one of my pet peeves!! schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Spamassassin 3.1 and improved bayes/sql.
I thought I would mention to the MD users that spamassassin 3.1 which is in rc1 has much better bayes/sql support. I was working with the SA guys and the people on the pgsql performance list where Tom Lane came up with a way to pass the tokens as an array via a pgsql proc. The result is grouping all of the tokens from an email into a single transaction which is the difference between pgsql being unusable to almost as fast as mysql. Here is the new benchmark: http://wiki.apache.org/spamassassin/BayesBenchmarkResults I have this running now and it's WAY better than the bdb solution. schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Spamassassin 3.1 and improved bayes/sql.
David F. Skoll wrote:
Matthew Schumacher wrote:
Have you thought about simply doing:
SELECT * FROM bayes WHERE token in ('tok1', 'tok2', ..., 'tokN')
It seems to me that should be just as fast, and not rely on PostgreSQL
features or stored procedures.
You have to be careful with messages that have extremely large numbers
of tokens; you might need to split the query into chunks of 1000
tokens each or something like that.
Yes, we tried that. I attached another version of the proc that gets
rid of the looping altogether, but believe it or not, it's slower.
The reason why we rely on procs is because the SA code doesn't have
transactions yet. By passing the tokens in as an array we get
transactions because pgsql treats each run of a proc as a trans.
Here is the new benchmark:
http://wiki.apache.org/spamassassin/BayesBenchmarkResults
Those results are extremely surprising. Our CanIt benchmarks show
Berkeley DB outperforming PostgreSQL by a factor of 6 to 10, but your
benchmarks show them about equal. Something is funny there... I
wonder if it could be that CanIt never locks the BDB files, whereas
SpamAssassin does? If that's the case, then there's still tremendous
room for improvement on the BDB side.
Yea the SA bdb code is a locking headache. I have never really coded
against bdb so I have no idea if it's good, bad, or otherwise.
Also, I don't think the fsync=false column should even be presented.
Nobody who cares about his/her data runs PostgreSQL like that, so the
timings in that column are unachievable in real-world situations.
I agree that it's useless in the real world, but it is interesting to
see how much time the sync takes. Before grouping tokens into procs
(read transactions) sync was 25x slower. Now that they are pretty close
speed wise, that tells you that we are not spending to much time doing a
sync.
Ironically, just as SpamAssassin is making strides with a centralized
SQL database, in CanIt, we've revised our thinking and started moving
to distributed BDB databases. :-)
You mean having a separate bdb for each key/val pair? I proposed this
to the sa people for the AWL and they shot it down. Like I said, I'm
ignorant when it comes to bdb.
As much as I love pgsql and think it's light years beyond mysql in
features, stability, flexibility, and even performance (when doing
complex queries against large tables), I am now testing mysql for my
bayes store. Mysql has one good thing going for it, raw speed on simple
queries, which is really the only requirement for bayes.
create index bayes_token_id_idx on bayes_token (id);
CREATE OR REPLACE FUNCTION put_tokens(inuserid INTEGER,
intokenary BYTEA[],
inspam_count INTEGER,
inham_count INTEGER,
inatime INTEGER)
RETURNS VOID AS '
BEGIN
UPDATE
bayes_token
SET
spam_count = greatest_int(spam_count + inspam_count, 0),
ham_count = greatest_int(ham_count + inham_count , 0),
atime = greatest_int(atime, inatime)
WHERE
id = inuserid
AND
token IN (SELECT bayes_token_tmp FROM bayes_token_tmp(intokenary));
UPDATE
bayes_vars
SET
token_count = token_count + (
SELECT
count(bayes_token_tmp)
FROM
bayes_token_tmp(intokenary)
WHERE
NOT EXISTS (SELECT token FROM bayes_token WHERE token = bayes_token_tmp)),
newest_token_age = greatest_int(newest_token_age, inatime),
oldest_token_age = least_int(oldest_token_age, inatime)
WHERE
id = inuserid;
INSERT INTO
bayes_token
SELECT
inuserid,
bayes_token_tmp,
inspam_count,
inham_count,
inatime
FROM
bayes_token_tmp(intokenary)
WHERE
(inspam_count 0 OR inham_count 0)
AND
NOT EXISTS (SELECT token FROM bayes_token WHERE token = bayes_token_tmp);
RETURN;
END;
' LANGUAGE 'plpgsql';
CREATE OR REPLACE FUNCTION bayes_token_tmp(intokenary BYTEA[]) RETURNS SETOF bytea AS
'
BEGIN
for i in array_lower(intokenary, 1) .. array_upper(intokenary, 1)
LOOP
return next intokenary[i];
END LOOP;
RETURN;
end
'
language 'plpgsql';
CREATE OR REPLACE FUNCTION greatest_int (integer, integer)
RETURNS INTEGER
IMMUTABLE STRICT
AS 'SELECT CASE WHEN $1 $2 THEN $2 ELSE $1 END;'
LANGUAGE SQL;
CREATE OR REPLACE FUNCTION least_int (integer, integer)
RETURNS INTEGER
IMMUTABLE STRICT
AS 'SELECT CASE WHEN $1 $2 THEN $1 ELSE $2 END;'
LANGUAGE SQL;
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Mail::SpamAssassin::BayesStore::SQL database persistence.
Sevo Stille wrote: Matthew Schumacher wrote: At any rate with my installation, SA/pg seems to be mostly slowed down by the fact that the database is poorly indexed - indexing the most significant search fields (bayes_seen(msgid), bayes_token(token) and id throughout all tables) and doing a vacuum optimize run did improve performance quite significantly. Would you mind sharing your updated schema so that we can see what indexing you did. My installation seems fine right up until it goes to expire data, then it goes into suck mode. schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Mail::SpamAssassin::BayesStore::SQL databasepersistence.
[EMAIL PROTECTED] wrote:
schu wrote:
So the question is, is there a way to make SA use persistent
connections? I read though the docs and didn't see anything.
Perhaps a way to create a persistent connection in MD and pass it to
the SA module?
Some thoughts...
1. spamd
spamassassin calls through the spamassassin command and the
Mail::SpamAssassin perl modules don't live long enough to hold on to a
persistent connection. But spamd threads do.
2. connection pooling
Perhaps a middle layer could be developed, consisting of a daemon that does
nothing but relay connection info - each connection could be kept open for a
certain time after it is nominally closed. If a request for a new connection
comes in, surviving connections could be reused. Windows has an option to do
this for ODBC connections on a per-driver level.
It looks like the code is supposed use persistent connections:
From SpamAssassin/BayesStore/SQL.pm on line 134
==
return 1 if ($self-{_dbh}); # already connected
my $main = $self-{bayes}-{main};
$self-read_db_configs();
# Turn off PrintError and explicitly set AutoCommit to off
my $dbh = DBI-connect($self-{_dsn}, $self-{_dbuser}, $self-{_dbpass},
{'PrintError' = 0, 'AutoCommit' = 1});
if (!$dbh) {
dbg(bayes: Unable to connect to database: .DBI-errstr());
return 0;
}
else {
dbg(bayes: Database connection established);
}
$self-{_dbh} = $dbh;
==
So the question is why doesn't it work? Does anyone know if this works
when using spamd?
schu
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Mail::SpamAssassin::BayesStore::SQL databasepersistence.
David F. Skoll wrote: Matthew Schumacher wrote: It looks like the code is supposed use persistent connections: That's bogus, though; there are many explicit calls to untie_db sprinkled in the code. It might just be a safety check in case someone calls tie_db twice. I should also note that there are many problems with the code, especially because it makes many assumptions about MySQL-like optimizations. For example, the code to check if too many tokens will be deleted by an expire run is just plain stupid. The query to check if it's OK to do the expiry takes about as long as the actual expiry itself! The private _get_oldest_token_age function in SQL.pm will also be dreadfully slow because it does a sequential scan. IMO, SpamAssassin's SQL.pm module is simply not production code. It will bog down any site doing a reasonable mail volume. Regards, David. I saw that they where doing min and max and thought about that, obviously they built the thing for mysql, but at this point I was getting a zillion Cannot open bayes databases /etc/mail/bayes/bayes_* R/W: lock failed: File exists error messages. Is there a good method for deploying bayes on a busy site that isn't slow or full of locking issues? schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Mail::SpamAssassin::BayesStore::SQL databasepersistence.
David F. Skoll wrote: I should also note that there are many problems with the code, especially because it makes many assumptions about MySQL-like optimizations. For example, the code to check if too many tokens will be deleted by an expire run is just plain stupid. The query to check if it's OK to do the expiry takes about as long as the actual expiry itself! The private _get_oldest_token_age function in SQL.pm will also be dreadfully slow because it does a sequential scan. One thing I did to make this a little better is I found everywhere in the code where min() and max() are used and created an index on the column used. In the case of _get_oldest_token_age simply creating an index on the atime column made this much difference: QUERY PLAN Aggregate (cost=220682.04..220682.04 rows=1 width=4) (actual time=12267.656..12267.657 rows=1 loops=1) - Seq Scan on bayes_token (cost=0.00..220542.42 rows=55844 width=4) (actual time=12267.645..12267.645 rows=0 loops=1) Filter: (id = 1) Total runtime: 12267.804 ms QUERY PLAN - Aggregate (cost=17947.15..17947.15 rows=1 width=4) (actual time=0.130..0.131 rows=1 loops=1) - Index Scan using bayes_token_pkey on bayes_token (cost=0.00..17936.02 rows=4451 width=4) (actual time=0.121..0.121 rows=0 loops=1) Index Cond: (id = 1) Total runtime: 0.361 ms (4 rows) I looked at the other queries and they aren't to bad, so I think I have the biggest problems pretty much solved except for database persistence. At this point it is holding up to my mail load, so I'm on the fence with going back to bayes and having locking problems or just sticking with this. schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Mail::SpamAssassin::BayesStore::SQL databasepersistence.
David F. Skoll wrote: Matthew Schumacher wrote: It looks like the code is supposed use persistent connections: That's bogus, though; there are many explicit calls to untie_db sprinkled in the code. It might just be a safety check in case someone calls tie_db twice. The only thing the untie_db code does in the SQL module is disconnect from the database. Since there is never a reason to do that unless we are exiting (in which case it's cleaned up anyway) I commented out the disconnect and undef statements. I can now see persistent db connections, so it works, but I wonder if there are any adverse affects that are eluding me. Can you think of any? schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Code to remove winmail.dat files and replace them with the attachments.
hans hm04 wrote:
hi Thomas
did you find a solution in the meantime ?
i noticed, not all winmail.dat-attachements do have
real data with included files.
some of them have only a pointer to a file of an external filesystem.
therefore nothing couldn't be extracted.
i am also interesting of automtic converting winmail.dat to
normal attachments - as we don't use microsoft
so i asked for a solution in 02/2004 - but no satisfied answers.
in the meantime there is a patch availabel - as i see.
are there success-stories with this patch too ?
best regards
hans
Whoa, someone is talking about my code and I'm not even paying attention
I have that code in prod and it works fine, but you must compile and
install the ytnef binary.
If you look at the code:
@new_files = `/usr/local/bin/ytnef -f Work $path`;
foreach my $file ( @new_files ){
chomp( $file );
push( @tnef_files, $file );
md_syslog( 'info', Found file $file in winmail.dat attachment );
}
It uses ytnef to extract the files, if there are files extracted they
are later attached, if not, the winmail.dat file is deleted and nothing
is attached.
Hope that helps,
schu
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Code to remove winmail.dat files and replace them with the attachments.
David F. Skoll wrote: Matthew Schumacher wrote: I have that code in prod and it works fine, but you must compile and install the ytnef binary. Is there a reason you didn't use Convert::TNEF? http://search.cpan.org/~dougw/Convert-TNEF-0.17/ Regards, David. No, not really, I am pretty sure I looked for a perl module, since that is the better way to do it, perhaps I missed it. I'll look into converting the code soon, but for now it does work, and I'm not sure I have the time to play with it right now. schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Mail::SpamAssassin::BayesStore::SQL database persistence.
List, I moved my SA bayes store into my postgres database because I was running into locking issues with the Berkeley DB. The database is up and running fine, but I noticed that that connections to the database are not persistent. As many of you know, postgres isn't the fastest database during startup so I am sure this is hurting performance quite a bit. In fact I looked that the performance benchmarks on the spamassassin site: http://wiki.apache.org/spamassassin/BayesBenchmarkResults and they show postgres to be a terrible performer which would be true if the are not using persistent database connections. So the question is, is there a way to make SA use persistent connections? I read though the docs and didn't see anything. Perhaps a way to create a persistent connection in MD and pass it to the SA module? schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] ClamAV's Worm/Trojan/Joke/W97M classifications
Les Mikesell wrote: On Thu, 2005-06-30 at 08:39, [EMAIL PROTECTED] wrote: I still think you'll change your mind the day your address is the one being forged and the target of a million bounces. You didn't answer when I asked this before so I'll try again. Viruses virtually always use legitimate addresses found in the local contact list or headers of received email - just not the real sender. How does your system ensure that rejections by the next hop can only be returned to the real sender, not a forged address? If you can't do this yourself, how can you expect the rest of the world to do it? If you can, I'd like to know how. Les and the others make valid points. Rejecting the message will still cause some other relay to generate a bounce message, that is a disadvantage of rejecting vs dropping. Personally I would rather give the legit user the benefit of the doubt. If someone does send a virus infected attachment, or if the virus scanner has a false positive the user needs to be notified. There is another case where rejecting is better that hasn't been bought up yet (or at least I didn't read it) password protected zip archives. On our mail system we call these viruses simply because they almost always are, but if we where silently dropping them then that would be a problem. One last issue is that of trust. I am simply not confident enough with virus scanners, password protected archives, and other false positives to start dropping email silently. schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] ClamAV's Worm/Trojan/Joke/W97M classifications
Chris Gauch wrote: Well, you have to accept the message data to scan it in the first place. Actually, you don't, you get a copy of it and can look at it before you decide, but you don't have to accept it. Since I already wasted my time scanning it with the virus scanner, I might as well take the nanosecond involved in accepting the message and then just throw it out. We scan for viruses before any spam scanning is performed. In my opinion, the virus has to meet its doom somewhere, that way I know it's gone and not floating around on the net causing problems for someone else (perhaps being a thorn in our side again). I think it's safe to say that don't use too many additional resources by throwing virus-infected mail into the bit bucket after it has already been identified as infected by our virus scanner. ;-) It's not an issue of processing time, or if the message should die, it's an issue of correctly reporting what happened. While some other MTA sending bounce notification emails may be causing a thorn in someone else's side because you wouldn't accept the message, it's quite possible that your policy of announcing that your 'OK' with the message then silently dropping it could be creating plenty of thorns of their own. I make strict policy to accept and deliver or reject regardless of whether the message is spam, over quota, invalid user, or otherwise. This cut and dry rule is easy for my customers to understand. Either we accepted the message and it's in your inbox, or we rejected the message in which case the sender will be notified of the problem by their relay. There is no possibility for lost email because nothing is ever dropped. ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Order of the fuctions?
John Scully wrote: We handle this a little differently. Instead of relying on the lowest threshold to determine how to han dle all recipients we pull the users settings during the recipient check in filter_recipient, and write them to our own file called RECIPIENT_SETTING in the working directory of the message (this is very fast since we have the MD dir on ramdisk). This includes exploding any multiple mailbox aliases into their individual users and saving those settings. In filter end, after scoring the message we loop through the recipient_settings file and based on the relative score and the per-user setting to mark, file or discard spam we use add_recipient and delete_recipient to make the changes. End result is that each user's message is handles based on their own settings for threshold and disposition, without the additional overhead of stream_by_recipient. John John, that is a very interesting way to do it, it solves the issue with each user getting their own settings, but the error reporting is a little odd since people sending email marked as spam will not get notified that their message was not delivered. Have you noticed that to be a problem or do you send a bounce message to those senders? Thanks, schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Order of the fuctions?
List, David,
Anyone have a list of all of the functions that mimedefang calls in the
filter in order?
I read though the man page and this is what I came up with:
filter_initialize() # called once for each child
foreach message {
filter_sender() # requires the -s flag to run
foreach recipient {
filter_recipient()# requires the -t flag to run
}
filter_create_parser()# parse mime or built in
filter_begin()# scan entire message
foreach part {
filter()# scan mime part
filter_multipart() # scan container parts
}
filter_end() # last bit of filtering
}
filter_cleanup()# right before the child dies
Does that look right? Am I missing some?
Also is it generally save to use filter_sender to flush vars so that
they don't carry over into the next message since it's the first thing
called on each message?
Thanks,
schu
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Order of the fuctions?
David F. Skoll wrote:
Matthew Schumacher wrote:
Do you know of a way to access the rcpt to: addresses in filter_end?
In the array @Recipients.
Please read
http://lists.roaringpenguin.com/pipermail/mimedefang/2005-June/027297.html
Many people lose sight of the fact that ALL information in
filter_{relay,sender,recipient} is available in filter_end too.
Right on!!! You thought of everything!!!
schu
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] smtp spoofing
David F. Skoll wrote:
Super-Dome wrote:
how can i disallow use of my domain from outside network using *From address
of my domain.
You cannot.
Regards,
David.
I have this running at one site:
if((exists $SendmailMacros{md_tag} and $SendmailMacros{md_tag} eq
'authenticated') or ($ip eq 127.0.0.1)){
$authenticated = TRUE;
md_syslog('info', user authenticated);
} else {
$authenticated = FALSE;
md_syslog('info', user is NOT authenticated);
}
if( ($sender =~ /domain.com/i) ($authenticated ne TRUE) ){
return ('REJECT', 'You must authenticate to send mail from an
domain.com account.');
}
This pretty much stops mail from our domain from being spoofed by users
that don't authenticate, then I turn off relaying for everything that
doesn't authenticate.
Viruses that spoof our domain pretty much go nowhere, but places like
ebay that spoof your address to send you email don't work either.
schu
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] smtp spoofing
David F. Skoll wrote: You can't prevent me from pretending to be [EMAIL PROTECTED] and e-mailing to [EMAIL PROTECTED] or [EMAIL PROTECTED]. Oh, I thought we was talking about excepting spoofed messages from his own domain. I didn't realize he was talking about other systems excepting spoofed messages. As you pointed out there isn't a way to block this in smtp, and I don't know of any other method that has enough support to be effective. schu ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Tie to aliases.db doesn't work
Jan Pieter Cornet wrote:
On Mon, May 23, 2005 at 06:37:05PM -0800, Matthew Schumacher wrote:
newaliases stores the keys (and values) with a trailing \0 character. Try:
print $aliases{root\0}, \n;
Also make sure to strip the \0 from the value if you're using it for
anything other than just printing, so the code above could also be
written as:
my $alias = root;
my $dest = $aliases{$alias\0};
$dest =~ s/\0$//;
print $dest, \n;
Thanks Jan, that helps a bunch.
schu
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Tie to aliases.db doesn't work
List,
I just can't figure this out. This code:
require DB_File;
use Fcntl;
tie %aliases, DB_File, /etc/mail/aliases.db, O_RDONLY;
while ( ($k,$v) = each %aliases ) {
print $k = $v\n;
}
finds all of the keys and values and works just fine:
root = [EMAIL PROTECTED]
@ = @
nobody = root
However if I try to pull out a specific value with:
require DB_File;
use Fcntl;
tie %aliases, DB_File, /etc/mail/aliases.db, O_RDONLY;
print $aliases{root};
It doesn't work. If I use `makemap hash aliases aliases` to make the
database all is well, but when I make it with newaliases I can't find a
value in the hash by key name.
Any ideas?
Thanks,
schu
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] How can I tell WHY this was rejected?
Lisa Casey wrote: According to my user, this e-mail consists of just a jpg image in the body of the email (and when I had him send the mail using a different mail server without mimedefang/spamassassin and CC a copy of it to me so I could take a look at it, sure enough that is what I got). Is there any way I can figure out WHY mimedefang is rejecting this e-mail so I can tell my user what he can change to keep this from happening again? Simple, mimedefang is rejecting this email because you told it to. For some reason spamassassin marks this email with a score over 6 which causes the bounce. The easiest way to troubleshoot is to quarantine the message with the spamassassin report so you can see what rules the message is matching. Put this before your action_bounce(): action_quarantine_entire_message($report); schu ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Thought I should share this - Reduced Junk traffic with md_check_against_smtp_server
Larry Starr wrote:
Having received requests, off list, for my solution, I'm posting it for
whoever want's it
Note:
1. my internal server - should be the name of the machine that your are
relaying for i.e. internalserver.mydomain.com
2. The RE for setting $relay mshould be your mail domain mydom.com
should, of course be your domain \bmydomain\.com\b
3.myfqdn should be the name of the server running mimedefang
i.e mail.mydomain.com
Larry,
Here is how I do it:
sub filter_initialize {
require DB_File;
use Fcntl;
tie %relaydomains, DB_File, /etc/mail/mailertable.db, O_RDONLY;
}
sub filter_recipient {
my($recip, $sender, $ip, $host, $first, $helo, $rcpt_mailer,
$rcpt_host, $rcpt_addr) = @_;
($user,$domain) = split /\@/, $recip;
if( $relaydomains{$domain} =~ /^smtp:\[(.+)\]/){
return md_check_against_smtp_server($sender, $recip,
filter.domain.com, $1);
} else {
return (CONTINUE, OK);
}
}
This is causes mimedefang to md_check_against_smtp_server() on all relay
domains listed in your mailertable.
schu
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Anti-virus software
[EMAIL PROTECTED] wrote: I must add, I have been using ClamAV on a mail gateway for 800 users and have found it very good. Stable while processing around 100,000 messages a month. We use this as a gateway to a Domino system with Symantec on the Domino servers. Not much gets through. We are using File:Scan as well. I cannot say exactly how many users we have, but it's a LOT more than 800. Anyway, I have found that clamd works well, but clamav-milter has given me problems on occasion. The most recent one was when I found that sendmail runs into blocking issues when it is calling clamav-milter over the inet socket, but seems to work perfectly when called over a local unix socket. As far as the updates go, I use slackware and maintain my own packages. So when a new clamav comes out, I run my package builder script on my build host, then copy the package to the production server. Once the package is on the server, it's one command to install, and another to restart clamd. I should also note that running MD and clamd is better for security anyway if you have md_copy_orig_msg_to_work_dir_as_mbox_file() in your filter_begin() before the virus scanner is called. This is because the message will be placed in the temp dir which will allow clamd's scanmail interface to parse the message, even though mimedefang has already broken out the mime parts. The idea is that two different mime parsers read each message which would make it more difficult to hide a virus in some broken mime. schu ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] MimeDefang vs clamav
Richard A Nelson wrote: I'm stumped... What difference does clamdscan see when invoked from clamav-milter and when invoked via mimedefang ? I'm using the stock mimedefang-filter I've just converted from running spamass-milter + clamav-milter to mimedefang (running all three at first, to ensure things are working). I've now removed spamass-milter as the mimedefang integration is workin well in that regard. I've not been able to remove the subsequent clamav-milter as of yet because it is catching viruses that it doesn't when invoked by mimedefang ?!? You need to look at your mimedefang-filter. Mime defang is just a perl program that breaks up email into mime parts and gives you the ability to do things with those mime parts. This is what makes mimedefang so wonderful, it gives you complete control over your email in perl syntax, and without the typical perl performance hit to boot. It's a beautiful thing!!! That said, if it's not doing what you want, then you need to update your mimedefang-filter and tell it what you want to do. Older versions of mimedefang passed each attachment to clamd for scanning, but there are viruses out there that break the MIME rfc so mimedefang doesn't break out the attachment because it isn't valid. Since the scanmail feature in clamd is specifically designed to detect viruses despite broken mime you may want to add: md_copy_orig_msg_to_work_dir_as_mbox_file(); to the top of your filter_begin() rule add enable scanmail in your clamd.conf. This will tell mime-defang to put the complete message in mbox format in the tmp extraction dir before calling clamd which will find the virus in the mbox file. This is really a better solution anyway because you have two mime implementations parsing the message. Another way around this (better?) is to use the action_rebuild() function to tell mimedefang to rebuild each message with a valid mime format. This may cause things to break for clients that do not produce valid mime messages, but it can stop a lot problems. It's your call (as David puts it). HTH, schu ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Can't get surbl to work inside MD but works fine in spamassassin -D
List, su - defang -c /usr/local/scripts/spamassassin -D /data/tmp/SPAM returns: X-Spam-Flag: YES X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on snow X-Spam-Level: ** X-Spam-Status: Yes, hits=10.6 required=5.0 tests=BAYES_99,HTML_50_60, HTML_MESSAGE,SPAMCOP_URI_RBL autolearn=no version=2.63 X-Spam-Report: * 0.1 HTML_MESSAGE BODY: HTML included in message * 5.4 BAYES_99 BODY: Bayesian spam probability is 99 to 100% * [score: 1.] * 0.1 HTML_50_60 BODY: Message is 50% to 60% HTML * 5.0 SPAMCOP_URI_RBL URI's domain appears in spamcop database at sc.surbl.org * [oPw0A.9f9D6QMPJ.mtqRTNgJ.enomypoises.org is] [blacklisted in URI RBL at sc.surbl.org] But inside of mimedefang I get: X-Spam-Score: * X-Spam-Report: 1.894 (5 req) HTML_50_60,HTML_FONT_FACE_BAD,HTML_FONT_FACE_ODD,HTML_IMAGE_ONLY_12,HTML_MESSAGE X-Scanned-By: MIMEDefang 2.43 I can't figure out why the mimedefang config is different, anyone have any ideas? schu ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Can't get surbl to work inside MD but works fine in spamassassin -D
Ole Craig wrote: The SURBL plugin is a network test. What's the value of $SADoNetTests in your MD filter file? If it's 0 (which is the default in the sample filter) then SpamAssassin won't do network tests, including SURBL. Ole Thanks for the reply, Where is the documentation for this? I don't see that var in any of the filters I have nor does it return anything in google. I thought that all of the spamassassin config was in the SA config files. schu ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Can't get surbl to work inside MD but works fine in spamassassin -D
Ole Craig wrote:
Urk. Sorry, that's a local permutation, which I've used for so
long it's second nature. The one you want is $SALocalTestsOnly, which
if set to 1 will *disable* network tests. Set it to 0 (before
calling spam_assassin_init()-compile_now(1), if you have that set in
filter_initialize) and you should be OK.
Ole,
I put the following into my rule:
if ($Features{SpamAssassin}) {
$SALocalTestsOnly = 0;
spam_assassin_init()-compile_now(1) if
defined(spam_assassin_init());
}
And I still don't get any network tests nor to I get any bayes filters,
so I think there is something more wrong here. Any other ideas before I
start digging into the code?
schu
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Wishlist for David
David, Mimedefang is a God send, it is a wonderful product that makes it possible to keep control of your email. Anyway, why don't you put up a amazon wish list of something, while I personally don't have a lot of money to give I want to at least buy you a book, dvd, or something. Thanks, schu ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] OT: Sendmail greet_pause feature
Michael Faurot wrote: In article [EMAIL PROTECTED] you wrote: On a related note, what are people setting their timeout value to for greet_pause? Great, exactly what I was looking for. I currently have my setting at 1 milliseconds. I don't want to go to crazy because of some discussion on the comp.mail.sendmail newsgroup about mail servers with low timeout values. I set mine to 10,000 (10 seconds) as well. Seems to be working okay, but it's only been half a day. I've already seen several log messages indicating it's doing its job. So far, I've seen no evidence to indicate this is hindering transfer of mail from desirable sources. Is their a way to omit this if the smtp client comes from some defined IP space? This would be great but I don't want all of the users complaining that the mail server takes for ever to send email. schu ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] OT: Sendmail greet_pause feature
[EMAIL PROTECTED] wrote: From: Matthew Schumacher [mailto:[EMAIL PROTECTED] Is their a way to omit this if the smtp client comes from some defined IP space? This would be great but I don't want all of the users complaining that the mail server takes for ever to send email. Michael Faurot already answered this in the thread: use access_db entries (from cf/README) GreetPause:my.domain0 GreetPause:example.com 5000 GreetPause:10.1.2 2000 GreetPause:127.0.0.10 Please excuse me as I lower my head in shame :) Next time I'll be a good poster and RTFM. Thanks for answering, schu ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] sendmail 8.13.0
David F. Skoll wrote:
Actually, it's far more than that. You can implement *any* Sendmail map
with a daemon, including mailertable, access_db, virtusertable, ...
That is cool!!!
MIMEDefang will allow you to define a function called filter_map
that will be called with two arguments: A map name, and a key. It can
then do whatever magic it likes to look up the key.
Here's an example I'm working on right now.
Sendmail config file:
V10/Berkeley
Kmysock socket unix:/var/spool/MIMEDefang/map.sock
The MIMEDefang filter:
sub filter_map ($$) {
my($mapname, $key) = @_;
my $ans;
$ans = reverse($key) . - . reverse($mapname);
return (OK, $ans);
}
Results of a sendmail -bt test session:
$ /usr/sbin/sendmail -bt -C ./map.cf
No local mailer defined
ADDRESS TEST MODE (ruleset 3 NOT automatically invoked)
Enter ruleset address
/map mysock Testing 1 2 3
map_lookup: mysock (Testing 1 2 3) returns 3 2 1 gnitseT - kcosym (0)
Isn't that so cool? :-) I defy anyone to write rules with Sendmail's .cf
language that can reverse a string.
Of course, a non-toy example would do something real, like looking things
up in an LDAP directory or a SQL database.
I already have sendmail rules to check valid email addresses against a
supported map backend (LDAP) so this basically a different way to do the
same thing but would be a bit more flexible. I wonder if their are any
performance differences between having mimedefang doing the ldap lookup
instead of sendmail. Have you noticed any performance differences in
your initial testing?
schu
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Poll: Time to drop Trophie support?
David F. Skoll wrote: Hi, all. Is anyone using Trophie with MIMEDefang? It looks like Trend Micro doesn't give out enough docs for the Trophie author to maintain it. If it's a dead end, I will drop Trophie support. Yell if that will hurt you! It seems like clamav is the best solution for mail systems anyway. I'm looking at replacing sohpie with clamav so I have both running right now and I am finding that nothing is getting though clamav to sophie (sophos). schu ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] email wire tap
David F. Skoll wrote:
Both of these can be done by setting up an archive account that uses procmail
or something similar to do the archiving. Then it's just a matter of:
if (the_right_conditions_are_met()) {
add_recipient('[EMAIL PROTECTED]');
}
This was an issue at my place of work so this is how I got around it:
# archive messages from domain.com
:0
* FROM ?? domain.com
{
:0 c
/var/mail/archive/$DATE-$FROM
}
# archive messages to domain.com
:0
* TO ?? domain.com
{
:0 c
/var/mail/archive/$DATE-$TO
}
This will break out the archive to each employee on each day. Every
night I compress and gpg encrypt the messages. If something happens and
someone needs to read some mail they must request the messages from one
person and the public key from another. This keeps people from spying
and insures a legit reason when looking though email.
David, not only is this legal in the US, but after Enron, the courts
encourage it!! Pretty lame if you ask me, but such is life.
schu
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] email wire tap
I call procmail in filter mode to do this because I didn't want to deal with the mailbox locking issues when trying to write email to a file from a mimedefang filter. I also wanted mbox format which procmail does very well. Another reason for this is the mail headers are not absolutely complete when the milter is called so mimedefang doesn't see the last received header which I wanted to capture anyway. If you already use procmail for your LDA then this is trivial, but I am using the cyrus LDA. As far as the performance hit goes, there is some, but I would MUCH rather call procmail than non persistent perl. schu Charles Mount wrote: I have two similar needs for sending mail to an additional recipient. Our security department has asked for the ability to have a copy of all mail to/from a given single address or entire domain sent to a security mailbox in addition to the original destination. The equivalent of an email wire tap I would also like to be able keep a copy of all mail received for a brief period of time to hand sort for initial Bayes training. Has anyone used MimeDefang to do something like this? This is a PRIVATE message. If you are not the intended recipient, please delete without copying and kindly advise us by e-mail of the mistake in delivery. NOTE: Regardless of content, this e-mail shall not operate to bind CSC to any order or other contract unless pursuant to explicit written agreement or government initiative expressly permitting the use of e-mail for such purpose. ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
