[Mimedefang] mimedefang not seeing clamd-defang
From: Tom H Subject: [Mimedefang] mimedefang not seeing clamd-defang Oh, and don't use Fedora package for mimedefang itself. They have messed up with the AV auto detection routines. Remove it, and install MD from source. It will start working, and it would be easier to upgrade when needed. When using Fedora, I suggest that you install from: clamav - source mimedefang - source spamassassin - source or CPAN. sendmail - yum package perl modules - yum or CPAN (or both). Good Luck Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] upgraded to current fedora 10 clamav packages
From: Tom H Subject: [Mimedefang] upgraded to current fedora 10 clamav packages, not sure about tokens I've recently got a fedora 10 with sendmail, mimedefang and clamav - however I am confused by the references to in the docs for the clamav-server setup. I get an error like so; [r...@vs802 init.d]# service clamd-service start Starting clamd.clamd_service: /bin/bash: clamd.clamd_service: command not found Is there a howto for mimedefang/clamav that refers to the current version of clamav that I should be using? HI. The Fedora clamav packages are really quite confusing. You can use the config files which I used in past versions (Fedora 6 - 8) but will probably be similar in F10. Here are some config files and script for install (you might need to tweak it up a bit): http://yizhar.mvps.org/temp/clamav-fedora-package.zip However, I recommend that you uninstall the clamav packages, and install it from source. For 2 reasons: 1. Main reason = Fedora do not update packages for new clamav versions, so you will stay behind with an old version in a short time. 2. In addition, it is simpler and easier to get the source packages install working and cooperating with MD. To do that, you can use the following config and script files (again, with few tweaks if needed): http://yizhar.mvps.org/temp/clamav-source-config.zip Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: MD-Quarantine and received by headers
Yizhar Hurwitz wrote: One thing that I found, is that all the "received by" headers are stripped out and not stored in the quarantine directory. But I would like to have that information for diagnostics. Me again - same issue, more details. After further investigation of quarantine messages, I now I understand that only the last "Received:" header is missing, i.e. the topmost Received line that is added by my own server. This is probably because sendmail adds the topmost received line only after MD finishes working on it. So I would still like to know if/how I can preserve this information in the quarantine directory, and try to do it efficiently as well by minimizing system calls from my custom filter. Currently I'm going to try the following in "mimedefang-filter": action_quarantine_entire_message( "probable_spam\nRelayAddr=$RelayAddr RelayHost=$RelayHostName Helo=$Helo\n$report\n" ); This will store the relay information in the file "MSG.0" What do you think? Is there a better way? Maybe a sendmail macro that I can use? Thanks, Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: MD-Quarantine and received by headers
Yizhar Hurwitz wrote: One thing that I found, is that all the "received by" headers are stripped out and not stored in the quarantine directory. But I would like to have that information for diagnostics. Oops... I was wrong, it does have the information. Maybe the specific message that I was testing was submitted from the server itself and didn't have additional "Received" headers. Please ignore my previos post... Yizhar ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] MD-Quarantine and received by headers
HI. Currently I'm using a spamdrop mailbox for probable spam, and I'm testing the alternative of using "action_quarantine_entire_message" instead. One thing that I found, is that all the "received by" headers are stripped out and not stored in the quarantine directory. But I would like to have that information for diagnostics. This is different from messages that are kept in spamdrop, where I can review that info. I understand that this is by design, and that if I later un-quarantine the message, the original "received by" information is irrelevant. So, I would like to ask: Can/How this information be stored in the quarantine as well, for example in an additional file? Thanks Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Spam filtered twice
HI. Some important information is still missing, so I will try to complete the picture by reading between the lines. Please correct me if I get anything wrong... > I have a Linux box which is used as a web server and mail server. It > is directly on the web and it serves roughly 60 different domains for > web and mail. So I guess that you probably: have webmail service for users to read+send mail. let users get mail via pop3 and/or imap. allow users to send outgoing mail (relay) from their clients, via your smtp server, probably using SMTP AUTH. Please confirm, am I right? > I am using clamav 0.88.4, Spamassassin 3.1.4, Mimedefang 2.57 and > sendmail 8.13.7. All on a slackware 2.4.32 Unrelated to the issue - I think you should try to keep up at least with latest clamav version. > The mimedefang-filter is quite standard as far as setup, nothing > really special. > Local mail is delivered through the standard procmail setup, and the > Spam is delivered to the spam box by one of the recipes. That works So procmail is looking for the X-Spam-Score header to process the incoming mail. > fine. But for outgoing mail, procmail is not used, so I need a way to > filter the outgoing mail and put it in a specific folder if it's > declared as spam. OK, let's see what we can do, or give you some tips and ides: * You should have a definition of what is considered by you as unacceptable spam. If one of your users sends a monthly newsletter to 200 customers, how do you define if it is spam or not? This policy should be delivered to your users - they should better know what is wrong and what is right, or at least you can feel that you did a good effort to let them know... * I would try to minimize end-users use of your smtp server for sending outgoing mail. It is best to instruct them to use their local ISP SMTP servers for that, unless they have a reason not to, or unless they are using your webmail. * Your users are assumed innocent by default, unless proven otherwise. Which means - you can allow them to send whatever they send, and you can decide that you only monitor that, so that if MD finds an outgoing message with a spam score higher then, let say 8, you will get a notification. How to do that? You can modify mimedefang-filter, so that if any mail comes from SMTP AUTH user, or from 127.0.0.1, and gets a spam score higher then X, it will write something to the logfile (using md_graphdefang_log function), and if you like you can use md_quarantine_entire_message in such case for diagnostic and further investigation. You can parse the logs (/var/log/maillog) on a daily basis looking the information that MD is writing to it. (And as mentioned above, you can configure MD to write whatever you think to those logs). You can scan the MD-Quarantine folder on a daily basis, if you have decided to use that method. * You can instruct MD to reject high scoring mail during SMTP session (either incoming or outgoing mail), this is one of the special things that you can do with MD because it scans during the delivery (it is a milter). This is done with: action_bounce... * You should read: man mimedefang-filter and also practice your perl skills. Using MD without basic perl knowledge is like going to a fancy restaurant and ordering bread and butter for 50$. (Well, I couldn't find a better example - other people are invited to give their own) * Mail sent from webmail can be easily identified, because the $RelayAddr is probably 127.0.0.1 . You can use that info in your filter if you wish. Mail submitted via SMTP AUTH can also be identified using sendmail macros. search the list archives and look on MD WIKI pages for more info about that. Mail submitted via regular SMTP should be inbound only, so isn't related to your question. Please note that I'm trying to give you tips and ideas, not exact instructions, so you can pick whatever suites your needs, and ignore the rest. > > I noticed that there is a procedure to dump the mail if it contains a > virus. That seems to work for both incoming and outgoing mail. I need > some You should simply learn perl basics, with patients and practice on regular scripts (print "hello world", etc), then when you feel confident enough you should start modifying mimedefang-filter to fit your own custom needs. Start here: perldoc.perl.org Then you will see that it is quite simple and powerful. BTW, did you read - The MIMEDefang HOWTO: http://www.mickeyhill.com/mimedefang-howto/ Good luck Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: On pinheaded ISP's that insist on a copy of Spam
HI.
> So I'd like to still make the remote end think that the message was
rejected
> (on the unlikely possibility that this will cause them to delete this
address from their database and go away permanently... probably just a
pipe-dream...)...
> but at the same time, move the message into another mailbox where I
can then forward it (along with a complaint) to the appropriate ISP.
One way to do it is quite simple:
add the command:
action_quarantine_entire_message('bla..bla');
Or maybe:
action_quarantine_entire_message($msgid);
In "mimedefang-filter", at filter_end, just before you have something like:
action_bounce('bye bye');
But cation - this will cause additional load on your server and consume
some disk space and file system directory entries,
so you might wish to quarantine only messages from specific ip
addresses, and/or run a scheduled job to clean old items from the
quarantine.
Yizhar Hurwitz
http://yizhar.mvps.org
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Spam through trusted mx relay
HI. From: "David Koski" <[EMAIL PROTECTED]> Subject: [Mimedefang] Spam through trusted mx relay I was wondering if anyone had any thoughts if there is a better way to deal with spam through a trusted mx and forwarded to my local mail server. Please read: man Mail::SpamAssassin::Conf and search for: trusted_networks internal_networks You should set them in your sa-mimedefang.cf file, and thus spamassassin running on your MD server will better know how to handle email from the upstream servers. It would do better job by searching RBL against the ip address of the originating smtp client, instead of the ip of your upstream mail server. Take a look here: [Mimedefang] md is not the first relay: http://lists.roaringpenguin.com/pipermail/mimedefang/2006-December/031437.html http://lists.roaringpenguin.com/pipermail/mimedefang/2006-December/031399.html Thanks to /Jan-Pieter Cornet /who gave me this tip. Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Spam filtered twice
HI.
From: Pierre Forget <[EMAIL PROTECTED]>
When my mail server receives an email which has already been filtered
by Mimedefang on another server (and considered Spam), it deletes the
X-Spam-Score line which have been installed by the other server. So,
the email goes through even though it's Spam.
Well, if you wish to keep the first header, you can do the following in
your mimedefang filter:
Use this:
action_add_header("X-Spam-Score", "$hits ($score) $names");
Instead of this:
action_change_header("X-Spam-Score", "$hits ($score) $names");
But anyway - you should normally trust and use your whatever score your
own server is calculating for the message,
and use that info to decide what to do with the message.
If you wish to filter spam on an upstream mail server - well, you should
simply change the configuration of that server to do so and so with
detected spam.
Do you get my point?
Which brings another question: how do I make sure that an outgoing
email from my server doesn't go out if Mimedefang considered it as Spam?
If the email is filterred by mimedefang, you can do whatever you decide
with it, optionally the same decisions and actions that apply to
incoming mail.
It is up to you to decide, and then implement it in your filter.
If you can provide more information and background about your question,
maybe we can better help you.
Please note that we don't know what you currently have in your own
custom mimedefang-filter file,
nor we cannot guess how your MD box fit in the whole picture
(is it a mail relay in front of other mail server, is it on the mail
server itself, are you an ISP or SMB etc).
Yizhar Hurwitz
http://yizhar.mvps.org
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: compare mimedefang to mailscanner
HI. John Rudd <[EMAIL PROTECTED]> wrote on 01/17/2007 07:11:51 PM: Dropping without notifying _anyone_ is "an even worse practice". You don't have to notify the sender, as long as you notify the recipient (and visa versa). Which is just another piece of annoying email in the inbox. Why bother removing the spam if your just going to deliver a message held email in its place? Here is my approach (I guess other implementations are similar): Known Virus = discard silently. Bad filename (or unknown virus) = replace the attachment with a warning. The recipient gets the message without the attachment. High score spam (score >10) = Reject message. Probable spam (5 < score < 10) = Quarantine the message in a spamdrop. However a daily report is sent to the end user, listing all the quarantined messages with information such as sender+subject. Other mail = let it through. So, if a user is receiving 100 spam messages, 90% of them are normally blocked as high score spam, and 10 "probable spam" go to the spamdrop. The user will get a day after only 1 email message with a short list of the 10 probable spam message, so he can look for false positive. That is 1 message per day for about 100 spam (10 probable spam) messages. Most spam is filtered, but in case of false positive either the sender or recipient has a chance to know about it. I think that this is a good trade-off for the end users and the sysadmin. Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: md_check_against_smtp_server question
HI. > When I tested I got smtp failure code 5.x.x for all "unkown user" messages, > which is a good sign, but bounce messages still got generated on my > mimedefang_server. How do I discard these "unkown user" messages? Maybe those messages are generated for older messages that were in your mail queue already, before you started using md_check_against_smtp. If this is correct, then cleaning your mail queue and/or waiting few days will solve the problem. If I'm wrong, please tell us... In addition, you can look at my implementation and reuse parts of the code: [Mimedefang] My semi-cached version of md_check_against_smtp_server http://lists.roaringpenguin.com/pipermail/mimedefang/2006-December/031463.html Take a look here as well: http://www.mimedefang.org/kwiki/index.cgi?RelayCheckAddresses And you can also consider other alternatives, such as: http://www.mimedefang.org/kwiki/index.cgi?Exchange2Access Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: OT: New Attack/Poor SPAMming programming?
HI. Here is a great article about sendmail time-outs (and other antispam tricks such as greet pause): http://www.acme.com/mail_filtering/sendmail_config_frameset.html Highly recommended for any sendmail admin. Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: replace_with_url with multiple small
d modified section. ###
return 1;
}
##
sub filter {
my($entity, $fname, $ext, $type) = @_;
my $FileSize;
my $detach_msg;
# Other code ...
$FileSize = (stat($entity->bodyhandle->path))[7];
if (($DetachBigEnable and ($FileSize >= $DetachBigSizeMB*100))
or ($DetachMultimediaEnable and ($ext =~ /^\.($DetachMultimediaExt)$/i))) {
md_graphdefang_log('detached', $fname, $FileSize);
$detach_msg = $DetachText;
$detach_msg =~ s/_FILENAME_/$fname/g;
$detach_msg =~ s/_FILESIZE_/$FileSize/g;
return custom_action_replace_with_url($entity, $DetachPath,
$DetachURL, $detach_msg, $fname);
}
# Other code ...
}
#***
# %PROCEDURE: list_detached_files
#***
sub list_detached_files ($) {
my($entity) = @_;
my $plain = $DetachListTextTop. join("\n", @DetachedFiles).
$DetachListTextBot;
my $html = $plain;
$html =~ s|(http://\S*)|$1|g;
$html =~ s/\n/\n/g;
append_text_boilerplate($entity, $plain, 0);
append_html_boilerplate($entity, $html, 0);
}
##
sub filter_end {
my($entity) = @_;
# No sense doing any extra work
return if message_rejected();
if (@DetachedFiles > 0) {
&list_detached_files($entity);
}
# Other code ...
}
=== mimedefang-filter
Comments are welcome.
BTW - Sample CGI scripts for downloading the files are welcome also,
as currently my users get the SHA1 filename instead of the original one.
I guess that it is easy but didn't play with it yet.
Yizhar Hurwitz
http://yizhar.mvps.org
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] dccproc of dccifd?
HI Ken. From: "Ken Menzel" <[EMAIL PROTECTED]> More specifically http://spamassassin.apache.org/full/3.1.x/doc/Mail_SpamAssassin_Plugin_DCC.html My config includes in init.pre # DCC - perform dcc check # loadplugin Mail::SpamAssassin::Plugin::DCC and local.cf includes use_dcc 1 dcc_timeout 10 dcc_path /usr/local/bin/dccproc dcc_home /usr/local/dcc dcc_dccifd_path /usr/local/dcc/dccifd Your comments were very helpful, especially the one about init.pre. It seems to work fine now. Few more tips for other who might look for the it: To run dccifd as a "service" on ntsysv systems: 1. edit /var/dcc/dcc_conf and enable dccifd. 2. copy /var/dcc/libexec/rcDCC to /etc/init.d 3. chkconfig rcDCC on 4. chkconfig rcDCC start Then check if you see the socket: /var/dcc/dccifd Thank you Yizhar Hurwitz ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] My semi-cached version of md_check_against_smtp_server
From: alan premselaar <[EMAIL PROTECTED]> I have some comments that hopefully you'll find useful. Thank you for your time and attention. Firstly, I would probably check_against_smtp_server before checking the cache, because you don't have any housecleaning code to handle the case where an account was deleted within the 30 days of the last cache store. No, the whole point of this is to avoid the overhead of establishing an SMTP session for each recipient. I don't need the housecleaning code, because I expire the cache after X days. Maybe 30 days is a big high, so I might lower the timeout. This could cause your system to potentially accept mail for an unknown user and thus have to generate an NDS and defeat the entire purpose of this feature. This is OK for me and "by design". Because I will need to send NDR only to a very small bunch of messages that are: sent to recipients that was valid not long ago, but is disabled now. passed virus and spam checks. So this leaves me with only few messages that are mostly legitimic and from real sender. Secondly, I'm assuming that you just haven't gotten around to writing the code to check the mailertable for the relay host. however, it doesn't appear that you have a contingency plan for if the host does not appear in the mailertable. In all of my "mail relay" installations the destination host is configurred in mailertable. I have some installation were the MD machine is also the mailbox server, but for such sites I don't need to use md_check_against_smtp_server at all, and therefor the main switch $CheckRecipientEnable will be set to zero 0. (i.e. what if all or some of the forwarding is handled by the virtusertable? what if it's aliases?) In that case I catch it with this line: if ($CheckRecipientEnable and ($rcpt_mailer ne 'local')) ... and therefor skip the check (it will be checked against the local users of sendmail). you may want to consider an assignable override variable as well so that way an administrator could give it a fixed value should they choose and still keep the code fairly portable. What exactly do you mean? I have this variable: my $CheckRecipientEnable = 1; It is actualy not in /etc/mail/mimedefang-filter, but in a separate config file that I use, which is called /etc/mail/mdf-config. This is where I store the site-specific configuration, such as spamdrop address, high score spam handling, admin name and address, etc... also, there is no guarantee that even if the hostname is configured in the mailertable that it will be enclosed in square brackets [] ... as, the absense of the brackets just tells sendmail to actually do a DNS lookup on the hostname whereas the brackets tell it NOT to do the lookup. You are correct, and there is also no guarantee that the destination server is configurred to reject un-known recipients, as by default MS Exchange servers will accept mail to bad recipient unless it is manually configured. This is up to me when I install and configure a mail relay system, to make sure that I configure both the destination server and the MD machine to do what I expect it to. By default, the check is disabled $CheckRecipientEnable = 0; and also in /etc/sysconfig/mimedefang I enable the recipient filter_recipient only for sites that will actually use it. Again, thank you for your comments. Yizhar Hurwitz ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] dccproc of dccifd?
Date: Tue, 5 Dec 2006 13:03:15 -0800 From: Kelsey Cummings <[EMAIL PROTECTED]> You really want to use dccifd. It is much faster. Can you help me find the exact "how-to" instructions? I did try to read "man dccifd" and "man Mail::SpamAssassin::Conf", but didn't find the exact instructions how to do it, what to put in sa-mimedefang.cf, etc. Thanks, Yizhar Hurwitz ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] My semi-cached version of md_check_against_smtp_server
HI.
Here is my cached implementation of md_check_against_smtp_server.
I publish it here for other to look at, and for tips on improving it.
General design goals and thoughts:
* I know about the option of using LDAP or other methods to get the
valid recipients list,
and I currently use exchange2access.pl on some sites,
but for some other sites I prefer to use md_check_against_smtp_server.
* Make it lite, simple and portable to different sites.
* No use of external DB software (sql server), unless I decide that I
really need it.
* Cache only positive response for valid recipients (CONTINUE).
This will give me the major benefits of the md_check_function,
the mail relay will accept mail for cached valid recipient even when
backend mail server is down.
When a new mailbox/address is configured on the mailbox server, it will
be available immediately.
* Use a combination of disk cache for writing changes, and ram cache for
reading them.
This will avoid possible locking issues or race conditions, because I
will very rarely write to the disk cache.
I'm taking advantage of the assumptions that most "filter_recipients"
calls in MD 2.57 and above would run on the save slave(s)
I think that I can afford storing the whole cache in RAM, because:
It is for use on small sites, with maximum of 200 recipients.
I cache only valid recipients.
I run only 2-6 slaves on each MD machine.
So if I assume that each email address will consume less then 50 bytes
of RAM, and I have less then 200 recipients,
I get less then 10kb spent RAM per slave, or did I miss anything?
* I'm currently using an SDBM file for the disk cache,
I really don't know much about all those different dbm file formats,
so if you can tell me about a better format I can try it.
* I've used the command: "tie .. or die ", so I can see if
something goes wrong.
Maybe I should change it to be more tolerant, but anyway it seems to
work fine with no errors so far.
* Take a look at how I get the destination server address to check
against, from $rcpt_host.
This makes the code portable from site to site, without the need to
manually specify the server.
What do you think? So far it seems to work for me.
It is designed for servers which are mail relay of incoming mail only.
OK, time for the real thing (relevant parts from
/etc/mail/mimedefang-filter):
# On/Off switch...
$CheckRecipientEnable = 1;
### Used for valid recipients cache:
use Fcntl;
use SDBM_File;
### Valid Recipients Cache:
my $vrc_filename = '/home/defang/vrc-sdbm';
my %vrc_disk;
my %vrc_ram;
sub filter_initialize {
if ($CheckRecipientEnable) {
my $valid_timestamp = time - 86400*30; ### Currently I'm caching
valid recipients for 30 days.
tie (%vrc_disk, 'SDBM_File', $vrc_filename, O_RDONLY|O_CREAT, 0666) or
die "Cannot tie VRC file, $!";
while (($key,$val) = each %vrc_disk) {
$vrc_ram{$key} = 1 if ($val >= $valid_timestamp);
}
untie (%vrc_disk);
}
}
sub filter_recipient
{
my($recip, $sender, $ip, $host, $first, $helo, $rcpt_mailer,
$rcpt_host, $rcpt_addr) = @_;
if ($CheckRecipientEnable and ($rcpt_mailer ne 'local')) {
if ($vrc_ram{$recip}) {
return ('CONTINUE', 'OK');
}
else {
### Check if $rcpt_host is in mailertable by looking for square
brackets []:
if ($rcpt_host =~ /^\[(.*)\]$/) {
my ($stat,$msg,$code) = md_check_against_smtp_server($sender,
$recip, $HostName, $1);
if ($stat eq 'CONTINUE') {
$vrc_ram{$recip} = 1;
tie (%vrc_disk, 'SDBM_File', $vrc_filename, O_RDWR, 0666) or die
"Cannot tie $vrc_file for write, $!";
$vrc_disk{$recip} = time;
untie (%vrc_disk);
}
return ($stat, $msg, $code);
}
}
}
return ('CONTINUE', 'OK');
}
Comments are welcome.
Yizhar Hurwitz.
http://yizhar.mvps.org
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Slave 0 stderr: ignoring text in character set WINDOWS-1255
HI.
I see many of the following errors in /var/log/maillog, on several
different MD machines:
Dec 7 19:53:11 mail mimedefang-multiplexor[1917]: Slave 0 stderr:
ignoring text in character set `WINDOWS-1255' at
/usr/lib/perl5/vendor_perl/5.8.8/MIME/Parser/Filer.pm line 660
Dec 7 19:53:11 mail mimedefang-multiplexor[1917]: Slave 0 stderr:
ignoring text in character set `WINDOWS-1255' at
/usr/lib/perl5/vendor_perl/5.8.8/MIME/Parser/Filer.pm line 534
All of the machines are in Israel, so WINDOWS-1255 (Hebrew) is expected
for regular messages,
and also for file names attached to them.
As far as I assume/guess/investigated/understand, it is related to
filenames in Hebrew, because I found the following:
/usr/lib/perl5/vendor_perl/5.8.8/MIME/Parser/Filer.pm line 534:
my $recommended = unmime $head->recommended_filename;
/usr/lib/perl5/vendor_perl/5.8.8/MIME/Parser/Filer.pm line 660:
my $fname = unmime $head->recommended_filename;
I then looked at the file which implements the "unmime" function
/usr/lib/perl5/vendor_perl/5.8.8/MIME/WordDecoder.pm
and found this at line #100:
### Standard handlers.
my %Handler =
(
KEEP => sub {$_[0]},
IGNORE => sub {''},
WARN => sub { carp "ignoring text in character set `$_[1]'\n" },
DIE=> sub { croak "can't handle text in character set `$_[1]'\n" },
);
Now, I understand that this is just an annoying message, but I would
like to avoid it,
to make my life easier when I "grep" the log for more important errors.
Or maybe it is important?
I also found this in Google:
http://lists.roaringpenguin.com/pipermail/mimedefang/2003-March/013629.html
On all machines that I see this message I have recent (probably latest)
versions of MD and MIME::Tools.
I did try to reproduce the error by sending an attachment with Hebrew
name using Thunderbird+MS Outlook Express,
but I didn't see the error.
I didn't try to reproduce with MS Outlook yet.
Can you help me troubleshoot that error?
Thanks
Yizhar Hurwitz
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] md is not the first relay
HI. > I am now using it and is seems to do the job. > I have also added the upstream mail relay to "internal_networks" for > example: > > trusted_networks a.b.c.d > internal_networks a.b.c.d > > This is what I understood from "man Mail::SpamAssassin::Conf" which is a bit > confusing, for me at least. You don't have to explicitly set internal_networks if it's the same as trusted_networks. internal_networks is supposed to be all of your MX hosts. trusted_networks may contain more than your MX hosts, if there are other hosts that you trust not to forge headers (eg: other mailservers you control, or that regularly forward mail to you, operated by trusted third parties). Does that make it clear? No, it is still confusing. This is what I read in "man Mail::SpamAssassin::Conf": trusted_networks ip.add.re.ss[/mask] ... (some text skipped)... MXes for your domain(s) and internal relays should also be speci- fied using the "internal_networks" setting. When there are trusted hosts that are not MXes or internal relays for your domain(s) they should only be specified in "trusted_networks". And this: internal_networks ip.add.re.ss[/mask] ... (default: none) What networks or hosts are internal in your setup. Internal means that relay hosts on these networks are considered to be MXes for your domain(s), or internal relays. This uses the same format as "trusted_networks", above. This value is used when checking dial-up or dynamic IP address blocklists, in order to detect direct-to-MX spamming. Trusted relays that accept mail directly from dial-up connections should not be listed in "internal_networks". List them only in "trusted_networks". So, as far as I understand from the above: The general rule is: an MX server should be listed in "trusted_networks" and also in "internatl_networks" An exception rule is: but if the MX server is also accepting direct connections from client (for example an ISP outgoing mail server), then it should be listed only in "trusted_networks". So in my case the general rule applies, because the MX server is used only for incoming mail, it is an ISP server dedicated for that purpose (as far as I know). dial up and home users of the ISP use a different server for sending their outbound mail. Am I correct? Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] dccproc of dccifd?
HI. I have several MD servers that I plan to install DCC on, and I would like to know what is the best way to do it. When using DCC via MimeDefang + SpamAssassin, is it better to use dccproc or dccifd? Any implementation tips regarding MD+SA+DCC would also help I have read the dcc faqs in dcc-servers.net site and spamassassin wiki, and I'm looking for those tips which are more specific to MD. Currently I have this in sa-mimedefang.cf: dcc timeout 8 dcc_home /var/dcc dcc_path /usr/local/bin/dccproc use_dcc 1 Do I need all of them? Did I miss something? I have also done: cd /var/dcc chown defang * Was that needed? Thanks, Yizhar Hurwitz ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] two md_check_against_smtp_server questions
HI.
> From: John Rudd <[EMAIL PROTECTED]>
> 2) Has anyone set up a means of caching results? I don't want to hit my
> back-line servers constantly with these requests. I would prefer to
> have results cached for, say, 2 hours. I'm trying to think of a good
> way to do this.
You can take a look here:
Ray Ferguson's fancy version:
http://www.mimedefang.org/kwiki/index.cgi?RecipientCheckBDBCache
But his version is quite complex and difficult to follow (at least for me).
Also - there is an important bug in the above version,
look at the lines (from Ray :
if ( $stat == "REJECT" ...
} elsif ( $stat == "CONTINUE" ...
The above is wrong because he is using numeric "==" for string comprasion,
and will produce the wrong results.
I will send him (Ray) an email about this off the list also.
I am in a similar boat, and have just started playing with my own cached
md_check_against_smtp version.
However my situation is different - I need it for a relativly low traffic
system, between 100-200 recipients, and no more then 6 concurent MD slaves.
Therefor I think that I can afford spending RAM and hold the recipients cache in
a simple perl hash in RAM of each slave, while I write updates to a file on disk.
I currently cache only positive results (CONTINUE, OK).
I have just started using something last week, and am still working on it.
I have not yet added timestamps to the cache and plan to add it later,
so the code below will cache positive response forever -
but this will be changed and fixed.
This is what I Currently have in mimedefang-filter:
~~
~~
### Valid Recipients Cache:
my $vrc_file= '/home/defang/vrc-sdbm';
my %vrc_disk;
my %vrc_ram;
#***
# %PROCEDURE: filter_initialize
#***
sub filter_initialize {
if ($CheckRecipientEnable) {
if (-e $vrc_file) {
### Load Valid Recipient Cache from file to ram:
tie (%vrc_disk, 'SDBM_File', $vrc_file, O_RDONLY, 0666) or die "Cannot
create VRC file, $_";
foreach (keys %vrc_disk) {
$vrc_ram{$_} = 1;
}
untie (%vrc_disk);
}
else {
### Create a new empty disk cache:
tie (%vrc_disk, 'SDBM_File', $vrc_file, O_RDWR|O_CREAT, 0666) or die "Cannot
create VRC file, $_";
untie (%vrc_disk);
}
}
}
#***
# %PROCEDURE: filter_recipient
#***
sub filter_recipient
{
my($recip, $sender, $ip, $host, $first, $helo, $rcpt_mailer, $rcpt_host,
$rcpt_addr) = @_;
if ($CheckRecipientEnable) {
if ($vrc_ram{$recip}) {
return ('CONTINUE', 'OK');
}
else {
my ($stat,$msg,$code) = md_check_against_smtp_server($sender, $recip,
$HostName, $CheckRecipientServer);
if ($stat eq 'CONTINUE') {
$vrc_ram{$recip} = 1;
tie (%vrc_disk, 'SDBM_File', $vrc_file, O_RDWR, 0666) or die "Cannot tie
$vrc_file for write, $_";
$vrc_disk{$recip} = 1;
untie (%vrc_disk);
}
return ($stat, $msg, $code);
}
}
return ('CONTINUE', 'OK');
}
~~
~~
I would like to add my own questions to the list about the same issue:
How efficient is a simple perl hash when accessing it?
i.e. when I write:
if ($vrc_ram{$recip}) .
So assuming that I have 100 keys (recipients) in the hash,
Does perl need to go over all the 100 keys in the hash,
or does it do some magic tricks and find it more efficiently?
Which db file type is best for storing and accessing such a cache:
for example:
DB_File
NDBM_File
SDBM_File
GDBM_File
I really don't understand the actual differences between them,
although I've read their man pages (but didn't googled yet)...
Is a disk cache using "tie" with one of the above database formats,
more efficient then a regular perl hash in RAM?
I do preffer to use a simple database without additional database software like
sql server which seems an overkill for my needs.
(I currently don't use db for other things in the filter).
Any comments are welcome...
Yizhar Hurwitz
http://yizhar.mvps.org
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Test message to the list .
HI. This is a test message. Yizhar ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re:GraphDefang question
HI.
From: Manoj Srivastava <[EMAIL PROTECTED]>
I now have graphdefang set up as a cron job generating pretty
graphs of incoming Spam. The very first graph is of Spam, Probable
Spam, Viruses, and Mail In -- and the Probable Spam number is always
0.
So I am not recording some event -- what exactly is
graphdefang looking for? Is it affected by the two optional
parameters in md_graphdefang_log?
Graphdefang simply looks for what you write to the logs.
It is all up to you to decide how to categorize your mail traffic,
and implement it in mimedefang-filter.
For example, below is the relevant code from my mimedefang-filter,
which is categorizing email traffic to 3 major types:
mail_in = spamscore below 5 (or no spam check).
This is passed on to the end user mailbox.
probable_spam = score between 5 to 10.
This is hold in a spamdrop folder,
and I send daily reports via cron job to the user with summary of probable_spam.
spam = score above 10.
This is rejected (or discarded) without any notification to the recipient.
You can see how I use md_graphdefang_log in the following code:
sub filter_end {
...
# Spam checks if SpamAssassin is installed
if (($Features{"SpamAssassin"}) and ($RelayAddr !~ /$SafeRelay/ )) {
if (-s "./INPUTMSG" < 100*1024) {
my($hits, $req, $names, $report) = spam_assassin_check();
my($score);
if ($hits < 10) {
$score = "*" x int($hits);
} else {
$score = "*" x 10;
}
action_add_header("X-Spam-Score", "$hits ($score) $names");
if ($hits >= $req) {
if ($hits >= $HighScore) {
md_graphdefang_log('spam', $hits, $RelayAddr);
if ($AutoRejectEnable) {
action_bounce('Message Rejected.');
return;
}
}
else {
md_graphdefang_log('probable_spam', $hits, $RelayAddr);
}
action_add_part($entity, "text/plain", "-suggest", "$report\n",
"SpamAssassinReport.txt", "inline");
if ($SpamDropEnable) {
action_add_header("X-Orig-Rcpts", join(", ", @Recipients));
foreach $recip (@Recipients) { delete_recipient($recip); }
add_recipient($SpamDropAddres);
}
if ($TagSubjectEnable) {
action_change_header('Subject', $TagSubjectText. $Subject);
}
return;
} else {
md_graphdefang_log('mail_in', $hits, $RelayAddr);
}
}
}
else {
### mail was not checked for spam:
md_graphdefang_log('mail_in', undef, $RelayAddr);
}
...
Yizhar Hurwitz
http://yizhar.mvps.org
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] md is not the first relay
HI. I am reposting this message, because it didn't get through before, so I'm sorry if you get it twice by accident. internet ==> mailrelay1 ==> my mimedefang server ==> internal mail server. How should I configure MD/SpamAssassin to use the HELO information from the top (or second?) "Received by" header? = From: Kees Theunissen <[EMAIL PROTECTED]> Does the section "PRESERVING RELAY INFORMATION" from the mimedefang-filter(5) manpage help you? No, it doesn't fit my situation because the upstream mail relay does not run mimedefang and is not under my control. = From: Jan-Pieter Cornet <[EMAIL PROTECTED]> Add the IP address of mailrelay1 (as seen by your mimedefang server) to the trusted_networks config in spamassassin. See: man Mail::SpamAssassin::Conf Yes, I think that this is exactly what I need. I did read about it before posting but didn't understand that it is what I'm looking for... Just to clarify again - does it mean that spamassassin on the mimedefang server will now start checking DNSBL against the ip of the relay that originated the email and sent it to mailrelay1 (This is what I want)? I am now using it and is seems to do the job. I have also added the upstream mail relay to "internal_networks" for example: trusted_networks a.b.c.d internal_networks a.b.c.d This is what I understood from "man Mail::SpamAssassin::Conf" which is a bit confusing, for me at least. Hartstikke bedankt. Yizhar Hurwitz, http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] md is not the first relay
HI. I have an MD installation that is behind another mail relay which I am not administaring. As far as I understand, in that scenario, some DNSBL checks do not work, for example checking if the email came directly from a dial-up connection, as well as some HELO checks. The configuration is: internet ==> mailrelay1 ==> my mimedefang server ==> internal mail server. How should I configure MD/SpamAssassin to use the HELO information from the top (or second?) "Received by" header? Thanks Yizhar Hurwitz ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] HOME=/var/spool/MIMEDefang
HI. I have several MD 2.57 installations, which I plan to upgrade to MD 2.58. All installs are on redhat systems (RHEL4, FC4, FC5, FC6). I would like to better understand the exact meaning of the new 2 lines in the suggested /etc/init.d/mimedefang file: HOME=/var/spool/MIMEDefang export HOME I know about the problem with spamassassin on redhat discussed in this group (the errors in /var/log/maillog), and I have also experienced it myself on some of these servers. My bayes and awl databased are stored at: /home/defang/.spamassassin In /etc/mail/sa-mimedefang.cf I have: auto_whitelist_path /home/defang/.spamassassin/auto-whitelist bayes_path /home/defang/.spamassassin/bayes BTW - On most of the servers, /var/spool/MIMEDefang is a ram drive. So my question is - should I use the new suggested /etc/init.d/mimedefang file "as is", or change it to: HOME=/home/defang (or /home/defang/.spamassassin) export HOME I simply don't understand what is the exact effect of this settings - is it used for temporary lock files and such, or is it also used for storing SA databases such as bayes and awl? Thanks Yizhar Hurwitz ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Help with white listing
Date: Mon, 13 Nov 2006 10:25:32 -0700 From: "Ashley M. Kirchner" <[EMAIL PROTECTED]> Subject: [Mimedefang] Help with white listing person who placed the order. The only thing I have to go by is that the messages get relayed through their server. So really I need to white list their relay server and I don't know if that's even possible. HI I think that you can try the following: man Mail::SpamAssassin::Conf Then search for: whitelist_from_rcvd For example (correct me if I'm wrong): whitelist_from_rcvd [EMAIL PROTECTED] mailserver.domain.name Yizhar ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] maintaining state - per message custom variables
HI.
I would like to start using my custom variables in mimedefang-filter.
I have read "man mimedefang-filter => MAINTAINING STATE" section,
and I would like to ask your confirmation if I got it right.
I plan to use a custom array, that I will:
1. clear in "filter_begin".
2. Optionally write something to it in "filter".
3. Check in "filter_end" if it is not empty, and do something.
As far as I understand, this is OK because all those functions will run
by the same process - is it right?
What I plan to do is my custom replacement to "action_replace_with_url",
which will do something like that:
filter_begin:
===
@detached_files = ();
sub filter:
==
if (..) {
# copy the file to someplace...
push (@detached_files, 'link to the file');
}
filter:
if (@detached_files > 0) {
# Use append_text_boilerplate + append_html_boilerplate to add links
to the detached files.
}
I need the links to files that I detach, to be visible *and clickable*
in MS Outlook 2003 clients,
and I have tested both "action_replace_with_url" and
"action_drop_with_warning" to find that they don't produce a clickable
inline link,
so I'm trying to do it my own way.
Did I understand correctly the "maintaining state" section, and does my
sample code look OK?
Thanks,
Yizhar Hurwitz
http://yizhar.mvps.org
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Separate config file for mimedefang-filter
HI. An update to my previous post about using separate file to store configuration for mimedefang-filter. I plan to do the following: Add the following line to the start of my custom standard mimedefang-filter file: require '/etc/mail/mdf-config'; And put my per-host configuration info in the /etc/mail/mdf-config file, for example: # mdf-config $SALocalTestsOnly = 0; $AdminName = 'local admin name'; # etc... 1; The goal is to create a custom standard mimedefang-filter file that will work on several different hosts, and put host/site specific parameters in the /etc/mail/mdf-config file. Does it look reasonable? Any catch or tips? Thanks, Yizhar Hurwitz ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Separate config file for mimedefang-filter
HI. I am managing several different MD installations. All of them are on Fedora/RHEL based systems. MD is installed from source, and init scripts are based on redhat samples. I would like to create a standard mimedefang-filter which would include my customized settings but be the same on all locations, and to use another config file which would hold any site specific configuration settings, such as: admin name and address. spamdrop enable/disable + spamdrop email address. enable/disable usage of replace_by_url for multimedia files (the actual implementation will be in mimedefang-filter). and some other variables. Again - the goal is a standard mimedefang-filter which would be the same for all installations, and move site specific variables to a different file. This will allow me to easily deploy updated versions of mimedefang-filter. I would like to get your suggestions, which looks like the best way to establish this. Should I create a separate perl file and include it in mimedefang-filter? Should I create a plain text config file, and parse it at the start of mimedefang-filter (outside of the procedures)? Should I use /etc/sysconfig/mimedefang (I guess not, but asking)? Should I use other method? (I don't currently use and don't wish to use db software for that). Note - I have basic perl skills, but certainly I'm not an advanced perl programmer, so any tips, samples or catches warnings about any optional solutions are highly welcome. Any comments (or request for more info) are welcome. Thanks, Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: lost input channel.
Date: Wed, 20 Sep 2006 17:19:05 +0200 From: "J.P van Oyen" <[EMAIL PROTECTED]> Subject: [Mimedefang] lost input channel. In my sendmail logs I see so now an than entries like: Sep 17 20:46:00 www sm-mta[28696]: k8IKiuOO018686: lost input channel from ppp-104-23.telesat.com.co [200.71.104.23] to MTA after mail HI. There are several possibles reasons for that. One of them is a layer 3 network problem - MTU related. One host is using PPPoE or similar connection that has an MTU of 1492. As you can see by the name, ppp-104-23.telesat.com.co It might be the case. The other host (your server) has MTU of 1500. This can cause problems because PMTU doesn't work over some routers firewall that block ICMP . Here is a nice article about this: Path MTU Discovery: http://www.sendmail.org/tips/pathmtu.html If this is the case (or might be the case), possible solutions/workarounds are: * ignore if it is rare and neglect able. * Set your host MTU to 1492 instead of the default 1500. * Disable PMTU on your server. * Investigate further, for example check if you can see anything common to the hosts that cause this. Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Problems Installing Mail::Audit perl module
HI. I am installing a new server following the MIMEDefang howto. http://www.mickeyhill.com/mimedefang-howto/#s5 OS = Fedora Core 6 test3 (this is a test server, clean and minimal install). Perl version 5.8.8 I had similar problems when I installed it on other redhat based systems (RHEL4, FC4, FC5), and as far as I remember I did "force install Mail::Audit" eventually. I would like to understand and solve this problem now. I did try to send email to "simon at cpan.org" (maintainer of Mail::Audit) but got an NDR back. I did not try yet to manually install it without using "cpan". I did successfully install other perl modules needed for MD as mentioned in the HOWTO. Here is the output of the command "cpan Mail::Audit". Manifying blib/man3/Mail::Audit::PGP.3pm /usr/bin/make -- OK Running make test PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'inc', 'blib/lib', 'blib/arch')" t/*.t t/basicNOK 8 # Failed test 'after accept without dest, emergency is maildir' # in t/basic.t at line 70. t/basicok 9/10# Looks like you failed 1 test of 10. t/basicdubious Test returned status 1 (wstat 256, 0x100) DIED. FAILED test 8 Failed 1/10 tests, 90.00% okay t/custom-acceptok t/dan-root.ok t/exit.ok t/ignore...ok t/is_mime..ok t/pod-coverage.skipped all skipped: Test::Pod::Coverage 1.08 required for testing POD coverage t/pod..skipped all skipped: Test::Pod 1.00 required for testing POD t/reject...ok t/shorthandok Failed Test Stat Wstat Total Fail Failed List of Failed --- t/basic.t 1 256101 10.00% 8 2 tests skipped. Failed 1/10 test scripts, 90.00% okay. 1/48 subtests failed, 97.92% okay. make: *** [test_dynamic] Error 255 /usr/bin/make test -- NOT OK Running make install make test had returned bad status, won't install without force My questions are: Can you help me install it properly? Should I use "force install"? Is it safe to use it? Is this module really needed by MIMEDefang? Thanks Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Reviewing SA analysis for quarantined mail
HI.
Does anyone have a script that reproduces the MD invocation of SA on a
quarantined message? What I'd like to do is see the SA report on false
positives.
When you quarantine a message in "mimedefang-filter", you can supply
additional info.
Examples:
action_quarantine_entire_message(join("\n", 'spam', $hits, $names));
Or even include the more detailes:
action_quarantine_entire_message(join("\n", 'spam', $hits, $names,
$report));
If you use the last example, it is best to customize the SA report to
make it a bit shorter.
This can be done in "sa-mimedefang":
clear_report_template
report SpamAssassin detailed report: (_SCORE_ points, _REQD_ required)
report _SUMMARY_
Yizhar Hurwitz
http://yizhar.mvps.org
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: no filtering happening
HI. From: "Kitione Lalakomacoi" <[EMAIL PROTECTED]> Subject: [Mimedefang] no filtering happening , no errors to be seen - how to troubleshoot? the milter is called in the .mc file as per the manual INPUT_MAIL_FILTER(`mimedefang', `S=unix:/var/spool/MIMEDefang/mimedefang.sock, F=T, T=S:60s;R:60s;E:5m')dnl (note F=T, this thing should be stopping all deliveries if the milter doesn't work, however its quite happy to deliver away) running md-mx-ctrl msgs tells me that no messages have passed through the filter, even though i've sent multiple e-mails through. Well, I think that the changes you've made to sendmail.mc, were not applied to sendmail.cf, which is the actual configuration file that sendmail uses. First, check the date/time stamp of both sendmail.mc and sendmail.cf in /etc/mail. sendmail.cf should be newer. If it is not, you should restart sendmail and/or run the command: m4 sendmail.mc > sendmail.cf Bye Yizhar ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Ignoring headers with sa-learn
HI. I'm currently using Mimedefang 2.52 with SA 3.0.4 . So far I have used SA rules and network tests without Bayes, and now I'm starting to use Bayes also. The bayes database (along with AWL) is at: /home/defang/.spamassassin I have already started using bayes by adding the following to sa-mimedefang: use_bayes 1 bayes_auto_learn 1 bayes_auto_learn_threshold_nonspam 0.5 bayes_auto_learn_threshold_spam 8.0 All seems to work fine and I have no problem with that (but you can tell me if you have any tips). I also have a corpus of manually sorted spam and ham messages that I have collected and will like to use sa-learn with them. The corpus is in specific mail folders on my Cyrus-imap server (single message per file). All of these messages have headers added by MD, such as: X-Spam-Score: ... X-Scanned-By: MIMEDefang ... Some of the messages (false positive) also have a SpamAssassinReport.txt attached to them that was added by MD. Those messages are in a separate corpus (mail folder) so I can start by learning only from messages that don't have that attachment. I have read the sa-learn man pages and perldoc Mail::SpamAssassin:Conf, and found the option: bayes_ignore_header But I'm not sure where (in which file) should I use it, because I run sa-learn outside of the scope of MD, so sa-mimedefang isn't probably the right place for it. So my questions are: * How should I tell sa-learn which headers to ignore? * Can I tell sa-learn to ignore the attachment SpamAssassinReport.txt , or can you suggest a script to easily remove that specific attachment? Thanks for any tips, Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Notifications to users, and message journals
HI. I'm using MD ver 2.52, on a Fedora Core 3 system, for small company (about 20 users, low volume of emails). I plan to install similar systems at other locations (for different customers) most of them to act as mail relay in front of MS Exchange servers, some of them as stand-alone mail servers. Everything is working fine so far, and I do the following for spam messages: messages with score higher then 8 ==> reject. messages with score higher then 5 ==> send to "antispam" mailbox as described here: How do I redirect spam to a spam maildrop? - MIMEDefang: http://www.mimedefang.org/node.php?id=35 Now, I would like to: 1. Send a daily report to each original recipient with a log of all the messages blocked, or at least those messages that were sent to the maildrop, so that the user can know what was filtered. Note: I want to continue to use a maildrop or other kind of central quarantine and I don't want to just tag the messages and let the user configure rules on the MUA. 2. I would also like to collect a server based log of the messages passed through it, containing information such as: date, time, sender, recipients, subject, size, spamscore, action taker, and other optional info such as attachments names. Can you recommend any scripts or software to do it. What are you doing with spam - do you send any notification/report to the end users in some way? Thanks in advance for any input, Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Accept mail before processing
HI.
As some have already suggested to you, I did the following on my small
MD server to improve performance of outgoing mail:
1. I have disabled antispam checking for outgoing mail, using the
following:
It is up to you to decide if you want to skip spamassassin check for
outgoing mail or not, and if so, which method or rules to use for that.
sub filter_end ($) {
my($entity) = @_;
# No sense doing any extra work
return if message_rejected();
# Spam checks if SpamAssassin is installed
if ($Features{"SpamAssassin"}) {
if ($RelayAddr =~ "^192\.168\.1"
or $RelayAddr eq "127.0.0.1") {
# note: You can add here logging or other commands to note that
SpamAssassin was skipped...
} else {
if (-s "./INPUTMSG" < 100*1024) {
# Only scan messages smaller than 100kB. Larger messages
# are extremely unlikely to be spam, and SpamAssassin is
# dreadfully slow on very large messages.
my($hits, $req, $names, $report) = spam_assassin_check();
etc
2. It would be a good idea to follow the performance tuning steps to
improve MD performance in general for any mail traffic.
One major step is to put /var/spool/MIMEDefang on a RAM drive.
Read here:
Creating a RAM-based spool directory - MIMEDefang:
http://www.mimedefang.org/node.php?id=27
And here (page 142):
http://www.mimedefang.org/static/mimedefang-lisa04.pdf
Good luck,
Yizhar Hurwitz
http://yizhar.mvps.org
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] URLing multimedia attachments
HI.
Matthew S. Cramer wrote:
Indeed. In a week we are implementing a feature to remove multimedia
files from emails using MIMEDefang and replace them with a URI and
some scare text to say "click here to get your media file, and be sure
you are complying with our acceptable use policy."
I have implemented this on my MD machine and would like to share the
code I used, especially the list of extensions in use.
I also do the same with files over a specific size (8MB), but these are
stored in different location.
This is what I have in "sub filter":
$size = (stat($entity->bodyhandle->path))[7];
if ($size > 800) {
return action_replace_with_url($entity,
"/var/www/mimedefang/bigfiles",
"http://x.x.x.x/bigfiles";,
"\nThe attached file was larger then 8MB.\n" .
"You can download it from the following URL:\n\n" .
"_URL_\n\n" .
"The original file name is:\n" . $fname . "\n\n" .
"File Size = " . $size . "\n\n" .
"Please note that the download process can take a long time.\n" .
"Please note that the file will later be deleted from the mail
server.\n"
);
}
$multimedia = '(asf|avi|mov|mpeg|mpg|wmv)';
if (re_match($entity,'\.' . $multimedia)) {
return action_replace_with_url($entity,
"/var/www/mimedefang/multimedia",
"http://x.x.x.x/multimedia";,
"\nThe attached multimedia file is available for download
here:\n\n" .
"_URL_\n\n" .
"Original file name:\n" . $fname . "\n\n" .
"File Size = " . $size . "\n\n" .
"Please note that the file will later be deleted from the mail
server.\n"
);
I currently have no statistics about this, but I can tell that it was
implemented 4 months ago
and I didn't got any complain or query from the users about this filter.
I haven't yet implemented a script to delete old files from there
because it currently doesn't take too much disk space.
You can consider doing the same with PPS and maybe other extensions.
FYI
Yizhar Hurwitz
http://yizhar.mvps.org
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Justifying greylisting to management
HI Again. I would like to share with you more information about the MS Exchange -> Greylisting issue I have described before. First, I'm not the only one... I found another discussion about the exact same problem: http://groups.google.com/group/microsoft.public.exchange.admin/browse_thread/thread/36cd5a8dabd3663d/09ff07ac14b116db I assume that other had or will have similar problems (either at the sender or recipient side). I have contacted a person in MS and asked him to check this issue - will tell you if I get some answers. Here are the workarounds that I did to prevent this problem, I will share them with you because although this is not an MS Exchange group, I think it will be of interest for all those that plan or currently use greylisting: * I have changed the default retry time outs on the SMTP Virtual server, here are the values that I currently use (the defaults are in brackets): First retry = 1 minute (default 10) Second retry = 2 minutes (default 10) Third retry = 15 minutes (default 10) Subsequent retry = 30 minutes (default 15) * I have configured a scheduled task that restarts the SMTP Service every day. * I have configured specific domains to route via ISP . Here are some answers to your comments: From: Jan Pieter Cornet <[EMAIL PROTECTED]> > However - the bottom line was the important emails (important for both > sender and recipient) where delayed for more then 1 week, without any > notification to sender nor recipient! That sounds like an enormous bug in the setup on the exchange side. I agree that it is a bug, however I think that it is a bug in the software itself and not in the specific setup. > * I assume that this is not a single specific issue but does/will > probably affect customers in other similar scenarios. I doubt it. To be blunt - it sounds like an incompetently managed Exchange server. Sure, some issue like this are likely to be present in more than one location - dumb admins are everywhere, and not only behind I agree that in first look is "sounds and looks" like a misconfiguration, but please believe me that it is not that simple. My skills are not the main issue here. winders machines. But that's the whole point. We detect spammers in basically two ways - by their breaking of RFCs, and by the content of their message. Greylisting falls in the first category. OK. Now I'm not too fond of Exchange, but I do know a little bit about MS Exchange, and I am positive that a properly configured exchange server has no trouble dealing with a greylisting mailserver. I thought so before until I had that problems. I assume and hope that a hotfix will be released in few weeks. Now, all exchange experts I've spoken to, agree that one of the cardinal mistakes you can make in setting up an exchange server is letting it talk directly to the internet at large - you should always put it behind a sendmail(or other unix MTA) box that does the actual mail receiving and transmitting into the whole bad world for it. (However, those deeply inundated with M$ will only very reluctantly admit this). It looks like in your situation you made at least this setup error. Exchange is sending/receiving directly to the Internet. This is a very common configuration in small businesses and also in larger deployments. I do try to convince my clients to install a mail relay (such as sendmail/MD/clamav) for incoming mail. This is not because of Exchange limitations but simply for additional security and filtering. I also try to configure most mail servers to relay via smart host at ISP - but this is not always applicable. > * My point is that you should also take into account that greylisting > might cause more severe problems and not only delays of few minutes, > and this should be added to the "cons" count against greylisting. I'd say that counts as one of the "cons" of incompetence :) Temporary failures do happen, occasionally, independent of greylisting. If your setup cannot handle that, then you have a problem. I agree that I have a problem. I wrote this email because I think that it is important and valuable information for you all. My main point is: If you are going to implement greylisting - go ahead, that is your choice. But you should be aware that in addition to the planned X minutes delay of email which is the direct result of it, you might encounter more severe problems like the one I described. And if by any chance this happens to you (weather you are managing the sender or recipient mail server), you will have more info to troubleshoot the problem with the administrator at the other side. In the specific incidents that I had, it caused important business emails to be delayed for days with no NDR nor delay notification. Again - I do agree that MS Exchange at my side seems to be the cause of the problem, not the greylisting recipient server. From: "David F. Skoll" <[EMAIL PROTECTED]> Now, there *are* some marginal SMTP servers that fail in
[Mimedefang] Re: Justifying greylisting to management
HI. I would like to share a specific issue that I had with greylisting at the sender side: I manage several mail servers, most of them with MS Exchange 2003. Some of the recipients that my customers send emails to, are using some sort of greylisting (I didn't check which method exactly). One of the recipients domain is "technion.ac.il" I have found that for some reason unknown yet, MS Exchange 2003 SP2 does not handle greylisting very well with default configuration, and in some scenarios the outbound mail to such domains is simply frozen and not sent. This could be due to timing issues or problems with specific greylisting method at the recipients side - I don't know the exact cause. I have found some workarounds at my side (sender) and tweaks to prevent this. However - the bottom line was the important emails (important for both sender and recipient) where delayed for more then 1 week, without any notification to sender nor recipient! I haven't asked MS to solve it yet because it is a bit difficult to reproduce the problems and describe to them, but even if/when the issue will be solved, it won't be automatically fixed on all similar systems. So: * Greylisting is a nice idea, but does not always work as planned. * I assume that this is not a single specific issue but does/will probably affect customers in other similar scenarios. * My point is that you should also take into account that greylisting might cause more severe problems and not only delays of few minutes, and this should be added to the "cons" count against greylisting. * You can say whatever you like or dislike about MS Exchange, but as we all know it is widely in use and non of us has control over other persons mail servers. * You can say: "that's a problem of the sending server, not mine (the recipient side)". I won't argue with that because I'm not sure what is the exact cause of such problems. But your customers (end users and management) might argue about important emails lost or delayed for days. Bottom line: * It's your choice weather to implement greylisting or not. I recommend avoiding it if applicable and if you can get reasonable spam filtering without it. The issue I have described should be counted as one of the "cons" against it. For Your Info. Yizhar http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Strange activity
HI. Date: Fri, 6 Jan 2006 12:58:22 -0500 From: [EMAIL PROTECTED] Subject: Re: [Mimedefang] Re: Strange activity "Yizhar Hurwitz" <[EMAIL PROTECTED]> wrote on 01/06/2006 11:58:27 AM: However, regardless if this is related or not, I suggest that mail servers will use by default a low MTU value, for example 1300 . As far as I understand, it can only improve performance and avoid some problems with no negative side effects. What is the benefit of this change? How does it improve performance? Well, as you know, many systems nowdays use xDSL lines, that some of them have lower MTU because of tunneling protocols (such as PPPoE). And also, many firewalls drop ICMP packets required for PMTU, so you cannot trust PMTU to find the best packet size. Some firewalls might also drop fragment packets. Therefore, manual tuning of MTU on Internet facing servers, can avoid dropped or fragmented packets. I did not make a statistical research, but the logic is: Lowering MTU from 1500 (Ethernet default) to something in the range 1300-1450 will have no negative impact (or neglectiable one) when connecting with hosts that can support MTU 1500 all the way, BUT will have a positive impact when connecting with hosts over lines that do need lower one. Or the logic again rephrased: Better be on the safe side - don't use MTU of 1500 when you know that it can cause problems with some hosts, even if your connection does support it. This article can help understanding part of the problem(s): Path MTU Discovery: http://www.sendmail.org/tips/pathmtu.html Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Strange activity
HI. From: "David F. Skoll" Subject: Re: [Mimedefang] Strange activity All I can check is the MTU of the Ethernet interface, which is 1500. I doubt it's an MTU issue, because lots of normal mail (including large messages) is flowing through perfectly well. Also, *all* of the I myself I also doubt if it's an MTU related issue. However, regardless if this is related or not, I suggest that mail servers will use by default a low MTU value, for example 1300 . As far as I understand, it can only improve performance and avoid some problems with no negative side effects. My 2 cents. Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Strange activity
HI. From: "David F. Skoll" Subject: [Mimedefang] Strange activity Has anyone noticed some strange activity lately? Specifically, one of our customers has been hit by hundreds or thousands of machines that open SMTP connections to his boxes and then just sit there, leaving the connection idle. This wreaks havoc by creating tons and tons of Sendmail processes. I would like to suggest another possible explanasion to the "buggy" spam bot. Issues like that might also be related to MTU . Have you checked the MTU value of your customer mail servers? Can you try to ping with different packet sizes to some of the sender IP addresses? I do agree that the most reasonable explanasion so far is a spam/dictioanry/other attack software, but in addition you should also consieder layer 3 networking issues, or a combination of attacks hanging due to layer 3 problems or client side limitted bandwidth. Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Mimedefang and clamd configuration problems
HI. I'm running clamd as the user 'defang', using the 'User' directive in the clamd.conf. Also, I've tried using clamav-milter and mimedefang in sendmail and sendmail would ONLY use clamd and NOT mimedefang. Perhaps I mis-configured the sendmail.mc file ?? You should use either clamav-milter *OR* mimedefang to scan for virusses using clamav. The common scenario is to scan from using mimedefang, so you should comment out and remove clamav-milter from your sendmail.mc Yizhar Hurwitz http://yizhar.mvps.org Date: Fri, 23 Dec 2005 14:02:04 -0500 From: [EMAIL PROTECTED] Subject: [Mimedefang] Mimedefang and clamd configuration problems To: mimedefang@lists.roaringpenguin.com Message-ID: <[EMAIL PROTECTED]> Content-Type: text/plain; charset="US-ASCII" I'm running clamd as the user 'defang', using the 'User' directive in the clamd.conf. Also, I've tried using clamav-milter and mimedefang in sendmail and sendmail would ONLY use clamd and NOT mimedefang. Perhaps I mis-configured the sendmail.mc file ?? Joseph Morin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Mail delay due to "Cannot mkdir(Work): No
HI. Try the command: ls /var/spool/MIMEDefang How many files do you have there? That folder is a temporary spool folder that should only contain few "pid/sock" files, and "working folders" for current in transit mail. But if you see a lot of old sub-folders, they are probably left overs from previos crashes. If you have a lot of subfolders there, you might wish to first clean it up: service sendmail stop service mimedefang stop cd /var/spool/MIMEDefang # rm -rf mdefang-* (Before running rm -rf, make sure that you are in the correct folder and using the correct syntax!!) Next stop is to start using a RAM Drive if possible, to speed up mimedefang and it might also help with your problem: Creating a RAM-based spool directory - MIMEDefang: http://www.mimedefang.org/node.php?id=27 Yizhar Hurwitz http://yizhar.mvps.org ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: performance tuning once again
HI. My main concern is that every know and then,i get these dreaded 'timeout before data read' errors (timeout of 10 minutes specified in sendmail.cf) In addition to MD related issues, you can also check for MTU related issues, because those can also casue timeouts which are a bit difficult to troubleshoot. I found that the best MTU value should be no more then 1300, instead of the default of 1500. Even if your internet connection can handle 1500 packets, they might be broken or even dropped by the other side, and it is best to prevent or minimize the problems in advanced. I don't know if it is related to your problem, but still it would be good idea to check it. Read this: http://www.sendmail.org/tips/pathmtu.html Yizhar Hurwitz http://yizhar.mvps.org ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Replace with URL (updated)
HI.
I have some updates and corrections to my previous post.
Here they are (lines without the > symbol are changes from previous post).
#***
# %PROCEDURE: action_replace_with_comment
# %ARGUMENTS:
# msg -- message
# %RETURNS:
# Nothing
# %DESCRIPTION:
# Makes a note to drop the current part and replace it with a comment
#
# To be used in replace_with_url,
# instead of the original replace_with_warning which is plain text only.
#
# Written by Yizhar Hurwitz, http://yizhar.mvps.org
# Version 1.0 , 13-Oct-2005
#
#***
sub action_replace_with_comment ($) {
my($msg) = @_;
return 0 if (!in_filter_context("action_replace_with_comment"));
$Actions{'replace_with_warning'}++;
$Action = "replace";
$msg =~ s/\n/\n/g;
$ReplacementEntity = MIME::Entity->build(Type => "text/html",
Encoding => "-suggest",
Data => [ "\n$msg\n<\/body><\/html>\n" ]);
$WarningCounter++;
$ReplacementEntity->head->mime_attr("Content-Type.name" => "comment$Warning
ounter.htm");
$ReplacementEntity->head->mime_attr("Content-Disposition" => "inline");
$ReplacementEntity->head->mime_attr("Content-Disposition.filename" => "comm
nt$WarningCounter.htm");
return 1;
}
=== And here are the changes to the existing function replace_with_url:
=
my($fname, $ext, $name, $url);
my($htmlurl);
my $extension = "";
[.]
$htmlurl = "$url<\/a>";
[.]
$msg =~ s/_URL_/$htmlurl/g;
action_replace_with_comment($msg);
return 1;
And in "/etc/mail/mimedefang-filter" I have done the following
==
$multimedia = '(avi|mpg|mov|pps|wmv)';
if (re_match($entity, '\.' . $multimedia)) {
return action_replace_with_url($entity,
"/var/tmp/mail_parts",
"http://10.0.0.4/mail_parts";,
"multimedia file converted to URL:\n_URL_\n"
);
}
Yizhar Hurwitz
http://yizhar.mvps.org
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Replace with URL
HI.
I was thinking about a hack to the "replace_with_url" function that will add a
text/html MIME part,
but I don't feel that I have the required skills to do that in Perl without
breaking something else.
Simply replace "text/plain" with "text/html" and then format the body
in HTML. No perl skills required :)
OK, I have changed "mimedefang.pl" and it seems to work fine and solve the
issue,
however I would like to ask you to review my code and tell me if you have any
comments about it.
I did the change in a lab environment, not in production yet.
Here are the relevant changes to the file "/usr/local/bin/mimedefang.pl":
= Added a new function: ==
#***
# %PROCEDURE: action_replace_with_comment
# %ARGUMENTS:
# msg -- message
# %RETURNS:
# Nothing
# %DESCRIPTION:
# Makes a note to drop the current part and replace it with a comment
#
# To be used in replace_with_url,
# instead of the original replace_with_warning which is plain text only.
#
# Written by Yizhar Hurwitz, http://yizhar.mvps.org
# Version 1.0 , 13-Oct-2005
#
#***
sub action_replace_with_comment ($) {
my($msg) = @_;
return 0 if (!in_filter_context("action_replace_with_comment"));
$Actions{'replace_with_warning'}++;
$Action = "replace";
$ReplacementEntity = MIME::Entity->build(Type => "text/html",
Encoding => "-suggest",
Data => [ "\n$msg\n<\/body><\/html>\n" ]);
$WarningCounter++;
$ReplacementEntity->head->mime_attr("Content-Type.name" => "comment$Warning
ounter.htm");
$ReplacementEntity->head->mime_attr("Content-Disposition" => "inline");
$ReplacementEntity->head->mime_attr("Content-Disposition.filename" => "comm
nt$WarningCounter.htm");
return 1;
}
=== And here are the changes to the existing function replace_with_url:
=
my($fname, $ext, $name, $url);
my($htmlurl);
my $extension = "";
[.]
$msg =~ s/_URL_/$htmlurl/g;
action_replace_with_comment($msg);
return 1;
And in "/etc/mail/mimedefang-filter" I have done the following
==
$multimedia = '(avi|mpg|mov|pps|wmv)';
if (re_match($entity, '\.' . $multimedia)) {
return action_replace_with_url($entity,
"/var/tmp/mail_parts",
"http://10.0.0.4/mail_parts";,
"multimedia file converted to URL:\n_URL_\n"
);
}
==
So - how does it look?
Any hidden mines or overlooked potential problems?
Thanks again.
Yizhar Hurwitz
http://yizhar.mvps.org
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: Replace with URL
HI. From: Rob MacGregor <[EMAIL PROTECTED]> Two options off the top of my head: Ugly: Send a text/html mail, with correct HTML formatting Can you explain what exactly do you mean? Send the email from mimedefang? how? I was thinking about a hack to the "replace_with_url" function that will add a text/html MIME part, but I don't feel that I have the required skills to do that in Perl without breaking something else. Ok: Wrap the url in angle brackets: <http://10.0.0.4/mail_parts/attachment.zip> I did try it already, and it doesn't help. From: <[EMAIL PROTECTED]> You'll need to hack the replace_with_url function (or make your own new one) to add the following HTML text (url goes here too) in HTML emails. I did try to do something similar and it didn't help (I did it in mimedefang-filter by changing the text parameters that I pass to the function). Can you help me with a sample code? Here are the steps to reproduce the problem if you wish to see it for yourself: 1. In mimedefang-filter, use "replace_with_url" function for specific file size or name. 2. Send a test message to yourself with attachment, using an email client that uses HTML format, such as Outlook Express. 3. The "warning1.txt" MIME part would appear at the end of the message as a "text/plain" part, and the URL link is unclickable (at least in Outlook Express). BTW - I have tested this with a different email client = Thunderbird, and Thunderbird does convert it to a clickable link, so I know that the issue is dependant on the recipient email client, But this information does not help me too much because I cannot force or control the mail clients used by the recipients. Thanks again for any comments and tips. Yizhar Hurwitz http://yizhar.mvps.org ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Replace with URL
HI.
I'm using Mimedefang version 2.52 on a mail server for small company (20 users).
Most of the users use MS Outlook Express (on WinXP), few of them use
Thunderbird (on Windows also).
I am using the function "replace_with_url" to replace files larger then a
specific files,
and also some multimedia files by extension.
My problem is that the user receiving the message, cannot click on the URL, but
needs to copy and paste it.
This happens if the original email is in HTML format, which is the default
format of our clients.
As far as I understand, this is probably because the function
"replace_with_warning" adds a text/plain part,
and some EMail clients (like MS Outlook Express) displays the link as regular plain text instead of creating a link from
it.
Here is an example from one of my test messages:
--=_NextPart_000_00CB_01C5CF72.0FF716C0
Content-Type: text/plain; name="warning1.txt"
Content-Disposition: inline; filename="warning1.txt"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-Mailer: MIME-tools 5.417 (Entity 5.417)
multimedia file converted to URL:
http://10.0.0.4/mail_parts/0297c3c84eb5e9f616825d87a0a721d5ae1b57d2.avi
Here is the relevant part from "mimedefang-filter":
---
$multimedia = '(avi|mov|mpg|pps|wmv)';
if (re_match($entity, '\.' . $multimedia)) {
return action_replace_with_url($entity,
"/var/tmp/mail_parts",
"http://10.0.0.4/mail_parts";,
"multimedia file converted to URL:\n_URL_\n");
}
---
So, my question is:
How can I use the function "replace_with_url", but make my best effort so that the recipient will be able to simply
click on the URL instead of needing to copy & paste it?
(Please let me know if I need to provide more details)
Thanks
Yizhar Hurwitz
http://yizhar.mvps.org
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] filter multimedia files and replace with url
HI.
I am managing a mail server with MimeDefang version 2.52
(Fedora Core 3 + Sendmail + Mimedefang + SpamAssasin + Clamav + Cyrus IMAP)
I'm currently using replace_with_url for files bigger then 8mb in the
production server,
and now I would like in addition to do it also by extension with some
multimedia files.
I have written the code to do it, implemented in a test machine similar to the
production.
It seems to work fine and do the job as expected.
However, I have no experience with Perl, so I would like you to take a look at
the code,
and just tell me if it is OK or if you have any comments about it.
Here it is (copied from the test machine):
===
$multimedia = '(avi|mpg|mov|pps|wmv)';
if (re_match($entity, '\.' . $multimedia)) {
return action_replace_with_url($entity,
"/var/tmp/mail_parts",
"http://10.0.0.4/mail_parts";,
"multimedia file converted to URL:\n_URL_\n");
}
===
The code is in the "filter" subroutine.
Is that correct?
Should it also be in filter_multipart?
Additional notes:
===
In the production server I will use a more descriptive text to the users, no
need to comment about this.
In the production server I will use a real FQDN of course instead of http://10.0.0.4 so you don't have to comment about
this, and also a different path instead of "/var/tmp/mail_parts".
I'm not interested in multimeda files inside ZIP - only regular file
attachments.
Thanks for any comments.
Yizhar Hurwitz
http://yizhar.mvps.org
___
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list
MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] RE:Email attachment size
HI. Have you considered the following as alternative: Limit maximum size in Sendmail to 50mb Use "action_replacewithurl" (I don't remember the exact syntax) for attachments sized 10-50 mb This can be done for all users, or fine tuned for specific users using your own customization. You can add a notice to the email that the attachment will be removed from server after X days/weeks, and run a cron job or manually clean the attachments directory periodically. Yizhar Hurwitz http://yizhar.mvps.org - Original Message - From: "Pramod Anugu" <[EMAIL PROTECTED]> To: Sent: Friday, September 09, 2005 11:23 PM Subject: [Mimedefang] RE:Email attachment size does anyone know if its possible (and how to accomplish) enforcing a size limitation on email . if I want to limit email to 5Mb or under, and then reject it. I wan to reject it as soon as 5.1Mb is received. is this possible?.Can i also limit based on the groups. For certain group the limit is 5 MB and other group it is 20 MB and for the other group it 50 MB. thanks ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Compiling MD on FC4 - warning: pointer targets diferin signedness
HI. Make sure that you edit your /etc/sysconfig/i18n file and change the line: I did change the file: /etc/sysconfig/i18n as you wrote, but it didn't change anything. When I run "make" I get the same errors. And yes, I did try to reboot, "make clean", and also to untar the source file from and start from scratch. I didn't try yet compiling an older version of MD. However, after writing the previos post, I did continue to install and configure MD, and so far it seems to work fine on my test system. So I assume that this is only a minor bug. Any additional comments about the issue? Thanks. Yizhar Hurwitz http://yizhar.mvps.org ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Compiling MD on FC4 - warning: pointer targets difer in signedness
HI. I'm getting warnings when running "make" to compile MimeDefang on Fedora Core 4. I have done the following: ./configure (no problems) make (Here I get the warnings). Here are some lines with warnings from the make command output: md-mx-ctrl.c: In function ?percent_decode': md-mx-ctrl.c:71: warning: pointer targets in passing argument 1 of ?sscanf' dif er in signedness md-mx-ctrl.c: In function ?buildCmd': md-mx-ctrl.c:176: warning: pointer targets in passing argument 1 of ?percent_en ode' differ in signedness md-mx-ctrl.c:176: warning: pointer targets in passing argument 2 of ?percent_en ode' differ in signedness md-mx-ctrl.c: In function ?doCmd': md-mx-ctrl.c:239: warning: pointer targets in passing argument 1 of ?percent_de ode' differ in signedness gcc -g -O2 -Wall -Wstrict-prototypes -o md-mx-ctrl md-mx-ctrl.o -lnsl test "" != "1" && strip md-mx-ctrl More information: This is my first Mimedefang installation. mimedefang version which I'm trying to install = 2.52 I am following the how to guide: http://www.mickeyhill.com/mimedefang-howto It is on a test machine (Virtual PC) with Fedora Core 4. Sendmail is installed from RPM and running (version 8.13.4-2) I have updated GCC and related packages from gcc 4.0.0 to 4.0.1 but it didn't change anything. "make" version is 3.80 Other installed packages (most of them from RPM either FC4 cdrom or "yum"): clamav-0.86.2-1 spamassassin-3.0.4-1.fc4 perl-5.8.6-15 Linux Kernel version = 2.6.11-1.1369_FC4 Afterwards I did "make install" and it seemed to work fine, but I don't know if it is OK or not. I have not completed the installation and configuration yet so I don't know if it is actually working. So, what can you tell me about these warnings: pointer targets in passing argument 1 of ... differ in signedness ? Thanks in advance. Yizhar Hurwitz http://yizhar.mvps.org ___ Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
