[Mimedefang] Strip DOC with macros
Hey Mimedefang listers, I wanted to know if I could use mimedefang to strip out .DOC, .DOCX, .XLS, and .XLSX files (or any applicable file type) if they contain a macro. --Justin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] learner indicated ham
On Sat, Aug 9, 2014 at 1:41 PM, G.W. Haywood mimedef...@jubileegroup.co.uk wrote: It wasn't all that vague. :) You guys do REJECT your spam, don't you? -- 73, Ged. Bill, Thank you very much for the response. The detail is much appreciated. As Ged mentioned, not vague, helpful to say the least. The part about highly trusted rules caught my attention: Another way to increase autolearning without going all the way to the learn on error behavior is to flag rules that you trust highly as autolearn_force so that messages matching them won't ever be excluded from autolearning based on the existing Bayes DB disagreeing with the deterministic rules. I think these will get me started: tflags URIBL_DBL_SPAM autolearn_force tflags URIBL_JP_SURBL autolearn_force tflags URIBL_BLACK autolearn_force tflags INVALID_DATE autolearn_force Any others that are definites? ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] learner indicated ham
Aug 8 12:00:53.067 [19948] dbg: learn: auto-learn: message score: 13.934, computed score for autolearn: 17.583 Aug 8 12:00:53.067 [19948] dbg: learn: auto-learn? ham=0, spam=7, body-points=7.448, head-points=5.511, learned-points=-1.9 Aug 8 12:00:53.067 [19948] dbg: learn: auto-learn: autolearn_force not flagged for a rule. Body Only Points: 7.448 (3 req'd) / Head Only Points: 5.511 (3 req'd) Aug 8 12:00:53.067 [19948] dbg: learn: auto-learn? no: scored as spam but learner indicated ham (-1.9 -1) Is this something that I can fix? I want stuff to be trained as spam but it doesn't seem to make it. I am thinking it's either a setting I am not aware of or I need to retrain my bayes DB ham. Any help would be great. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Mimedefang/Multiplexor wrong score. Stops running tests randomly
Steffen and Stephen, From a combination of your responses I was able to shed some light on a few things. Firewall outbound was blocking Pyzor/Razor and Spamassassin for a few IPs. I originally allowed the traffic during testing, but to one external IP that connects to spamassassin. The command run as defang, adding a shell, was the most helpful. I was able to see the score that defang would see. Thanks for that tip. su defang -s /bin/bash -c 'spamassassin -x -p /etc/mail/sa-mimedefang.cf -D' spam.eml Thanks again for your help. It has been greatly appreciated. On Fri, Jul 18, 2014 at 10:54 AM, Stephen Johnson (DIS) stephen.john...@arkansas.gov wrote: On Thu, 2014-07-17 at 18:51 -0400, Justin Edmands wrote: Hey, Mimedefang is not appending the appropriate score to our messages. An example would be a message manually run through spamassassin produces a 17.6 score. This same message processed by the mimedefang filter only produces a 0.698. This is all run on the same server. What the heck? It only runs those tests? It runs random tests sometimes. I have no idea why. Does it have a max process time or something causing it to stop running tests after X time? Anyways... You are misunderstanding how Mimedefang uses spamassassin. Spamassassins's rewriting of e-mail headers is done when it's used after the MTA has accepted delivery of the e-mail. Mimedefang runs as a milter (mail filter) within sendmail itself. That means that an incoming e-mail is still in the processing of being received when Mimedefang get's called be sendmail. The e-mail can't be rewritten by spamassassin. The only way to modify the incoming e-mails is via milter API calls. And only Mimedefang itself has to do the rewrites. Spamassassin in this scenario is only used to run the tests. If you are using the default Mimedefang filter (/etc/mail/mimedefang-filter), you will see some rewriting code happening in the fitler_end() function. And in terms of how spamassassin works espceically inside run within Mimedefang. Spamassassin data (e.g. bayes filter database, autowhitelist database, etc), the data is stored on a per user basis. That means the spamassassin runs its tests using data stored in the user id that Mimedefang runs under. Running the same e-mail on a different user it will result in different test scores. If you want a semi-accurate spamassasin check of an e-mail as Mimedefang sees it, it has to be done under the Mimedefang user id. -- Stephen L Johnson stephen.john...@arkansas.gov Unix Systems Administrator / DNS Hostmaster Department of Information Systems State of Arkansas 501-682-4339 ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] how do I train bayes MySQL when relayed
Hey, Seems like lots of spam is slipping past. In turn, I would like to train/retrain my bayes database for the defang user. This is certainly just a relay so the mail is in and out without being stored. How do I train the database when it's MySQL. Do I need to go to my MDA and pull the .msg files and feed them to the sa-learn program? Also, in the actual database I wanted to see the spam and ham count. Seems like so much ham and not much spam collected. Any reason this is incorrect?: mysql select id,username,spam_count,ham_count,token_count from spamassassin.bayes_vars; ++--++---+-+ | id | username | spam_count | ham_count | token_count | ++--++---+-+ | 1 | defang |404 | 15794 | 203108 | ++--++---+-+ These might be dumb questions...sorry if RTFM is the only solution and I missed it somehow. ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] multiplexor - No DNS servers available!
I am trying to fix our setup. What needs to exist for this to work? Jun 4 23:49:49 relay2 mimedefang-multiplexor[2199]: s553nbRf003041: Slave 1 stderr: plugin: eval failed: available_nameservers: No DNS servers available! Jun 4 23:49:49 relay2 mimedefang-multiplexor[2199]: s553nbRf003041: Slave 1 stderr: rules: failed to run NO_DNS_FOR_FROM RBL test, skipping: Jun 4 23:49:49 relay2 mimedefang-multiplexor[2199]: s553nbRf003041: Slave 1 stderr: (available_nameservers: No DNS servers available!) Jun 4 23:49:50 relay2 mimedefang-multiplexor[2199]: s553nbRf003041: Slave 1 stderr: spf: lookup failed: available_nameservers: No DNS servers available! Jun 4 23:49:50 relay2 mimedefang-multiplexor[2199]: s553nbRf003041: Slave 1 stderr: spf: lookup failed: available_nameservers: No DNS servers available! and another request for DKIM stuff: Jun 4 23:59:29 relay2 mimedefang-multiplexor[2199]: s553xJiS003650: Slave 0 stderr: plugin: eval failed: available_nameservers: No DNS servers available! Jun 4 23:59:29 relay2 mimedefang-multiplexor[2199]: s553xJiS003650: Slave 0 stderr: rules: failed to run NO_DNS_FOR_FROM RBL test, skipping: Jun 4 23:59:29 relay2 mimedefang-multiplexor[2199]: s553xJiS003650: Slave 0 stderr: (available_nameservers: No DNS servers available!) Jun 4 23:59:30 relay2 mimedefang-multiplexor[2199]: s553xJiS003650: Slave 0 stderr: rules: failed to run DKIM_ADSP_DISCARD test, skipping: Jun 4 23:59:30 relay2 mimedefang-multiplexor[2199]: s553xJiS003650: Slave 0 stderr: (available_nameservers: No DNS servers available! Jun 4 23:59:30 relay2 mimedefang-multiplexor[2199]: s553xJiS003650: Slave 0 stderr: ) Jun 4 23:59:30 relay2 mimedefang-multiplexor[2199]: s553xJiS003650: Slave 0 stderr: spf: lookup failed: available_nameservers: No DNS servers available! Jun 4 23:59:30 relay2 mimedefang-multiplexor[2199]: s553xJiS003650: Slave 0 stderr: spf: lookup failed: available_nameservers: No DNS servers available! ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] mimedefang with spamassassin -- incorrect score assessed
Mimedefang list,
We currently use mimedefang and spamassassin on our relays. It appears
that recently the relays stopped assessing a proper spam score. Some
spam will get through, while others with the same format and will be
blocked. I am making an assumption about wrong score based on a spam
message not being detected and then copying the source(headers etc) to
http://spamcheck.postmarkapp.com/ to test the score. I'll see some
messages pass that are in the 10's. super spam, but still gets
through.
I have everything setup in /etc/mail/sa-mimedefang.cf. Originally it
appeared that I needed to flush out the
/etc/mail/spamassassin/bayes_{toks,seen,journal} files to allow it to
regenerate a new DB for spam scores.
All files in /etc/mail/spamassassin are defang:defang. I have to fix
these on the bayes_ files from time to time. Any idea why these change
to root:root every night? I assume cron job, etc. Not sure outside of
that.
/etc/mail/sa-mimedefang.cf:
required_score 3.4
ok_locales en
skip_rbl_checks 0
skip_uribl_checks 0
#Custom Rules
score ALL_TRUSTED 0.0 0.0 0.0 0.0
score AWL 0.0 0.0 0.0 0.0
#Bayesian auto-learn config
bayes_path /etc/mail/spamassassin/bayes
auto_whitelist_path /etc/mail/spamassassin/auto-whitelist
bayes_file_mode 0644
auto_whitelist_file_mode 0644
bayes_learn_to_journal 1
bayes_journal_max_size 102400
bayes_ignore_header X-Spam-Score
bayes_ignore_header X-Scanned-By
bayes_auto_learn_threshold_nonspam 0.0
bayes_auto_learn_threshold_spam 7.0
...
...
whitelist stuff
...
...
blacklist stuff
...
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.
Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] ./mimedefang: line 1: [: missing `]'
On 1/2/06, George [EMAIL PROTECTED] wrote: Hi, I got this error when I try to start mimedefang using /etc/init.d/mimedefang start Starting mimedefang-multiplexor: ./mimedefang: line 1: [: missing `]' [ OK ] Starting mimedefang: ./mimedefang: line 1: [: missing `]' [ OK ] I get a similar message with the freebsd port's init script since 2.54... Starting mimedefang-multiplexor:[: missing ] [ OK ] Starting mimedefang:[: missing ] [ OK ] Haven't had a chance to determine why yet. I don't think the init script is throwing the error but i haven't yet determined where the error comes from. I think it may have something to do with require lines, but that's mostly a guess. -Justin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] ./mimedefang: line 1: [: missing `]'
On 1/2/06, David F. Skoll [EMAIL PROTECTED] wrote: It's a typo in the init script. Add a space before any ] that lacks a space before it. Ah, on the two new spooldir tests... adding the space works a charm. Cheers! -Justin ___ NOTE: If there is a disclaimer or other legal boilerplate in the above message, it is NULL AND VOID. You may ignore it. Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] temp failing - got code?
On Wed, Jan 05, 2005 at 09:38:30AM -0800, Gary Funck wrote: Would like to implement temp failing/grey listing along the following lines: I haven't implemented it yet, but Anthony Howe has a milter for greylisting ... www.milter.info. Also, Evan Harris lists a bunch of greylisting implementations, including a mimedefang one, at projects.puremagic.com/greylisting/links.html. --j ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Undesired consequence of stream_by_recipient
On Mon, 5 Apr 2004, David F. Skoll wrote:
On Mon, 5 Apr 2004, Jim Hatfield wrote:
I didn't appreciate that stream_by_recipient causes mail to be accepted
before the validity of the recipient addresses has been checked.
Actually, this is a milter issue.
[snip]
You can fix this by adding a filter_recipient routine that checks if an
address is valid and returns ('REJECT', User unknown) if it is not.
[snip]
Just want to confirm... I'm correctly reading that stream_by_recipient()
does not accept for all recipients before filter_recipient has a chance to
[temp]fail?
And this means white/black/greylisting implementations that operate in
filter_recipient should still work - without sendmail having to generate a
bounce?
-Justin
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Greylist DB addition fails silently?
On Wed, 23 Jun 2004, David F. Skoll wrote: On Wed, 23 Jun 2004, -ray wrote: Yes i saw very similar problems when trying to implement greylisting using Jonas' code. I ended up NOT using greylisting because of this problem, I've had endless problems with Berkeley DB and Perl. Our greylisting implementation uses PostgreSQL; I'm sure it wouldn't be too hard to write a DBI version that can use MySQL or PostgreSQL. I have modified Steven Rocha's implementation (http://lists.roaringpenguin.com/pipermail/mimedefang/2004-February/020126.html) which I believe is a modification of Jonas' implementation. My modified version uses a PostgreSQL database in place of Berkley DB and allows you to specify action to take (white/black/grey) based on cidr/host address, using subnet 0/0 as the default action. I will clean it up and post if there's interest. -Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Greylist DB addition fails silently?
On Thu, 24 Jun 2004, Roland Pope wrote:
- Original Message -
From: Justin [EMAIL PROTECTED]
I have modified Steven Rocha's implementation
(http://lists.roaringpenguin.com/pipermail/mimedefang/2004-February/020126.html)
which I believe is a modification of Jonas' implementation. My modified
version uses a PostgreSQL database in place of Berkley DB and allows
you to specify action to take (white/black/grey) based on cidr/host
address, using subnet 0/0 as the default action.
I will clean it up and post if there's interest.
I would be very interested in a copy of this as I have wanted to use
greylists, but needed to have a shared DB as I have multiple MX's.
Be careful with single point of failure. I believe many here have noted
that it's better to just have an independent greylist db on each relay.
The attached snippet of filter should get you going with greylisting and
postgresql. Note that it also includes some popauthdb code adapted from
Kevin McGrail's example.
And while I'm at it let me make sure I give credit to:
jonas - for the original work
steven- for the modified version i adapted
kevin - for popauthdb bit
puremagic - gl_triplets table layout (adapted from their mysql table)
david - mimedefang
You'll need a postgresql database setup with tables as defined
something like this (beware of occasional line wrap):
Table public.gl_triplets
Column |Type | Modifiers
+-+-
id | integer | not null default
nextval('public.gl_triplets_id_seq'::text)
relay_ip | inet|
mail_from | character varying(255) |
rcpt_to| character varying(255) |
block_expires | timestamp without time zone | not null
record_expires | timestamp without time zone | not null
blocked_count | bigint | not null default 0
passed_count | bigint | not null default 0
aborted_count | bigint | not null default 0
create_time| timestamp without time zone | not null
last_update| timestamp without time zone | not null
Indexes:
triplet_key primary key, btree (id)
ip_from_to btree (relay_ip, mail_from, rcpt_to)
Table public.md_subnet_rules
Column | Type | Modifiers
+---+---
subnet | inet | not null
action | character varying(10) |
Indexes:
md_subnet_rules_pkey primary key, btree (subnet)
Some more columns might be helpful, such as created_date and a comment
area on md_subnet_rules, but these two are all the attached filter snippet
requires.
You should populate the md_subnet_rules table with something like the
following as a beginning:
subnet | action
-+
10.1.1.0/24 | white
127.0.0.1 | white
0.0.0.0/0 | grey
The record at the end is very important, and is used to define the default
action you wish to take. If you wished, you could set the 0/0 record to
white, and then only greylist specific subnets, like comcast, apnic, etc.
It's up to you.
Drop a note if there are any problems with the code.
-Justin
#*
# Greylist
#Settings for greylisting.
#
# For an explanation of what the purpose of this is, and maybe a hint as to
# what values to enter, check http://projects.puremagic.com/greylisting/;.
# I think they recommend something like this:
# $gdb_black = 1*60*60;
# $gdb_grey = 5*60*60;
# $gdb_white = 36*24*60*60;
# $gdb_subnet = 1;
#
#
# If $greylist is 1, greylisting will be used.
#
# Greylisting is done on a triplet of sending hosts IP, mail from: and
# rcpt to:.
#
# When a session with a new triplet arrives, all sessions with that
# triplet will be tempfailed for $gdb_black seconds.
# After $gdb_black seconds it will be white-listed for $gdb_grey
# seconds.
# If a session for the triplet arrives within the $gdb_grey white-listing
# period, it will then be white-listed for $gdb_white seconds.
# If a session for a triplet arrives within the $gdb_white white-listing
# period, it will be white listed for another $gdb_white seconds.
#
# If $gdb_subnet is true, only the first 3 octes of the IP-addresses will be
# used in the greylist.
# If $gdb_from_domain is true, only the domain part of the mail from: address
# will be used in the greylist.
# If $gdb_to_domain is true, only the domain part of the rcpt to: address
# will be used in the greylist.
# If $gdb_from_strip is true, some stuff in the user part of the mail from:
# address will be replaced in order to handle mailinglists and some other
# stuff better.
# If $gdb_to_strip is true, some stuff in the user part of the rcpt to:
# address will be replaced in order to handle use parameters and some other
# stuff better.
#
#
# Make sure you set $gdb_dsn, $gdb_username
Re: [Mimedefang] Can I test a filter before I reload the rules
On Mon, 5 Apr 2004, Mark Penkower wrote: Is it possible to test a new filter to see if it works (perhaps using a test email account) before putting it into production and reloading the rules? mimedefang.pl -f test-filter -test -Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] tmpfs queue directories
Howdy all. I'm building a new mail system and have a question about the tmpfs sizing recommendation. We're also evaluating Can-It Pro so I need to keep its needs in mind at the same time. http://www.mimedefang.org/node.php?id=28 In that FAQ David recommends 2.5 to 3 times the max message size x the maximum number of MD slaves. Does this still hold true with the newest versions of MD? That FAQ entry is almost a year old. Also is it best to configure MD to have as many slaves available as I've configured Sendmail to have? If Sendmail has more than MD, does Sendmail simply wait on MD to free up a slave or does Sendmail tempfail the message? We recently were on the receiving end of a massive spam flood. This happened a few days after MD permanently broke on our server so we weren't able to perform any AV/spam checks on the incoming flood of crap. At that time our mail server had to be reconfigured to allow up to 600 slaves (from 60 previously) just to withstand the flood and still allow customers to relay messages throught the server. We also allow attachments of up to 15MB. 15MB attachment x 60 slaves = 900MB tmpfs volume 900MB tmpfs volume x 2.5 recommened by David = 2250MB tmpfs volume That's a lot of RAM to spend on a temp mail processing. Is this accurate? What's the general recommendation of the group on the RAM needs? Our new system is 2 frontend boxes running dual 2.8 Xeons and a backend box running dual 3.06 Xeons. All 3 have 2GB of at present. Will this be adequate? As far as mail load goes, I'm not sure what the current numbers are. I was brought back in to handle this project after not administrating the mail system for the better part of a year. At the time I was maintaining these boxes they were handling anywhere between 25k-65k messages a day and rejecting 260k pieces of spam per week (access list and numerous DNSBLs w/ Sendmail). I know that the load has significantly increased over this past year though. Suggestions? Thanks Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] tmpfs queue directories
I of course thought of another point to ask about after I sent my first message. :) Is it considered acceptable to only perform AV checks on messages smaller than a certain threshold? We only scan for spam on messages smaller than 100KB. What are the odds of an email virus being found in a 15MB attachment? I know that a 15MB Word document could very well have a macro virus. I'm wondering if it would still be worthwhile to only scan attachments under, say, 500KB for viruses. Would that be a good idea? That would certainly lower my ram disk requirements. Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] SA suddenly not catching spam
On Wed, 24 Mar 2004, Gwendolynn ferch Elydyr wrote: Further poking about yesterday showed that SA alone seems to be handing out fairly reasonable scores, but SA in combination with MD is seeing hideously low scoring. It doesn't look to me as though I've turned off any SA rules via MD - bayes, dns and rbl checks are all enabled - but even after a restart, I'm not having much luck here. Are the differences in scores the same as the difference in scores pre-defined in SA for use when the calling instance meets a certain requirement or requirements? Ie network test are enabled, bayes is enabled, or bayes and network tests are enabled. The heuristic tests pick different scores for 4 different scenarios. If Bayes isn't enabled when calling from MD but it is when called with spamc then there will be a definite difference in scores. http://www.spamassassin.org/tests.html Is anybody running with a spam threshold hovering around 1 or 2 ? Nope, or at least they shouldn't be. The hueristic tests were run on the basis that 5 was the spam/ham threshold. If you want to raise the scores to tag more spam, add more tests like network tests and bayes. HTH Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] More Problems with Libraries
On Mon, 22 Mar 2004, David F. Skoll wrote: Beware... this is a bashism. We GNU users are spoiled. :-) Genuine sh will barf at this... you need: LC_ALL=C; export LC_ALL I vote for just fixing it at the source (no, not bombing the RH engineering team!): # /etc/sysconfig/i18n LANG=C SUPPORTED=C:en_US.UTF-8:en_US:en SYSFONT=latarcyrheb-sun16 That's something like step #2 on my Undo All the Stupid Redhat Things list. :) Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Latest MIME-Tools
On Mon, 22 Mar 2004, Nels Lindquist wrote: Which RedHat distribution are you using? PostgreSQL RPMs have been included on the CD (though not necessarily installed by default) at least as far back as 6.2, and if you want a more recent version, binary and source RPMs can be downloaded directly from http://www.postgresql.org/. Our new mail system is unfortunately using RH9. I would have preferred 7.3 but kernel drivers with 9 were required by some of the hardware. I almost never use any outward facing daemons that aren't compiled from source. I compile just about everything from source if given the opportunity. RH is terrible about keeping up with the latest greatest unless it involves a critical security fix. RH is also terrible about using the compile-time options that I want. When was the last time you used an RPM to install Apache or MIMEDefang? :) Better is somewhat subjective. :-) Having to recompile from source every time there's, let's say, a security patch for glibc doesn't strike me as necessarily superior to a binary package when you're talking about server maintenance. Well, anything is better than RH9. This is the 4th RH9 box I've set up in as many months and I'm left wondering if the RH engineers actually woke up in the morning and thought to themselves Where can we randomly put libraries today and how many oddball things can we statically link to them? LOL. I swear half the job of getting a new RH9 server online is undoing all of the oddball RH intricacies. Why in the world did they put kerberos in /usr/kerberos? Why UTF-8? Why is pine statically linked to an old version of SASL? Why does RPM have to be so darned obtuse? I'm just getting sick of RH. 7.3 was good enough for most of my servers after I was done with them. RH9 is the final stray for me though. I've heard a lot of good things about Gentoo so I'm going to at least give it a try. I'm not sure what to consider after that. Debian perhaps. I actually compile much of my system from source now. A new release of sendmail doesn't compell me to wait a couple days for RH to back port the fix like the frequently do. Instead I download the latest source, configure, compile and install it myself. That's just the way I've always done it. I'll do the same with PostgreSQL like I planned on doing with MySQL. Thanks for the reply Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: OT: Gentoo, Red Hat, etc. (was Re: [Mimedefang] Latest MIME-Tools)
On Mon, 22 Mar 2004, David F. Skoll wrote: In that case, use the latest PostgreSQL 7.4.x. It's much better than the 7.3.x series (faster, and better at reclaiming space during a VACUUM.) It sounds like it's worth it. I'll see what I can do about compiling that shortly. I'll see if I can jumpstart the demo process soon too. Fedora Core 1's actually not too bad. I have Gentoo on a laptop, but compiling *everything* from source pretty soon gets tiresome. I've heard some good things about Fedora. I've also heard it's basically RH9 with improvememnts (not a bad thing). It was proposed that we switch to RH Enterprise. The question I posed back to the person that suggested that was when was the last time we actually called RH for tech support. Well, after some pondering I finally just told them: we've never called them for tech support cause we've never needed it (or could afford to do it their way so they'd support our efforts). Really a good admin should be able to support him or herself with the available online resources and groups. That was the end of the RH Enterprise notion. I'm really looking forward to trying Gentoo on one of my boxes. All my boxes are SMP boxes and reasonably fast at that so compile times shouldn't be too bad. I'm really looking forward to what could quite possibly be a much better package manager. RPM is really quite lame. If you ever want to really annoy RPM uninstall the very dated version of Perl and all it's various modules that come with RH and compile and install the latest greatest from source. RPM will never forgive you that one. :) I'm looking forward to trying Gentoo and trying Can-It Pro. I can't hardly wait until we're ready. Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: OT: Gentoo, Red Hat, etc. (was Re: [Mimedefang] Latest MIME-Tools)
On Mon, 22 Mar 2004, Kelson Vibber wrote: I don't think it's a failing of RPM so much as it's a failing of package managers in general - namely, if you install anything that the PM doesn't know about, it acts as if it isn't there. The only way you can get around that is if you can override the PM and tell it, Look, Perl's really installed. I know I can't tell you in detail where all the files are, or what libraries and utilities it depends on, but it's installed, honest! See I was thinking Portage could do just this. Note however that I haven't yet gotten a chance to try it, but I'm getting closer. Perhaps it would be better to build a system that maintains a detail file on what the package manage expects to find (or has to find) to be able to say that yes, Package A is actually installed so I can add it to my internal DB. That way even if you download and compile by hand you could tell your package manager that Package A is really installed and the package manager could use the detail file to confirm that. It's a possibility. This way the detail file is distributed in portage and portage has a chance to confirm that the package is installed rather than taking our word for it. Hmm... interesting... Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Latest MIME-Tools
Does anyone know if MIME-tools-6.200_02 has the RP patches for MIME-tools-5.411a? I'm doing a new MD installation and am installing the lastest greatest of everything this time around. Also, does anyone happen to know if Can-It Pro has any additional requirements that aren't on the MD requirements list? We're getting ready to demo Can-It Pro and hopefully buy it as well. Thanks Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] Re: delete_recipient does not work for mixed case recipients
On Fri, Feb 13, 2004 at 03:23:19PM -0500, Justin Michael wrote: Hi, I'm still having a problem with mimedefang's delete_recipient not working with a mixed case recipient. It looks to me that mimedefang is sending the recipient in all lower case, but that sendmail is looking for exact case. I looked at mimedefang.pl and mimedefang.c and since mimedefang.c relies on smfi_delrcpt. In libmilter smfi_delrcpt calls mi_wr_cmd. In milter.c, milter_delrcpt calls removefromlist In recipient.c, removefromlist calls sameaddr In parseaddr.c, sameaddr uses bitset to compare userids and then strcmp to compare domains. In the logs, mimedefang reports the recipient as all lower case, no matter how RCPT TO: is reported to sendmail. And sendmail, when it logs the e-mail recipients, lists the recipient in mixed case. Suggestions? Thanks! Justin Feb 13 14:46:53 mail3 mimedefang.pl[27999]: i1DJkfpe032375: Moved SPAM: 20.829 [EMAIL PROTECTED] [EMAIL PROTECTED] Feb 13 14:46:53 mail3 sendmail[32381]: i1DJkfpe032375: [EMAIL PROTECTED], delay=00:00:00, xdelay=00:00:00, mailer=esmtp, pri=30488, relay=mta.mydomain.net. [IP.ADR.HRE.XX], dsn=2.0.0, stat=Sent (i1DJkrku029201 Message accepted for delivery) Feb 13 14:46:53 mail3 sendmail[32381]: i1DJkfpe032375: to=[EMAIL PROTECTED], delay=00:00:00, xdelay=00:00:00, mailer=esmtp, pri=30488, relay=myserver.mydomain.com. [IP.NO.WAS.HRE], dsn=2.0.0, stat=Sent (Message accepted for delivery) ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] delete_recipient does not work for mixed case recipients
Hi,
I'm still having a problem with mimedefang's delete_recipient
not working with a mixed case recipient.
sendmail 8.12.10
mimedefang 2.38
Here's my sample spam delivered via telneting to my host:
helo myserver
mail from: [EMAIL PROTECTED]
rcpt to: [EMAIL PROTECTED]
data
Date: February 13, 2004
From: J M [EMAIL PROTECTED]
To: [EMAIL PROTECTED] [EMAIL PROTECTED]
Subject: Test of spam analysis 4848
THIS SHOULD LOOK LIKE SPAM SPAM SPAM.
YOU WILL STILL GET IT PER YOUR REQUEST
IT HAS UNDISCLOSED RECIPIENTS
CALL TOLL FREE TO REMOVE
YOU HAVE REQUESTED THIS SPAM
BUY THIS NOW AT 1-800-232-3323 for $10,000,000.00
VIAGRA
CLICK HERE TO REMOVE a href=mailto: [EMAIL PROTECTED] HERE /a
a href=http://www.msn.fullfeed.com;click here/a
.
quit
Here's the code from mimedefang-filter:
if ($MOVESPAM $hits = $SAMoveScore) {
action_add_header(X-Spam-Rec, @Recipients);
my ($neworig);
my ($ok2add);
$ok2add=0;
foreach $neworig (@Recipients) {
delete_recipient(lc($neworig));
delete_recipient(uc($neworig));
delete_recipient($neworig);
$neworig=\.$neworig.\;
delete_recipient(lc($neworig));
delete_recipient(uc($neworig));
delete_recipient($neworig);
} # end of recipient loop
add_recipient([EMAIL PROTECTED]);
} # end of movespam conditional
Here's the log:
Feb 13 14:46:53 mail3 sendmail[32375]: i1DJkfpe032375: [EMAIL PROTECTED], size=488,
class=0, nrcpts=1, msgid=[EMAIL PROTECTED], proto=SMTP, daemon=MTA,
relay=myserver.mydomain.com [MY.IP.ADR.HRE]
Feb 13 14:46:53 mail3 mimedefang.pl[27999]: i1DJkfpe032375: Moved SPAM: 20.829 [EMAIL
PROTECTED] [EMAIL PROTECTED]
Feb 13 14:46:53 mail3 sendmail[32381]: i1DJkfpe032375: [EMAIL PROTECTED],
delay=00:00:00, xdelay=00:00:00, mailer=esmtp, pri=30488, relay=mta.mydomain.net.
[IP.ADR.HRE.XX], dsn=2.0.0, stat=Sent (i1DJkrku029201 Message accepted for delivery)
Feb 13 14:46:53 mail3 sendmail[32381]: i1DJkfpe032375: to=[EMAIL PROTECTED],
delay=00:00:00, xdelay=00:00:00, mailer=esmtp, pri=30488, relay=myserver.mydomain.com.
[IP.NO.WAS.HRE], dsn=2.0.0, stat=Sent (Message accepted for delivery)
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] PING: MD filter_end problem with my SA checks
It looks like my message got lost in the high list traffic on the day I sent it. Does anyone have any thoughts on the matter? I've tried the default filter and sa-mimedefang.cf to no avail. Something is seriously broken here. Any pointers would be very welcomed. Thanks Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] memory leak?
On Mon, 9 Feb 2004, Stephen Smoogen wrote: also realize that x86 hardware has to do various internal hacks to deal with memory above 2-4 Gigs. [2 gigs for some types of hardware and 4 gigs for other types.] Also check for things in your /proc/slabinfo to see if anything is full.. or if dmesg is reporting some sort of network issue. This would be one very good reason for stepping up to a 2.6 kernel. I've been running them on a couple of my servers since the rc's and have been most impressed. Support for larger amounts of memory has been greatly improved. $.02 before taxes, Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] milter timing out.
On Thu, 5 Feb 2004, Lucas Albers wrote: Justin said: On Wed, 4 Feb 2004, Edmund wrote: I'm having the same problem here. A co-worker was trying to send a big file to another co-worker and the mail server kept on throwing a Error 451. Unfortunately, I don't know how big the file is. then returns a 451. I'm going to inquire with the list about this under separate cover tomorrow, after I test a few more things. I've had to disable MD in the mean time. It's very odd. I run sendmail on a 7.3 box, mimedefang 2.33, sendmail 8.12.8, and it has never had timeout issues from large file size. I handle 15 meg files routinelly. I turned off spam checking for authenticated and internally sent mail, as some clients were suffering timeouts. you can just swatch your log, and only restart mimedefang/sendmail when it is suffering a timeout. My guess is something in your filter is taking a long time to process the email. Thanks for the reply. These messages weren't of any abnormal length. In fact the majority of them weren't as long as this message is. MD did regularly temp fail mail to my usenet alias. The only mail I ever received there was infected mail. It temp failed just about anything though regardless of size. I still have that box and the installation intact. It doesn't however handle mail anymore and it wouldn't be easy to set it up to handle mail at the present time either. Perhaps after I move I can revive that machine and try to figure out what went wrong. My current server is what I'm focusing on at the moment though. Thanks Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] no free slaves...
On Tue, 3 Feb 2004, David F. Skoll wrote: The hardware you mentioned seems pretty fast to me. You should easily be able to handle 2 million messages/day on four boxes like the ones you have. With 4GB of RAM, I would bump MX_MAXIMUM up to 100, and set MX_MINIMUM to around 40. I have a question. On dedicated mail relays, what is the benefit to those kind of MX_MIN and MX_MAX values versus setting them equal (say 75 for this person) based on interpretation of `md-mx-ctrl histo` and your available RAM. -Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
[Mimedefang] MD filter_end problem with my SA checks
=~ /^64\.71\.99\./ or
$RelayAddr =~ /^64\.71\.100\./ or
$RelayAddr =~ /^64\.71\.101\./ or
$RelayAddr =~ /^64\.71\.102\./ or
$RelayAddr =~ /^64\.71\.103\./ or
$RelayAddr =~ /^64\.71\.104\./ or
$RelayAddr =~ /^64\.71\.105\./ or
$RelayAddr =~ /^64\.71\.106\./ or
$RelayAddr =~ /^64\.71\.107\./ or
$RelayAddr =~ /^64\.71\.108\./ or
$RelayAddr =~ /^64\.71\.109\./ or
$RelayAddr =~ /^64\.71\.110\./ or
$RelayAddr =~ /^64\.71\.111\./
) {
# Disabled extension checks by returning immediately.
return 0;
} else {
if (-s ./INPUTMSG 100*1024) {
# Only scan messages smaller than 100kB. Larger messages
# are extremely unlikely to be spam, and SpamAssassin is
# dreadfully slow on very large messages.
my($hits, $req, $names, $report) = spam_assassin_check();
my($score);
if ($hits 40) {
$score = * x int($hits);
} else {
$score = * x 40;
} # end of $hits 150
# We add a header which looks like this:
# X-Spam-Score: 6.8 (**) NAME_OF_TEST,NAME_OF_TEST
# The number of asterisks in parens is the integer part
# of the spam score clamped to a maximum of 40.
# MUA filters can easily be written to trigger on a
# minimum number of asterisks...
if ($hits = $req) {
# Delete any existing X- Spam-related headers?
# action_delete_header(X-Spam-Score);
# action_delete_header(X-Spam-Report);
# action_delete_header(X-Spam-Status);
# action_delete_header(X-Scanned-By);
# my $fixed_report = $report;
# $fixed_report =~ s/\n+\z//g;# fixes for multiline
header
# $fixed_report =~ s/\n/\n\t/g; # to stop sendmail
complaining
action_change_header(X-Spam-Score, $score ($hits)
$names);
# action_change_header(X-Spam-Report, $fixed_report);
# action_add_header(X-Spam-Report, $fixed_report);
md_graphdefang_log('spam', $hits, $RelayAddr);
# Change the Subject line if $hits = 10
# if ($hits = 10) {
# action_add_header(X-Original-Subject, $Subject);
# action_change_header(Subject, ***SPAM*** $Subject);
# }
} else {
# Delete any existing X-Spam-Score header?
action_delete_header(X-Spam-Score);
# action_delete_header(X-Spam-Report);
# action_delete_header(X-Spam-Status);
# action_delete_header(X-Scanned-By);
} #end of $hits = $req
} # end of size check
} # end of $RelayAddress check
} # end of Features[SpamAssassin]
# I HATE HTML MAIL! If there's a multipart/alternative with both
# text/plain and text/html parts, nuke the text/html. Thanks for
# wasting our disk space and bandwidth...
# If you don't mind HTML mail, comment out the next line.
remove_redundant_html_parts($entity);
md_graphdefang_log('mail_in');
} # sub filter_end
If anyone wants the full mimedefang-filter I'll send it separately. I
didn't want to waste the list's bandwidth if I didn't need to. I'm pretty
sure the problem is with the SA checks. I can't think of any other reason
why mail could get through when my $RelayAddr check excluded the source of
my mail from SA checks (although they weren't working) but fails as soon
as I remove that check. The only thing I haven't yet done is revert to
the example mimedefang-filter and example sa-mimedefang.cf. I'll try both
after I send this message. Any other ideas would be much appreciated.
Thanks
Justin
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] milter timing out.
On Wed, 4 Feb 2004, Edmund wrote: I'm having the same problem here. A co-worker was trying to send a big file to another co-worker and the mail server kept on throwing a Error 451. Unfortunately, I don't know how big the file is. I read on this thread that by resetting MD it should return to normal. I did that, and it seems to have worked. But I'm certain that it'd do that again sooner or later. The problem I used to have on my RH 7.3 box was fixed in the same way. I had a cronjob restart sendmail and MD every 5 minutes just to keep things mostly working. I never did find the cause of that problem. My current problem is with MD going completely brain dead after receiving a message. I mean it accepts the message (and any new messages) for processing but it will do nothing. It simply waits for sendmail to time it out and sendmail then returns a 451. I'm going to inquire with the list about this under separate cover tomorrow, after I test a few more things. I've had to disable MD in the mean time. It's very odd. Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] making spamassassin less sensitive
On Wed, 4 Feb 2004, Muhammad Talha wrote:
RAM 256MB
Ouch. This is going to hurt you.
Swap 512
Yikes!
Sendmail-8.11.12
8.12.11?
Clamav-0.65
Upgrade to the latest snapshot. 0.65 is buggy.
Due to lack of RAM system sometime take all the Ram+Swap and i have to
restart mimedefang
Ok, you definitely need more RAM. Since you have a PIII, is it safe to
assume your box takes PC133 (not always the case but the odds are good).
RAM is ubber cheap. Buy a lot of it because you'll need it. If you go
over a certain threshold you'll need to roll a new kernel to take
advantage of it. It's not really that hard.
i want to make Spamassain less sensitive so that i can save more memory.
Steps i want to take
1) Change mimedefang-filter
if (-s ./INPUTMSG 100*1024) {
TO
if (-s ./INPUTMSG 50*1024) {
This isn't a bad idea if your spam is usually under 50k. You need to
build up a large corpus of spam (procmail is your friend) to examine for
your average spam message size to be certain.
3) change mount /var/spool/MIMEDefang as tmpfs ( size 256m )
Good idea. Disks are slow. How much mail do you normally handle though?
What's the max attachment size you've configured Sendmail to allow?
What's the maximum number of sendmail slaves you allow? Max MD slaves?
Simply put you need as much space in your spool as the largest attachment
size Sendmail will allow times the maximum number of MD slaves, and then
some. I say 'and then some' because decompressing attachments for virus
scanning will add to this. Failed messages saved in the spool will also
add to this (if you save any). tmpfs is a good thing but don't be afraid
to go overboard. You can't put too much RAM in a box. Well, I suppose
it's possible that you could hit the upper limit that your specific
machine can recognize but... This page will help you.
http://www.mimedefang.org/node.php?id=28
5) Should i had to remove some rules from /usr/share/spamasssain ??
If you're running with the stock MD filter and SA rules then none of the
network tests (DNSBL, Razor, Pyzor, DCC) checks are being run. These
usually add network delay but they can also affect overall performance.
Really there isn't anything that you can drop at this point to lessen the
load. Consider getting and SMP box. I don't build or buy any single
processor boxes any more since SMP boxes can be had for very little more.
Moral of the story: RAM, RAM and more RAM. When you think you have
enough, double it. Also use the available limits within Sendmail and MD
to prevent possible runaway train wrecks with respect to your available
resources. You can never be too safe.
HTH
Justin
___
Visit http://www.mimedefang.org and http://www.canit.ca
MIMEDefang mailing list
[EMAIL PROTECTED]
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
RE: [Mimedefang] Security note: Open port 25 on internal mail se rvers
On Wed, 4 Feb 2004 [EMAIL PROTECTED] wrote: In fact, relay-test all your machines that listen on port 25 as a matter of habit. I happened to relay check our entire campus once back when I first started my previous job. I found an entire lab of around a dozen SGI's that were of course open relays (among other things), a Sparc 20 that was an open relay (which the FBI later confiscated), and an AIX machine that was an open relay. I knew where the first two groups were but didn't know where the second was. After searching our switches SAT's for that MAC I was surprised to see that it was in the office next to mine. The office of my super, the lead sysadm. Whoops! OT story but I still find it funny. :) Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Re: [Mimedefang] Razor question
Howdy, Jos. On Wed, 28 Jan 2004, Jos De Graeve wrote: We're using v2.34: # razor-check -v Razor Agents 2.34, protocol version 3 First things first. Upgrade to the latest greatest. 2.36 was released back in May. http://razor.sourceforge.net/ That may very well fix your problem right there. Next enable DNS query logging on your primary NS for the box that's supposed to be making Razor queries. Run Razor queries by hand to see if they are successful. Update your Razor server list. If you're running an outdated copy of Razor then you probably also have an outdated server list. razor-admin -discover Check ownership and permissions and identity locations. I've always had trouble getting everything just right. Best of luck Justin ___ Visit http://www.mimedefang.org and http://www.canit.ca MIMEDefang mailing list [EMAIL PROTECTED] http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
