subject:"plugins"

> you dont need this

I see, I stand corrected!

> maybe ask how to configure extracttext ?

Sure, I'd be happy to see some examples.  The man page looks pretty
straight forward.

I see it depends on some external tools like tesseract and odt2txt so
I had better install those first.

I have not had good luck with tesseract out of the box, I wonder if
there's some options to tune it to make it work better.  Is there
anything better?

To see how well this is working, I am hoping to be able to see the
output of these tools with -D so I can write some rules.

Similarly, is there a way to see the 'body' text that is fed into the
rules?  I don't see that in the output of -D.  By 'body', I mean the
text with the html cleaned out of it plus the subject line.  I have a
message and I want to write a new body rule, I want to see what
spamassassin is using as the 'body' so I can write the regex.  I don't
see the body text in -D.




signature.asc
Description: PGP signature

Re: installing spamassassin plugins on debian

2023-03-17 Thread Benny Pedersen


Michael Grant via users skrev den 2023-03-17 17:24:


I want to try the ExtractText plugin.


is in core spamassassin 4


What if I just install this from CPAN?  It installs in
/usr/share/perl5/Mail/SpamAssassin/Plugin/ which looks correct.


dont install spamassassin core plugins from cpan unless you install all 
spamassassin from cpan



It was also recommended to me maybe use cpan2deb and install that, but
then I'm maintaining my own private debian package which I really did
not want to do.  What's wrong with just installing from CPAN in this 
case?


you dont need this

maybe ask how to configure extracttext ?

Re: installing spamassassin plugins on debian

2023-03-17 Thread Matus UHLAR - fantomas


Michael Grant via users skrev den 2023-03-17 09:52:
> What do people do to keep things up to date easily?



On Fri, Mar 17, 2023 at 04:03:03PM +0100, Benny Pedersen wrote:

i just use gentoo, or freebsd, not a precompiled problems (hehe)

but what plugin do you need with spamassassin 4 now ?

are you willing to apt maintain a custom plugin in debian ?, i see no
problem if you do this :)


On 17.03.23 12:24, Michael Grant via users wrote:

I want to try the ExtractText plugin.

What if I just install this from CPAN?  It installs in
/usr/share/perl5/Mail/SpamAssassin/Plugin/ which looks correct.


use SA 4 which has ExtractText built in.


It was also recommended to me maybe use cpan2deb and install that, but
then I'm maintaining my own private debian package which I really did
not want to do.  What's wrong with just installing from CPAN in this case?


I recommend not to mix CPAN with anything package-based.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
2B|!2B, that's a question!

Re: installing spamassassin plugins on debian

> I guess you didn't notice that you are actually installing SpamAssassin
> 4.0.0, since that's what you are looking at from CPAN?  It's part of the
> official SA package starting from 4.0.0, not a standalone plugin.

Thank you!  I did not notice that, now I see its there.  I know why, I
have 2 boxes, one with the older 3.4 and a newer one with 4.0.0.  So
that little problem is now a non-issue!




signature.asc
Description: PGP signature

Re: installing spamassassin plugins on debian

2023-03-17 Thread Henrik K

On Fri, Mar 17, 2023 at 12:24:39PM -0400, Michael Grant via users wrote:
> On Fri, Mar 17, 2023 at 04:03:03PM +0100, Benny Pedersen wrote:
> > Michael Grant via users skrev den 2023-03-17 09:52:
> > 
> > > What do people do to keep things up to date easily?
> > 
> > i just use gentoo, or freebsd, not a precompiled problems (hehe)
> > 
> > but what plugin do you need with spamassassin 4 now ?
> > 
> > are you willing to apt maintain a custom plugin in debian ?, i see no
> > problem if you do this :)
> 
> I want to try the ExtractText plugin.
> 
> What if I just install this from CPAN?  It installs in
> /usr/share/perl5/Mail/SpamAssassin/Plugin/ which looks correct.
> 
> It was also recommended to me maybe use cpan2deb and install that, but
> then I'm maintaining my own private debian package which I really did
> not want to do.  What's wrong with just installing from CPAN in this case?

https://metacpan.org/pod/Mail::SpamAssassin::Plugin::ExtractText

I guess you didn't notice that you are actually installing SpamAssassin
4.0.0, since that's what you are looking at from CPAN?  It's part of the
official SA package starting from 4.0.0, not a standalone plugin.

With luck the plugin will work even on 3.4 without troubles, but uhh yeah..

Re: installing spamassassin plugins on debian

On Fri, Mar 17, 2023 at 04:03:03PM +0100, Benny Pedersen wrote:
> Michael Grant via users skrev den 2023-03-17 09:52:
> 
> > What do people do to keep things up to date easily?
> 
> i just use gentoo, or freebsd, not a precompiled problems (hehe)
> 
> but what plugin do you need with spamassassin 4 now ?
> 
> are you willing to apt maintain a custom plugin in debian ?, i see no
> problem if you do this :)

I want to try the ExtractText plugin.

What if I just install this from CPAN?  It installs in
/usr/share/perl5/Mail/SpamAssassin/Plugin/ which looks correct.

It was also recommended to me maybe use cpan2deb and install that, but
then I'm maintaining my own private debian package which I really did
not want to do.  What's wrong with just installing from CPAN in this case?

signature.asc
Description: PGP signature

Re: installing spamassassin plugins on debian

2023-03-17 Thread Benny Pedersen


Michael Grant via users skrev den 2023-03-17 09:52:


What do people do to keep things up to date easily?


i just use gentoo, or freebsd, not a precompiled problems (hehe)

but what plugin do you need with spamassassin 4 now ?

are you willing to apt maintain a custom plugin in debian ?, i see no 
problem if you do this :)

Re: installing spamassassin plugins on debian

2023-03-17 Thread Henrik K

On Fri, Mar 17, 2023 at 05:34:37AM -0400, Michael Grant via users wrote:
> On Fri, Mar 17, 2023 at 11:26:21AM +0200, Henrik K wrote:
> > On Fri, Mar 17, 2023 at 04:52:41AM -0400, Michael Grant via users wrote:
> > > Is there a recommended way of installing a spamassassin plugin on
> > > debian (or ubuntu) such that the plugin gets updated via say apt?  I'm
> > > guessing no because I don't see many spamassassin plugins when I do an
> > > "apt search".
> > > 
> > > Up to now, I have been manually putting things in /etc/spamassassin/
> > > but I feel like there has to be a better way to manage these.
> > > 
> > > What do people do to keep things up to date easily? 
> > 
> > There is no automated handling of third party plugins.  It's up the
> > maintainers to provide or not provide any support.  Which usually just means
> > monitoring some github repo etc.
> 
> What about CPAN?  Do people use that?  It seems like there's quite a
> few modules in CPAN already.  I will admit that if I see a debian
> package, I go for that, I rarely if ever install stuff from CPAN but I
> could be convinced to use it more if this created some order out of
> the chaos.

Again, it's up to the plugin developer to publish it in CPAN or not, some
can be found there.  But it really isn't any different or more safe than
wgetting some Plugin.pm file manually from Github.  It's not recommended to
automate either way, since you could just be downloading some incompatible
or worst case, a malicious file.  Always audit and test updates manually.

Re: installing spamassassin plugins on debian

On Fri, Mar 17, 2023 at 11:26:21AM +0200, Henrik K wrote:
> On Fri, Mar 17, 2023 at 04:52:41AM -0400, Michael Grant via users wrote:
> > Is there a recommended way of installing a spamassassin plugin on
> > debian (or ubuntu) such that the plugin gets updated via say apt?  I'm
> > guessing no because I don't see many spamassassin plugins when I do an
> > "apt search".
> > 
> > Up to now, I have been manually putting things in /etc/spamassassin/
> > but I feel like there has to be a better way to manage these.
> > 
> > What do people do to keep things up to date easily? 
> 
> There is no automated handling of third party plugins.  It's up the
> maintainers to provide or not provide any support.  Which usually just means
> monitoring some github repo etc.

What about CPAN?  Do people use that?  It seems like there's quite a
few modules in CPAN already.  I will admit that if I see a debian
package, I go for that, I rarely if ever install stuff from CPAN but I
could be convinced to use it more if this created some order out of
the chaos.



signature.asc
Description: PGP signature

Re: installing spamassassin plugins on debian

2023-03-17 Thread Henrik K

On Fri, Mar 17, 2023 at 04:52:41AM -0400, Michael Grant via users wrote:
> Is there a recommended way of installing a spamassassin plugin on
> debian (or ubuntu) such that the plugin gets updated via say apt?  I'm
> guessing no because I don't see many spamassassin plugins when I do an
> "apt search".
> 
> Up to now, I have been manually putting things in /etc/spamassassin/
> but I feel like there has to be a better way to manage these.
> 
> What do people do to keep things up to date easily? 

There is no automated handling of third party plugins.  It's up the
maintainers to provide or not provide any support.  Which usually just means
monitoring some github repo etc.

installing spamassassin plugins on debian

Is there a recommended way of installing a spamassassin plugin on
debian (or ubuntu) such that the plugin gets updated via say apt?  I'm
guessing no because I don't see many spamassassin plugins when I do an
"apt search".

Up to now, I have been manually putting things in /etc/spamassassin/
but I feel like there has to be a better way to manage these.

What do people do to keep things up to date easily? 


signature.asc
Description: PGP signature

Re: Install plugins into embedded spamassassin

2023-02-27 Thread hg user

Hi Riccardo,
thank you.

Yes, the directories are those, I was expecting something different
but actually in this way everything is in a user-controlled dir.

Re: Install plugins into embedded spamassassin

2023-02-27 Thread Riccardo Alfieri

If you'd like you can take inspiration from the instructions the Zimbra 
people wrote for our own plugin here: 
https://wiki.zimbra.com/wiki/Spamhaus_HBL


Just use the correct file name for yours and you should be fine

On 25/02/23 15:30, hg user wrote:

Hi,
I'd like to install at least one plugin in my embedded spamassassin, 
installed inside Zimbra.

I'm a bit afraid of breaking stuff, about missing dependencies and so on.

I'm on SA 3.4.5 and - as a test - I'd like to install ESP plugin.


--
Best regards,
Riccardo Alfieri

Spamhaus Technology
https://www.spamhaus.com/

Re: Install plugins into embedded spamassassin

2023-02-26 Thread hg user

Grazie Giovanni,
bundled is probably a better word than embedded.

Probably the dirs are different but the steps are those. I suppose the RBLs
used by esp plugin are free to use... are them?

Antony, zimbra people will come next week to clean up some errors in the
setup they left after an upgrade :-) so I'd like to understand myself what
to do.

What plugins should be "mandatory" in 2023 ? And also useful for the
italian language?

On Sun, Feb 26, 2023 at 4:30 PM Giovanni Bechis  wrote:

> On Sat, Feb 25, 2023 at 03:30:13PM +0100, hg user wrote:
> > Hi,
> > I'd like to install at least one plugin in my embedded spamassassin,
> > installed inside Zimbra.
> > I'm a bit afraid of breaking stuff, about missing dependencies and so on.
> >
> > I'm on SA 3.4.5 and - as a test - I'd like to install ESP plugin.
> Zimbra uses standard SA, it's just bundled in their software.
> To install an additional plugin you should create
> /etc/mail/spamassassin/ESP.pre
> file with this content:
> loadplugin Mail::SpamAssassin::Plugin::Esp Esp.pm
> And add Esp.pm and Esp.cf to /etc/mail/spamassassin/.
> Same for other plugins you might need.
> Zimbra uses amavisd-new, so you need to reload amavisd-new as well when
> you change SpamAssassin configurations.
>
>  Giovanni
>

Re: Install plugins into embedded spamassassin

2023-02-26 Thread Giovanni Bechis

On Sat, Feb 25, 2023 at 03:30:13PM +0100, hg user wrote:
> Hi,
> I'd like to install at least one plugin in my embedded spamassassin,
> installed inside Zimbra.
> I'm a bit afraid of breaking stuff, about missing dependencies and so on.
> 
> I'm on SA 3.4.5 and - as a test - I'd like to install ESP plugin.
Zimbra uses standard SA, it's just bundled in their software.
To install an additional plugin you should create /etc/mail/spamassassin/ESP.pre
file with this content:
loadplugin Mail::SpamAssassin::Plugin::Esp Esp.pm
And add Esp.pm and Esp.cf to /etc/mail/spamassassin/.
Same for other plugins you might need.
Zimbra uses amavisd-new, so you need to reload amavisd-new as well when
you change SpamAssassin configurations.

 Giovanni


signature.asc
Description: PGP signature

Re: Install plugins into embedded spamassassin

2023-02-25 Thread Antony Stone

On Saturday 25 February 2023 at 15:30:13, hg user wrote:

> Hi,
> I'd like to install at least one plugin in my embedded spamassassin,
> installed inside Zimbra.
> I'm a bit afraid of breaking stuff, about missing dependencies and so on.
> 
> I'm on SA 3.4.5 and - as a test - I'd like to install ESP plugin.

You might well be better off asking the Zimbra people, assuming that this 
"embedding" was done by them.

People here will know about "standard SA" but how it's been integrated into 
another product is going to be that other product's area of expertise.

Antony.

-- 
The GNU General Public Licence was first published on this day in 1989
https://www.gnu.org/licences/gpl.html

   Please reply to the list;
 please *don't* CC me.

Install plugins into embedded spamassassin

2023-02-25 Thread hg user

Hi,
I'd like to install at least one plugin in my embedded spamassassin,
installed inside Zimbra.
I'm a bit afraid of breaking stuff, about missing dependencies and so on.

I'm on SA 3.4.5 and - as a test - I'd like to install ESP plugin.

Re: sa-update --allow-plugins [was: sa-update not properly parsing urls in MIRRORED.BY files?]

2019-01-12 Thread Henrik K

On Sat, Jan 12, 2019 at 02:10:37PM -0500, listsb wrote:
> 
> that said, i don't quite follow the second statement, if i'm honest.  i
> suppose that some people may run sa-update or spamassassin as root, but i
> don't, and would be filing bugs against any packagers or distributors that
> were delivering it this way.

Things like virtual users or privileged ports require starting as root. 
Even if it is configured to switch users, it can run some linting/compiling
stuff from plugins as root.

> that said, i would think that if there were to be any channel that should
> be trusted to deliver safe plugins [regardless of if the code involved
> were to run as either a privileged or non-privileged user], it would be
> the official channel, wouldn't it?

Sure, but I think it's just a legacy extra feature that has never been used,
and in my opinion there's no reason to use it ever.  Most people don't have
the option enabled anyway, so it would be pointless to distribute anything,
that's what version updates are for..

sa-update --allow-plugins [was: sa-update not properly parsing urls in MIRRORED.BY files?]

2019-01-12 Thread listsb

> On Jan 11, 2019, at 10.55, Henrik K  wrote:
> 
> On Wed, Jan 09, 2019 at 11:59:36PM -0500, listsb wrote:
>> 
>>> sa-update -vvv --allowplugins ...
> 
> Just a general note, I would never ever use --allowplugins unless it's your
> personal channel.  There is no reason why official channels should ever
> distribute plugins as it would be basically remote code run as root.

thanks for mentioning this.  i'd wondered about that - the documentation 
["Allow downloaded updates to activate plugins."] doesn't quite express what 
exactly --allowplugins does/means, imho.  i would like to better understand 
this.

that said, i don't quite follow the second statement, if i'm honest.  i suppose 
that some people may run sa-update or spamassassin as root, but i don't, and 
would be filing bugs against any packagers or distributors that were delivering 
it this way.  that said, i would think that if there were to be any channel 
that should be trusted to deliver safe plugins [regardless of if the code 
involved were to run as either a privileged or non-privileged user], it would 
be the official channel, wouldn't it?

take a look @ 2 great plugins

2016-10-03 Thread Nicola Piazzi

http://saplugin.16mb.com/

And tell me how it works 

Nicola Piazzi
CED - Sistemi
COMET s.p.a.
Via Michelino, 105 - 40127 Bologna - Italia
Tel.  +39 051.6079.293
Cell. +39 328.21.73.470
Web: www.gruppocomet.it
[Descrizione: gc]

2 Plugins

2016-08-30 Thread Nicola Piazzi

Here 2 plugins selfmade
http://saplugin.16mb.com/

If someone send me a feedback it will be appreciate

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

2016-02-08 Thread Henrik K


On Tue, Feb 09, 2016 at 08:27:08AM +0100, Benning, Markus wrote:
> On 2016-02-05 16:25, Henrik K wrote:
> >You should start using the bundled Mail/SpamAssassin/Util/TinyRedis.pm
> >instead of cpan Redis module..
> 
> I didn't notice theres a Redis module included with SpamAssassin.
> It seems like the module is undocumented.
> 
> What does the SA Redis module different from the mainline module?

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6972

It's made by Mark so quality is assured and it's always bad to rely on
external modules.

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

2016-02-08 Thread Benning, Markus


On 2016-02-05 16:25, Henrik K wrote:

You should start using the bundled Mail/SpamAssassin/Util/TinyRedis.pm
instead of cpan Redis module..


I didn't notice theres a Redis module included with SpamAssassin.
It seems like the module is undocumented.

What does the SA Redis module different from the mainline module?

Markus

--
https://markusbenning.de/

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

2016-02-05 Thread Henrik K

On Fri, Feb 05, 2016 at 03:17:21PM +0100, Benning, Markus wrote:
> On 2016-02-04 22:42, Skeffling wrote:
> >Sorry to reply to an old thread, but is it possible for the the
> >RedisAWL
> >plugin to specify a redis password too? (I'm looking to use TxRep
> >with a
> >redis backend...)
> 
> Just released version 1.001:
> 
> commit 4f7138e4823379f7728bcca005611d2a6640cf93
> Author: Markus Benning 
> Date:   Fri Feb 5 15:11:45 2016 +0100
> 
> Additional redis connection parameters
> 
> The followin configuration parameters have been added:
> 
>   * auto_whitelist_redis_password
>   * auto_whitelist_redis_database
>   * auto_whitelist_redis_debug

You should start using the bundled Mail/SpamAssassin/Util/TinyRedis.pm
instead of cpan Redis module..

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

2016-02-05 Thread Benning, Markus


On 2016-02-04 22:42, Skeffling wrote:
Sorry to reply to an old thread, but is it possible for the the 
RedisAWL
plugin to specify a redis password too? (I'm looking to use TxRep with 
a

redis backend...)


Just released version 1.001:

commit 4f7138e4823379f7728bcca005611d2a6640cf93
Author: Markus Benning 
Date:   Fri Feb 5 15:11:45 2016 +0100

Additional redis connection parameters

The followin configuration parameters have been added:

  * auto_whitelist_redis_password
  * auto_whitelist_redis_database
  * auto_whitelist_redis_debug

- Markus


--
https://markusbenning.de/

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

2016-02-04 Thread Skeffling

On 16/07/2015 19:58, Benning, Markus wrote:
> Hi Patrik,
> 
> i just pushed Version 1.002 to github and CPAN:
> 
> -- 
> The following new features have been added:
> 
>   - New option: timing_redis_password allows to specifiy a redis password

Sorry to reply to an old thread, but is it possible for the the RedisAWL
plugin to specify a redis password too? (I'm looking to use TxRep with a
redis backend...)

Thanks.

s.

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

2015-09-15 Thread Martin Gregorie

On Tue, 2015-09-15 at 08:41 -0700, Ian Zimmerman wrote:
> On 2015-06-09 17:57 +0200, Benning, Markus wrote:
> 
> > RuleTimingRedis - collect SA rule timings in redis
> 
> I'm trying this out.  I have a little annoying problem: the logs
> beginning on line 178 seem to go to stdout or stderr as well as
> syslog.
> The result is that cron sends me email every time spamd is restarted
> (after every rule update).  Do you know how to change that?  I find
> nothing about logging in perldoc Mail::SpamAssassin::Conf.
> 
Assuming your sa_update script is the same as mine (I run Fedora Linux)
The obvious quick fix is to change line 2 in case 0) to read:

   service spamassassin restart;;

and remove lines 1 and 3, which (in my script) are both 'echo' commands
reporting that the update completed and that Spamassassin was
restarted. If you're still seeing the unwanted messages, replace the
remaining line with:

   service spamassassin restart 1>/dev/null 2>/dev.null;; 

will will suppress anything that Spamassassin sends to stdout and
stderr. The sa_update script I have would still report updating errors
and the fact that there was no rules update because these are handled
by case 1, 2 and * and so can't be silenced by this edit.

> I suppose I could just delete those lines from the module :-)  But 
> then I would have extra work when I merge with any new versions you
> have.
> 
Doing what I suggest would be easier than editing SA itself. Keep a
copy of your cron script when its doing what you want so, if an upgrade
slots in a new sa_update script that lets the unwanted text through
again, you can simply slot your modified version back in.


Martin

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

2015-09-15 Thread Ian Zimmerman

On 2015-06-09 17:57 +0200, Benning, Markus wrote:

> RuleTimingRedis - collect SA rule timings in redis

I'm trying this out.  I have a little annoying problem: the logs
beginning on line 178 seem to go to stdout or stderr as well as syslog.
The result is that cron sends me email every time spamd is restarted
(after every rule update).  Do you know how to change that?  I find
nothing about logging in perldoc Mail::SpamAssassin::Conf.

I suppose I could just delete those lines from the module :-)  But then
I would have extra work when I merge with any new versions you have.

Thanks for your ideas.

-- 
Please *no* private copies of mailing list or newsgroup messages.
Rule 420: All persons more than eight miles high to leave the court.

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

2015-07-16 Thread Patrick Ben Koetter

Markus,

* Benning, Markus i...@markusbenning.de:
 Hi Patrik,
 
 i just pushed Version 1.002 to github and CPAN:
 
 --
 The following new features have been added:
 
   - New option: timing_redis_password allows to specifiy a redis
 password
 
   - New option: timing_redis_exclude_re excludes rules from timing
 statistics. By default set to '^__' which will exclude all sub-rules
 
   - New option: timing_redis_database allows to select a non-default
 database in redis. (redis SWITCH call)
 
   - New option: timing_redis_bulk_update will queue timing updates
 before sending them to redis and execute them in a bulk via a
 single call to a server-side script. By default this option is set
 to 50 entries. Set to 0 do disable. Requires redis = 2.6.0 and a
 Redis perl = 1.954 module.
 --
 
 I'm currently not using it on a system where the overhead is
 relevant for me, but
 i tried to reduce the number of redis command executed.
 I hope this will reduce the overhead significant.


that's great news. Thanks!

 Feedback and test results welcome.

I will, as soon as I have something to share!

p@rick


 Am 2015-07-15 23:22, schrieb Patrick Ben Koetter:
 Markus,
 
 are you planning to add 'password' and 'database ID' support for redis
 connects to RuleTimingRedis?
 
 What's your experience regarding Timing overhead? My simple tests
 on the
 commandlne show about 1 second overhead when RuleTimingRedis is added:
 
 # Without RuleTimingRedis
 mail# time spamassassin --lint
 
 real0m1.975s
 user0m1.852s
 sys 0m0.116s
 
 # Enable RuleTimingRedis
 mail# vim /etc/mail/spamassassin/init.pre
 
 # With RuleTimingRedis
 mail# time spamassassin --lint
 
 real0m2.828s
 user0m2.128s
 sys 0m0.392s
 
 p@rick
 
 
 
 * Benning, Markus i...@markusbenning.de:
 Hello,
 
 i want to announce the release of the SpamAssassin Plugins:
 
 RedisAWL - redis support for spamassassin AWL/TxRep
 RuleTimingRedis - collect SA rule timings in redis
 
 Both can be downloaded from CPAN or GitHub:
 
 https://metacpan.org/author/BENNING
 
 https://github.com/benningm
 
 Timings gathered with the RuleTimingRedis plugin can be used in
 collectd
 with the Collectd-Plugins-RedisClient module also available from CPAN.
 
  Markus
 
 --
 Markus Benning, https://markusbenning.de/
 
 -- 
 Markus Benning, https://markusbenning.de/

-- 
[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München
 
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

2015-07-16 Thread Benning, Markus


Hi Patrik,

i just pushed Version 1.002 to github and CPAN:

--
The following new features have been added:

  - New option: timing_redis_password allows to specifiy a redis 
password


  - New option: timing_redis_exclude_re excludes rules from timing
statistics. By default set to '^__' which will exclude all sub-rules

  - New option: timing_redis_database allows to select a non-default
database in redis. (redis SWITCH call)

  - New option: timing_redis_bulk_update will queue timing updates
before sending them to redis and execute them in a bulk via a
single call to a server-side script. By default this option is set
to 50 entries. Set to 0 do disable. Requires redis = 2.6.0 and a
Redis perl = 1.954 module.
--

I'm currently not using it on a system where the overhead is relevant 
for me, but

i tried to reduce the number of redis command executed.
I hope this will reduce the overhead significant.

Feedback and test results welcome.

Markus

Am 2015-07-15 23:22, schrieb Patrick Ben Koetter:

Markus,

are you planning to add 'password' and 'database ID' support for redis
connects to RuleTimingRedis?

What's your experience regarding Timing overhead? My simple tests on 
the

commandlne show about 1 second overhead when RuleTimingRedis is added:

# Without RuleTimingRedis
mail# time spamassassin --lint

real0m1.975s
user0m1.852s
sys 0m0.116s

# Enable RuleTimingRedis
mail# vim /etc/mail/spamassassin/init.pre

# With RuleTimingRedis
mail# time spamassassin --lint

real0m2.828s
user0m2.128s
sys 0m0.392s

p@rick



* Benning, Markus i...@markusbenning.de:

Hello,

i want to announce the release of the SpamAssassin Plugins:

RedisAWL - redis support for spamassassin AWL/TxRep
RuleTimingRedis - collect SA rule timings in redis

Both can be downloaded from CPAN or GitHub:

https://metacpan.org/author/BENNING

https://github.com/benningm

Timings gathered with the RuleTimingRedis plugin can be used in 
collectd

with the Collectd-Plugins-RedisClient module also available from CPAN.

 Markus

--
Markus Benning, https://markusbenning.de/


--
Markus Benning, https://markusbenning.de/

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

2015-07-15 Thread Patrick Ben Koetter

Markus,

are you planning to add 'password' and 'database ID' support for redis
connects to RuleTimingRedis?

What's your experience regarding Timing overhead? My simple tests on the
commandlne show about 1 second overhead when RuleTimingRedis is added:

# Without RuleTimingRedis
mail# time spamassassin --lint

real0m1.975s
user0m1.852s
sys 0m0.116s

# Enable RuleTimingRedis
mail# vim /etc/mail/spamassassin/init.pre 

# With RuleTimingRedis
mail# time spamassassin --lint

real0m2.828s
user0m2.128s
sys 0m0.392s

p@rick



* Benning, Markus i...@markusbenning.de:
 Hello,
 
 i want to announce the release of the SpamAssassin Plugins:
 
 RedisAWL - redis support for spamassassin AWL/TxRep
 RuleTimingRedis - collect SA rule timings in redis
 
 Both can be downloaded from CPAN or GitHub:
 
 https://metacpan.org/author/BENNING
 
 https://github.com/benningm
 
 Timings gathered with the RuleTimingRedis plugin can be used in collectd
 with the Collectd-Plugins-RedisClient module also available from CPAN.
 
  Markus
 
 -- 
 Markus Benning, https://markusbenning.de/

-- 
[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München
 
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein

Re: Must-Have Plugins?

2015-06-23 Thread Reindl Harald




Am 24.06.2015 um 02:00 schrieb Philip Prindeville:



On 06/19/2015 01:07 PM, Dianne Skoll wrote:

On Fri, 19 Jun 2015 12:51:28 -0600
Philip Prindeville philipp_s...@redfish-solutions.com wrote:

[stuff]


With this, we avoid ever accepting about 98% of the SPAM that we’d
otherwise receive.

Really?  98%?  I find that surprising.  We get quite a lot of spam
from gmail, hotmail, yahoo etc. that would pass all of your tests.

Regards,

Dianne.


I should have mentioned we also blacklist yahoo... and are thinking
about blocking google, too


don't forget facebbok (xfdw) and gmail, but why not just blcklist 
anything except whitelisted? :-)




signature.asc
Description: OpenPGP digital signature

Re: Must-Have Plugins?

2015-06-23 Thread Philip Prindeville




On 06/19/2015 01:07 PM, Dianne Skoll wrote:

On Fri, 19 Jun 2015 12:51:28 -0600
Philip Prindeville philipp_s...@redfish-solutions.com wrote:

[stuff]


With this, we avoid ever accepting about 98% of the SPAM that we’d
otherwise receive.

Really?  98%?  I find that surprising.  We get quite a lot of spam
from gmail, hotmail, yahoo etc. that would pass all of your tests.

Regards,

Dianne.


I should have mentioned we also blacklist yahoo... and are thinking 
about blocking google, too.

Re: Must-Have Plugins?

2015-06-23 Thread Dianne Skoll

On Tue, 23 Jun 2015 18:00:27 -0600
Philip Prindeville philipp_s...@redfish-solutions.com wrote:

 I should have mentioned we also blacklist yahoo... and are thinking 
 about blocking google, too.

I see.  If we did this, then yes, we'd probably stop a lot of spam
(though nowhere near 98%) but we'd also lose 98% of our customers,
or more like 99.999%.

Implementing a spam filter for 20 probably highly-techie recipients
imposes vastly different constraints than doing it for hundreds of
thousands of mostly non-technical people. :)

Regards,

Dianne.

Re: Must-Have Plugins?

2015-06-19 Thread David Jones

 From: Philip Prindeville philipp_s...@redfish-solutions.com

 On Jun 9, 2015, at 12:29 PM, John Hardin jhar...@impsec.org wrote:

 On Tue, 9 Jun 2015, David Jones wrote:

 Some of the best and easiest things you can enable to block spam are
 outside of SpamAssassin at your MTA (sendmail, postfix, etc.).

 - Enable greylisting.  This is just about the only way you can block
 zero-hour spam from compromised accounts that come from legit mail
 servers before they get listed in RBLs.

 Just bear in mind some commercial organizations may be very hostile to 
 anything that delays delivery of mail, regardless of how much it would 
 reduce spam.

 Two things that I have found very useful at the MTA level are:

 (1) Delay sending your SMTP banner a second or two and reject any sender 
 that starts sending information before that. This is a built-in option in 
 Sendmail, google greet_pause.

 (2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e. 
 it's not got any periods at all). This probably shouldn't be done on mail 
 originating locally, but for mail coming in from the Internet the other 
 MTA should always be sending a FQDN in the HELO. A non-FQDN HELO is a 
 pretty good sign of a spambot sending from a compromised workstation or PC 
 directly to your MTA.

 I have some other MTA checks in place, but these two block the most.


 We use MimeDefang and it actually calls SpamAssassin.  But before we accept 
 connections (or email), we make a bunch of tests.

 We use Geo::IP to block a bunch of ISP’s and countries that are hostile in 
 filter_relay().

 These include ColoCrossing, Enzu, Datashack, Proxad, WholesaleInternet, etc.

 Then we apply more tests in filter_helo().  Note, these are only for 
 connections that arrive on port 25, not on port 587
 (which is for local clients to submit, and is authenticated with a 
 username/password pair).

 We accept connections from 127.0.0.1 with no further checks.

 We block anyone saying HELO \d+\.\d+\.\d+\.\d+ because it’s missing the 
 required brackets.  We block any bracketed dotted quads that have an 
 invalid quad (i.e. 300.300.300.300) or where the address they claim to be 
 from is different from the actual address we’re seeing (no one behind a 
 NATting firewall should be connecting to us unless they know their 
 [static] external address).

 We block anyone listed by ZenBL.

 We block (a) anyone claiming to be us, (b) anyone claiming to be 
 “localhost” or “localhost.localdomain”.

 We block a list of names that are always fraudulent, like “paypal.com” 
 (without any subdomains), or “smtp.communitel.net” which seems to get 
 spoofed a LOT, or “mail.com” or “mail.ru”.  We block hostnames that don’t 
 have a domain (no dots).

 Lastly, we TEMPFAIL hosts that don’t have valid rDNS mappings (including 
 the A and PTR records not agreeing).

 With this, we avoid ever accepting about 98% of the SPAM that we’d 
 otherwise receive.

 Good information.  Thanks for taking the time to document it for this list.
 How many mailboxes do you filter with this configuration?

Not that many.  About 20.

That's cool.  Mail filtering (probably any filtering) is not a 
one-size-fits-all.  I am able to do many of the things
you mentioned above but some things will break too many legit senders with 
incorrect FCrDNS (PTR = HELO).

But I’m on a LOT of high volume mailing lists (like mozilla-general and 
netdev) that get heavily spammed.

Filtering mailing lists is a slightly different ballgame than filtering regular 
email.  Some of the items listed above
don't apply to or won't work with mailing lists (as Dianne Skoll mentioned) 
since they are like proxies of the
original sender's mail server.

Dave

Re: Must-Have Plugins?


On Jun 19, 2015, at 2:35 PM, David Jones djo...@ena.com wrote:

 
 But I’m on a LOT of high volume mailing lists (like mozilla-general and 
 netdev) that get heavily spammed.
 
 Filtering mailing lists is a slightly different ballgame than filtering 
 regular email.  Some of the items listed above
 don't apply to or won't work with mailing lists (as Dianne Skoll mentioned) 
 since they are like proxies of the
 original sender's mail server.
 
 Dave

Sorry, I also meant that many of those mailing lists are harvested… so my 
address has been bought and sold many, many times.

Re: Must-Have Plugins?

2015-06-19 Thread David Jones

From: Philip Prindeville philipp_s...@redfish-solutions.com
Sent: Friday, June 19, 2015 3:53 PM
To: David Jones
Cc: users@spamassassin.apache.org
Subject: Re: Must-Have Plugins?

On Jun 19, 2015, at 2:35 PM, David Jones djo...@ena.com wrote:

 But I’m on a LOT of high volume mailing lists (like mozilla-general and 
 netdev) that get heavily spammed.

 Filtering mailing lists is a slightly different ballgame than filtering 
 regular email.  Some of the items listed above
 don't apply to or won't work with mailing lists (as Dianne Skoll mentioned) 
 since they are like proxies of the
 original sender's mail server.

 Dave

Sorry, I also meant that many of those mailing lists are harvested… so my 
address has been bought and sold many, many times.

I see.  For email addresses that have gotten on those lists, what I have found 
to be effective is to
focus more on the reputation of the sending mail server.  Some mail servers 
like mailchimp,
sendgrid, constant contact, etc. will get these addresses but you can safely 
unsubscribe from
them and eventually get off of their lists over a few weeks.
Most of the other mail servers can be blocked by the major RBLs and the other 
techniques you
mentioned in your original post.  There are safe ways to whitelist specific 
sending IPs for domains
where you don't have to put in a risky whitelist entry at the MTA level that 
will open you up to
spoofing problems.  This usually requires a little scripting to pull data that 
has been vetted by the
SA level then tie it back to the MTA level.
The rest that gets to SA needs to be handled by properly trained Bayes, DBLs, 
custom rules
like KAM.cf, etc.

Re: Must-Have Plugins?




On 06/10/2015 04:34 AM, Amir Caspi wrote:

On Jun 10, 2015, at 12:32 AM, Matus UHLAR - fantomas uh...@fantomas.sk wrote:


FEATURE(`block_bad_helo')
define(`confALLOW_BOGUS_HELO', `False')

Argh, unfortunately, that feature is only on sendmail 8.14 and higher, which 
means RHEL/CentOS 6 or higher.  For those of us running RHEL/CentOS 5, that's 
only available via a custom install. =(

Does anyone know of a reputable RPM distro for sendmail 8.14+ for CentOS 5?  I 
can't find anything decent via Google, everything is for CentOS 6 or higher.

(My server setup requires RPMs, so I can't build from a source tarball. I could 
potentially use the source RPM from CentOS 6 to get a custom RPM for 5, 
although even that is problematic.)

Bleh.  I wish I could upgrade this server to a newer OS, but various 
circumstances prevent that right now.

--- Amir



Given how many vulnerabilities CentOS 5 has, why would you want to keep 
running that?

Re: Must-Have Plugins?

On Jun 19, 2015, at 3:28 PM, David Jones djo...@ena.com wrote:

 From: Philip Prindeville philipp_s...@redfish-solutions.com
 Sent: Friday, June 19, 2015 3:53 PM
 To: David Jones
 Cc: users@spamassassin.apache.org
 Subject: Re: Must-Have Plugins?

 On Jun 19, 2015, at 2:35 PM, David Jones djo...@ena.com wrote:

 But I’m on a LOT of high volume mailing lists (like mozilla-general and 
 netdev) that get heavily spammed.

 Filtering mailing lists is a slightly different ballgame than filtering 
 regular email.  Some of the items listed above
 don't apply to or won't work with mailing lists (as Dianne Skoll mentioned) 
 since they are like proxies of the
 original sender's mail server.

 Dave

 Sorry, I also meant that many of those mailing lists are harvested… so my 
 address has been bought and sold many, many times.

 I see.  For email addresses that have gotten on those lists, what I have 
 found to be effective is to
 focus more on the reputation of the sending mail server.  Some mail servers 
 like mailchimp,
 sendgrid, constant contact, etc. will get these addresses but you can safely 
 unsubscribe from
 them and eventually get off of their lists over a few weeks.
 Most of the other mail servers can be blocked by the major RBLs and the other 
 techniques you
 mentioned in your original post.  There are safe ways to whitelist specific 
 sending IPs for domains
 where you don't have to put in a risky whitelist entry at the MTA level that 
 will open you up to
 spoofing problems.  This usually requires a little scripting to pull data 
 that has been vetted by the
 SA level then tie it back to the MTA level.
 The rest that gets to SA needs to be handled by properly trained Bayes, DBLs, 
 custom rules
 like KAM.cf, etc.

Well that was interesting.  How long at Hetzner been hosting one of the spam 
assassin.apache.org mirrors?

Because they have a very low reputation with us.

So your message, which includes the text “spamassassin.apache.org” got flagged 
and quarantined.

More than a little ironic…

Re: Must-Have Plugins?

2015-06-19 Thread Amir Caspi

On Jun 19, 2015, at 6:02 PM, Philip Prindeville 
philipp_s...@redfish-solutions.com wrote:
 Given how many vulnerabilities CentOS 5 has, why would you want to keep 
 running that?

Because, while I wish I could upgrade ... various circumstances prevent that 
right now.

It is fully patched, FWIW.

--- Amir
thumbed via iPhone

Re: Must-Have Plugins?


On Jun 19, 2015, at 1:01 PM, David Jones djo...@ena.com wrote:

 From: Philip Prindeville philipp_s...@redfish-solutions.com
 
 On Jun 9, 2015, at 12:29 PM, John Hardin jhar...@impsec.org wrote:
 
 On Tue, 9 Jun 2015, David Jones wrote:
 
 Some of the best and easiest things you can enable to block spam are
 outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
 
 - Enable greylisting.  This is just about the only way you can block
 zero-hour spam from compromised accounts that come from legit mail
 servers before they get listed in RBLs.
 
 Just bear in mind some commercial organizations may be very hostile to 
 anything that delays delivery of mail, regardless of how much it would 
 reduce spam.
 
 Two things that I have found very useful at the MTA level are:
 
 (1) Delay sending your SMTP banner a second or two and reject any sender 
 that starts sending information before that. This is a built-in option in 
 Sendmail, google greet_pause.
 
 (2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e. 
 it's not got any periods at all). This probably shouldn't be done on mail 
 originating locally, but for mail coming in from the Internet the other MTA 
 should always be sending a FQDN in the HELO. A non-FQDN HELO is a pretty 
 good sign of a spambot sending from a compromised workstation or PC 
 directly to your MTA.
 
 I have some other MTA checks in place, but these two block the most.
 
 
 We use MimeDefang and it actually calls SpamAssassin.  But before we accept 
 connections (or email), we make a bunch of tests.
 
 We use Geo::IP to block a bunch of ISP’s and countries that are hostile in 
 filter_relay().
 
 These include ColoCrossing, Enzu, Datashack, Proxad, WholesaleInternet, etc.
 
 Then we apply more tests in filter_helo().  Note, these are only for 
 connections that arrive on port 25, not on port 587
 (which is for local clients to submit, and is authenticated with a 
 username/password pair).
 
 We accept connections from 127.0.0.1 with no further checks.
 
 We block anyone saying HELO \d+\.\d+\.\d+\.\d+ because it’s missing the 
 required brackets.  We block any bracketed dotted quads that have an 
 invalid quad (i.e. 300.300.300.300) or where the address they claim to be 
 from is different from the actual address we’re seeing (no one behind a 
 NATting firewall should be connecting to us unless they know their [static] 
 external address).
 
 We block anyone listed by ZenBL.
 
 We block (a) anyone claiming to be us, (b) anyone claiming to be “localhost” 
 or “localhost.localdomain”.
 
 We block a list of names that are always fraudulent, like “paypal.com” 
 (without any subdomains), or “smtp.communitel.net” which seems to get 
 spoofed a LOT, or “mail.com” or “mail.ru”.  We block hostnames that don’t 
 have a domain (no dots).
 
 Lastly, we TEMPFAIL hosts that don’t have valid rDNS mappings (including the 
 A and PTR records not agreeing).
 
 With this, we avoid ever accepting about 98% of the SPAM that we’d otherwise 
 receive.
 
 Good information.  Thanks for taking the time to document it for this list.
 How many mailboxes do you filter with this configuration?


Not that many.  About 20.

But I’m on a LOT of high volume mailing lists (like mozilla-general and netdev) 
that get heavily spammed.

-Philip


 
 -Philip

Re: Must-Have Plugins?

2015-06-19 Thread David Jones

From: Philip Prindeville philipp_s...@redfish-solutions.com

On Jun 9, 2015, at 12:29 PM, John Hardin jhar...@impsec.org wrote:

 On Tue, 9 Jun 2015, David Jones wrote:

 Some of the best and easiest things you can enable to block spam are
 outside of SpamAssassin at your MTA (sendmail, postfix, etc.).

 - Enable greylisting.  This is just about the only way you can block
  zero-hour spam from compromised accounts that come from legit mail
  servers before they get listed in RBLs.

 Just bear in mind some commercial organizations may be very hostile to 
 anything that delays delivery of mail, regardless of how much it would 
 reduce spam.

 Two things that I have found very useful at the MTA level are:

 (1) Delay sending your SMTP banner a second or two and reject any sender 
 that starts sending information before that. This is a built-in option in 
 Sendmail, google greet_pause.

 (2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e. 
 it's not got any periods at all). This probably shouldn't be done on mail 
 originating locally, but for mail coming in from the Internet the other MTA 
 should always be sending a FQDN in the HELO. A non-FQDN HELO is a pretty 
 good sign of a spambot sending from a compromised workstation or PC directly 
 to your MTA.

 I have some other MTA checks in place, but these two block the most.


We use MimeDefang and it actually calls SpamAssassin.  But before we accept 
connections (or email), we make a bunch of tests.

We use Geo::IP to block a bunch of ISP’s and countries that are hostile in 
filter_relay().

These include ColoCrossing, Enzu, Datashack, Proxad, WholesaleInternet, etc.

Then we apply more tests in filter_helo().  Note, these are only for 
connections that arrive on port 25, not on port 587
(which is for local clients to submit, and is authenticated with a 
username/password pair).

We accept connections from 127.0.0.1 with no further checks.

We block anyone saying HELO \d+\.\d+\.\d+\.\d+ because it’s missing the 
required brackets.  We block any bracketed dotted quads that have an invalid 
quad (i.e. 300.300.300.300) or where the address they claim to be from is 
different from the actual address we’re seeing (no one behind a NATting 
firewall should be connecting to us unless they know their [static] external 
address).

We block anyone listed by ZenBL.

We block (a) anyone claiming to be us, (b) anyone claiming to be “localhost” 
or “localhost.localdomain”.

We block a list of names that are always fraudulent, like “paypal.com” 
(without any subdomains), or “smtp.communitel.net” which seems to get spoofed 
a LOT, or “mail.com” or “mail.ru”.  We block hostnames that don’t have a 
domain (no dots).

Lastly, we TEMPFAIL hosts that don’t have valid rDNS mappings (including the A 
and PTR records not agreeing).

With this, we avoid ever accepting about 98% of the SPAM that we’d otherwise 
receive.

Good information.  Thanks for taking the time to document it for this list.
How many mailboxes do you filter with this configuration?

-Philip

Re: Must-Have Plugins?

2015-06-19 Thread Dianne Skoll

On Fri, 19 Jun 2015 12:51:28 -0600
Philip Prindeville philipp_s...@redfish-solutions.com wrote:

[stuff]

 With this, we avoid ever accepting about 98% of the SPAM that we’d
 otherwise receive.

Really?  98%?  I find that surprising.  We get quite a lot of spam
from gmail, hotmail, yahoo etc. that would pass all of your tests.

Regards,

Dianne.

Re: Must-Have Plugins?

On Jun 9, 2015, at 12:29 PM, John Hardin jhar...@impsec.org wrote:

On Tue, 9 Jun 2015, David Jones wrote:

Some of the best and easiest things you can enable to block spam are
outside of SpamAssassin at your MTA (sendmail, postfix, etc.).

- Enable greylisting. This is just about the only way you can block
zero-hour spam from compromised accounts that come from legit mail
servers before they get listed in RBLs.

Just bear in mind some commercial organizations may be very hostile to
anything that delays delivery of mail, regardless of how much it would reduce
spam.

Two things that I have found very useful at the MTA level are:

(1) Delay sending your SMTP banner a second or two and reject any sender that
starts sending information before that. This is a built-in option in
Sendmail, google greet_pause.

(2) Check the HELO the other guy sends and reject if it's not a FQDN (i.e.
it's not got any periods at all). This probably shouldn't be done on mail
originating locally, but for mail coming in from the Internet the other MTA
should always be sending a FQDN in the HELO. A non-FQDN HELO is a pretty good
sign of a spambot sending from a compromised workstation or PC directly to
your MTA.

I have some other MTA checks in place, but these two block the most.

We use MimeDefang and it actually calls SpamAssassin. But before we accept
connections (or email), we make a bunch of tests.

We use Geo::IP to block a bunch of ISP’s and countries that are hostile in
filter_relay().

These include ColoCrossing, Enzu, Datashack, Proxad, WholesaleInternet, etc.

Then we apply more tests in filter_helo(). Note, these are only for
connections that arrive on port 25, not on port 587 (which is for local clients
to submit, and is authenticated with a username/password pair).

We accept connections from 127.0.0.1 with no further checks.

We block anyone saying HELO \d+\.\d+\.\d+\.\d+ because it’s missing the
required brackets. We block any bracketed dotted quads that have an invalid
quad (i.e. 300.300.300.300) or where the address they claim to be from is
different from the actual address we’re seeing (no one behind a NATting
firewall should be connecting to us unless they know their [static] external
address).

We block anyone listed by ZenBL.

We block (a) anyone claiming to be us, (b) anyone claiming to be “localhost” or
“localhost.localdomain”.

We block a list of names that are always fraudulent, like “paypal.com” (without
any subdomains), or “smtp.communitel.net” which seems to get spoofed a LOT, or
“mail.com” or “mail.ru”. We block hostnames that don’t have a domain (no dots).

Lastly, we TEMPFAIL hosts that don’t have valid rDNS mappings (including the A
and PTR records not agreeing).

With this, we avoid ever accepting about 98% of the SPAM that we’d otherwise
receive.

-Philip

Re: Must-Have Plugins?

2015-06-15 Thread Matus UHLAR - fantomas

On 10.06.15 04:34, Amir Caspi wrote:

To: Matus UHLAR - fantomas uh...@fantomas.sk
Cc: users@spamassassin.apache.org

pleaase, avoid personal mail. The list is for public discussion.

Subject: Re: Must-Have Plugins?

On Jun 10, 2015, at 12:32 AM, Matus UHLAR - fantomas uh...@fantomas.sk wrote:

FEATURE(`block_bad_helo')
define(`confALLOW_BOGUS_HELO', `False')

Argh, unfortunately, that feature is only on sendmail 8.14 and higher,
which means RHEL/CentOS 6 or higher.  For those of us running RHEL/CentOS
5, that's only available via a custom install.  =(

It was available before and I have used it as HACK()

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
To Boot or not to Boot, that's the question. [WD1270 Caviar]

Re: Must-Have Plugins?

2015-06-11 Thread RW

On Wed, 10 Jun 2015 18:45:10 -0400
Michael B Allen wrote:

 On Wed, Jun 10, 2015 at 9:56 AM, David Jones djo...@ena.com wrote:
  given that install unbound as local resolver takes 2 minutes it's
  even not worth to argue on that topic and a spamfilter without
  RBL's and URIBL's is just nonsense
 
 I have installed a caching DNS server before (albeit probably about
 15 years ago). But it just shouldn't be necessary.
 
  It can be necessary if you have enough mail volume.
 
 That's not what I'm saying. It should not be necessary to run a
 full-blown DNS server for SA to do it's queries. 

You don't need a  full-blown DNS server, you just need a resolver which
is typically ~ 100kB plus whatever space you want for caching.
Mine is currently using 9MB of resident memory compared with  103MB
for a single spamd child process. This handles DNS  more efficiently
than SpamAssassin could. 


The closest SA itself could get is to have a dedicated process -
essentially reinventing the wheel with a heavyweight perl resolver. But
that isn't a general solution because spamd isn't the only way of
using the SA libraries. The generic solution would be to use a shared
database with all the overheads and complications that that implies.

Re: Must-Have Plugins?

2015-06-11 Thread Michael B Allen

On Thu, Jun 11, 2015 at 10:03 AM, RW rwmailli...@googlemail.com wrote:
 On Wed, 10 Jun 2015 18:45:10 -0400
 Michael B Allen wrote:

 On Wed, Jun 10, 2015 at 9:56 AM, David Jones djo...@ena.com wrote:
  given that install unbound as local resolver takes 2 minutes it's
  even not worth to argue on that topic and a spamfilter without
  RBL's and URIBL's is just nonsense
 
 I have installed a caching DNS server before (albeit probably about
 15 years ago). But it just shouldn't be necessary.
 
  It can be necessary if you have enough mail volume.

 That's not what I'm saying. It should not be necessary to run a
 full-blown DNS server for SA to do it's queries.

 You don't need a  full-blown DNS server, you just need a resolver which
 is typically ~ 100kB plus whatever space you want for caching.
 Mine is currently using 9MB of resident memory compared with  103MB
 for a single spamd child process. This handles DNS  more efficiently
 than SpamAssassin could.

 The closest SA itself could get is to have a dedicated process -

No. You don't need a process at all. It would just be some functions
that are called (preferably async but that's easy to make optional).
Depending on the size of the cache it could be in memory or a disk
file. Then it would be properly librarified and isolated.

But this thread is getting a little wonky so I apologize in advance
for not respond further.

Mike

Re: Must-Have Plugins?

2015-06-11 Thread David Jones

 given that install unbound as local resolver takes 2 minutes it's even not
 worth to argue on that topic and a spamfilter without RBL's and URIBL's is
 just nonsense

I have installed a caching DNS server before (albeit probably about 15
years ago). But it just shouldn't be necessary.

 It can be necessary if you have enough mail volume.

That's not what I'm saying. It should not be necessary to run a
full-blown DNS server for SA to do it's queries. It should be possible
to call a library and create a DNS context that has all of it's own
parameters and then use that in an isolated way. Then other services
on the system are completely unaffected. Don't tell me someone has
never tweaked some parameter in your supposedly caching-only
nameserver and inadvertantly broken something or wished they could
tweak something and can't because of the dependencies. And it's very
possible that the queries might be for different names using custom
query parameters in an async way and so on in which case the system
resolver API might not be ideal.

You missed my point which I clarified yesterday in a previous post.

I'm not pooh-poohing your advice. I'm just saying the DNS bits should
be librarified so that these things don't even need extra thought.
This stuff might be what you do all the time but I don't. I do this
once every few years. This is the sort of thing that makes people
switch to cloud services.

If you don't do this kind of work often, I completely understand it's
hard to keep up with everything.  I am sure I can't keep up with
some of the stuff that you do everyday.  One option is to use
something like http://efa-project.org/ so it can be handled for
you automatically by smart people like Shawn that do this everyday.

Caching nameserver vs. resolver library (was Re: Must-Have Plugins?)

2015-06-11 Thread Dianne Skoll

[I have lost the attribution, but someone wrote:]

 That's not what I'm saying. It should not be necessary to run a
 full-blown DNS server for SA to do it's queries. It should be
 possible to call a library and create a DNS context that has all of
 it's own parameters and then use that in an isolated way.

That's silly.  The whole point of a caching DNS server is that *all*
applications on the system benefit from the cache.  If you cache within
each process, then you waste memory and make redundant DNS lookups.

 Then other services on the system are completely unaffected. Don't
 tell me someone has never tweaked some parameter in your supposedly
 caching-only nameserver and inadvertantly broken something or
 wished they could tweak something and can't because of the
 dependencies.

Umm... that has never happened to me and I've been administering UNIX
systems since 1990.  The whole point of caching nameservers is that
they're dead easy to set up and once they're running, you just leave
them alone.

Regards,

Dianne.

Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)

2015-06-11 Thread Bill Cole


On 10 Jun 2015, at 10:26, Kevin A. McGrail wrote:


On 6/10/2015 10:18 AM, Dianne Skoll wrote:
I'm not disputing that running a caching DNS server is a good idea, 
but
you may be quite surprised at the low cache hit rate for IP-based 
DNSBLs.
IMO, the primary goal of a caching-only nameserver is in fact, not the 
caching, but rather the unique source IP so as to avoid running into 
DNS limits placed on RBL queries from some BL providers that you can 
run afoul of when sharing a DNS server.


Caching is really just icing on the cake coupled with the simplest way 
to get a local DNS server up and running, no?


Not at all scales and styles of mail system. The MTA does lookups at 
connect and at each command that mostly block progress, and them if the 
message makes it to SA, virtually all of those lookups and often closely 
related ones will be done again, often in another process running as a 
different user which might (OS-dependent) mean that a record in the 
meager cache kept by the OS won't be used for the second lookup.


I no longer have access to the data I gathered on this when I was 
handling a big-ish system with multiple then-hefty MX gateways doing 
spam filtering, but my memory is still sound enough that I can say the 
difference between (1) talking to The Official Enterprise DNS Server on 
the other side of a router that handled all recursive resolution and (2) 
using a machine-local caching forwarder on each MX forwarding to a 
shared caching recursive resolver on a common LAN was most of the median 
SMTP session life. My recollection is that (1) meant most sessions took 
~7 seconds or longer, (2) dropped it to near 3 seconds. A number of 
things have changed in the past decade that might substantially change 
that effect even in a similar site, but I think most of the effect 
(proliferation of DNS-based tactics like SPF  DKIM and many more usable 
DNSBLs and particularly URIBLs) can only make a cache more helpful, even 
if the help is marginal. On the other hand, even legitimate operations 
seem to think every DNS record should expire before today's close of 
business, and that makes caching less possible.


Also, a smaller site gets less benefit at all from a DNS cache. If 
you've got a few dozen users getting the same mail simultaneously in 
parallel, you win. If you don't HAVE a few dozen users and most of your 
users get no spam and little mail, you have a cache that's pretty 
sparse. You still avoid the problems of looking like part of an abusive 
behemoth when you forward to Google or of getting self-serving lies from 
the local ISP browser-aid resolver, so it remains worthwhile.

Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)

2015-06-11 Thread Noel Butler

 

On 11/06/2015 00:18, Dianne Skoll wrote: 

 On Wed, 10 Jun 2015 13:56:49 +
 David Jones djo...@ena.com wrote:
 
 [One should run a caching DNS server on a mail server.]
 
 We are giving you solid advice based on real experiences where we
 ran into problems and worked around them. Just try to enable RBLs
 and see how it works for you.
 
 I'm not disputing that running a caching DNS server is a good idea, but
 you may be quite surprised at the low cache hit rate for IP-based DNSBLs.
 Spamhaus, for example, has a TTL of 1 minute on its A records. (Check
 out host -v 2.0.0.127.sbl.spamhaus.org if you don't believe me.)
 
 Quite a number of years ago, I ran an analysis of the mail logs on a
 very busy server and found an abysmally low cache hit rate (about 30%)
 and that was in the day when Spamhaus had a 15-minute TTL.

30% is an excellent hit rate, however - 

The longer the TTL the higher the cache hit 
The longer the TTL the higher the collateral damage 

It's why most run 1-10 min TTL's, might not seem much, but take for
example in the mid 90's when AOL was useless at dealing with spam
issues, a listing of 10 mins could deny thousands of messages back then,
and that helped prompt them into getting their act together,
especially when a number of DNSBL's were doing it, so they kicked off
their user (who often retuned 30 mins later courtesy of AOL's world wide
flood of freebie CD's), and blocks where removed quick enough to
minimise more innocents getting caught up.

Re: Must-Have Plugins?

2015-06-11 Thread Bill Cole


On 10 Jun 2015, at 10:55, Alex Regan wrote:


Hi,


Not everyone is running a dedicated mail server. My server is an
everything-server running on a hosted VPS that only has a few 
users

that get significant amounts of email. I'm not sure I want another
daemon that can break or take up clock cycles and memory on a system
processing 10 spams / hour (of which the DNSBL service might catch
2?). At least not yet, but I suppose I could change my mind. At the
moment not that many spams are getting through.



Mike


You asked for help, we provided it.  It's fine if you ignore the 
advice,
but it's good advice if you want to take your mail filtering to the 
next

level in the future.


And it's not just for mail filtering. Unless you have the smallest of 
systems, it's good general practice to implement a caching name server 
for authentication, the email system itself, and general queries from 
other services as well.


Right. A modern MTA using very simple internal spam exclusion tactics 
(validity of sender domains, existence of valid PTR for the client IP, 
etc) does multiple DNS queries per message. A local cache entry is 
always an order of magnitude faster to get than one on the other side of 
some router, so unless you're using an OS with a system name cache (e.g 
Solaris' nscd, MacOS' Directory Services, etc.) that can handle many 
records, your MTA mostly waits for DNS. Adding in DNSBLs used by the MTA 
and variations like URIBLs used in SA, and you can end up doing the same 
queries because of the same message far enough apart between the MTA and 
the filtering that both go off to the net for resolution because the 
system cache is anemic or absent.


But there's an even better reason: trustworthiness. ISPs and others 
providing 'free access' resolvers have a history of shaping their 
replies in their own interest. Sometimes this is also well-intentioned 
(e.g. redirecting bad  domain names) but while such tactics can be 
protective for web browsing users, it can cripple mail servers trying to 
block spam.

Re: Must-Have Plugins?

2015-06-10 Thread Kevin A. McGrail


On 6/10/2015 12:45 AM, Michael B Allen wrote:

But I just can't
bring myself to install a caching DNS server and run everything
through localhost. This is why software should be librarified.

I strongly advise you to install a caching DNS server and using a few RBLs.

regards,
KAM

Re: Must-Have Plugins?



Am 10.06.2015 um 13:17 schrieb Kevin A. McGrail:

On 6/10/2015 2:32 AM, Matus UHLAR - fantomas wrote:



I'm not sure whether or not I have enabled requiring valid rDNS... given
how many legitimate mailservers out there don't have proper rDNS,


how many? I'm happy to block them for years...

 From what I've see, the effectivness and false positives depends mostly
on the countries sending email to your users.

Off the cuff, I'd say it has it's roots in the mid 90's when AOL was a
1000lb gorilla and pushed the rptr for mail servers.  I think much of
Europe and NA complied.

However, I see a TON of legit mail, for example, coming out of the
Pacific Rim countries with no rptr's


well, combine strict PTR rules with DNSWL and SPF, the one which have 
*something* sane are not hurted and the rest need to learn some basics




signature.asc
Description: OpenPGP digital signature

Re: Must-Have Plugins?




Am 10.06.2015 um 13:21 schrieb Kevin A. McGrail:

On 6/10/2015 12:45 AM, Michael B Allen wrote:

But I just can't
bring myself to install a caching DNS server and run everything
through localhost. This is why software should be librarified.

I strongly advise you to install a caching DNS server and using a few RBLs


+1

i can't understand I just can't bring myself to install a caching DNS 
server and run everything through localhost because that is the way to 
go to avoid exceed RBL limits and bad resolvers



This is why software should be librarified is nonsense in that context 
- the library also needs to ask a dns server at the end of the day and 
the server needs to be 100% trustable when it comes to email


given that install unbound as local resolver takes 2 minutes it's even 
not worth to argue on that topic and a spamfilter without RBL's and 
URIBL's is just nonsense




signature.asc
Description: OpenPGP digital signature

Re: Must-Have Plugins?

2015-06-10 Thread Amir Caspi

On Jun 10, 2015, at 12:32 AM, Matus UHLAR - fantomas uh...@fantomas.sk wrote:

 FEATURE(`block_bad_helo')
 define(`confALLOW_BOGUS_HELO', `False')

Argh, unfortunately, that feature is only on sendmail 8.14 and higher, which 
means RHEL/CentOS 6 or higher.  For those of us running RHEL/CentOS 5, that's 
only available via a custom install. =(

Does anyone know of a reputable RPM distro for sendmail 8.14+ for CentOS 5?  I 
can't find anything decent via Google, everything is for CentOS 6 or higher.

(My server setup requires RPMs, so I can't build from a source tarball. I could 
potentially use the source RPM from CentOS 6 to get a custom RPM for 5, 
although even that is problematic.)

Bleh.  I wish I could upgrade this server to a newer OS, but various 
circumstances prevent that right now.

--- Amir

Re: Must-Have Plugins?

2015-06-10 Thread Kevin A. McGrail


On 6/10/2015 2:32 AM, Matus UHLAR - fantomas wrote:



I'm not sure whether or not I have enabled requiring valid rDNS... given
how many legitimate mailservers out there don't have proper rDNS,


how many? I'm happy to block them for years...
From what I've see, the effectivness and false positives depends mostly 
on the countries sending email to your users.


Off the cuff, I'd say it has it's roots in the mid 90's when AOL was a 
1000lb gorilla and pushed the rptr for mail servers.  I think much of 
Europe and NA complied.


However, I see a TON of legit mail, for example, coming out of the 
Pacific Rim countries with no rptr's.


regards,
KAM

Re: Must-Have Plugins?

 Some of the best and easiest things you can enable to block spam are
 outside of SpamAssassin at your MTA (sendmail, postfix, etc.).
 - Enable RBLs and DBLs.  zen.spamhaus.org is the best way to block the
   majority of junk before it reaches SA.  Just make sure you are below their
   free threshold limit.  One important way to do this is to make sure your
   SA server isn't pointed to an Internet caching DNS server that would join
   your queries with others.  Install a local caching DNS server that does not
   forward to another caching DNS server and change /etc/resolv.conf to use
   127.0.0.1.

Well that sounds like a must-have feature to me. But I just can't
bring myself to install a caching DNS server and run everything
through localhost. This is why software should be librarified.

What OS are you running?  It's normally a very simple install to get a caching
DNS server running locally since the default configurations usually come ready
to do exactly what you need in this case.  Google caching dns server howto
plus your OS and you will see it's pretty easy.

You can try using RBLs with your existing DNS server configuration.  If it's a
dedicated DNS server for your network, then you have a good chance of
staying below the free thresholds for a low volume server.  If it's a major
ISP's DNS server then the odds will be against you.
http://www.spamhaus.org/faq/section/DNSBL%20Usage#366
Try dig 138.178.203.192.zen.spamhaus.org or nslookup to see if you get
back a response of 127.0.0.4.  If so, you should be good.

 - Enable DNS checks:
   Make sure the sending mail server's SMTP HELO is a valid domain.
   Make sure the sender address (MAIL FROM) is a valid domain.
   Make sure the sending mail server has a PTR record.  Some can go farther 
 with
   this one and require the PTR match the SMTP HELO for FCrDNS but there are
   many legit mail servers out there that don't have this setup properly so I 
 can
   only check to make sure a PTR record exists.  Later in SA I add points for 
 rule
   RDNS_NONE that penalizes for incorrect FCrDNS.

Is this done with postfix rules or SA rules? Where can I learn more
about this? Doesn't SA already do this stuff?

This should be done in your MTA if possible before it's handed off to SA.  Some 
of
this information is exposed to SA in headers but some isn't.  High volume 
servers
need this logic at the MTA to keep processing times low.  Only a small 
percentage
of mail makes it to SA in my environment and most of that is going to be clean.
In a low volume environment, you can send most of the mail through SA and still
keep the processing times down low.  My MailScanner batches average about 5 to
10 messages and complete in 4 to 5 seconds normally with ClamAV and Eset
Nod32 AV scanners.

Re: Must-Have Plugins?

 - Enable RBLs and DBLs.  zen.spamhaus.org is the best way to block the
majority of junk before it reaches SA.  Just make sure you are below their
free threshold limit.  One important way to do this is

One important way to do this in terms of the Spamhaus threshold limit
is to not be such a tightwad and poney up for the Spamhaus commercial
service.  ;-)

They do a cheaper version than the RSync feed, you can just query their
servers directly.

Spamhaus do a fantastic job.  They deserve charitable donations from
generous mail sysadmins !!!

We filter a lot of mailboxes and pay spamhaus several thousands of dollars each 
year.

The invaluement.com RBL is very effective and only costs a few hundred dollars 
a year.

 - Enable greylisting.

Ewww...

I hate people who operate graylisting.   Its a lazy tarnish everyone
with the same brush approach to anti-spam.

In this day and age, you don't need it.  Decent network checks, properly
configured Spamassassin and you should be able to achieve a very
respectable spam catching rate.

I respectfully disagree.  I too hated greylisting for years until recently when 
I found a
way to slowly ease it in for my users who didn't even detect it.  There is no 
other way
to block brand new spam campaigns from compromised accounts.  These mail servers
normally have a good reputation so they are not on any RBLs yet.  Greylisting 
puts a
speed bump in place so the RBLs have time to catch up.
Spammers pay sweat shops to devise new spam that will get through the major 
filters
like SA and commercial products as zero-hour spam.   Bayes is often ineffective 
against
these new campaigns.  When a new spam campaign like this hits the Internet, the 
world
takes a little while to detect and react to it so you have to put some kind of 
buffer like
greylisting in place.

I whitelist a lot of major ISPs and known good senders to bypass greylisting so 
this only
impacts mostly small mail servers that don't have any compromised account 
detection.

My company's support team used to get several calls a week calls about 
blacklisting
senders that turned out to be compromised accounts but now we might get one or 
two
every 3 months now.  By the time  the blacklist entry was added, RBLs had 
already been
blocking them so I just remove the new entry to keep my lists clean.

If you have a large mail filtering environment with a lot of very old email 
accounts that have
become bought and sold on spammer lists, this is a must.  I guess if you are 
only filtering
for a few hundred accounts, you can do a lot of things differently and be fine.

Re: Must-Have Plugins?

2015-06-10 Thread Ben




- Enable RBLs and DBLs.  zen.spamhaus.org is the best way to block the
   majority of junk before it reaches SA.  Just make sure you are below their
   free threshold limit.  One important way to do this is



One important way to do this in terms of the Spamhaus threshold limit 
is to not be such a tightwad and poney up for the Spamhaus commercial 
service.  ;-)


They do a cheaper version than the RSync feed, you can just query their 
servers directly.


Spamhaus do a fantastic job.  They deserve charitable donations from 
generous mail sysadmins !!!





- Enable greylisting.


Ewww...

I hate people who operate graylisting.   Its a lazy tarnish everyone 
with the same brush approach to anti-spam.


In this day and age, you don't need it.  Decent network checks, properly 
configured Spamassassin and you should be able to achieve a very 
respectable spam catching rate.

Re: Must-Have Plugins?



Am 10.06.2015 um 15:49 schrieb Michael B Allen:

By librarified I mean the DNS server is just a code context that
can be constructed with it's own config precisely and only as needed
by the software that will be querying it (possibly temporarily if it's
just client-only activity like a barrage of DNS queries fired in
reaction to an email that fails other spam tests). It should not be
necessary to change the resolver configuration or behavior of the
entire system and everything running on it if only one component in
the system needs this special feature (in this case a query limit and
private cache). That is just bad programming philosophy and it the
source of a lot of bad behavior in software (and DNS is a very good
example of this actually)


1: SA has a option to define the nameserver without touch resolv.conf
2: why would somebody not re-use already cached data for any software



signature.asc
Description: OpenPGP digital signature

Re: Must-Have Plugins?

 given that install unbound as local resolver takes 2 minutes it's even not
 worth to argue on that topic and a spamfilter without RBL's and URIBL's is
 just nonsense

I have installed a caching DNS server before (albeit probably about 15
years ago). But it just shouldn't be necessary.

It can be necessary if you have enough mail volume.

By librarified I mean the DNS server is just a code context that
can be constructed with it's own config precisely and only as needed
by the software that will be querying it (possibly temporarily if it's
just client-only activity like a barrage of DNS queries fired in
reaction to an email that fails other spam tests). It should not be
necessary to change the resolver configuration or behavior of the
entire system and everything running on it if only one component in
the system needs this special feature (in this case a query limit and
private cache). That is just bad programming philosophy and it the
source of a lot of bad behavior in software (and DNS is a very good
example of this actually).

You obviously don't understand how DNS works in relation to RBLs.

We are giving you solid advice based on real experiences where we
ran into problems and worked around them.  Just try to enable RBLs
and see how it works for you.

Not everyone is running a dedicated mail server. My server is an
everything-server running on a hosted VPS that only has a few users
that get significant amounts of email. I'm not sure I want another
daemon that can break or take up clock cycles and memory on a system
processing 10 spams / hour (of which the DNSBL service might catch
2?). At least not yet, but I suppose I could change my mind. At the
moment not that many spams are getting through.

Mike

You asked for help, we provided it.  It's fine if you ignore the advice,
but it's good advice if you want to take your mail filtering to the next
level in the future.

DNSBLs and cache hit rate (was Re: Must-Have Plugins?)

2015-06-10 Thread Dianne Skoll

On Wed, 10 Jun 2015 13:56:49 +
David Jones djo...@ena.com wrote:

[One should run a caching DNS server on a mail server.]

 We are giving you solid advice based on real experiences where we
 ran into problems and worked around them.  Just try to enable RBLs
 and see how it works for you.

I'm not disputing that running a caching DNS server is a good idea, but
you may be quite surprised at the low cache hit rate for IP-based DNSBLs.
Spamhaus, for example, has a TTL of 1 minute on its A records.  (Check
out host -v 2.0.0.127.sbl.spamhaus.org if you don't believe me.)

Quite a number of years ago, I ran an analysis of the mail logs on a
very busy server and found an abysmally low cache hit rate (about 30%)
and that was in the day when Spamhaus had a 15-minute TTL.

Anyway, run through the exercise yourself; it's eye-opening.
My original post was here (back when I used to be David, so don't
let the signature confuse you...)

http://spamassassin.1065346.n5.nabble.com/Fwd-Asrg-draft-levine-iprangepub-01-tp28778p28802.html

Regards,

Dianne.

Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)

2015-06-10 Thread Kevin A. McGrail


On 6/10/2015 10:18 AM, Dianne Skoll wrote:

I'm not disputing that running a caching DNS server is a good idea, but
you may be quite surprised at the low cache hit rate for IP-based DNSBLs.
IMO, the primary goal of a caching-only nameserver is in fact, not the 
caching, but rather the unique source IP so as to avoid running into DNS 
limits placed on RBL queries from some BL providers that you can run 
afoul of when sharing a DNS server.


Caching is really just icing on the cake coupled with the simplest way 
to get a local DNS server up and running, no?


Regards,
KAM

Re: Must-Have Plugins?

2015-06-10 Thread Michael B Allen

On Wed, Jun 10, 2015 at 7:25 AM, Reindl Harald h.rei...@thelounge.net wrote:


 Am 10.06.2015 um 13:21 schrieb Kevin A. McGrail:

 On 6/10/2015 12:45 AM, Michael B Allen wrote:

 But I just can't
 bring myself to install a caching DNS server and run everything
 through localhost. This is why software should be librarified.

 I strongly advise you to install a caching DNS server and using a few RBLs


 +1

 i can't understand I just can't bring myself to install a caching DNS
 server and run everything through localhost because that is the way to go
 to avoid exceed RBL limits and bad resolvers


 This is why software should be librarified is nonsense in that context -
 the library also needs to ask a dns server at the end of the day and the
 server needs to be 100% trustable when it comes to email

 given that install unbound as local resolver takes 2 minutes it's even not
 worth to argue on that topic and a spamfilter without RBL's and URIBL's is
 just nonsense

I have installed a caching DNS server before (albeit probably about 15
years ago). But it just shouldn't be necessary.

By librarified I mean the DNS server is just a code context that
can be constructed with it's own config precisely and only as needed
by the software that will be querying it (possibly temporarily if it's
just client-only activity like a barrage of DNS queries fired in
reaction to an email that fails other spam tests). It should not be
necessary to change the resolver configuration or behavior of the
entire system and everything running on it if only one component in
the system needs this special feature (in this case a query limit and
private cache). That is just bad programming philosophy and it the
source of a lot of bad behavior in software (and DNS is a very good
example of this actually).

Not everyone is running a dedicated mail server. My server is an
everything-server running on a hosted VPS that only has a few users
that get significant amounts of email. I'm not sure I want another
daemon that can break or take up clock cycles and memory on a system
processing 10 spams / hour (of which the DNSBL service might catch
2?). At least not yet, but I suppose I could change my mind. At the
moment not that many spams are getting through.

Mike

Re: Must-Have Plugins?

2015-06-10 Thread Bill Cole


On 9 Jun 2015, at 14:39, Matus UHLAR - fantomas wrote:


On 09.06.15 11:29, John Hardin wrote:

Two things that I have found very useful at the MTA level are:

(1) Delay sending your SMTP banner a second or two and reject any 
sender that starts sending information before that. This is a 
built-in option in Sendmail, google greet_pause.


even 15...


Based on the implementation in Sendmail on a variety of systems with up 
to a few million sessions/day, Postfix's 'postscreen' implementation on 
smaller (mostly *much* smaller) sites and CommunigatePro's 
implementation on a couple of middling sites, the value of adding 
greet_pause time starts dropping fast around 3 seconds and there's no 
point going past 5. You can actually start seeing occasional collateral 
damage around 10 seconds with some high-volume legitimate senders timing 
out their 'first try' runs aggressively. 15 is another breakpoint, 
apparently because there is widespread lore in high-speed sending that 
says it is not worth waiting any longer for a greeting banner. For 
high-volume sites it is also important to understand that SMTP session 
lives from SYN-ACK to FIN without a greeting pause are mostly under 10 
seconds; adding 3 seconds to that means you must support MANY more 
concurrent SMTP connections.


FWIW, the best implementation of fast-talk detection is Postfix's 
postscreen. It is optimized to put this and other low-cost exclusion 
tricks outside of a relatively heavy smtpd and was designed with an eye 
on the many years of demonstration by Sendmail and others of how to 
minimize the impact of adding delays. By not delaying clients that have 
recently passed a delay, it assures that most of your high-volume 
legitimate senders don't contribute to performance issues.




(2) Check the HELO the other guy sends and reject if it's not a FQDN 
(i.e. it's not got any periods at all).


or if it's your FQDN, or your IP - they should use their FQDN, not 
yours.


And if you don't/can't use a greeting pause, these are useful in 
catching many of the bots that fast-talk. Also useful are checks for 
improper use of IP HELOs (e.g. unbracketed IPs, not valid IP literals) 
and checking for *.local and *.localdomain HELO args. A handful of 
innocent morons run MTAs that use such names, but they tend not to 
survive long doing so.

Re: Must-Have Plugins?

2015-06-10 Thread Alex Regan


Hi,


Not everyone is running a dedicated mail server. My server is an
everything-server running on a hosted VPS that only has a few users
that get significant amounts of email. I'm not sure I want another
daemon that can break or take up clock cycles and memory on a system
processing 10 spams / hour (of which the DNSBL service might catch
2?). At least not yet, but I suppose I could change my mind. At the
moment not that many spams are getting through.



Mike


You asked for help, we provided it.  It's fine if you ignore the advice,
but it's good advice if you want to take your mail filtering to the next
level in the future.


And it's not just for mail filtering. Unless you have the smallest of 
systems, it's good general practice to implement a caching name server 
for authentication, the email system itself, and general queries from 
other services as well.


Regards,
Alex

Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)

2015-06-10 Thread Dianne Skoll

On Wed, 10 Jun 2015 14:56:40 +
David Jones djo...@ena.com wrote:

 My point was that running a local caching server is the only way one
 can know exactly how the lookups are happening.

Ah, true.  I missed that point I guess.

Regards,

Dianne.

Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)

[One should run a caching DNS server on a mail server.]

 We are giving you solid advice based on real experiences where we
 ran into problems and worked around them.  Just try to enable RBLs
 and see how it works for you.

I'm not disputing that running a caching DNS server is a good idea, but
you may be quite surprised at the low cache hit rate for IP-based DNSBLs.
Spamhaus, for example, has a TTL of 1 minute on its A records.  (Check
out host -v 2.0.0.127.sbl.spamhaus.org if you don't believe me.)

Quite a number of years ago, I ran an analysis of the mail logs on a
very busy server and found an abysmally low cache hit rate (about 30%)
and that was in the day when Spamhaus had a 15-minute TTL.

My point was that running a local caching server is the only way one
can know exactly how the lookups are happening.  If you point to a
DNS server that you don't manage, it could be forwarding to an ISP's
DNS caches which will aggregate your queries in with others and could
cause unexpected results for those RBLs that limit queries.

I have 8 mail filters that run a local caching DNS server which forward
to a pair of DNS caches running rbldnsd for a local copy of a number
of RBL zones including my own private RBL.  This configuration has to
provide some caching benefits when I get blasted by mass marketing
campaigns.  Postfix should keep my local cache populated so when SA
asks for the same DNS information it would be a few milliseconds
response.

I should take some time to do some real analysis as you have done.
Thanks for the info and link.

Re: Must-Have Plugins?

2015-06-10 Thread Matus UHLAR - fantomas


On Jun 9, 2015, at 12:29 PM, John Hardin jhar...@impsec.org wrote:

(2) Check the HELO the other guy sends and reject if it's not a FQDN
(i.e.  it's not got any periods at all).  This probably shouldn't be done
on mail originating locally, but for mail coming in from the Internet the
other MTA should always be sending a FQDN in the HELO.  A non-FQDN HELO
is a pretty good sign of a spambot sending from a compromised workstation
or PC directly to your MTA.


On 09.06.15 12:48, Amir Caspi wrote:

Do you have a sendmail line (or lines) that we can pretty much copy/paste
to implement this, for those of us who are not total sendmail experts and
too lazy (or busy, or incompetent) to Google something like this?  =)


FEATURE(`block_bad_helo')
define(`confALLOW_BOGUS_HELO', `False')


I'm not sure whether or not I have enabled requiring valid rDNS... given
how many legitimate mailservers out there don't have proper rDNS,


how many? I'm happy to block them for years...

--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
We are but packets in the Internet of life (userfriendly.org)

Re: Must-Have Plugins?

2015-06-10 Thread John Hardin


On Wed, 10 Jun 2015, Kevin A. McGrail wrote:


On 6/10/2015 12:45 AM, Michael B Allen wrote:

 But I just can't
 bring myself to install a caching DNS server and run everything
 through localhost. This is why software should be librarified.


I strongly advise you to install a caching DNS server and using a few RBLs.


Just a minor nit: It's not the caching part that's important here, it's 
the not forwarding part. If you set up a caching DNS server that just 
forwards to your ISP's DNS servers, you haven't addressed the BL blocked 
due to query volume problem.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  ...much of our country's counterterrorism security spending is not
  designed to protect us from the terrorists, but instead to protect
  our public officials from criticism when another attack occurs.
-- Bruce Schneier
---

Re: Must-Have Plugins?

2015-06-10 Thread John Hardin


On Wed, 10 Jun 2015, Bill Cole wrote:

  (2) Check the HELO the other guy sends and reject if it's not a FQDN 
  (i.e. it's not got any periods at all).


 or if it's your FQDN, or your IP - they should use their FQDN, not yours.


And if you don't/can't use a greeting pause, these are useful in catching 
many of the bots that fast-talk.


Absolutely. I see a lot of instances where the first couple of tries from 
a given IP are blocked by greet-pause, and then after a bit there are 
several more from the same IP blocked by invalid HELO.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  ...much of our country's counterterrorism security spending is not
  designed to protect us from the terrorists, but instead to protect
  our public officials from criticism when another attack occurs.
-- Bruce Schneier
---

Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)

2015-06-10 Thread David B Funk


On Wed, 10 Jun 2015, David Jones wrote:


[One should run a caching DNS server on a mail server.]


My point was that running a local caching server is the only way one
can know exactly how the lookups are happening.  If you point to a
DNS server that you don't manage, it could be forwarding to an ISP's
DNS caches which will aggregate your queries in with others and could
cause unexpected results for those RBLs that limit queries.


One other technical benefit to running a local caching server is that if
SA is configured to talk to it va the localhost (loopback) interface there
are MTU advantages.
Most loopback interfaces have a MTU of 16K (or bigger) and will handle large
UDP packets without fragementation. In general DNS transactions are fastest
via UDP if you don't have fragementation issues.

--
Dave Funk  University of Iowa
dbfunk (at) engineering.uiowa.eduCollege of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include std_disclaimer.h
Better is not better, 'standard' is better. B{

Re: Must-Have Plugins?

2015-06-10 Thread Michael B Allen

On Wed, Jun 10, 2015 at 9:56 AM, David Jones djo...@ena.com wrote:
 given that install unbound as local resolver takes 2 minutes it's even not
 worth to argue on that topic and a spamfilter without RBL's and URIBL's is
 just nonsense

I have installed a caching DNS server before (albeit probably about 15
years ago). But it just shouldn't be necessary.

 It can be necessary if you have enough mail volume.

That's not what I'm saying. It should not be necessary to run a
full-blown DNS server for SA to do it's queries. It should be possible
to call a library and create a DNS context that has all of it's own
parameters and then use that in an isolated way. Then other services
on the system are completely unaffected. Don't tell me someone has
never tweaked some parameter in your supposedly caching-only
nameserver and inadvertantly broken something or wished they could
tweak something and can't because of the dependencies. And it's very
possible that the queries might be for different names using custom
query parameters in an async way and so on in which case the system
resolver API might not be ideal.

I'm not pooh-poohing your advice. I'm just saying the DNS bits should
be librarified so that these things don't even need extra thought.
This stuff might be what you do all the time but I don't. I do this
once every few years. This is the sort of thing that makes people
switch to cloud services.

Re: DNSBLs and cache hit rate (was Re: Must-Have Plugins?)