Re: Can SA be used to implement greylisting?

[Logan Shaw]

On Tue, 20 Jun 2006, jdow wrote:

A one time one hour delay for a given source is no big deal.


That's a value judgement.  Not universally true for everyone.

Probably true for lots of people, in which case ideas on how
to minimize the negatives of greylisting will be worthless.

For others, eliminating delays where possible would be
worthwhile.  As someone else pointed out, you'll never be able
to eliminate 100% of delays; that is the nature of e-mail.
But there are situations where eliminating delays as much as
possible would be worthwhile, perhaps for no other reason than
because it increases user acceptance of your spam filtering
strategy.

  - Logan

Re: Can SA be used to implement greylisting?

[jdow]

From: "Rick Macdougall" <[EMAIL PROTECTED]>


jdow wrote:

From: "Jonas Eckerman" <[EMAIL PROTECTED]>

Logan Shaw wrote:


Basically, greylisting has an achilles heel:  legit messages
from unknown senders are delayed a long time.  This is fine for


Only if you use a long delay.

In my experience a grey period of 3 minutes is enough to stop most spam 
and viruses. Especially if you combine it with some anti-dictionary stuff.


<> There are indications that this is no longer the case. The
spams apparently come in repeated bursts of four or five about 5
minutes apart. Since it is common for spam to use the same header
values except sometimes "To:" for all messages sent this trips up
greylisting. I ran across a fellow on the FC users list (I believe
it was) that remarked about seeing this and it defeating his greylist.

{^_^}


Hi jdow,

Haven't seen that here.  I used to get 300+ spam a day (dns, postmaster, 
abuse, my own 10+ year email address) and with greylisting I now get 1 - 
3 spams a week, usually 419 spams from free account sites.  Most, if not 
all, the spam that does get through is marked up by SA and learned as spam.


Oh, and I use a 1 minute greylist time.


I believe he was an ISP sort of person from the numbers he cited. I
personally get about 150 to 300 spams a day depending on who was
incarcerated or successfully sued last. I usually figure 250. And
depending on list activity I get 500 to 1500 total messages a day.
Fetchmail does not allow greylisting. But good rules and SpamAssassin
have left me with 10 or 11 spams that got away in the last couple
months. None of them will repeat. (Um, none of them was an image
spam, either.) Make that three in the last few weeks. Even Bayes
tags the image spams these days.

{^_^}

Re: Can SA be used to implement greylisting?

[Rick Macdougall]

jdow wrote:

From: "Jonas Eckerman" <[EMAIL PROTECTED]>

Logan Shaw wrote:


Basically, greylisting has an achilles heel:  legit messages
from unknown senders are delayed a long time.  This is fine for


Only if you use a long delay.

In my experience a grey period of 3 minutes is enough to stop most spam 
and viruses. Especially if you combine it with some anti-dictionary stuff.


<> There are indications that this is no longer the case. The
spams apparently come in repeated bursts of four or five about 5
minutes apart. Since it is common for spam to use the same header
values except sometimes "To:" for all messages sent this trips up
greylisting. I ran across a fellow on the FC users list (I believe
it was) that remarked about seeing this and it defeating his greylist.

{^_^}


Hi jdow,

Haven't seen that here.  I used to get 300+ spam a day (dns, postmaster, 
abuse, my own 10+ year email address) and with greylisting I now get 1 - 
3 spams a week, usually 419 spams from free account sites.  Most, if not 
all, the spam that does get through is marked up by SA and learned as spam.


Oh, and I use a 1 minute greylist time.

Regards,

Rick

Re: Can SA be used to implement greylisting?

[jdow]

From: "Kelson" <[EMAIL PROTECTED]>


Jonas Eckerman wrote:

Logan Shaw wrote:


Basically, greylisting has an achilles heel:  legit messages
from unknown senders are delayed a long time.  This is fine for


Only if you use a long delay.


Or if the sender has a long retry period.  Yeah, you can expect them to 
retry in 10-15 minutes, but you have no control over how long it'll 
really take for them to try again.


A one time one hour delay for a given source is no big deal.
{^_^}

Re: Can SA be used to implement greylisting?

[jdow]

From: "Jonas Eckerman" <[EMAIL PROTECTED]>

Logan Shaw wrote:


Basically, greylisting has an achilles heel:  legit messages
from unknown senders are delayed a long time.  This is fine for


Only if you use a long delay.

In my experience a grey period of 3 minutes is enough to stop most spam and viruses. 
Especially if you combine it with some anti-dictionary stuff.


<> There are indications that this is no longer the case. The
spams apparently come in repeated bursts of four or five about 5
minutes apart. Since it is common for spam to use the same header
values except sometimes "To:" for all messages sent this trips up
greylisting. I ran across a fellow on the FC users list (I believe
it was) that remarked about seeing this and it defeating his greylist.

{^_^} 

Re: Can SA be used to implement greylisting?

[Kelson]

Jonas Eckerman wrote:

Logan Shaw wrote:


Basically, greylisting has an achilles heel:  legit messages
from unknown senders are delayed a long time.  This is fine for


Only if you use a long delay.


Or if the sender has a long retry period.  Yeah, you can expect them to 
retry in 10-15 minutes, but you have no control over how long it'll 
really take for them to try again.


--
Kelson Vibber
SpeedGate Communications 

Re: Can SA be used to implement greylisting?

[Jonas Eckerman]

Logan Shaw wrote:


Basically, greylisting has an achilles heel:  legit messages
from unknown senders are delayed a long time.  This is fine for


Only if you use a long delay.

In my experience a grey period of 3 minutes is enough to stop most spam and 
viruses. Especially if you combine it with some anti-dictionary stuff.

And (also in my experience), greylisting is still pretty effective even if you 
simply whitelist the sending host for a week (or longer) after a mail has made 
it past the greylist. This way only a few mails from each hosts will be 
delayed, and usually they will be delayed about 10 minutes.

This might be more acceptable than long delays for most mail...

Some people even do away with the delay and let everything through on the 
second attempt. This still stops a lot.


on it.  If it comes up with a very low score (almost definitely
not spam), let it pass.  If it comes up with a very high score
(almost definitely spam), drop it right away.  If it comes up with
an indeterminate score, apply the greylisting approach and delay
it until later.


This can be done with MIMEDefang without implementing the actual greylist in 
SA. I suspect there are other solutions as well that can do this.

Sure, it could be done in SA, but are the benefits big enough for enough people?

I don't know. Maybe they are.

Regards
/Jonas

--
Jonas Eckerman, FSDB & Fruktträdet
http://whatever.frukt.org/
http://www.fsdb.org/
http://www.frukt.org/

Re: Can SA be used to implement greylisting?

[John D. Hardin]

On Mon, 19 Jun 2006, Logan Shaw wrote:

> > Consider the case of a spammer whose software *does* retry, but
> > retries every two or three minutes until delivery is accepted or
> > PERMFAILed. I have seen this in my greylist logs. Do you really want
> > SA + AV + whatever to completely process this message a half-dozen
> > times before making a permanent decision at the end of the delay
> > period?
> 
> That's a possible problem.  One solution is that there is
> already a database that tells you when not to delay; this could
> be extended to tell you when not to run content-based checking
> again.  So, you'd only have to run the expensive content-based
> checks the first time you get a message from an unknown source.

Okay, I'll buy that. IP + MSGID, store the spam score and don't scan
it again.

One problem with that is you may lose extra spam points from a URL
that goes into the URIBLs between the time the message is first
scanned and when the greylist delay expires. Or you could limit it to
two scans, one initial and one at delay expiration, and suppress the
scans on intermediate delivery attempts.

Another alternative is no timeout. If you are just saying "I want to
see whether they will retry at all," then accept the message on the
second delivery attempt whenever that occurs. Scan both times. The
other greylist packages may have good reasons they don't do this,
though.

--
 John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/
 [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED]
 key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  You are in a maze of twisty little protocols,
  all written by Microsoft.
--

Re: Can SA be used to implement greylisting?

[David B Funk]

On Mon, 19 Jun 2006, Rick Macdougall wrote:

> JamesDR wrote:
> > 1) Message comes in, check against AWL, if sender/ip pair do not exist,
> > send the tempfail, if sender/ip pair do exist:
> > 2) Check the average score against some threshold (say 4 points as a
> > figure.) If sender's score is over this value (still at the header
> > stage) send tempfail, if the sender's average score is below:
> > 3) Send the message on through to the AV's, SA, etc..
> > 4) SA will adjust the totals, rinse and repeat.
>
> If sender/ip pair is in AWL, it's most likely in the greylisting
> database as well and will be allowed in.

You've missed a point. In the traditional greylisting database you
list by sender/ip -and- recipient address. This is to prevent a
machine-gunning spammer from getting a 'bye' for hitting previous
targets. (IE you only do the greylist pass if the triple of
sender & recipient & ip address match).

If you do the suggested AWL lookup then you can use the sender/ip
score entries as an indication of "credibility" for sending to other
recipients; all done at the SMTP header stage so low overhead.

You could also use this as a selective greylist system:
1) If there is no AWL/greylisting-database entry for the sender
   or a 'good' score in AWL, don't greylist, just pass thru SA as a
   normal message (but also do -not- add to greylisting-database).
2) If 'bad' AWL score and not in greylisting-database, greylist delay.
3) If 'bad' AWL score and in greylisting-database pass thru to SA.

So if the sender is unknown to you, you give them the benefit of the
doubt (no greylist delay) but do the SA score to get a ranking.

If known 'bad' then delay, if known 'good' no delay.


-- 
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Re: Can SA be used to implement greylisting?

[Logan Shaw]

On Mon, 19 Jun 2006, John D. Hardin wrote:

On Mon, 19 Jun 2006, Logan Shaw wrote:


If it comes up with a very high score (almost definitely spam),
drop it right away.  If it comes up with an indeterminate score,
apply the greylisting approach and delay it until later.


What's the point? You've already *got* the entire message, at that
point why tell the sender "I don't want it right now, try again
later"?


The point is, in doing so you will see whether they actually
*do* try again later.  Because greylisting is known to be an
effective tool against spam, we can conclude that it is useful
information.  Because SpamAssassin gave indeterminate results
(by definition), any information you can get is useful.


By definition, SpamAssassin by itself is insufficient
in these cases, so any extra information you can gather (i.e. whether
the sender retries) is valuable information.


At the cost of receiving and processing those messages at least twice.


Yes, I agree with that.  It does use more resources.  So,
fundamentally, this is a question of cost vs. benefits.
Is it worth using resources to get this extra information?
It might be to some, and it might not be to others.


Consider the case of a spammer whose software *does* retry, but
retries every two or three minutes until delivery is accepted or
PERMFAILed. I have seen this in my greylist logs. Do you really want
SA + AV + whatever to completely process this message a half-dozen
times before making a permanent decision at the end of the delay
period?


That's a possible problem.  One solution is that there is
already a database that tells you when not to delay; this could
be extended to tell you when not to run content-based checking
again.  So, you'd only have to run the expensive content-based
checks the first time you get a message from an unknown source.
Of course, that complicates the software a little bit, but we
are back to cost vs. benefits again.


Email is a store-and-forward best-attempt unguaranteed delivery
system, *not* Instant Messaging. The perception that a fifteen-minute
delay in delivery of a message is not acceptable is unrealistic.


I agree with that too, but that doesn't mean it wouldn't be
worthwhile to do something to reduce the negative effects of
greylisting while still getting most of its benefits.

  - Logan

Re: Can SA be used to implement greylisting?

[Rick Macdougall]

JamesDR wrote:

Logan Shaw wrote:

On Mon, 19 Jun 2006, David B Funk wrote:


On Mon, 19 Jun 2006, Justin Mason wrote:





[snip]
1) Message comes in, check against AWL, if sender/ip pair do not exist, 
send the tempfail, if sender/ip pair do exist:
2) Check the average score against some threshold (say 4 points as a 
figure.) If sender's score is over this value (still at the header 
stage) send tempfail, if the sender's average score is below:

3) Send the message on through to the AV's, SA, etc..
4) SA will adjust the totals, rinse and repeat.


If sender/ip pair is in AWL, it's most likely in the greylisting 
database as well and will be allowed in.


I considered adding this to simscan as well but for reasons mentioned 
before I found it to be overly burdensome on the mail server and really 
didn't add any true value.


I ended up just whitelisting major ISP's in my greylisting database and 
went from there.


Regards,

Rick

Re: Can SA be used to implement greylisting?

[JamesDR]

Logan Shaw wrote:

On Mon, 19 Jun 2006, David B Funk wrote:


On Mon, 19 Jun 2006, Justin Mason wrote:





[snip]




OK, if that's the case, let me offer my own personal justification of
why it might be worthwhile to combine greylisting with SpamAssassin.

Basically, greylisting has an achilles heel:  legit messages
from unknown senders are delayed a long time.  This is fine for
certain types of organizations, but what if all the accounts on
your mail server are for salespeople?  They're constantly trying to
generate new contacts and leads, and they really don't want their
communications to be delayed.  This is just one example; there are
surely other people who don't want any delays that can be avoided.

So, what does this have to do with greylisting + content-based
filtering?  It's simple: if you receive a message from an unknown
sender / domain / IP / whatever, you can then do a spamassassin run
on it.  If it comes up with a very low score (almost definitely
not spam), let it pass.  If it comes up with a very high score
(almost definitely spam), drop it right away.  If it comes up with
an indeterminate score, apply the greylisting approach and delay
it until later.

What does this buy you?  Two things.  The first is that low-risk
messages (based on content) go right through, eliminating much
of the downside of greylisting.  The second is that for messages
which SpamAssassin is unsure about, you get the added benefit of
greylisting.  By definition, SpamAssassin by itself is insufficient
in these cases, so any extra information you can gather (i.e. whether
the sender retries) is valuable information.

To put it another way, greylisting has a high cost in terms of
convenience.  So, apply greylisting only when SpamAssassin is not
confident in its judgement; in those cases, you can easily justify
the cost, and in the other cases, you can avoid the cost of
greylisting completely.

  - Logan



I've been following this thread closely... and I have a few thoughts on 
how SA could be used... as a feed back loop device.


Currently in SA is the AWL, we all know this is a table of senders, 
ip's, mail count, and total scores. If one has a good AWL database - 
then this could be used in part of the graylist decision. In my mind, 
how this could work is:


1) Message comes in, check against AWL, if sender/ip pair do not exist, 
send the tempfail, if sender/ip pair do exist:
2) Check the average score against some threshold (say 4 points as a 
figure.) If sender's score is over this value (still at the header 
stage) send tempfail, if the sender's average score is below:

3) Send the message on through to the AV's, SA, etc..
4) SA will adjust the totals, rinse and repeat.

This to me has 2 good effects:
1) Uses graylisting as an efficient means to stop spam (early on in the 
SMTP conversation.)
2) Integrates SpamAssassin in what it does best, scoring. Thus feeds 
back to the graylisting mech.


Of course, this assume that the standard table of senders, ip's, and 
their last try time are kept. Along with whitelist/blacklist entries. I 
think this is a good use of SpamAssassin, graylisting and our tight IT 
budgets for processor time.


My $0.25 worth.
--
Thanks,
JamesDR


smime.p7s
Description: S/MIME Cryptographic Signature

Re: Can SA be used to implement greylisting?

[John D. Hardin]

On Mon, 19 Jun 2006, Logan Shaw wrote:

> If it comes up with a very high score (almost definitely spam),
> drop it right away.  If it comes up with an indeterminate score,
> apply the greylisting approach and delay it until later.

What's the point? You've already *got* the entire message, at that
point why tell the sender "I don't want it right now, try again
later"? Instead of SMTP TMPFAILing the message, why not just add a few
SA points for "never seen this sender before"?

> What does this buy you?  Two things.  The first is that low-risk
> messages (based on content) go right through, eliminating much
> of the downside of greylisting.  The second is that for messages
> which SpamAssassin is unsure about, you get the added benefit of
> greylisting.  By definition, SpamAssassin by itself is insufficient
> in these cases, so any extra information you can gather (i.e. whether
> the sender retries) is valuable information.

At the cost of receiving and processing those messages at least twice.

Consider the case of a spammer whose software *does* retry, but
retries every two or three minutes until delivery is accepted or
PERMFAILed. I have seen this in my greylist logs. Do you really want
SA + AV + whatever to completely process this message a half-dozen
times before making a permanent decision at the end of the delay
period?
 
> To put it another way, greylisting has a high cost in terms of
> convenience.

Email is a store-and-forward best-attempt unguaranteed delivery
system, *not* Instant Messaging. The perception that a fifteen-minute
delay in delivery of a message is not acceptable is unrealistic. And
if such a delay *is* unacceptable, then you need to use something
other than email to communicate.

Most greylisting tools can be configured to bypass greylisting for
specified recipient addresses. In your example of salespeople not
wanting to have their messages from potential customers delayed, just
bypass greylisting for them and leave the standard behavior in place
for everybody else.

--
 John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/
 [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED]
 key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
What nuts do with guns is terrible, certainly. But what evil or crazy
people do with *anything* is not a valid argument for banning that item.
  -- John C. Randolph <[EMAIL PROTECTED]>
---

Re: Can SA be used to implement greylisting?

[John D. Hardin]

On Mon, 19 Jun 2006, Steven W. Orr wrote:
> 
> And this is my point. SA *DOESN'T* work on messages after they
> have been received. Since I use spamass-milter, SA sees the
> messages before reception is completed.

So, you're passing just the message headers through SA?

Using a milter doesn't magically mean that you haven't received the
body of the message yet, it just means that a third-party extension to
Sendmail is allowed to review the message before Sendmail acknowledges
receipt and attempts delivery to the next stage (procmail, relay,
whatever). You've already spent the time and bandwidth and system
resources to receive the entire message and process it through SA.

So: why not save the resources consumed by receiving and processing
the entire message *multiple times* (because legitimate messages will
have to be almost completely transmitted at least twice in this model)
and use an existing tool (greylist-milter) to do the greylisting early
in the SMTP exchange?

--
 John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/
 [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED]
 key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
What nuts do with guns is terrible, certainly. But what evil or crazy
people do with *anything* is not a valid argument for banning that item.
  -- John C. Randolph <[EMAIL PROTECTED]>
---

Re: Can SA be used to implement greylisting?

[Logan Shaw]

On Mon, 19 Jun 2006, David B Funk wrote:

On Mon, 19 Jun 2006, Justin Mason wrote:



Yep -- that's the key point -- as far as I know it's illegal (in
SMTP terms) to offer a 421 after DATA.


RFC-2821 section 3.9:

  An SMTP server MUST NOT intentionally close the connection except:

  -  After receiving a QUIT command and responding with a 221 reply.

  -  After detecting the need to shut down the SMTP service and
 returning a 421 response code.  This response code can be issued
 after the server receives any command or, if necessary,
 asynchronously from command receipt (on the assumption that the
 client will receive it after the next command is issued).

So anytime is 421 time. ;)

also a 451 is explicitly listed as an acceptable error response
to a DATA.


OK, if that's the case, let me offer my own personal justification of
why it might be worthwhile to combine greylisting with SpamAssassin.

Basically, greylisting has an achilles heel:  legit messages
from unknown senders are delayed a long time.  This is fine for
certain types of organizations, but what if all the accounts on
your mail server are for salespeople?  They're constantly trying to
generate new contacts and leads, and they really don't want their
communications to be delayed.  This is just one example; there are
surely other people who don't want any delays that can be avoided.

So, what does this have to do with greylisting + content-based
filtering?  It's simple: if you receive a message from an unknown
sender / domain / IP / whatever, you can then do a spamassassin run
on it.  If it comes up with a very low score (almost definitely
not spam), let it pass.  If it comes up with a very high score
(almost definitely spam), drop it right away.  If it comes up with
an indeterminate score, apply the greylisting approach and delay
it until later.

What does this buy you?  Two things.  The first is that low-risk
messages (based on content) go right through, eliminating much
of the downside of greylisting.  The second is that for messages
which SpamAssassin is unsure about, you get the added benefit of
greylisting.  By definition, SpamAssassin by itself is insufficient
in these cases, so any extra information you can gather (i.e. whether
the sender retries) is valuable information.

To put it another way, greylisting has a high cost in terms of
convenience.  So, apply greylisting only when SpamAssassin is not
confident in its judgement; in those cases, you can easily justify
the cost, and in the other cases, you can avoid the cost of
greylisting completely.

  - Logan

Re: Can SA be used to implement greylisting?

[David B Funk]

On Mon, 19 Jun 2006, Justin Mason wrote:

> Yep -- that's the key point -- as far as I know it's illegal (in
> SMTP terms) to offer a 421 after DATA.
>
> --j.

RFC-2821 section 3.9:

   An SMTP server MUST NOT intentionally close the connection except:

   -  After receiving a QUIT command and responding with a 221 reply.

   -  After detecting the need to shut down the SMTP service and
  returning a 421 response code.  This response code can be issued
  after the server receives any command or, if necessary,
  asynchronously from command receipt (on the assumption that the
  client will receive it after the next command is issued).

So anytime is 421 time. ;)

also a 451 is explicitly listed as an acceptable error response
to a DATA.



-- 
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Re: Can SA be used to implement greylisting?

[Jonas Eckerman]

Steven W. Orr wrote:

I'm running sendmail here on a home server. I've been looking for a good 
greylist package and I frankly have not found one. There are a couple out 
there but they work in memory and don't maintain their tables in a 
database.


My greylist code  using the 
milter MIMEDefang  keeps the tables in a SQLite database 
(wich means it shouldn't be too hard porting to a server based SQL database).

I'm also running spamass-milter which is set to reject mail ifd SA says 
it's spam. Is it worthwhile to try to convince the SA dev crowd to add 
greylist functionality?


One of the key benefits of "normal" greylisting is that it you don't need to 
call SpamAssassin, anti virus stuff, etc, at all for the messages that never makes it 
through a SMTP level grey list.

Also, SpamAssassin cannot itself refuse to accept a mail, so the SMTP software 
still has to be involved for grey listing to work.

/Jonas

--
Jonas Eckerman, FSDB & Fruktträdet
http://whatever.frukt.org/
http://www.fsdb.org/
http://www.frukt.org/

Re: Can SA be used to implement greylisting?

[David B Funk]

On Mon, 19 Jun 2006, Steven W. Orr wrote:

> And this is my point. SA *DOESN'T* work on messages after they have been
> received. Since I use spamass-milter, SA sees the messages before
> reception is completed. (You're free to do otherwise.) Then when SA
> decides that the message doesn't conform to its high standards, the report
> of that fact goes back to spamass-milter which then returns status back to
> sendmail. The current result is a reject 5xx status. So all we need is for
> SA to manage one extra table and to allow some sort of reportage that
> spamass-milter could be mucked to understand.
>
> Is this making sense?

Yes, but IMHO you are trying to use the wrong tool for the job.

greylisting is a relativly lightweight task and can be done quickly
(IE no DNS/network lookups needed, only need envelope-from, envelope-to,
sending host IP addr, no need to absorb the whole message body,
very small CPU load ,etc) so it should be done -before- you waste
resources running a full SA scan.

I would suggest sendmail milter configured so that first you run the
greylisting milter, then the virus-scan milter and finally the SA
milter.

I can see your argument for needing some kind of persistant table
backend for greylisting but that looks like an arugment for building
a MySQL backend for greylisting, not trying to mung greylisting into SA.

The only reason that I can see for trying to combine greylisting & SA
is to have an adaptive greylist, one that only kicks in if the message
has a high-enough score (but still lower than spam-tagging threshold).
However you would loose the load reduction benefit of the previously
mentioned config.

Dave

-- 
Dave Funk  University of Iowa
College of Engineering
319/335-5751   FAX: 319/384-0549   1256 Seamans Center
Sys_admin/Postmaster/cell_adminIowa City, IA 52242-1527
#include 
Better is not better, 'standard' is better. B{

Re: Can SA be used to implement greylisting?

[Justin Mason]

John D. Hardin writes:
> On Mon, 19 Jun 2006, Steven W. Orr wrote:
> 
> > =>> Is it worthwhile to try to convince the SA dev 
> > =>> crowd to add greylist functionality?
> 
> > Neither. What I'm looking for is a rubust way to say: "I haven't
> > seen mail from this guy before so I'm going to reject his email
> > with a 450 error code. If email from him tries for delivery after
> > (let's say) four hours, then I will accept it and nevermore will
> > this guy have a delay in sending me mail."
> 
> That's the common definition of "greylisting".
> 
> I think the suggestion to add it to SA misses a basic fact: the
> mail has *already been received in its entirety* by the time SA gets a
> chance to see it. What's the point in greylisting then?
> 
> Proper greylisting is done early in the SMTP exchange, at the point
> the DATA command is sent and the sender and recipients are known but
> before the message itself has been received. I use milter-greylist to
> do this and it works well.

Yep -- that's the key point -- as far as I know it's illegal (in
SMTP terms) to offer a 421 after DATA.

--j.

Re: Can SA be used to implement greylisting?

[Andy Jezierski]

[snip]

> And this is my point. SA *DOESN'T* work on messages
after they have been 
> received. Since I use spamass-milter, SA sees the messages before

> reception is completed. (You're free to do otherwise.) Then when SA

> decides that the message doesn't conform to its high standards, the
report 
> of that fact goes back to spamass-milter which then returns status
back to 
> sendmail. The current result is a reject 5xx status. So all we need
is for 
> SA to manage one extra table and to allow some sort of reportage that

> spamass-milter could be mucked to understand.
> 
> Is this making sense?
> 

Why re-invent the wheel. While I'm sure most of the
greylist milters out there are similar, I can only comment on milter-greylist.
It will do almost everything you're looking for. The first time it receives
a message it will send out a 451, it'll continue rejecting messages until
a user defined time limit is reached. Two minutes in our case, although
I suppose you could specify 4 hours as you stated, but I don't know why
you'd want your mail delayed that long.  I've noticed that most servers
will try to resend their messages within 5-15 minutes. 

Once the second message is received, the tuple is
stored in memory, and the table is dumped to disk at a user defined time
interval. When you re-start the software, it'll attempt to read in the
last dumped file to rebuild it's in memory table.  You also, define
how long each entry is valid in memory, 7 days in our case, since yours
is a home server, you should be able to bump this limit way up. As long
as you keep receiving mail from a particular sender within this time period,
their mail will not be delayed.  If no messages are received in that
time period, the entry is dropped and they'll get a 451 the next time.
 If you know specific email addresses, or domains, or mail servers
that you never want to delay, you can white list them ahead of time using
the config files.

Practically everything you want, and nothing needs
to be re-coded.

Andy

RE: Can SA be used to implement greylisting?

[John D. Hardin]

On Mon, 19 Jun 2006, Steven W. Orr wrote:

> =>> Is it worthwhile to try to convince the SA dev 
> =>> crowd to add greylist functionality?

> Neither. What I'm looking for is a rubust way to say: "I haven't
> seen mail from this guy before so I'm going to reject his email
> with a 450 error code. If email from him tries for delivery after
> (let's say) four hours, then I will accept it and nevermore will
> this guy have a delay in sending me mail."

That's the common definition of "greylisting".

I think the suggestion to add it to SA misses a basic fact: the
mail has *already been received in its entirety* by the time SA gets a
chance to see it. What's the point in greylisting then?

Proper greylisting is done early in the SMTP exchange, at the point
the DATA command is sent and the sender and recipients are known but
before the message itself has been received. I use milter-greylist to
do this and it works well.

--
 John Hardin KA7OHZICQ#15735746http://www.impsec.org/~jhardin/
 [EMAIL PROTECTED]FALaholic #11174pgpk -a [EMAIL PROTECTED]
 key: 0xB8732E79 - 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
What nuts do with guns is terrible, certainly. But what evil or crazy
people do with *anything* is not a valid argument for banning that item.
  -- John C. Randolph <[EMAIL PROTECTED]>
---

Re: Can SA be used to implement greylisting?

[Ron Johnson]

Steven W. Orr writes:
> 
> 
> And this is my point. SA *DOESN'T* work on messages after they have been 
> received. Since I use spamass-milter, SA sees the messages before 
> reception is completed. (You're free to do otherwise.) Then when SA 
> decides that the message doesn't conform to its high standards, the report 
> of that fact goes back to spamass-milter which then returns status back to 
> sendmail. The current result is a reject 5xx status. So all we need is for 
> SA to manage one extra table and to allow some sort of reportage that 
> spamass-milter could be mucked to understand.
> 
> Is this making sense?
> 
Yes. And if you can manage to do the heavy lifting a lot of
people will thank you. (I have little problem accepting that
you can do pretty much anything via milter. But there's enough 
stuff that looks tricky that I'd be surprised if it goes high
on anybody's todo list)

To get back to your original point (memory resident versus databases)
there are some interesting setups that you can look at using
mimedefang.

http://whatever.frukt.org/mimedefangfilter.text.shtml

and from the wiki:

http://www.mimedefang.com/kwiki/index.cgi?Greylisting

I understand, you're probably not interested in mimedefang.
Still worth a look IMO.

Re: Can SA be used to implement greylisting?

[Steven W. Orr]

On Monday, Jun 19th 2006 at 10:24 -0700, quoth Bill Landry:

=>- Original Message - From: "Steven W. Orr" <[EMAIL PROTECTED]>
=>
=>> On Monday, Jun 19th 2006 at 11:40 -0400, quoth Chris Santerre:
=>> 
=>> =>
=>> =>
=>> =>> -Original Message-
=>> =>> From: Steven W. Orr [mailto:[EMAIL PROTECTED]
=>> =>> Sent: Monday, June 19, 2006 9:08 AM
=>> =>> To: spamassassin-users
=>> =>> Subject: Can SA be used to implement greylisting?
=>> =>>
=>> =>>
=>> =>> I'm running sendmail here on a home server. I've been looking
=>> =>> for a good
=>> =>> greylist package and I frankly have not found one. There are
=>> =>> a couple out
=>> =>> there but they work in memory and don't maintain their tables in a
=>> =>> database.
=>> =>
=>> =>grey.uribl.com  ???
=>> =>
=>> =>> I'm also running spamass-milter which is set to reject mail
=>> =>> ifd SA says
=>> =>> it's spam. Is it worthwhile to try to convince the SA dev
=>> =>> crowd to add
=>> =>> greylist functionality? I know it would be easy to modify
=>> =>> spamass-milter
=>> =>> to return the needed info to sendmail. It would require a new table.
=>> =>>
=>> =>> Does this make sense?
=>> =>
=>> =>Not really. Are you talking about greylisting as in a greet pause, or a
=>> =>"This is a spamish domain."?
=>> =>
=>> =>Greet pause would be used in Sendmail. grey.uribl.com would be used for
=>> the
=>> =>later.
=>> =>
=>> =>http://www.uribl.com/usage.shtml
=>> 
=>> Neither. What I'm looking for is a rubust way to say: "I haven't seen mail
=>> from this guy before so I'm going to reject his email with a 450 error
=>> code. If email from him tries for delivery after (let's say) four
=>> hours, then I will accept it and nevermore will this guy have a delay in
=>> sending me mail."
=>> 
=>> It's not a spam identifying technique but it does eliminate about 90% of
=>> the spam. The question is, is this worthwhile exploring as adjunct
=>> functionality to SA?
=>> 
=>> Yes, I understand that SA does not have any ability to reject mail, much
=>> less specify an SMTP error code. Is this clearly out of bounds for what SA
=>> should be doing?
=>
=>Yes, this has to happen before SA gets the message, as SA works on messages
=>after they have been fully received.  Greylisting needs to happen at the MTA
=>level, before the message is received.  Depending on what MTA you are using,
=>most support greylisting plug-ins.
=>
=>Bill 

And this is my point. SA *DOESN'T* work on messages after they have been 
received. Since I use spamass-milter, SA sees the messages before 
reception is completed. (You're free to do otherwise.) Then when SA 
decides that the message doesn't conform to its high standards, the report 
of that fact goes back to spamass-milter which then returns status back to 
sendmail. The current result is a reject 5xx status. So all we need is for 
SA to manage one extra table and to allow some sort of reportage that 
spamass-milter could be mucked to understand.

Is this making sense?

-- 
Time flies like the wind. Fruit flies like a banana. Stranger things have  .0.
happened but none stranger than this. Does your driver's license say Organ ..0
Donor?Black holes are where God divided by zero. Listen to me! We are all- 000
individuals! What if this weren't a hypothetical question?
steveo at syslang.net

Re: Can SA be used to implement greylisting?

[Bill Landry]

- Original Message - 
From: "Steven W. Orr" <[EMAIL PROTECTED]>



On Monday, Jun 19th 2006 at 11:40 -0400, quoth Chris Santerre:

=>
=>
=>> -Original Message-
=>> From: Steven W. Orr [mailto:[EMAIL PROTECTED]
=>> Sent: Monday, June 19, 2006 9:08 AM
=>> To: spamassassin-users
=>> Subject: Can SA be used to implement greylisting?
=>>
=>>
=>> I'm running sendmail here on a home server. I've been looking
=>> for a good
=>> greylist package and I frankly have not found one. There are
=>> a couple out
=>> there but they work in memory and don't maintain their tables in a
=>> database.
=>
=>grey.uribl.com  ???
=>
=>> I'm also running spamass-milter which is set to reject mail
=>> ifd SA says
=>> it's spam. Is it worthwhile to try to convince the SA dev
=>> crowd to add
=>> greylist functionality? I know it would be easy to modify
=>> spamass-milter
=>> to return the needed info to sendmail. It would require a new table.
=>>
=>> Does this make sense?
=>
=>Not really. Are you talking about greylisting as in a greet pause, or a
=>"This is a spamish domain."?
=>
=>Greet pause would be used in Sendmail. grey.uribl.com would be used for 
the

=>later.
=>
=>http://www.uribl.com/usage.shtml

Neither. What I'm looking for is a rubust way to say: "I haven't seen mail
from this guy before so I'm going to reject his email with a 450 error
code. If email from him tries for delivery after (let's say) four
hours, then I will accept it and nevermore will this guy have a delay in
sending me mail."

It's not a spam identifying technique but it does eliminate about 90% of
the spam. The question is, is this worthwhile exploring as adjunct
functionality to SA?

Yes, I understand that SA does not have any ability to reject mail, much
less specify an SMTP error code. Is this clearly out of bounds for what SA
should be doing?


Yes, this has to happen before SA gets the message, as SA works on messages 
after they have been fully received.  Greylisting needs to happen at the MTA 
level, before the message is received.  Depending on what MTA you are using, 
most support greylisting plug-ins.


Bill 

RE: Can SA be used to implement greylisting?

[Steven W. Orr]

On Monday, Jun 19th 2006 at 11:40 -0400, quoth Chris Santerre:

=>
=>
=>> -Original Message-
=>> From: Steven W. Orr [mailto:[EMAIL PROTECTED]
=>> Sent: Monday, June 19, 2006 9:08 AM
=>> To: spamassassin-users
=>> Subject: Can SA be used to implement greylisting?
=>> 
=>> 
=>> I'm running sendmail here on a home server. I've been looking 
=>> for a good 
=>> greylist package and I frankly have not found one. There are 
=>> a couple out 
=>> there but they work in memory and don't maintain their tables in a 
=>> database.
=>
=>grey.uribl.com  ???
=> 
=>> I'm also running spamass-milter which is set to reject mail 
=>> ifd SA says 
=>> it's spam. Is it worthwhile to try to convince the SA dev 
=>> crowd to add 
=>> greylist functionality? I know it would be easy to modify 
=>> spamass-milter 
=>> to return the needed info to sendmail. It would require a new table.
=>> 
=>> Does this make sense?
=>
=>Not really. Are you talking about greylisting as in a greet pause, or a
=>"This is a spamish domain."? 
=>
=>Greet pause would be used in Sendmail. grey.uribl.com would be used for the
=>later. 
=>
=>http://www.uribl.com/usage.shtml

Neither. What I'm looking for is a rubust way to say: "I haven't seen mail 
from this guy before so I'm going to reject his email with a 450 error 
code. If email from him tries for delivery after (let's say) four 
hours, then I will accept it and nevermore will this guy have a delay in 
sending me mail." 

It's not a spam identifying technique but it does eliminate about 90% of 
the spam. The question is, is this worthwhile exploring as adjunct 
functionality to SA?

Yes, I understand that SA does not have any ability to reject mail, much 
less specify an SMTP error code. Is this clearly out of bounds for what SA 
should be doing?

-- 
steveo at syslang dot net TMMP1 http://frambors.syslang.net/
Do you have neighbors who are not frambors?

RE: Can SA be used to implement greylisting?

[Chris Santerre]

Title: RE: Can SA be used to implement greylisting?







> -Original Message-
> From: Steven W. Orr [mailto:[EMAIL PROTECTED]]
> Sent: Monday, June 19, 2006 9:08 AM
> To: spamassassin-users
> Subject: Can SA be used to implement greylisting?
> 
> 
> I'm running sendmail here on a home server. I've been looking 
> for a good 
> greylist package and I frankly have not found one. There are 
> a couple out 
> there but they work in memory and don't maintain their tables in a 
> database.


grey.uribl.com  ???
 
> I'm also running spamass-milter which is set to reject mail 
> ifd SA says 
> it's spam. Is it worthwhile to try to convince the SA dev 
> crowd to add 
> greylist functionality? I know it would be easy to modify 
> spamass-milter 
> to return the needed info to sendmail. It would require a new table.
> 
> Does this make sense?


Not really. Are you talking about greylisting as in a greet pause, or a "This is a spamish domain."? 


Greet pause would be used in Sendmail. grey.uribl.com would be used for the later. 


http://www.uribl.com/usage.shtml


Chris Santerre
SysAdmin and SARE/URIBL ninja
http://www.uribl.com
http://www.rulesemporium.com