Re: Yet another simple question - how to reprocess an email

2019-11-28 Thread Joe Acquisto-j4
>>>

>> On Thu, 2019-11-28 at 18:38 -0500, Joe Acquisto-j4 wrote:
>> 
> > Is there any tangent down this path were I can get the dropped
>>> > > "test" message to actually flow through, in "normal" fashion?
>>> 
>>> > . . . 
>> My set up is a little odd in that my pipeline used getmail to retrieve
>> mail from my ISP's smarthost and precedes my MTA with a pipeline like
>> this, where 'spamkiller' is a simple C program that looks at the spam
>> headers to see whether its spam or ham. Spam is sent to a holding area
>> and ham is passed to Postfix for delivery. Here's a diagram:
>> 
>> getmail --> spamc --> spamkiller --ham--> sendmail --> postfix
>>|
>>+--spam--> spam quarantine store
>> 
>> Most people simply splice spamc into Postfix's internal pipeline,
>> defined in master.cf, which connects its mail reception process to its
>> delivery process. 
>> 
>> Martin
> 
> Oh, now you are asking me to think, or, revisit stuff I setup a looong time 
> ago.
> 
> I use fetchmail on a different box to pull mail from several accounts at an 
> ISP and send those
> messages to the SA/postfix box.
> 
> I can post the spamassassin line from master.cf if needed but basically SA 
> and sendmail/postfix are on the same box,
> and when anti-spam processing is complete postfix then delivers to yet 
> another box for final disposal.
> 
> joe a.
> 
> 

Well, success, I think anyway.  This seems to do it from the command line"

/usr/bin/spamc -s 75 < test.txt | /usr/sbin/sendmail -t -i "$@"

Thanks for the kick.

joe a/

-- 
+++
 joea@@j4computers.com
  https://www.j4computers.com
   845-687-3734
+++


Re: Yet another simple question - how to reprocess an email

2019-11-28 Thread Joe Acquisto-j4
>>>
> On Thu, 2019-11-28 at 18:38 -0500, Joe Acquisto-j4 wrote:
> 
>> > > Is there any tangent down this path were I can get the dropped
>> > > "test" message to actually flow through, in "normal" fashion?
>> 
>> > From logs I can see that spamd does seem to give the message a
>> > taste,  as I can follow /
>> var/log/mail and see activity at the precise time I feed it, but the
>> message  does not seem to be delivered.
>> 
>> I've tried some other off box methods as well, including using CURL
>> which is purported to have smtp ability, yet I get syntax
>> errors or invalid option as the touted features do not exist in
>> versions in use here.
>> 
> What are you doing with the message after it comes back from spamc? 
> 
> spamc should be in some sort of pipeline that grabs the message after it
> has had X-Spam headers inserted and pass it to whatever will queue it
> for the intended recipient's MUA.
> 
> My set up is a little odd in that my pipeline used getmail to retrieve
> mail from my ISP's smarthost and precedes my MTA with a pipeline like
> this, where 'spamkiller' is a simple C program that looks at the spam
> headers to see whether its spam or ham. Spam is sent to a holding area
> and ham is passed to Postfix for delivery. Here's a diagram:
> 
> getmail --> spamc --> spamkiller --ham--> sendmail --> postfix
>|
>+--spam--> spam quarantine store
> 
> Most people simply splice spamc into Postfix's internal pipeline,
> defined in master.cf, which connects its mail reception process to its
> delivery process. 
> 
> Martin

Oh, now you are asking me to think, or, revisit stuff I setup a looong time ago.

I use fetchmail on a different box to pull mail from several accounts at an ISP 
and send those
messages to the SA/postfix box.

I can post the spamassassin line from master.cf if needed but basically SA and 
sendmail/postfix are on the same box,
and when anti-spam processing is complete postfix then delivers to yet another 
box for final disposal.

joe a.


-- 
+++
 joea@@j4computers.com
  https://www.j4computers.com
   845-687-3734
+++


Re: Yet another simple question - how to reprocess an email

2019-11-28 Thread Martin Gregorie
On Thu, 2019-11-28 at 18:38 -0500, Joe Acquisto-j4 wrote:

> > > Is there any tangent down this path were I can get the dropped
> > > "test" message to actually flow through, in "normal" fashion?
> 
> > From logs I can see that spamd does seem to give the message a
> > taste,  as I can follow /
> var/log/mail and see activity at the precise time I feed it, but the
> message  does not seem to be delivered.
> 
> I've tried some other off box methods as well, including using CURL
> which is purported to have smtp ability, yet I get syntax
> errors or invalid option as the touted features do not exist in
> versions in use here.
> 
What are you doing with the message after it comes back from spamc? 

spamc should be in some sort of pipeline that grabs the message after it
has had X-Spam headers inserted and pass it to whatever will queue it
for the intended recipient's MUA.

My set up is a little odd in that my pipeline used getmail to retrieve
mail from my ISP's smarthost and precedes my MTA with a pipeline like
this, where 'spamkiller' is a simple C program that looks at the spam
headers to see whether its spam or ham. Spam is sent to a holding area
and ham is passed to Postfix for delivery. Here's a diagram:

getmail --> spamc --> spamkiller --ham--> sendmail --> postfix
   |
   +--spam--> spam quarantine store

Most people simply splice spamc into Postfix's internal pipeline,
defined in master.cf, which connects its mail reception process to its
delivery process. 

Martin




Re: Yet another simple question - how to reprocess an email

2019-11-28 Thread Martin Gregorie
On Thu, 2019-11-28 at 22:34 +, RW wrote:
> A lot of us rely on SA stipping X-Spam-* headers, so header-based
> filtering into a spam folder works correctly. This includes numerous
> mail hosting and freemail providers. 
> 
Interesting: I've just rechecked this using your last message in this
conversation and it has worked as you expected it to: 

- saved as an mbox format file by Evolution
- fed to spamd using spamc and grabbed on its return by less for
  inspection
- the original three X-Spam headers were replaced by three new ones
  recognised by a different hostname (expected) and slightly different 
  SA score.

I don't know when this started to work: I've been using SA since around
2004 and back then I was certainly seeing both the old and new sets of
X-Spam headers if I ran a message through my SA test systrem, otherwise
I'd never have written the 'cleaner' script.

As it happens, 'cleaner' does nothing to a message except copy it if
there are no X-Spam headers in the file, so no harm done apart from a
few wasted cycles.

Thanks for prodding me into checking it just now.

Martin





Re: Yet another simple question - how to reprocess an email

2019-11-28 Thread Joe Acquisto-j4
>>>

>> On Thu, 2019-11-28 at 11:56 -0500, Joe Acquisto-j4 wrote:
I want to be able to reprocess a particular email, marked as SPAM,
>>> after making some SA tweaks.   
>>> 
>> I do something similar with with collection of test messages, mostly
>> received spam, that I use to test my local SA rule set.
>> 
>> Essentially, all I do is:
>> 
>> 1) remove all headers starting with 'X-Spam', otherwise the X-Spam
>>headers injected when the piece if spam was received will still be
>>there after I've run the test. This is confusing rather than harmful,
>>but the cleaner is just a script using awk. See below.
>> 
>> 2) pass the message through spamd by running:
>> 
>>  spamc --max-size=200 > 
>>and examine the result
>> 
> 
> Thanks. Helpful.   I did not test the cleaner as I had already, in effect, 
> "sanitized" the message.
> 
> What I had hoped to do was allow this to flow though and be delivered to the 
> end point, as if under normal circumstances.
> 
> joe a. 
> 

Gentlefolks . . .

Is there any tangent down this path were I can get the dropped "test" message 
to actually flow through, in "normal" fashion?

>From logs I can see that spamd does seem to give the message a taste,  as I 
>can follow /
var/log/mail and see activity at the precise time I feed it, but the message  
does not seem to be delivered.

I've tried some other off box methods as well, including using CURL which is 
purported to have smtp ability, yet I get syntax
errors or invalid option as the touted features do not exist in versions in use 
here.





-- 
+++
 joea@@j4computers.com
  https://www.j4computers.com
   845-687-3734
+++


Re: Yet another simple question - how to reprocess an email

2019-11-28 Thread RW
On Thu, 28 Nov 2019 21:32:29 +
Martin Gregorie wrote:

> On Thu, 2019-11-28 at 21:00 +, RW wrote:
> 
> > There shouldn't be any need for this as SA strips such headers
> > itself. 
> Yes, I've seem that said several times, BUT every time I capture some
> spam from Evolution by using "File:Save as mbox" to capture it as a
> .txt file and then feed it into SA as I showed:
> 
> $ spamc --max-size=200  
> what is displayed always contains both the X-Spam headers added as the
> message hit my system and a second set of X-Spam headers added by that
> test shot. 


That's strange:

$ printf 'X-Spam-Status: 12345\n' | spamc -u test | grep X-Spam-Status 
X-Spam-Status: Yes, score=11.4 required=5.0 autolearn=disabled

$ printf 'X-Spam-Foo: 12345\n' | spamc -u test | grep X-Spam-Foo
$

A lot of us rely on SA stipping X-Spam-* headers, so header-based
filtering into a spam folder works correctly. This includes numerous
mail hosting and freemail providers. 

My guess is that when you saw this you had corrupted the headers in some
way.

 



Re: Yet another simple question - how to reprocess an email

2019-11-28 Thread Martin Gregorie
On Thu, 2019-11-28 at 21:00 +, RW wrote:

> There shouldn't be any need for this as SA strips such headers itself.
>
Yes, I've seem that said several times, BUT every time I capture some
spam from Evolution by using "File:Save as mbox" to capture it as a .txt
file and then feed it into SA as I showed:

$ spamc --max-size=200  domainhost -> ISP --getmail--> SA --sendmail--> Postfix

because this allows me to send and receive mail without having any ports
open in my firewall.
 
Martin




Re: Yet another simple question - how to reprocess an email

2019-11-28 Thread RW
On Thu, 28 Nov 2019 17:42:34 +
Martin Gregorie wrote:


> I do something similar with with collection of test messages, mostly
> received spam, that I use to test my local SA rule set.
> 
> Essentially, all I do is:
> 
> 1) remove all headers starting with 'X-Spam', otherwise the X-Spam
>headers injected when the piece if spam was received will still be
>there after I've run the test. This is confusing rather than
> harmful, but the cleaner is just a script using awk. See below.

There shouldn't be any need for this as SA strips such headers itself.


Re: Getting spamass-milter to work with postfix

2019-11-28 Thread Linkcheck

Thanks for that. I wasn't sure which config to add it to.

I now have the following in /etc/default/spamass-milter...

  OPTIONS="-u spamass-milter -i 127.0.0.1 -- -s 800"

... which works.


Re: Getting spamass-milter to work with postfix

2019-11-28 Thread Matus UHLAR - fantomas

On 28.11.19 16:40, Linkcheck wrote:
Belated thans again to all who helped me with this problem. It has 
been running reasonably well since my last posting. I now have only a 
couple of minor problems left, one of which (I think) more suited to 
the postfix forum.


The other one is max-size. I had gradually uprated this to 800 in 
the content filter .sh file, called via master.cf. I have tried 
setting it in /etc/default/spamassassin as --max-size=800 (which 
accepts it but ignores it) and in spamass-milter which does not accept 
either the long or short version (which is expected from the docs).


Where should I specify this and how, please?


spamass-milter calls spamc, which by default avoids mail bigger than 500KB.
you must provide extra spamc parameters in /etc/default/spamass-milter.

see man spamass-milter.


--
Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Windows 2000: 640 MB ought to be enough for anybody


Re: Yet another simple question - how to reprocess an email

2019-11-28 Thread Joe Acquisto-j4
>>>
> On Thu, 2019-11-28 at 11:56 -0500, Joe Acquisto-j4 wrote:
>> I want to be able to reprocess a particular email, marked as SPAM,
>> after making some SA tweaks.   
>> 
> I do something similar with with collection of test messages, mostly
> received spam, that I use to test my local SA rule set.
> 
> Essentially, all I do is:
> 
> 1) remove all headers starting with 'X-Spam', otherwise the X-Spam
>headers injected when the piece if spam was received will still be
>there after I've run the test. This is confusing rather than harmful,
>but the cleaner is just a script using awk. See below.
> 
> 2) pass the message through spamd by running:
> 
>   spamc --max-size=200  
>and examine the result
> 

Thanks. Helpful.   I did not test the cleaner as I had already, in effect, 
"sanitized" the message.

What I had hoped to do was allow this to flow though and be delivered to the 
end point, as if under normal circumstances.

joe a. 

-- 
+++
 joea@@j4computers.com
  https://www.j4computers.com
   845-687-3734
+++


Re: Yet another simple question - how to reprocess an email

2019-11-28 Thread Martin Gregorie
On Thu, 2019-11-28 at 11:56 -0500, Joe Acquisto-j4 wrote:
> I want to be able to reprocess a particular email, marked as SPAM,
> after making some SA tweaks.   
> 
I do something similar with with collection of test messages, mostly
received spam, that I use to test my local SA rule set.

Essentially, all I do is:

1) remove all headers starting with 'X-Spam', otherwise the X-Spam
   headers injected when the piece if spam was received will still be
   there after I've run the test. This is confusing rather than harmful,
   but the cleaner is just a script using awk. See below.

2) pass the message through spamd by running:

spamc --max-size=200 temp.txt
mv temp.txt $1
}

if [ $# -gt 0 ]
then
for f in $*
do
clean $f
done
else
for f in data/*.txt
do
clean $f
done
fi
== end of cleaner ===

Martin




Yet another simple question - how to reprocess an email

2019-11-28 Thread Joe Acquisto-j4
Well, here goes, asbestos pants on.  I did, honest, do some searching before 
asking this.

I want to be able to reprocess a particular email, marked as SPAM, after making 
some SA tweaks.   

Basically I have saved the email, which was received as an attachment, as a 
text file.   Thinking to simply drop this file into a queue somewhere in the 
receive process and just let her rip.  However I was given pause by the message 
numbering that sendmail seems to use. 

The process is mail is delivered by a "fetching" server, to a sendmail box, 
which also hosts SA, after processing is delivered to another on prem box.   

Thanks for any assistance.

-- 
+++
 joea@@j4computers.com
  https://www.j4computers.com
   845-687-3734
+++


Re: Rule to catch a certain email adress?

2019-11-28 Thread Axb

On 2019-11-28 13:43, Anders Gustafsson wrote:

Assume I want to give extra points to e-p...@pedago.fi? This is our
adress as given on our wesite so many spammers harvest that. I waht to
bump it sligtly, but have been unable to write a regexp that catches it.
Can anyone help?



this could work:

blacklist_to e-p...@pedago.fi

# SET SCORE TO WHATEVER YOU WANT
score USER_IN_BLACKLIST_TO  3.0

h2h


Re: Getting spamass-milter to work with postfix

2019-11-28 Thread Linkcheck
Belated thans again to all who helped me with this problem. It has been 
running reasonably well since my last posting. I now have only a couple 
of minor problems left, one of which (I think) more suited to the 
postfix forum.


The other one is max-size. I had gradually uprated this to 800 in 
the content filter .sh file, called via master.cf. I have tried setting 
it in /etc/default/spamassassin as --max-size=800 (which accepts it 
but ignores it) and in spamass-milter which does not accept either the 
long or short version (which is expected from the docs).


Where should I specify this and how, please?



Re: shortcircuit on alread x-spam-flag: yes

2019-11-28 Thread John Hardin

On Wed, 27 Nov 2019, Philipp Ewald wrote:


Hi Tobi,

we only want to trust "X-Spam-Flag: YES" or why should someone (spammer, 
other mailserver with outgoing spamfilter) set this Flag to Yes?


but like RW wrote:

If you want to
match on such a header you need to rewrite it before SA sees it.


i thought shortcircuit will test before any other tests but header was remove 
before shortcircuit :(

I have a lot to learn...

Thanks for help maybe i try this again... later :-)


The proper place to bypass SpamAssassin processing for any reason is in 
your glue layer.


How is SA hooked into your MTA?

Look into that, and see if there's a way to tell the glue to skip SA 
entirely if that header already exists.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Never forget, even for an instant, that the one and only reason
  anyone has for taking your gun away is to make you weaker than
  he is, so he can do something to you that you wouldn’t let him
  do if you were equipped to prevent it. This goes for burglars,
  muggers, and rapists, and even more so for policemen,
  bureaucrats, and politicians. -- Alexander Pope
---
 973 days since the first commercial re-flight of an orbital booster (SpaceX)

Re: shortcircuit on alread x-spam-flag: yes

2019-11-28 Thread Henrik K


It makes no difference for your network traffic, only SA 4.0 / trunk handles
shortcircuiting and network lookups properly.  But sure, marginal CPU
savings..


On Thu, Nov 28, 2019 at 01:50:31PM +0100, Philipp Ewald wrote:
> Hi Benny,
> 
> thanks for your link! ( i did not follow any BOFH Rules from this site ;-) )
> 
> i check headers and if "X-SPam-Flag: YES" is set, i write a custom Header 
> from postfix.
> 
> and in Spamassassin i search this custom header in shortcircuit.
> 
> It works!
> X-Spam-Status: Yes, score=98.7 tagged_above=- required=5
>   tests=[RCVD_IN_DNSWL_MED=-2.3, SHORTCIRCUIT=100, SpamFlag=1]
>   autolearn=disabled
> 
> i set this priority lower then DNSWL so save some network traffic
> 
> kind regards
> Philipp
> 
> 
> Am 27.11.19 um 18:30 schrieb Benny Pedersen:
> >On 2019-11-27 17:56, Philipp Ewald wrote:
> >
> >>we only want to trust "X-Spam-Flag: YES" or why should someone
> >>(spammer, other mailserver with outgoing spamfilter) set this Flag to
> >>Yes?
> >
> >trustness
> >
> >https://www.techiepark.com/tutorials/blocking-spam-using-postfix-header_checks-and-spamassassin/
> > bad example on what not to do :)
> >
> >http://www.techiepark.com/resources/postfix-header-checks/ really want to 
> >make postfix a spam filter ?
> >
> >bettr is to use fuglu.org as a before queue content filter with then can 
> >reject spam :=)
> >
> >i have still not seen mimedefang working
> >
> >
> 
> -- 
> Philipp Ewald
> Administrator
> 
> DigiOnline GmbH, Probsteigasse 15 - 19, 50670 Köln
> Telefon: +49 221 6500-532, Fax: +49 221 6500-690, E-Mail: 
> philipp.ew...@digionline.de
> 
> AG Köln HRB 27711, St.-Nr. 5215 5811 0640
> Geschäftsführer: Werner Grafenhain
> 
> Informationen zum Datenschutz: www.digionline.de/ds


Re: shortcircuit on alread x-spam-flag: yes

2019-11-28 Thread Philipp Ewald

Hi Benny,

thanks for your link! ( i did not follow any BOFH Rules from this site ;-) )

i check headers and if "X-SPam-Flag: YES" is set, i write a custom Header from 
postfix.

and in Spamassassin i search this custom header in shortcircuit.

It works!
X-Spam-Status: Yes, score=98.7 tagged_above=- required=5
tests=[RCVD_IN_DNSWL_MED=-2.3, SHORTCIRCUIT=100, SpamFlag=1]
autolearn=disabled

i set this priority lower then DNSWL so save some network traffic

kind regards
Philipp


Am 27.11.19 um 18:30 schrieb Benny Pedersen:

On 2019-11-27 17:56, Philipp Ewald wrote:


we only want to trust "X-Spam-Flag: YES" or why should someone
(spammer, other mailserver with outgoing spamfilter) set this Flag to
Yes?


trustness

https://www.techiepark.com/tutorials/blocking-spam-using-postfix-header_checks-and-spamassassin/
 bad example on what not to do :)

http://www.techiepark.com/resources/postfix-header-checks/ really want to make 
postfix a spam filter ?

bettr is to use fuglu.org as a before queue content filter with then can reject 
spam :=)

i have still not seen mimedefang working




--
Philipp Ewald
Administrator

DigiOnline GmbH, Probsteigasse 15 - 19, 50670 Köln
Telefon: +49 221 6500-532, Fax: +49 221 6500-690, E-Mail: 
philipp.ew...@digionline.de

AG Köln HRB 27711, St.-Nr. 5215 5811 0640
Geschäftsführer: Werner Grafenhain

Informationen zum Datenschutz: www.digionline.de/ds


Rule to catch a certain email adress?

2019-11-28 Thread Anders Gustafsson
Assume I want to give extra points to e-p...@pedago.fi? This is our
adress as given on our wesite so many spammers harvest that. I waht to
bump it sligtly, but have been unable to write a regexp that catches it.
Can anyone help?


>>> RW  2019-11-28 03:30 >>>
On Wed, 27 Nov 2019 12:59:47 +0100
Tobi  wrote:

> Hi,
> 
> is there any specific reason why the two tags mentioned in subject
are
> not set in SA? It took me a while to find out why an askdns test was
> not running. The test relies on _LASTEXTERNALRDNS_ but after running
> with ‑‑debug I found that those tags are not set by SA.


What version are you running? I just tried adding _LASTEXTERNALRDNS_ 
in
a header on 3.4.2 and it worked.






Re: LASTEXTERNALRDNS and LASTEXTERNALHELO not set in PerMsgStatus.pm ?

2019-11-28 Thread Tobi
Henrik,

thanks a lot, can confirm your fix works in my tests :-)

Cheers

tobi

Am 28.11.19 um 11:09 schrieb Henrik K:
>
> Fixed:
> http://svn.apache.org/viewvc?view=revision=1870552
>
> On Thu, Nov 28, 2019 at 11:29:19AM +0200, Henrik K wrote:
>>
>> Trunk has already many revamps and fixes for these codes, works there.  It
>> seems 3.4 askdns has trouble using those, investigating minimal fix..
>>
>>
>> On Thu, Nov 28, 2019 at 10:20:39AM +0100, Tobi  wrote:
>>> Henrik
>>>
>>> But my current testing clearly shows that without the explicit set_tag
>>> _LASTEXTERNALRDNS_ and _LASTEXTERNALHELO_ won't be set.
>>> I'm testing this using spamassassin in debug mode to scan a testmessage.
>>> As soon as I re-add the set_tag to PerMsgStatus.pm the tags are set and
>>> tests based on that tags are run. Removing the lines from pm and debug
>>> shows that the tests are **not** run anymore.
>>>
>>> So I somewhat doubt that the set_tag "code is redundant and should be
>>> removed" :-)
>>>
>>> Using: SA 3.4.2 on centos7 / perl 5.16.3
>>>
>>>
>>> Cheers
>>>
>>> tobi
>>>
>>> Am 28.11.19 um 08:36 schrieb Henrik K:

 There is nothing missing.  All the LASTEXT* tags are already dynamically
 returning functions.  If anything, that if ($lasthop) set_tag code is
 completely redundant and should be removed.


 BEGIN {
 LASTEXTERNALIP => sub {
   my $pms = shift;
   my $lasthop = $pms->{relays_external}->[0];
   $lasthop ? $lasthop->{ip} : '';
 },

 LASTEXTERNALRDNS => sub {
   my $pms = shift;
   my $lasthop = $pms->{relays_external}->[0];
   $lasthop ? $lasthop->{rdns} : '';
 },

 LASTEXTERNALHELO => sub {
   my $pms = shift;
   my $lasthop = $pms->{relays_external}->[0];
   $lasthop ? $lasthop->{helo} : '';
 },


 On Wed, Nov 27, 2019 at 10:18:20AM -0500, Kevin A. McGrail wrote:
> After a 10 minute or so study of the issue and comparing 3.4 and trunk,
> it definitely looks like the code is missing.  I am not 100% sure your
> fix works but it's better than it currently exists :-)
>
> Have you been using that change in production?
>
> Regards,
>
> KAM
>
> On 11/27/2019 6:59 AM, Tobi  wrote:
>> Hi,
>>
>> is there any specific reason why the two tags mentioned in subject are
>> not set in SA? It took me a while to find out why an askdns test was not
>> running. The test relies on _LASTEXTERNALRDNS_ but after running with
>> --debug I found that those tags are not set by SA. Although
>> _LASTEXTERNALIP_ is set. Also the man page states that the two tags
>> should exist.
>>
>> In PerMsgStatus.pm I saw that everything is prepared for the two tags
>> but they finally not set (around line 1721). So I changed to
>>
>>> if ($lasthop) {
>>>$self->set_tag('LASTEXTERNALIP',  $lasthop->{ip});
>>>$self->set_tag('LASTEXTERNALRDNS', $lasthop->{rdns});
>>>$self->set_tag('LASTEXTERNALHELO', $lasthop->{helo});
>>>  }
>>
>> and --debug shows that the tags are now set and the askdns test is run.
>>
>> So I wonder if the code has just been forgotten or if there are deeper
>> reasons to not set the tags?
>> If no deeper reasons exist it would be nice to have those two tags set
>> as default in PerMsgStatus.pm
>>
>>
>> Cheers
>>
>> --
>>
>> tobi
>
> --
> Kevin A. McGrail
> kmcgr...@apache.org
>
> Member, Apache Software Foundation
> Chair Emeritus Apache SpamAssassin Project
> https://www.linkedin.com/in/kmcgrail - 703.798.0171

>


Re: LASTEXTERNALRDNS and LASTEXTERNALHELO not set in PerMsgStatus.pm ?

2019-11-28 Thread Kevin A. McGrail
I never would have found that.  Nice job.

Any chance you can create a regression test for the issue ?

On Thu, Nov 28, 2019, 05:09 Henrik K  wrote:

>
> Fixed:
> http://svn.apache.org/viewvc?view=revision=1870552
>
> On Thu, Nov 28, 2019 at 11:29:19AM +0200, Henrik K wrote:
> >
> > Trunk has already many revamps and fixes for these codes, works there.
> It
> > seems 3.4 askdns has trouble using those, investigating minimal fix..
> >
> >
> > On Thu, Nov 28, 2019 at 10:20:39AM +0100, Tobi  wrote:
> > > Henrik
> > >
> > > But my current testing clearly shows that without the explicit set_tag
> > > _LASTEXTERNALRDNS_ and _LASTEXTERNALHELO_ won't be set.
> > > I'm testing this using spamassassin in debug mode to scan a
> testmessage.
> > > As soon as I re-add the set_tag to PerMsgStatus.pm the tags are set and
> > > tests based on that tags are run. Removing the lines from pm and debug
> > > shows that the tests are **not** run anymore.
> > >
> > > So I somewhat doubt that the set_tag "code is redundant and should be
> > > removed" :-)
> > >
> > > Using: SA 3.4.2 on centos7 / perl 5.16.3
> > >
> > >
> > > Cheers
> > >
> > > tobi
> > >
> > > Am 28.11.19 um 08:36 schrieb Henrik K:
> > > >
> > > > There is nothing missing.  All the LASTEXT* tags are already
> dynamically
> > > > returning functions.  If anything, that if ($lasthop) set_tag code is
> > > > completely redundant and should be removed.
> > > >
> > > >
> > > > BEGIN {
> > > > LASTEXTERNALIP => sub {
> > > >   my $pms = shift;
> > > >   my $lasthop = $pms->{relays_external}->[0];
> > > >   $lasthop ? $lasthop->{ip} : '';
> > > > },
> > > >
> > > > LASTEXTERNALRDNS => sub {
> > > >   my $pms = shift;
> > > >   my $lasthop = $pms->{relays_external}->[0];
> > > >   $lasthop ? $lasthop->{rdns} : '';
> > > > },
> > > >
> > > > LASTEXTERNALHELO => sub {
> > > >   my $pms = shift;
> > > >   my $lasthop = $pms->{relays_external}->[0];
> > > >   $lasthop ? $lasthop->{helo} : '';
> > > > },
> > > >
> > > >
> > > > On Wed, Nov 27, 2019 at 10:18:20AM -0500, Kevin A. McGrail wrote:
> > > >> After a 10 minute or so study of the issue and comparing 3.4 and
> trunk,
> > > >> it definitely looks like the code is missing.  I am not 100% sure
> your
> > > >> fix works but it's better than it currently exists :-)
> > > >>
> > > >> Have you been using that change in production?
> > > >>
> > > >> Regards,
> > > >>
> > > >> KAM
> > > >>
> > > >> On 11/27/2019 6:59 AM, Tobi  wrote:
> > > >>> Hi,
> > > >>>
> > > >>> is there any specific reason why the two tags mentioned in subject
> are
> > > >>> not set in SA? It took me a while to find out why an askdns test
> was not
> > > >>> running. The test relies on _LASTEXTERNALRDNS_ but after running
> with
> > > >>> --debug I found that those tags are not set by SA. Although
> > > >>> _LASTEXTERNALIP_ is set. Also the man page states that the two tags
> > > >>> should exist.
> > > >>>
> > > >>> In PerMsgStatus.pm I saw that everything is prepared for the two
> tags
> > > >>> but they finally not set (around line 1721). So I changed to
> > > >>>
> > >  if ($lasthop) {
> > > $self->set_tag('LASTEXTERNALIP',  $lasthop->{ip});
> > > $self->set_tag('LASTEXTERNALRDNS', $lasthop->{rdns});
> > > $self->set_tag('LASTEXTERNALHELO', $lasthop->{helo});
> > >   }
> > > >>>
> > > >>> and --debug shows that the tags are now set and the askdns test is
> run.
> > > >>>
> > > >>> So I wonder if the code has just been forgotten or if there are
> deeper
> > > >>> reasons to not set the tags?
> > > >>> If no deeper reasons exist it would be nice to have those two tags
> set
> > > >>> as default in PerMsgStatus.pm
> > > >>>
> > > >>>
> > > >>> Cheers
> > > >>>
> > > >>> --
> > > >>>
> > > >>> tobi
> > > >>
> > > >> --
> > > >> Kevin A. McGrail
> > > >> kmcgr...@apache.org
> > > >>
> > > >> Member, Apache Software Foundation
> > > >> Chair Emeritus Apache SpamAssassin Project
> > > >> https://www.linkedin.com/in/kmcgrail - 703.798.0171
> > > >
>


Re: LASTEXTERNALRDNS and LASTEXTERNALHELO not set in PerMsgStatus.pm ?

2019-11-28 Thread Henrik K


Fixed:
http://svn.apache.org/viewvc?view=revision=1870552

On Thu, Nov 28, 2019 at 11:29:19AM +0200, Henrik K wrote:
> 
> Trunk has already many revamps and fixes for these codes, works there.  It
> seems 3.4 askdns has trouble using those, investigating minimal fix..
> 
> 
> On Thu, Nov 28, 2019 at 10:20:39AM +0100, Tobi  wrote:
> > Henrik
> > 
> > But my current testing clearly shows that without the explicit set_tag
> > _LASTEXTERNALRDNS_ and _LASTEXTERNALHELO_ won't be set.
> > I'm testing this using spamassassin in debug mode to scan a testmessage.
> > As soon as I re-add the set_tag to PerMsgStatus.pm the tags are set and
> > tests based on that tags are run. Removing the lines from pm and debug
> > shows that the tests are **not** run anymore.
> > 
> > So I somewhat doubt that the set_tag "code is redundant and should be
> > removed" :-)
> > 
> > Using: SA 3.4.2 on centos7 / perl 5.16.3
> > 
> > 
> > Cheers
> > 
> > tobi
> > 
> > Am 28.11.19 um 08:36 schrieb Henrik K:
> > >
> > > There is nothing missing.  All the LASTEXT* tags are already dynamically
> > > returning functions.  If anything, that if ($lasthop) set_tag code is
> > > completely redundant and should be removed.
> > >
> > >
> > > BEGIN {
> > > LASTEXTERNALIP => sub {
> > >   my $pms = shift;
> > >   my $lasthop = $pms->{relays_external}->[0];
> > >   $lasthop ? $lasthop->{ip} : '';
> > > },
> > >
> > > LASTEXTERNALRDNS => sub {
> > >   my $pms = shift;
> > >   my $lasthop = $pms->{relays_external}->[0];
> > >   $lasthop ? $lasthop->{rdns} : '';
> > > },
> > >
> > > LASTEXTERNALHELO => sub {
> > >   my $pms = shift;
> > >   my $lasthop = $pms->{relays_external}->[0];
> > >   $lasthop ? $lasthop->{helo} : '';
> > > },
> > >
> > >
> > > On Wed, Nov 27, 2019 at 10:18:20AM -0500, Kevin A. McGrail wrote:
> > >> After a 10 minute or so study of the issue and comparing 3.4 and trunk,
> > >> it definitely looks like the code is missing.  I am not 100% sure your
> > >> fix works but it's better than it currently exists :-)
> > >>
> > >> Have you been using that change in production?
> > >>
> > >> Regards,
> > >>
> > >> KAM
> > >>
> > >> On 11/27/2019 6:59 AM, Tobi  wrote:
> > >>> Hi,
> > >>>
> > >>> is there any specific reason why the two tags mentioned in subject are
> > >>> not set in SA? It took me a while to find out why an askdns test was not
> > >>> running. The test relies on _LASTEXTERNALRDNS_ but after running with
> > >>> --debug I found that those tags are not set by SA. Although
> > >>> _LASTEXTERNALIP_ is set. Also the man page states that the two tags
> > >>> should exist.
> > >>>
> > >>> In PerMsgStatus.pm I saw that everything is prepared for the two tags
> > >>> but they finally not set (around line 1721). So I changed to
> > >>>
> >  if ($lasthop) {
> > $self->set_tag('LASTEXTERNALIP',  $lasthop->{ip});
> > $self->set_tag('LASTEXTERNALRDNS', $lasthop->{rdns});
> > $self->set_tag('LASTEXTERNALHELO', $lasthop->{helo});
> >   }
> > >>>
> > >>> and --debug shows that the tags are now set and the askdns test is run.
> > >>>
> > >>> So I wonder if the code has just been forgotten or if there are deeper
> > >>> reasons to not set the tags?
> > >>> If no deeper reasons exist it would be nice to have those two tags set
> > >>> as default in PerMsgStatus.pm
> > >>>
> > >>>
> > >>> Cheers
> > >>>
> > >>> --
> > >>>
> > >>> tobi
> > >>
> > >> --
> > >> Kevin A. McGrail
> > >> kmcgr...@apache.org
> > >>
> > >> Member, Apache Software Foundation
> > >> Chair Emeritus Apache SpamAssassin Project
> > >> https://www.linkedin.com/in/kmcgrail - 703.798.0171
> > >


Re: LASTEXTERNALRDNS and LASTEXTERNALHELO not set in PerMsgStatus.pm ?

2019-11-28 Thread Henrik K


Trunk has already many revamps and fixes for these codes, works there.  It
seems 3.4 askdns has trouble using those, investigating minimal fix..


On Thu, Nov 28, 2019 at 10:20:39AM +0100, Tobi  wrote:
> Henrik
> 
> But my current testing clearly shows that without the explicit set_tag
> _LASTEXTERNALRDNS_ and _LASTEXTERNALHELO_ won't be set.
> I'm testing this using spamassassin in debug mode to scan a testmessage.
> As soon as I re-add the set_tag to PerMsgStatus.pm the tags are set and
> tests based on that tags are run. Removing the lines from pm and debug
> shows that the tests are **not** run anymore.
> 
> So I somewhat doubt that the set_tag "code is redundant and should be
> removed" :-)
> 
> Using: SA 3.4.2 on centos7 / perl 5.16.3
> 
> 
> Cheers
> 
> tobi
> 
> Am 28.11.19 um 08:36 schrieb Henrik K:
> >
> > There is nothing missing.  All the LASTEXT* tags are already dynamically
> > returning functions.  If anything, that if ($lasthop) set_tag code is
> > completely redundant and should be removed.
> >
> >
> > BEGIN {
> > LASTEXTERNALIP => sub {
> >   my $pms = shift;
> >   my $lasthop = $pms->{relays_external}->[0];
> >   $lasthop ? $lasthop->{ip} : '';
> > },
> >
> > LASTEXTERNALRDNS => sub {
> >   my $pms = shift;
> >   my $lasthop = $pms->{relays_external}->[0];
> >   $lasthop ? $lasthop->{rdns} : '';
> > },
> >
> > LASTEXTERNALHELO => sub {
> >   my $pms = shift;
> >   my $lasthop = $pms->{relays_external}->[0];
> >   $lasthop ? $lasthop->{helo} : '';
> > },
> >
> >
> > On Wed, Nov 27, 2019 at 10:18:20AM -0500, Kevin A. McGrail wrote:
> >> After a 10 minute or so study of the issue and comparing 3.4 and trunk,
> >> it definitely looks like the code is missing.  I am not 100% sure your
> >> fix works but it's better than it currently exists :-)
> >>
> >> Have you been using that change in production?
> >>
> >> Regards,
> >>
> >> KAM
> >>
> >> On 11/27/2019 6:59 AM, Tobi  wrote:
> >>> Hi,
> >>>
> >>> is there any specific reason why the two tags mentioned in subject are
> >>> not set in SA? It took me a while to find out why an askdns test was not
> >>> running. The test relies on _LASTEXTERNALRDNS_ but after running with
> >>> --debug I found that those tags are not set by SA. Although
> >>> _LASTEXTERNALIP_ is set. Also the man page states that the two tags
> >>> should exist.
> >>>
> >>> In PerMsgStatus.pm I saw that everything is prepared for the two tags
> >>> but they finally not set (around line 1721). So I changed to
> >>>
>  if ($lasthop) {
> $self->set_tag('LASTEXTERNALIP',  $lasthop->{ip});
> $self->set_tag('LASTEXTERNALRDNS', $lasthop->{rdns});
> $self->set_tag('LASTEXTERNALHELO', $lasthop->{helo});
>   }
> >>>
> >>> and --debug shows that the tags are now set and the askdns test is run.
> >>>
> >>> So I wonder if the code has just been forgotten or if there are deeper
> >>> reasons to not set the tags?
> >>> If no deeper reasons exist it would be nice to have those two tags set
> >>> as default in PerMsgStatus.pm
> >>>
> >>>
> >>> Cheers
> >>>
> >>> --
> >>>
> >>> tobi
> >>
> >> --
> >> Kevin A. McGrail
> >> kmcgr...@apache.org
> >>
> >> Member, Apache Software Foundation
> >> Chair Emeritus Apache SpamAssassin Project
> >> https://www.linkedin.com/in/kmcgrail - 703.798.0171
> >


Re: LASTEXTERNALRDNS and LASTEXTERNALHELO not set in PerMsgStatus.pm ?

2019-11-28 Thread Tobi
Henrik

But my current testing clearly shows that without the explicit set_tag
_LASTEXTERNALRDNS_ and _LASTEXTERNALHELO_ won't be set.
I'm testing this using spamassassin in debug mode to scan a testmessage.
As soon as I re-add the set_tag to PerMsgStatus.pm the tags are set and
tests based on that tags are run. Removing the lines from pm and debug
shows that the tests are **not** run anymore.

So I somewhat doubt that the set_tag "code is redundant and should be
removed" :-)

Using: SA 3.4.2 on centos7 / perl 5.16.3


Cheers

tobi

Am 28.11.19 um 08:36 schrieb Henrik K:
>
> There is nothing missing.  All the LASTEXT* tags are already dynamically
> returning functions.  If anything, that if ($lasthop) set_tag code is
> completely redundant and should be removed.
>
>
> BEGIN {
> LASTEXTERNALIP => sub {
>   my $pms = shift;
>   my $lasthop = $pms->{relays_external}->[0];
>   $lasthop ? $lasthop->{ip} : '';
> },
>
> LASTEXTERNALRDNS => sub {
>   my $pms = shift;
>   my $lasthop = $pms->{relays_external}->[0];
>   $lasthop ? $lasthop->{rdns} : '';
> },
>
> LASTEXTERNALHELO => sub {
>   my $pms = shift;
>   my $lasthop = $pms->{relays_external}->[0];
>   $lasthop ? $lasthop->{helo} : '';
> },
>
>
> On Wed, Nov 27, 2019 at 10:18:20AM -0500, Kevin A. McGrail wrote:
>> After a 10 minute or so study of the issue and comparing 3.4 and trunk,
>> it definitely looks like the code is missing.  I am not 100% sure your
>> fix works but it's better than it currently exists :-)
>>
>> Have you been using that change in production?
>>
>> Regards,
>>
>> KAM
>>
>> On 11/27/2019 6:59 AM, Tobi  wrote:
>>> Hi,
>>>
>>> is there any specific reason why the two tags mentioned in subject are
>>> not set in SA? It took me a while to find out why an askdns test was not
>>> running. The test relies on _LASTEXTERNALRDNS_ but after running with
>>> --debug I found that those tags are not set by SA. Although
>>> _LASTEXTERNALIP_ is set. Also the man page states that the two tags
>>> should exist.
>>>
>>> In PerMsgStatus.pm I saw that everything is prepared for the two tags
>>> but they finally not set (around line 1721). So I changed to
>>>
 if ($lasthop) {
$self->set_tag('LASTEXTERNALIP',  $lasthop->{ip});
$self->set_tag('LASTEXTERNALRDNS', $lasthop->{rdns});
$self->set_tag('LASTEXTERNALHELO', $lasthop->{helo});
  }
>>>
>>> and --debug shows that the tags are now set and the askdns test is run.
>>>
>>> So I wonder if the code has just been forgotten or if there are deeper
>>> reasons to not set the tags?
>>> If no deeper reasons exist it would be nice to have those two tags set
>>> as default in PerMsgStatus.pm
>>>
>>>
>>> Cheers
>>>
>>> --
>>>
>>> tobi
>>
>> --
>> Kevin A. McGrail
>> kmcgr...@apache.org
>>
>> Member, Apache Software Foundation
>> Chair Emeritus Apache SpamAssassin Project
>> https://www.linkedin.com/in/kmcgrail - 703.798.0171
>