from:"\-"

Re: [Mimedefang] Is this list still alive?

2021-04-01 Thread Kenneth Porter

--On Thursday, April 01, 2021 7:16 PM -0700 Kenneth Porter 
 wrote:



[retry assuming greylisting]


BTW, Comcast never automatically retries on graylisting. I have to manually 
re-send from my Sent folder. (My sendmail-based home server is set to 
smarthost to Comcast's server.) It's not just this listserv. I also have 
the problem with the CentOS list.



___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Is this list still alive?

2021-04-01 Thread Joseph Brennan

Test test test



On Thu, Apr 1, 2021 at 11:19 AM Dianne Skoll via MIMEDefang <
mimedefang@lists.roaringpenguin.com> wrote:

> Original Message Attached ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍
> ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍
> ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍
> ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍
> ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍
> ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍ ‍
> ‍ ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
> ZjQcmQRYFpfptBannerEnd
>
>
> Original Message Attached
>
>
>
> -- Forwarded message --
> From: Dianne Skoll via MIMEDefang 
> To: 
> Cc:
> Bcc:
> Date: Thu, 1 Apr 2021 11:19:18 -0400
> Subject: Re: [Mimedefang] Is this list still alive?
> On Thu, 01 Apr 2021 11:13:48 -0400
> "Bill Cole"  wrote:
>
> > There was an issue opened a month ago at
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_The-2DMcGrail-2DFoundation_MIMEDefang_issues_25=DwIGaQ=009klHSCxuh5AI1vNQzSO0KGjl4nbi2Q0M1QLJX9BeE=ZxkJnEk7djUruVXXQ1YhtyPk0yO9wbHUi6rXFRThw2k=hSBrrF3152dlzlKD90IDGIG7j7UH_IHYDOUMwFA7APU=vGvWh7Tm6-hUlYE12mtYTobwRA-TUWwxTBjeVJiXKpw=
> > stating that the list was down.
>
> The list is still being run by AppRiver (Zix, now, I guess.)
>
> > I attempted to post a test on March 3 which never showed up on the
> > list.
>
> Huh, weird.  It's unfortunately out of my hands now. :(
>
> Regards,
>
> Dianne.
>
>
> ___
> NOTE: If there is a disclaimer or other legal boilerplate in the above
> message, it is NULL AND VOID.  You may ignore it.
>
> Visit
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.mimedefang.org=DwIGaQ=009klHSCxuh5AI1vNQzSO0KGjl4nbi2Q0M1QLJX9BeE=ZxkJnEk7djUruVXXQ1YhtyPk0yO9wbHUi6rXFRThw2k=hSBrrF3152dlzlKD90IDGIG7j7UH_IHYDOUMwFA7APU=ZhpeHy_uqANQ-sctXCt_lWcbmOR1xpn-ojB4OaQdrgs=
> and
> https://urldefense.proofpoint.com/v2/url?u=http-3A__www.roaringpenguin.com=DwIGaQ=009klHSCxuh5AI1vNQzSO0KGjl4nbi2Q0M1QLJX9BeE=ZxkJnEk7djUruVXXQ1YhtyPk0yO9wbHUi6rXFRThw2k=hSBrrF3152dlzlKD90IDGIG7j7UH_IHYDOUMwFA7APU=x7LMhhlEIYnWfOYC2v5qMDEnr_gd6oXjKg6u0SSlox4=
> MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
>
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.roaringpenguin.com_mailman_listinfo_mimedefang=DwIGaQ=009klHSCxuh5AI1vNQzSO0KGjl4nbi2Q0M1QLJX9BeE=ZxkJnEk7djUruVXXQ1YhtyPk0yO9wbHUi6rXFRThw2k=hSBrrF3152dlzlKD90IDGIG7j7UH_IHYDOUMwFA7APU=EST9t_iXdcoUg5k-GRpsthvPiiuFETDZP9vBrl1_9i8=
>


-- 
Joseph Brennan
Lead, Email and Systems Applications
Columbia University Information Technology
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Is this list still alive?

2021-04-01 Thread Mark Costlow

I got this message, and I've seen your replies to a couple of other
list members, but I didn't see any of the messages direct from those
people.  Maybe they replied to you off-list, but if they did reply
to the list then you might be the only one whose messages are 
getting through.

If these words come back to me as a list posting, I will eat them.

Thanks,

Mark


On Thu, Apr 01, 2021 at 10:33:24AM -0400, Dianne Skoll via MIMEDefang wrote:
> I haven't seen traffic since December 2020... just pinging the list...
> 
> Regards,
> 
> Dianne.
> ___
> NOTE: If there is a disclaimer or other legal boilerplate in the above
> message, it is NULL AND VOID.  You may ignore it.
> 
> Visit http://www.mimedefang.org and http://www.roaringpenguin.com
> MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
> http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
> 
> 

-- 
Mark Costlow| Southwest Cyberport | Fax:   +1-505-232-7975
che...@swcp.com | Web:   www.swcp.com | Voice: +1-505-232-7992
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Is this list still alive?

2021-04-01 Thread wayne

Funny I had the same question this morning. 
My question also was what is going to happen to the data feeds we are getting. 
Someone going to migrate them after Appriver decides to shut them down? What is 
the future and what are others doing? I don't see a good option for small ISPs 
that don't have a budget for spam filtering at scale.

Wayne

-Original Message-
From: MIMEDefang  On Behalf Of 
Dianne Skoll via MIMEDefang
Sent: Thursday, April 1, 2021 9:33 AM
To: mimedefang@lists.roaringpenguin.com
Subject: [Mimedefang] Is this list still alive?

I haven't seen traffic since December 2020... just pinging the list...

Regards,

Dianne.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above message, 
it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang 
mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Is this list still alive?

2021-04-01 Thread Bill Cole


On 1 Apr 2021, at 10:33, Dianne Skoll via MIMEDefang wrote:


I haven't seen traffic since December 2020... just pinging the list...


I got your message.

There was an issue opened a month ago at 
https://github.com/The-McGrail-Foundation/MIMEDefang/issues/25 stating 
that the list was down.


I attempted to post a test on March 3 which never showed up on the list.

--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Is this list still alive?

2021-04-01 Thread Brown, William

"We are here! We are Here!"


--
William Brown
WNYRIC/Erie 1 BOCES
716-821-7285

SharePoint, Eforms, Email, Spam Filtering Please reach out to messag...@e1b.org
Immediate Needs Call our Service Desk at 716-821-7171

-Original Message-
From: MIMEDefang  On Behalf Of 
Dianne Skoll via MIMEDefang
Sent: Thursday, April 1, 2021 10:33 AM
To: mimedefang@lists.roaringpenguin.com
Subject: [Mimedefang] Is this list still alive?

 This email originated from outside of the organization. Use caution 
when replying, opening attachment(s), and/or clicking on URL's. 


I haven't seen traffic since December 2020... just pinging the list...

Regards,

Dianne.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above message, 
it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com MIMEDefang 
mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang
Confidentiality Notice: This electronic message and any attachments may contain 
confidential or privileged information, and is intended only for the individual 
or entity identified above as the addressee. If you are not the addressee (or 
the employee or agent responsible to deliver it to the addressee), or if this 
message has been addressed to you in error, you are hereby notified that you 
may not copy, forward, disclose or use any part of this message or any 
attachments. Please notify the sender immediately by return e-mail or telephone 
and delete this message from your system.

___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Is this list still alive?

2021-04-01 Thread Kenneth Porter


[retry assuming greylisting]

I sent a message on Feb 26 that got eaten. See here:



___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Is this list still alive?

2021-04-01 Thread Ben


On 4/1/21 10:19 AM, Dianne Skoll via MIMEDefang wrote:

On Thu, 01 Apr 2021 11:13:48 -0400
"Bill Cole"  wrote:


There was an issue opened a month ago at
https://github.com/The-McGrail-Foundation/MIMEDefang/issues/25
stating that the list was down.

The list is still being run by AppRiver (Zix, now, I guess.)


I attempted to post a test on March 3 which never showed up on the
list.

Huh, weird.  It's unfortunately out of my hands now. :(

Regards,


I got this email (both of them) from today.

but I also noticed all was quiet.

  -Ben


--
Ben Kamen - O.D.T., S.P.
--
eMail: ben AT benjammin DOT net   http://www.benjammin.net
Fortune says:
Vulcans never bluff.
-- Spock, "The Doomsday Machine", stardate 4202.1
-  -
NOTICE: All legal disclaimers sent to benjammin.net/benkamen.net
or any of it's affiliated domains are rendered null and void on
receipt of communications and will be handled/considered as such.

___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] [External] Re: Is this list still alive?

2021-04-01 Thread Kevin A. McGrail

> I'm not sure what data feeds you're referring to.  If you are 
referring to
> the Roaring Penguin IP blocklists and/or the CanIt Bayes data feeds, 
those

> are controlled by AppRiver and if AppRiver decides to shut them down...
> there's not much anyone can do because they are fed by CanIt 
installations.


Thanks for the reminder on this.  The pandemic just keeps ruining plans 
but getting the mailing list onto a platform that doesn't need kicking 
is on our list.


And the RPBL is also something where I don't believe there is a risk of 
it being shutdown without it being transferred to the McGrail Foundation 
first.  We've talked about it and need to get some of the other 
waterfowl aligned.  We do have a repo and patches are going into it and 
a release is imminent too.


Regards,

KAM

___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Is this list still alive?

2021-04-01 Thread Dianne Skoll via MIMEDefang

On Thu, 1 Apr 2021 09:43:21 -0500
 wrote:

> My question also was what is going to happen to the data feeds we are
> getting.

I'm not sure what data feeds you're referring to.  If you are referring to
the Roaring Penguin IP blocklists and/or the CanIt Bayes data feeds, those
are controlled by AppRiver and if AppRiver decides to shut them down...
there's not much anyone can do because they are fed by CanIt installations.

> I don't see a good option for small ISPs that don't have a budget
> for spam filtering at scale.

It is a problem indeed. :(  Many ISPs are in the same boat.

Regards,

Dianne.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Is this list still alive?

2021-04-01 Thread Dianne Skoll via MIMEDefang

On Thu, 01 Apr 2021 11:13:48 -0400
"Bill Cole"  wrote:

> There was an issue opened a month ago at
> https://github.com/The-McGrail-Foundation/MIMEDefang/issues/25
> stating that the list was down.

The list is still being run by AppRiver (Zix, now, I guess.)

> I attempted to post a test on March 3 which never showed up on the
> list.

Huh, weird.  It's unfortunately out of my hands now. :(

Regards,

Dianne.


___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

[Mimedefang] Is this list still alive?

2021-04-01 Thread Dianne Skoll via MIMEDefang

I haven't seen traffic since December 2020... just pinging the list...

Regards,

Dianne.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Mimedefang to strip HTML via sieve

2020-12-21 Thread Bill Cole


On 18 Dec 2020, at 8:21, @lbutlr wrote:

Does anyone have a sieve setup that removes html enclosures (or 
converts them to plain text, even better)? I want to do this for 
specific mails, not globally for the server, so I think sieve is the 
way to go here rather than a milter.


So, why ask on this list?

Obviously, MD can do this. remove_redundant_html_parts() (documented in 
the mimedefang-filter man page) exists. MD can select messages to act 
upon by any criteria you like.


If there is a way to edit the HTML of specific messages, that would 
also be useful. (For example, changing text colors to black instead of 
carrying shades of light grey).


Sieve seems like an entirely wrong place to do that sort of surgery on 
messages. It is entirely feasible to programmatically edit the HTML of a 
message in MD, which is designed for doing radical message surgery. MD 
passes each of the filter* subroutines a Mime::Entity object containing 
the relevant MIME object (whole message or sub-part)  and metadata. You 
can modify it at will.




--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Mimedefang to strip HTML via sieve

2020-12-21 Thread Kris Deugau


@lbutlr wrote:

Does anyone have a sieve setup that removes html enclosures (or converts them 
to plain text, even better)? I want to do this for specific mails, not globally 
for the server, so I think sieve is the way to go here rather than a milter.


To my understanding Sieve deliberately restricts calling arbitrary 
external programs.  I think there's a loophole if you build a whole 
Sieve plugin to wrap the program, but digging in to that is still on my 
"need to read more" list.


-kgd
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

[Mimedefang] Mimedefang to strip HTML via sieve

2020-12-18 Thread @lbutlr

Does anyone have a sieve setup that removes html enclosures (or converts them 
to plain text, even better)? I want to do this for specific mails, not globally 
for the server, so I think sieve is the way to go here rather than a milter.

If there is a way to edit the HTML of specific messages, that would also be 
useful. (For example, changing text colors to black instead of carrying shades 
of light grey).

Postfix/dovecot/mimedfang all current versions as of right now, on FreeBSD 
12.1-RELEASE

-- 
"I am" is reportedly the shortest sentence in the English language.
Could it be that "I do" is the longest sentence?


___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] [External] Re: Help with header checking

2020-12-01 Thread Kevin A. McGrail


On 12/1/2020 1:10 PM, Joseph Brennan wrote:

We used this to create an array of header information:

if (open(IN, '<', './HEADERS')) {
 while() {
 chomp;
 if (/^(\S+):\s*(.*)/) {
 my $label = $1;
 my $data  = $2;
 $label = lc($label);
 $Header{$label} = $data;
 if ($label eq 'received') {
 push(@Received,$data);
 }
 }
 close(IN);
 }


So following this, if for example we want to test the From: header, we
check $Header{'from'}
Received: was handled specially. It could end up an empty list. I
don't think we checked for that!

Hmm... If HEADERS was not there (??) it fails silently and
$Header{'anything'} is empty. The rest of filter still works,
but maybe we should have logged that case.


Does using MIME::Tools and $entity->head->get('Received'), for example, 
work?  Not sure how it parses through the various received headers.


For example, I convert Date's to unix epoch time format with something 
like this and the Date::Parse module:


$date_unixtime = Date::Parse::str2time($entity->head->get('Date'));

Regards,

KAM

___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-12-01 Thread Dianne Skoll via MIMEDefang

On Tue, 1 Dec 2020 13:10:43 -0500
Joseph Brennan  wrote:

> We used this to create an array of header information:

> if (open(IN, '<', './HEADERS')) {
> while() {
> chomp;
> if (/^(\S+):\s*(.*)/) {
> my $label = $1;
> my $data  = $2;
> $label = lc($label);
> $Header{$label} = $data;
> if ($label eq 'received') {
> push(@Received,$data);
> }
> }
> close(IN);
> }

Hmm.  Received: isn't the only header that can permit multiple instances.
You may lose some information here.

Why would you not just use the MIME::Head object that you can
get from $entity->head?  It has all the headers pre-parsed and has
decent accessors.

(I assume this is all academic (heh...) as Columbia looks to be
using Proofpoint now.)

Regards,

Dianne.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-12-01 Thread Joseph Brennan

We used this to create an array of header information:

if (open(IN, '<', './HEADERS')) {
while() {
chomp;
if (/^(\S+):\s*(.*)/) {
my $label = $1;
my $data  = $2;
$label = lc($label);
$Header{$label} = $data;
if ($label eq 'received') {
push(@Received,$data);
}
}
close(IN);
}


So following this, if for example we want to test the From: header, we
check $Header{'from'}
Received: was handled specially. It could end up an empty list. I
don't think we checked for that!

Hmm... If HEADERS was not there (??) it fails silently and
$Header{'anything'} is empty. The rest of filter still works,
but maybe we should have logged that case.



-- 
Joseph Brennan
Lead, Email and Systems Applications
Columbia University Information Technology
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-11-26 Thread Dianne Skoll via MIMEDefang

On Thu, 26 Nov 2020 12:47:56 -0500
"Bill Cole"  wrote:

[snip]
> That Received header is added by Sendmail *AFTER* all milters have
> done their end-of-data work. It is not present when MD sees the
> message.

So as per Bill Cole's analysis, it seems there were indeed no
Received: headers on the mail as seen by MIMEDefang.

As for why SpamAssassin is not running, Bill's guess (size limit) is a
pretty good one, but we can't know for sure without seeing your
filter code.

Regards,

Dianne.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-11-26 Thread Andrea Venturoli


On 11/26/20 6:47 PM, Bill Cole wrote:

That Received header is added by Cyrus during delivery, so obviously 
it's not present when MD sees the message.


Of course...




That Received header is added by Sendmail *AFTER* all milters have done 
their end-of-data work. It is not present when MD sees the message.


Oh!
That's the part I was missing and I think this answers my first question.
Thank you very much!




A common reason for SA not being called by MD is a size limit. The 
example mimedefang-filter script includes a limit that made more sense 
15 years ago than it does today, when spammers routinely send huge 
garbage.


I had already rised this, so I don't think it's the reason.
I'll keep an eye on this anyway, as I had forgot about this limit and ut 
might have been hit in some other cases.

Thanks a lot again.



 bye
av.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-11-26 Thread Bill Cole


On 26 Nov 2020, at 11:55, Andrea Venturoli wrote:
[...]

Anyway... you need to capture the message (or at least the headers)
so we can analyze what's going on.


Here's a sample:

Return-Path: 
Received: from soth.netfence.it ([unix socket])
 by mailserver.netfence.it (Cyrus 3.0.14) with LMTPA;
 Wed, 25 Nov 2020 03:45:44 +0100


That Received header is added by Cyrus during delivery, so obviously 
it's not present when MD sees the message.


X-Cyrus-Session-Id: 
mailserver.netfence.it-557-1606272344-1-3657946293514545252

X-Sieve: CMU Sieve 3.0
Received: from poeconomico.casa (vds74451.mgn-host.ru 
[89.191.230.250] (may be forged))

by soth.netfence.it (8.16.1/8.16.1) with ESMTPS id 0AP2jef2000844
	(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 
verify=NO)

for ; Wed, 25 Nov 2020 03:45:43 +0100 (CET)
(envelope-from pag...@poeconomico.casa)


That Received header is added by Sendmail *AFTER* all milters have done 
their end-of-data work. It is not present when MD sees the message.




Authentication-Results: soth.netfence.it;
	dkim=pass (2048-bit key) header.d=poeconomico.casa 
header.i=pag...@poeconomico.casa header.b=cGnTmyJh
X-Authentication-Warning: soth.netfence.it: Host vds74451.mgn-host.ru 
[89.191.230.250] (may be forged) claimed to be poeconomico.casa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=key1; 
d=poeconomico.casa;

 h=Message-ID:From:To:Subject:Date:MIME-Version:Content-Type;
 i=pag...@poeconomico.casa;
 bh=xbJLlOE1CWUnav77hJisuzISPwtefQrfatVm8E+8Sow=;
 b=cGnTmyJh1B9VDyiBCFcRI2pVOQqJ+fw65kJL6vCU15L3GTJXXNxpgd0HHyeFDlXYj/1o+HHX3mkt
   m1YEVxiN/83OcZzQGMRhFLk6rVtoTMARuN/uO1fYAaxcCLqpsM5YLyU6NPIwsYsCkZx0pz4vCtMo
   Scl4h3E9zx52tto+NClcudYfpP+NW8QkC1J3Wu3ZkwGcBE2HkxsX7TOkR0OAk8ottDAu3OThcvCL
   SCuDoaaZxBxok24KZUJ663tjzPFMPih+Lna0Gx7bmYi//3mvI+7vkwQNMztima+51SQiI+UI77Ro
   H/M9ke7T0CNZfImI7dd+x4KluyNSe4dyH83DKQ==
Message-ID: <2a3970dc95e4cec62a2f9935fd496366a1e...@poeconomico.casa>
From: accountant 
To: xxx...@netfence.it
Subject: Ho trovata la tua email attraverso il servizio di 
appuntamenti "meetic.it".

Date: Wed, 25 Nov 2020 02:44:07 +0100
MIME-Version: 1.0
Content-Type: multipart/related; 
boundary="2bd19889d80c22e13d3871e175a182d1cd7a"

X-Scanned-By: MIMEDefang 2.83


As you can see:
_ there's no X-Spam-Score header: either SpamAssassin didn't detect 
this or it wasn't even launched;
_ if it ran, it should have added 100 points alone since "*.casa" is 
blacklisted; that alone should have been enough;


A common reason for SA not being called by MD is a size limit. The 
example mimedefang-filter script includes a limit that made more sense 
15 years ago than it does today, when spammers routinely send huge 
garbage. Somewhere in the filter() or filter_end() subroutine in your 
mimedefang-filter there's a conditional code structure that governs 
whether SA is called, and that is where to look for the failure. If your 
code has retained the check from the distribution example, it will look 
something like this:


# Spam checks if SpamAssassin is installed
if ($Features{"SpamAssassin"}) {
if (-s "./INPUTMSG" < 100*1024) {
# Only scan messages smaller than 100kB.  Larger messages
# are extremely unlikely to be spam, and SpamAssassin is
# dreadfully slow on very large messages.
my($hits, $req, $names, $report) = spam_assassin_check();


Note also that the comment is a bit outdated. We've done a LOT of 
improvement in SA's performance with large messages, mostly by 
eliminating the use of ".*" in rules except when absolutely necessary.


_ also 89.191.230.250 range is in my personal DNSBL and again this 
alone should have been enough.


If that DNSBL is being used directly from Sendmail, that's a different 
failure. If it's being used via SpamAssassin, it's also due to not 
calling SA from MD.




N.B.
Running spamassassin on the command line effectively gives the score I 
expect, so I just *think* it's not called. What in the end I'm trying 
to see is why.
Calling spamassassin and the code I posted are two different things: 
but I see the latter is also failing and I thought that might give 
some hint.
If there's a better way to see why spamassassin fails it would 
probably enough (although curiosity... :).


Look for the call to "spam_assassin_check()" in mimedefang-filter and 
work backwards.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] [Off-Topic] Happy Thanksgiving and Announcing the Apache SpamAssassin Channel for the KAM Rule Set

2020-11-26 Thread Kenneth Porter


On CentOS and other RH-based distros, does MD get restarted when SA
channels get updated?



___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-11-26 Thread Giovanni Bechis


On 11/25/20 11:45 AM, Andrea Venturoli wrote:
[...]
> Are Mail::Header and Mail::Field still current? Or are they obsolete? If so, 
> what should be used as a replacement?
> 
> For these messages, I see in the logs:
>> mimedefang.pl[23042]: 0AP2jef2000844: head = %Mail::Header=HASH(0x1fe80ae0)
>> mimedefang.pl[23042]: 0AP2jef2000844: RecHeads = %
> 
> So it seems Mail::Header parsed the message, but either found no Received 
> header (and they are there!) or it's Mail::Field that fails.
> How would I log more data to know what's going on?
> 
> I tried adding:
>>     foreach $key (keys %$head)
>>     {
>>     md_syslog('warning',"$key is $head{$key}");
>>     }
> Alas, only the keys are printed, not the values.
> 
> Any hint?
> 
this very simple parser seems to correctly parse the sample headers you posted, 
maybe there is something wrong inside the "foreach my $rechead (@recheads)"
portion of the code.

 Giovanni


received_hdrs.pl
Description: Perl program
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-11-26 Thread Andrea Venturoli

On 11/26/20 3:47 PM, Dianne Skoll via MIMEDefang wrote:

You shouldn't really print "head = %$head" because you don't know the
implementation details of Mail::Header.  Treat it as an opaque object.

So how do I know why/how it failed?

I suggest calling action_quarantine_entire_message() if you get no
received headers so you can preserve a copy of the original message for
analysis.

Sorry, I think I wasn't clear: I have such messages!
They are spam getting through, so I've been able to examine them.
What I don't understand is why they weren't blocked, when, IMNSHO, there 
are more than a reason for them to be.

It should be $head->{$key} and is unlikely to be useful; you're
printing instance variables on the Mail::Header object, which most
likely do not correspond to actual headers.

> If you want to log @recheads, I would do it like this:
>
> md_syslog('warning', 'recheads = ' . join(', ', @recheads));

Thanks.
What I'm now getting is something like:

mimedefang.pl[86009]: 0AQGZBmM087137: head = %Mail::Header=HASH(0x1617cac8)
mimedefang.pl[86009]: 0AQGZBmM087137: mail_hdr_modify is 0
mimedefang.pl[86009]: 0AQGZBmM087137: mail_hdr_foldlen is 79
mimedefang.pl[86009]: 0AQGZBmM087137: mail_hdr_lengths is HASH(0x19404798)
mimedefang.pl[86009]: 0AQGZBmM087137: mail_hdr_list is ARRAY(0x15e19060)
mimedefang.pl[86009]: 0AQGZBmM087137: mail_hdr_hash is HASH(0x178b94b0)
mimedefang.pl[86009]: 0AQGZBmM087137: mail_hdr_mail_from is KEEP
mimedefang.pl[86009]: 0AQGZBmM087137: recheads = 
Mail::Field::Received=HASH(0x180a4630)

It's better than nothing.
Now I'm waiting for such spam to arrive, so I can see in what way it's 
different.
I think I first need to understand if it's Mail::Header (earlier) or 
Mail::Field (later) that is failing to parse.

Anyway... you need to capture the message (or at least the headers)
so we can analyze what's going on.

Here's a sample:

Return-Path: 
Received: from soth.netfence.it ([unix socket])
 by mailserver.netfence.it (Cyrus 3.0.14) with LMTPA;
 Wed, 25 Nov 2020 03:45:44 +0100
X-Cyrus-Session-Id: mailserver.netfence.it-557-1606272344-1-3657946293514545252
X-Sieve: CMU Sieve 3.0
Received: from poeconomico.casa (vds74451.mgn-host.ru [89.191.230.250] (may be 
forged))
by soth.netfence.it (8.16.1/8.16.1) with ESMTPS id 0AP2jef2000844
(version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO)
for ; Wed, 25 Nov 2020 03:45:43 +0100 (CET)
(envelope-from pag...@poeconomico.casa)
Authentication-Results: soth.netfence.it;
dkim=pass (2048-bit key) header.d=poeconomico.casa 
header.i=pag...@poeconomico.casa header.b=cGnTmyJh
X-Authentication-Warning: soth.netfence.it: Host vds74451.mgn-host.ru 
[89.191.230.250] (may be forged) claimed to be poeconomico.casa
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; s=key1; 
d=poeconomico.casa;
 h=Message-ID:From:To:Subject:Date:MIME-Version:Content-Type;
 i=pag...@poeconomico.casa;
 bh=xbJLlOE1CWUnav77hJisuzISPwtefQrfatVm8E+8Sow=;
 b=cGnTmyJh1B9VDyiBCFcRI2pVOQqJ+fw65kJL6vCU15L3GTJXXNxpgd0HHyeFDlXYj/1o+HHX3mkt
   m1YEVxiN/83OcZzQGMRhFLk6rVtoTMARuN/uO1fYAaxcCLqpsM5YLyU6NPIwsYsCkZx0pz4vCtMo
   Scl4h3E9zx52tto+NClcudYfpP+NW8QkC1J3Wu3ZkwGcBE2HkxsX7TOkR0OAk8ottDAu3OThcvCL
   SCuDoaaZxBxok24KZUJ663tjzPFMPih+Lna0Gx7bmYi//3mvI+7vkwQNMztima+51SQiI+UI77Ro
   H/M9ke7T0CNZfImI7dd+x4KluyNSe4dyH83DKQ==
Message-ID: <2a3970dc95e4cec62a2f9935fd496366a1e...@poeconomico.casa>
From: accountant 
To: xxx...@netfence.it
Subject: Ho trovata la tua email attraverso il servizio di appuntamenti 
"meetic.it".
Date: Wed, 25 Nov 2020 02:44:07 +0100
MIME-Version: 1.0
Content-Type: multipart/related; boundary="2bd19889d80c22e13d3871e175a182d1cd7a"
X-Scanned-By: MIMEDefang 2.83

As you can see:
_ there's no X-Spam-Score header: either SpamAssassin didn't detect this 
or it wasn't even launched;
_ if it ran, it should have added 100 points alone since "*.casa" is 
blacklisted; that alone should have been enough;
_ also 89.191.230.250 range is in my personal DNSBL and again this alone 
should have been enough.

N.B.
Running spamassassin on the command line effectively gives the score I 
expect, so I just *think* it's not called. What in the end I'm trying to 
see is why.
Calling spamassassin and the code I posted are two different things: but 
I see the latter is also failing and I thought that might give some hint.
If there's a better way to see why spamassassin fails it would probably 
enough (although curiosity... :).

 bye & Thanks
av.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

[Mimedefang] [Off-Topic] Happy Thanksgiving and Announcing the Apache SpamAssassin Channel for the KAM Rule Set

2020-11-26 Thread Kevin A. McGrail


Morning all,
I wanted to share the news from 
https://mcgrail.com/newsmanager/news_article.cgi?template=news.template_id=11 
with you all.  We'll also have a mailing list up soon too.
Thanks to the sponsors and to Georgia Smith and Karsten Bräckelmann who 
worked hard on setting up the infrastructure for this.


Happy Thanksgiving,
KAM


 Announcing the Apache SpamAssassin Channel for the KAM Rule Set

Nov 26, 2020
Happy Thanksgiving,

The McGrail Foundation is proud to announce the immediate availability 
of the channel for the KAM rule set.


The rule set has been free and available to improve Apache SpamAssassin 
installations for going on 17 years now. It includes rules for common 
spam as well as contributed rules plus tweaks to help make things faster 
and more efficient with the stock rules without lowering the efficacy.


The KAM rule set is authored by Kevin A. McGrail with contributions from 
Joe Quinn, Karsten Bräckelmann, Bill Cole, and Giovanni Bechis. It is 
maintained by The McGrail Foundation.


The KAM channel is made possible with the support of hosting from Linode 
and help from PCCC & cPanel. More information about our sponsors can be 
found at our Sponsor's Page  at 
https://mcgrail.com/template/sponsors


To enable the KAM rule set via an sa-update channel see the channel page 
 at 
https://mcgrail.com/template/kam.cf_channel

___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-11-26 Thread Giovanni Bechis

On 11/26/20 11:10 AM, Andrea Venturoli wrote:
> On 11/26/20 8:46 AM, Giovanni Bechis wrote:
>> both Mail::Headers and Mail::Field are still current, are you able to provide
>> a sample "./HEADERS" file ?
>> Just add this snippet after the open(2) call:
>> open(LOGH, '>', "/tmp/headers.log") or die $!;
>> print LOGH ;
>> close LOGH;
> 
> Wouldn't this overwrite the same file again and again?
> I'd need to let it run until that kind of mail arrives and not loose it at 
> the next one.
> Can't it be written to syslog in some way?
> 
sorry, I thought it was reproducible in some way, I think calling 
action_quarantine_entire_message()
as Diane suggested is the correct thing to do in this case.

 Giovanni
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-11-26 Thread Dianne Skoll via MIMEDefang

On Wed, 25 Nov 2020 11:45:11 +0100
Andrea Venturoli  wrote:

> > if (open(HF, "<./HEADERS")) {
> > my $head = Mail::Header->new([]);
> > close HF;
> > md_syslog('warning',"head = %$head");

You shouldn't really print "head = %$head" because you don't know the
implementation details of Mail::Header.  Treat it as an opaque object.

> The purpose is to extract Received fields, so to reconstruct the path
> a message underwent from server to server.
> It has worked 99.99% of the times for years; lately, however it fails
> on some very frequent spam messages and I have a hard time
> understanding why.

I suggest calling action_quarantine_entire_message() if you get no
received headers so you can preserve a copy of the original message for
analysis.

> I tried adding:
> > foreach $key (keys %$head)
> > {
> > md_syslog('warning',"$key is $head{$key}");
> > }  

It should be $head->{$key} and is unlikely to be useful; you're
printing instance variables on the Mail::Header object, which most
likely do not correspond to actual headers.

If you want to log @recheads, I would do it like this:

md_syslog('warning', 'recheads = ' . join(', ', @recheads));

Anyway... you need to capture the message (or at least the headers)
so we can analyze what's going on.

Regards,

Dianne.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-11-26 Thread Andrea Venturoli


On 11/26/20 8:46 AM, Giovanni Bechis wrote:

both Mail::Headers and Mail::Field are still current, are you able to provide
a sample "./HEADERS" file ?
Just add this snippet after the open(2) call:
open(LOGH, '>', "/tmp/headers.log") or die $!;
print LOGH ;
close LOGH;


Wouldn't this overwrite the same file again and again?
I'd need to let it run until that kind of mail arrives and not loose it 
at the next one.

Can't it be written to syslog in some way?

 bye
av.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Help with header checking

2020-11-25 Thread Giovanni Bechis

On Wed, Nov 25, 2020 at 11:45:11AM +0100, Andrea Venturoli wrote:
> Hello.
> 
> I'm a long time MIMEDefang user, but my PERL knowledge is about zero.
> 
> A long time ago I found some sample code (from a site which does not 
> even exist anymore) and managed to adapt it to my situation.
> 
> Here it is:
> > if (open(HF, "<./HEADERS")) {
> > my $head = Mail::Header->new([]);
> > close HF;
> > md_syslog('warning',"head = %$head");
> > if ($head) {
> > my @recheads = Mail::Field->extract('Received',$head);
> > md_syslog('warning',"RecHeads = %@recheads");
> > foreach my $rechead (@recheads) {
> > ...
> > }
> > }
> > }
> 
both Mail::Headers and Mail::Field are still current, are you able to provide
a sample "./HEADERS" file ?
Just add this snippet after the open(2) call:
open(LOGH, '>', "/tmp/headers.log") or die $!;
print LOGH ;
close LOGH;

> The purpose is to extract Received fields, so to reconstruct the path a 
> message underwent from server to server.
> It has worked 99.99% of the times for years; lately, however it fails on 
> some very frequent spam messages and I have a hard time understanding why.
> 
> So, I'm asking for help from someone who knows PERL.
> 
> Are Mail::Header and Mail::Field still current? Or are they obsolete? If 
> so, what should be used as a replacement?
> 
> For these messages, I see in the logs:
> > mimedefang.pl[23042]: 0AP2jef2000844: head = %Mail::Header=HASH(0x1fe80ae0)
> > mimedefang.pl[23042]: 0AP2jef2000844: RecHeads = %
> 
> So it seems Mail::Header parsed the message, but either found no 
> Received header (and they are there!) or it's Mail::Field that fails.
> How would I log more data to know what's going on?
> 
> I tried adding:
> > foreach $key (keys %$head)
> > {
> > md_syslog('warning',"$key is $head{$key}");
> > }
> Alas, only the keys are printed, not the values.
> 
> Any hint?
> 
>   bye & Thanks
>   av.
> 
> P.S. I'm using PERL 5.32.0.
> ___
> NOTE: If there is a disclaimer or other legal boilerplate in the above
> message, it is NULL AND VOID.  You may ignore it.
> 
> Visit http://www.mimedefang.org and http://www.roaringpenguin.com
> MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
> http://lists.roaringpenguin.com/mailman/listinfo/mimedefang


signature.asc
Description: PGP signature
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

[Mimedefang] Help with header checking

2020-11-25 Thread Andrea Venturoli


Hello.

I'm a long time MIMEDefang user, but my PERL knowledge is about zero.

A long time ago I found some sample code (from a site which does not 
even exist anymore) and managed to adapt it to my situation.


Here it is:

if (open(HF, "<./HEADERS")) {
my $head = Mail::Header->new([]);
close HF;
md_syslog('warning',"head = %$head");
if ($head) {
my @recheads = Mail::Field->extract('Received',$head);
md_syslog('warning',"RecHeads = %@recheads");
foreach my $rechead (@recheads) {
...
}
}
}


The purpose is to extract Received fields, so to reconstruct the path a 
message underwent from server to server.
It has worked 99.99% of the times for years; lately, however it fails on 
some very frequent spam messages and I have a hard time understanding why.


So, I'm asking for help from someone who knows PERL.

Are Mail::Header and Mail::Field still current? Or are they obsolete? If 
so, what should be used as a replacement?


For these messages, I see in the logs:

mimedefang.pl[23042]: 0AP2jef2000844: head = %Mail::Header=HASH(0x1fe80ae0)
mimedefang.pl[23042]: 0AP2jef2000844: RecHeads = %


So it seems Mail::Header parsed the message, but either found no 
Received header (and they are there!) or it's Mail::Field that fails.

How would I log more data to know what's going on?

I tried adding:

foreach $key (keys %$head)
{
md_syslog('warning',"$key is $head{$key}");
}

Alas, only the keys are printed, not the values.

Any hint?

 bye & Thanks
av.

P.S. I'm using PERL 5.32.0.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Future development

2020-11-23 Thread Anne Bennett


> The Perl code is atrocious.

Give yourself a break.  ;-)

> mimedefang.pl is over 7500 lines of
> Perl in one ugly file with a ton of global variables.  It needs a major
> overhaul.

For me as a filter writer, problems have been:

  - Initially (years ago), it was a bit hard to keep track of the bits
and pieces, because mimedefang.pl calls my filter file, and my
filter file calls various utilities provided by mimedefang.pl.
Any code re-org that would make it easier to untangle this would
help, even something as simple as putting the user-filter-called
facilities into their own Perl module.

  - I like to run with Perl taint checks on, but mimedefang.pl
doesn't cooperate well with that.

  - Picking up needed information (i.e., from the envelope and
message, and QID) seems to be different in each filter_*
subroutine.  I don't know how this would affect performance,
but if there were a hash containing all available header,
envelope, message status so far (e.g., did we mark as to
be rejected?), and QID information, and maybe even headers
in concatenated and decoded form, that would make writing
filter_* subroutines easier.

> I would start by writing a MD::Filter base class
> and storing state in the MD::Filter object.  Callbacks could be implemented
> as methods and users could subclass MD::Filter to implement their behaviour.

As another poster mentioned, that would indeed require more
advanced Perl chops from the end user (postmaster), though
well-chosen examples might go a long way toward alleviating
that requirement.

> The actual filter file would be a very simple thing that simply
> instantiates a (subclass of) MD::Filter and calls $filter->run() to
> run the main mimedefang.pl loop.

Well, there's still potentially a fair amount of homegrown code that
people can add.  I'm assuming that this would somehow fit into "users
could subclass MD::Filter to implement their behaviour".

It is definitely the case that backwards compatibility would be an
issue for those of us with thousands of lines of locally written code.



Anne.
-- 
Ms. Anne Bennett, Senior Sysadmin, ENCS, Concordia University, Montreal H3G 1M8
a...@encs.concordia.ca+1 514 848-2424 x2285
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Future development

2020-11-23 Thread Dianne Skoll via MIMEDefang

On Mon, 23 Nov 2020 12:41:00 -0500
Anne Bennett  wrote:

> > The Perl code is atrocious.  

> Give yourself a break.  ;-)

Well, OK. :)

The code itself is not terrible.  I like to write clear code that any
newbie can understand (for "newbie", read "myself in 6 months time...")

What I mean is the *design* is atrocious, given that MIMEDefang is the
result of acretion rather than design.

(OTOH, I have worked on systems with beautiful designs but very tricky
and "clever" code where most of my time is spent reverse-engineering.
We don't want that either.)

>   - Initially (years ago), it was a bit hard to keep track of the bits
> and pieces, because mimedefang.pl calls my filter file, and my
> filter file calls various utilities provided by mimedefang.pl.
> Any code re-org that would make it easier to untangle this would
> help, even something as simple as putting the user-filter-called
> facilities into their own Perl module.

Right.  Tracking the flow of control when callbacks are involved is tricky.

>   - I like to run with Perl taint checks on, but mimedefang.pl
> doesn't cooperate well with that.

Oh, hmm. :)  That would be a lot of work.

>   - Picking up needed information (i.e., from the envelope and
> message, and QID) seems to be different in each filter_*
> subroutine.  I don't know how this would affect performance,
> but if there were a hash containing all available header,
> envelope, message status so far (e.g., did we mark as to
> be rejected?), and QID information, and maybe even headers
> in concatenated and decoded form, that would make writing
> filter_* subroutines easier.

Yes, some sort of context object that held all of this would be good.

> > I would start by writing a MD::Filter base class
> > and storing state in the MD::Filter object.  Callbacks could be
> > implemented as methods and users could subclass MD::Filter to
> > implement their behaviour.

> As another poster mentioned, that would indeed require more
> advanced Perl chops from the end user (postmaster), though
> well-chosen examples might go a long way toward alleviating
> that requirement.

In my ideal world, you would be able to take an existing mimedefang-filter
script, and add two lines to the top:

package MyFilter;
use base 'MD::Filter::Compat';

and then have it Just Work.  I don't know if this is realistic or a pipe-dream.

> Well, there's still potentially a fair amount of homegrown code that
> people can add.  I'm assuming that this would somehow fit into "users
> could subclass MD::Filter to implement their behaviour".

Yes, exactly.

Regards,

Dianne.
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Future development

2020-11-23 Thread Jan-Pieter Cornet

On 22-11-20 22:21, Richard Laager wrote:

On 11/22/20 12:32 PM, Dianne Skoll via MIMEDefang wrote:

mimedefang.pl is over 7500 lines of
Perl in one ugly file with a ton of global variables. It needs a major
overhaul. I would start by writing a MD::Filter base class
and storing state in the MD::Filter object. Callbacks could be implemented
as methods and users could subclass MD::Filter to implement their behaviour.

I've written an OO-like wrapper around current mimedefang ages ago, which is
still in use today. Documented here:

https://johnpc.home.xs4all.nl/mimedefang-modular/

Feel free to use any way you like.

If the "mimedefang-filter-select" and "Mimedefang.pm" is incorporated into the
main program, then example modules might even be easier to maintain because different things (eg
blocking bad extensions vs stripping HTML) are in different files, and can be turned off completely
just by removing one line from MailFilter::Settings.pm.

Just a suggestion... and making MD even more modern with better OO sounds good
to me. I've basically done this to make it a bit more manageable, back in the
days when I started using this.

--
Jan-Pieter Cornet
"Any sufficiently advanced incompetence is indistinguishable from malice."
- Grey's Law

OpenPGP_signature
Description: OpenPGP digital signature
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID. You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Future development

2020-11-22 Thread Dianne Skoll via MIMEDefang

On Sun, 22 Nov 2020 15:21:04 -0500
"Bill Cole"  wrote:

[snipsnip...]

> Obviously OOPifying MD would be a v3 project, as it would break all 
> existing mimedefang-filter.pl scripts.

That is an issue.  Perhaps making an MD::Filter::Compat class that
lets old filter scripts keep working with minimal changes could be
a solution?

(It is a fair bit of work, of course...)

> A nearer-term step to clean up the code a bit would be to break out 
> existing code to distinct modules that may (or may not) be the basis
> for future object structures. This could result in supporting
> existing configurations with a code base that is easier to maintain
> and understand.

Yes, incremental cleanups could help too, but they would still likely
break existing filter scripts.

Regards,

Dianne.

___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] Future development

2020-11-22 Thread Bill Cole


On 22 Nov 2020, at 13:32, Dianne Skoll via MIMEDefang wrote:


On Sun, 22 Nov 2020 13:24:24 -0500
Dianne Skoll  wrote:


Anyway... I'd really like to see MIMEDefang development activity pick
up again.  I'm hoping The McGrail Foundation will attract more
interest in the project.


Specifically... these are things I would do differently if I were
starting MIMEDefang now instead of 20 years ago. :)

The C code is mostly fine; the milter and the multiplexor work
pretty well.

The Perl code is atrocious.


That's a bit harsh...

It is solidly non-OOP and clearly organically grown over a long time, 
but I've worked with worse code that was more carefully designed.



mimedefang.pl is over 7500 lines of
Perl in one ugly file with a ton of global variables.  It needs a 
major

overhaul.  I would start by writing a MD::Filter base class
and storing state in the MD::Filter object.  Callbacks could be 
implemented
as methods and users could subclass MD::Filter to implement their 
behaviour.


PLUS: A more elegant design. Probably easier to maintain long-term. More 
accessible/attractive to trained developers who are more comfortable 
with OOP.


MINUS: Raises the bar for Perl skill needed to implement local 
customizations. Less accessible for people who only understand 
procedural programming.



Functions like md_check_against_smtp_server, etc. should
be broken out into helper modules.


Yes. Having done some work in and around md_check_against_smtp_server, I 
agree that breaking out those sorts of utilities into other modules 
apart from the core functionality would be a good idea, even if it's 
just into a MD::Utils module full of subroutines.



The actual filter file would be a very simple thing that simply
instantiates a (subclass of) MD::Filter and calls $filter->run() to
run the main mimedefang.pl loop.

Comments/thoughts/etc?


Obviously OOPifying MD would be a v3 project, as it would break all 
existing mimedefang-filter.pl scripts.


A nearer-term step to clean up the code a bit would be to break out 
existing code to distinct modules that may (or may not) be the basis for 
future object structures. This could result in supporting existing 
configurations with a code base that is easier to maintain and 
understand.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire
___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

Re: [Mimedefang] [External] Re: Test

2020-11-22 Thread Kevin A. McGrail


On 11/22/2020 12:42 PM, Fredrik Pettai wrote:

Have you looked at Orange Spam?  That was effectively a lightweight rewrite of 
SA in Python.  Not as feature rich but I believe rulesets were compatible.  
They got bought by SolarWinds so I don't know if the project is alive any more.

This one?https://github.com/SpamExperts/OrangeAssassin  ?
It looked alive (it had recent commits), but a closer look shows its automation 
jobs doing all commits & pull requests etc. the last year(s).
I think so yes, but the website has been pulled.  I didn't think to 
check for a repo. I know that SpamExperts was bought by SolarWinds and 
that Dreas is no longer with the firm though.  With it, I think this 
python-based project became dormant.

(but then again, SpamAssassin isn’t fast moving target these days…)
SA is a mature product with gazillions of users so it doesn't have much 
change really.  But it is alive, features being added, bugs being fixed, 
etc.  I'd like to get more releases done though the repo code is usually 
quite suitable for production environments believe it or not.

PS. Is there a (hidden) pun in “orange assassin” that non-native English 
speakers don’t understand? Or perhaps it’s something Dutch… (SpamExperts 
originate from Holland I believe DS.


I believe it's an homage to Dreas and team from Amsterdam and the House 
of Orange.


Regards,
KAM

___
NOTE: If there is a disclaimer or other legal boilerplate in the above
message, it is NULL AND VOID.  You may ignore it.

Visit http://www.mimedefang.org and http://www.roaringpenguin.com
MIMEDefang mailing list MIMEDefang@lists.roaringpenguin.com
http://lists.roaringpenguin.com/mailman/listinfo/mimedefang

1 2 3 4 5 6 7 8 9 10 >

1 - 100 of 13378 matches

Mail list logo