[Mailman-Users] Re: Change moderator or admin password (random) different for each list they were an admin or moderator for - Mailman 2.1

[Stephen J. Turnbull]

Hi, Daniel

Mark Sapiro writes:

 > On 3/4/21 7:06 AM, Daniel Botting wrote:

 > > My script and usage so far can be found below:



 > This was asked and answered at
 > 
 > 
 > The answer there was correct and is you are generating the password once
 > as part of the withlist command line and passing the one single output
 > of `pwgen -sB 15 1` as the password argument to the script. You need to
 > generate a password within the script.

I think

newpassword = subprocess.run(["pwgen", "-sB", "15", "1"],
 capture_output=True,
 text=True).stdout

will do the trick (be careful about text=True, though; you may want
the default text=False so that stdout will be bytes instead of str).

https://docs.python.org/3/library/subprocess.html#subprocess.run

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

[Stephen J. Turnbull]

Executive summary: -0.8 for changing Scrubber (don't think there's a
benefit either way => don't bother), +0.5 for .txt (not sure it's
worth the effort), plus an amusing (YMMV) story.

Mark Sapiro writes:
 > On 3/4/21 7:05 PM, Stephen J. Turnbull wrote:
 > > Mark Sapiro writes:
 > 
 > > Does Scrubber really do that?  Per RFC, the two Content-Type fields
 > > have exactly the same semantics: "it is plain text, encoded as ASCII."
 > 
 > Yes, scrubber really does this. This dates back to Tokio and various
 > Japanese language emails which presumably weren't all ascii. Perhaps we
 > should revisit this, but you know much more about Japanese language
 > emails than I do.

I don't think there's a need to change nowadays.  The problems I see
now are (a) spam where the rules never did apply anyway and (b) in
application/* attachments such as zipfiles.  Common MUAs all do the
right things with text/*, and there are very few folks who bother to
write their own (or use netcat to talk to port 587 ;-) anymore.

 > Apparently, Tokio thought it was better to scrub such a part than to
 > treat it as ascii when it wasn't.

It certainly was back then.  A little earlier than that, I was at Ohio
State and they used a Prime minicomputer for email and secretary-
supported wordprocessing.  The professor who was head of the Econ Dept
computer committee got full of himself and sent me a huge warlording
.sig full of VT-220 (maybe even VT-320) control sequences for flashing
and reverse video and the like, containing the message "Beware the
Wizard!"

It was indeed very bright! and shiny!, so I sent it back to him with
compliments (something like "A sufficiently advanced technology is
indistinguishable from magic, but I have the VT-220 programming
manual").  Silly Rabbit, Trix are for kids -- he had ripped off the
fancy large screen console terminal for use in his personal office.
(I can't blame him, it was *very* nice and it's not like you need more
than a VT-52 or so for the console.)  It was *not* VT-compatible, and
apparently that warlord sig not only reprogrammed its function keys,
but proceeded to invoke them, sending a message to the host which
promptly crashed hard.  (I didn't notice the crash as I was using a PC
for real work and email is asynchronous so nothing out of the ordinary
there either.  I heard about it the next day at the faculty meeting
where "The Wizard" got a dressing-down for misappropriating department
property. :-)

I imagine that terminal lockups etc, if not full-blown system crashes,
were reasonably common with Shift JIS too.

 > Perhaps a compromise is to give scrubbed text/plain attachments a
 > .txt. extension rather than taking the first item returned by
 > mime_types.guess_all_extensions.

I'm not sure.  Nowadays I'm not so worried about harming the machine
as the possibility of malware (eg, URLs with Greek or Cyrillic
cognates of ASCII characters directing you to a phishing site when you
think you're linking to Amazon).

On the other hand, I guess if you're going to fall for that with .txt,
you'll probably think "computers are weird" and fall for it with
Content-Type: application/octet-stream; name="inline-text.ksh"
too. :-(

So, yeah, if you want to go to the trouble, I think defaulting to .txt
for text/plain with no filename specified is more user-friendly.
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

[Mark Sapiro]

On 3/4/21 8:36 PM, Mark Dale via Mailman-Users wrote:
> 
> Reading Steve's reply just now makes me look suspiciously at the Perl
> X-Mailer: MIME::Lite that is sending the email to the list. My
> understanding is the list owner has scheduled a Perl script to export
> from a database and post the resulting export.


I've looked at the code more carefully, and I see there are two
conditions for the text/plain part to be scrubbed. One is the lack of a
charset= parameter, but the other is that the part is not the only body
part or maybe the first part of a multipart body.

However, the way the code determines if the part is the body vs. being
an attachment is the presence of a Content-Disposition: header. Your
message has a Content-Disposition: inline header and while this is
explicitly allowed by RFC 2183, it is unusual for a single part
text/plain message.

If the perl script that generates this message can not include that
header, I don't thing the part will be scrubbed.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

[Mark Sapiro]

On 3/4/21 8:36 PM, Mark Dale via Mailman-Users wrote:
> 
> Thanks. I've implemented your script patch (LISTNAME needed to be in
> quotes otherwise the server spat the dummy).

Sorry about that ...

> The next scheduled post to
> the list will be in about 10 hours so I'll have a result to look at then.


Cool.


> Reading Steve's reply just now makes me look suspiciously at the Perl
> X-Mailer: MIME::Lite that is sending the email to the list. My
> understanding is the list owner has scheduled a Perl script to export
> from a database and post the resulting export.
> 
> Anyways, I'll see what the result gets written by scripts/post  in the
> morning.

OK.

I'm just curious, but if the body is scrubbed as an attachment with a
.txt extension instead of .ksh would that help?


-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

[Mark Dale via Mailman-Users]

 Original Message 
From: Mark Sapiro [mailto:m...@msapiro.net]
Sent: Friday, March 5, 2021, 01:29 UTC
To: mailman-users@python.org
Subject: [Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment


> Here's something you can try.
> 
> Create a file somewhere and give the Mailman's group write permission on
> it. Then apply the following patch to Mailman's scripts/post
> 
>> === modified file 'scripts/post'
>> --- scripts/post 2018-06-17 23:47:34 +
>> +++ scripts/post 2021-03-05 01:16:56 +
>> @@ -58,8 +58,12 @@
>>  # some MTAs have a hard limit to the time a filter prog can run.  
>> Postfix
>>  # is a good example; if the limit is hit, the proc is SIGKILL'd giving 
>> us
>>  # no chance to save the message.
>> +content = sys.stdin.read()
>> +if listname == LISTNAME:
>> +with open('PATH_TO_FILE', 'a') as fp:
>> +fp.write(content)
>>  inq = get_switchboard(mm_cfg.INQUEUE_DIR)
>> -inq.enqueue(sys.stdin.read(),
>> +inq.enqueue(content,
>>  listname=listname,
>>  tolist=1, _plaintext=1)
>>  
>>
> 
> 
> where LISTNAME is the problem list's name and PATH_TO_FILE is the path
> to the file you created. This file will accumulate all the incoming
> messages posted to the list, exactly as they are received by Mailman.
> 
> Then we can at least know for sure what that message looks like.
> 
> Of course, once you have captured one such message, you can revert the
> patch.
> 

Thanks. I've implemented your script patch (LISTNAME needed to be in
quotes otherwise the server spat the dummy). The next scheduled post to
the list will be in about 10 hours so I'll have a result to look at then.

Reading Steve's reply just now makes me look suspiciously at the Perl
X-Mailer: MIME::Lite that is sending the email to the list. My
understanding is the list owner has scheduled a Perl script to export
from a database and post the resulting export.

Anyways, I'll see what the result gets written by scripts/post  in the
morning.

Cheers,
Mark
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Change moderator or admin password (random) different for each list they were an admin or moderator for - Mailman 2.1

[Mark Sapiro]

On 3/4/21 7:06 AM, Daniel Botting wrote:
> Hi,
> 
> I have been using the withlist command to delete a given administrator
> or moderator from all mailman 2.1 lists.
> 
> I also require to change the moderator or admin password for all lists
> that they have been removed from, but have it so that it is a different
> randomly generated password for each list they are removed from and then
> have this emailed to the other moderators or admins who are then left or
> a default address if none are left (I have managed so far that it is
> changed to the same password for each list they have been removed from).
> I am currently at the beginning of my Python learning so this is why I'm
> asking for help.
> 
> My script and usage so far can be found below:
> 
> Usage:
> 
> withlist --all --run script_name.change_administrator_password
> firstname.lastn...@domain.co.uk `pwgen -sB 15 1`
> 
> The script is saved at:
> 
> /usr/sbin/
> 
> Function defined in the script:
> 
> import sha
> 
> def change_administrator_password(mlist, owner, newpasswd):
> mlist.Lock()
> try:
> if mlist.owner[mlist.owner.index(owner)]:
> mlist.password = sha.new(newpasswd).hexdigest()
>    print (newpasswd)
> 
> except:
>   print("password not changed for", (mlist.real_name, owner))
> mlist.Save()
> mlist.Unlock()
> 
> This will go through all lists and change the administrator password to
> a randomly generated one, but as advised above it will set the same
> password (not what I want) I've put a print in for debugging so I can
> see what it was.

This was asked and answered at


The answer there was correct and is you are generating the password once
as part of the withlist command line and passing the one single output
of `pwgen -sB 15 1` as the password argument to the script. You need to
generate a password within the script.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

[Mark Sapiro]

On 3/4/21 7:05 PM, Stephen J. Turnbull wrote:
> Mark Sapiro writes:

> 
> Does Scrubber really do that?  Per RFC, the two Content-Type fields
> have exactly the same semantics: "it is plain text, encoded as ASCII."


Yes, scrubber really does this. This dates back to Tokio and various
Japanese language emails which presumably weren't all ascii. Perhaps we
should revisit this, but you know much more about Japanese language
emails than I do.

Apparently, Tokio thought it was better to scrub such a part than to
treat it as ascii when it wasn't.

Perhaps a compromise is to give scrubbed text/plain attachments a .txt.
extension rather than taking the first item returned by
mime_types.guess_all_extensions.

Note that this is a Mailman 2.1 issue only. In MM 3 scrubber is only
invoked for plain text digests and just removes with notice all
non-text/plain elemental parts and for text/plain treats missing or
unknown charsets as ascii.


-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Change moderator or admin password (random) different for each list they were an admin or moderator for - Mailman 2.1

[Daniel Botting]

Hi,

I have been using the withlist command to delete a given administrator 
or moderator from all mailman 2.1 lists.


I also require to change the moderator or admin password for all lists 
that they have been removed from, but have it so that it is a different 
randomly generated password for each list they are removed from and then 
have this emailed to the other moderators or admins who are then left or 
a default address if none are left (I have managed so far that it is 
changed to the same password for each list they have been removed from). 
I am currently at the beginning of my Python learning so this is why I'm 
asking for help.


My script and usage so far can be found below:

Usage:

withlist --all --run script_name.change_administrator_password 
firstname.lastn...@domain.co.uk `pwgen -sB 15 1`


The script is saved at:

/usr/sbin/

Function defined in the script:

import sha

def change_administrator_password(mlist, owner, newpasswd):
mlist.Lock()
try:
if mlist.owner[mlist.owner.index(owner)]:
mlist.password = sha.new(newpasswd).hexdigest()
   print (newpasswd)

except:
  print("password not changed for", (mlist.real_name, owner))
mlist.Save()
mlist.Unlock()

This will go through all lists and change the administrator password to 
a randomly generated one, but as advised above it will set the same 
password (not what I want) I've put a print in for debugging so I can 
see what it was.


Thanks

Daniel

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
   https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

[Stephen J. Turnbull]

Mark Sapiro writes:

 > >> Content-Disposition: inline
 > >> Content-Type: text/plain; charset="us-ascii"
 > >> Content-Transfer-Encoding: 7bit
 > >>
 > >> And in the mbox file below (of the same message), I see:
 > >>
 > >> Content-Disposition: inline
 > >> Content-Type: text/plain
 > >> Content-Transfer-Encoding: quoted-printable
 > 
 > OK, that's the issue. The message in the .mbox is after various list
 > manipulations, but before scrubbing for the pipermail archive, and
 > somehow the '; charset="us-ascii"' has been lost from the Content-Type:
 > header, which is why Scrubber  scrubs it.

Does Scrubber really do that?  Per RFC, the two Content-Type fields
have exactly the same semantics: "it is plain text, encoded as ASCII."

I would hope instead that it's the non-ASCII content that triggered
something (are we sure it's Mailman? could be an MTA somewhere along
the line) to qp-encode.

For example, the original mail may have included directed quotes or
similar hard-to-distinguish "fancy punctuation", but the composing MUA
didn't notice them and just randomly set charset=us-ascii.  Is there
quoted-printable (easily recognized by the "=" + 2 hex digits syntax)
in that MIME body in the mbox?  Another possibility was that it was a
very long line and it was qp-encoded ("=" CRLF inserted after a space)
to conform to RFC 822.

 > > FWIW, using Thunderbird I posted the contents of the original email to a
 > > test list (on the same server, with the same lists configs) and as
 > > expected the archived message displays correctly as a plaintext
 > > email.

How did the contents get into the message in Thunderbird?  Copy-paste?
Yank from mailbox?  Forward?  If it's anything but the last,
Thuderbird almost surely massaged it on the way in.

 > > The headers this time show as:
 > > 
 > > === FROM EMAIL HEADER
 > > MIME-Version: 1.0
 > > Content-Type: text/plain; charset="us-ascii"
 > > Content-Transfer-Encoding: 7bit
 > > 
 > > === FROM MBOX
 > > MIME-Version: 1.0
 > > Content-Type: text/plain; charset=utf-8
 > > Content-Transfer-Encoding: 7bit
 > 
 > This is what should happen. I'm not sure which handler changed the
 > charset to utf-8, but I don't think that's significant.

This is RFC-ly bizarre.  Why would anything change the charset, unless
there were non-ASCII octets?  If something does make that change
despite the body being pure ASCII, I would argue that's a bug (very
old MUAs might refuse to display the message), although it's probably
irrelevant nowadays.

But if there *are* non-ASCII octets, the Content-Transfer-Encoding is
a lie.  I think the original message was probably broken as sent.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

[Mark Sapiro]

On 3/4/21 4:20 PM, Mark Dale via Mailman-Users wrote:
>>
>> Thanks Mark. I've copied the message from mbox file and pasted below.
>> (the client's info is marked as redacted).
>>
>> Something I did notice: in the original received email headers I see:


Original as received by the list or from the list?


>> Content-Disposition: inline
>> Content-Type: text/plain; charset="us-ascii"
>> Content-Transfer-Encoding: 7bit
>>
>> And in the mbox file below (of the same message), I see:
>>
>> Content-Disposition: inline
>> Content-Type: text/plain
>> Content-Transfer-Encoding: quoted-printable

OK, that's the issue. The message in the .mbox is after various list
manipulations, but before scrubbing for the pipermail archive, and
somehow the '; charset="us-ascii"' has been lost from the Content-Type:
header, which is why Scrubber  scrubs it.


...
> 
> FWIW, using Thunderbird I posted the contents of the original email to a
> test list (on the same server, with the same lists configs) and as
> expected the archived message displays correctly as a plaintext email.
> 
> The headers this time show as:
> 
> === FROM EMAIL HEADER
> MIME-Version: 1.0
> Content-Type: text/plain; charset="us-ascii"
> Content-Transfer-Encoding: 7bit
> 
> === FROM MBOX
> MIME-Version: 1.0
> Content-Type: text/plain; charset=utf-8
> Content-Transfer-Encoding: 7bit


This is what should happen. I'm not sure which handler changed the
charset to utf-8, but I don't think that's significant.

Clearly there is something different in either the two lists, or the
message that arrived at the lists, but My only guess is the message that
arrived at the original list was already missing the charset="us-ascii".

Here's something you can try.

Create a file somewhere and give the Mailman's group write permission on
it. Then apply the following patch to Mailman's scripts/post

> === modified file 'scripts/post'
> --- scripts/post  2018-06-17 23:47:34 +
> +++ scripts/post  2021-03-05 01:16:56 +
> @@ -58,8 +58,12 @@
>  # some MTAs have a hard limit to the time a filter prog can run.  Postfix
>  # is a good example; if the limit is hit, the proc is SIGKILL'd giving us
>  # no chance to save the message.
> +content = sys.stdin.read()
> +if listname == LISTNAME:
> +with open('PATH_TO_FILE', 'a') as fp:
> +fp.write(content)
>  inq = get_switchboard(mm_cfg.INQUEUE_DIR)
> -inq.enqueue(sys.stdin.read(),
> +inq.enqueue(content,
>  listname=listname,
>  tolist=1, _plaintext=1)
>  
> 


where LISTNAME is the problem list's name and PATH_TO_FILE is the path
to the file you created. This file will accumulate all the incoming
messages posted to the list, exactly as they are received by Mailman.

Then we can at least know for sure what that message looks like.

Of course, once you have captured one such message, you can revert the
patch.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

[Mark Dale via Mailman-Users]

 Original Message 
From: Mark Dale via Mailman-Users [mailto:mailman-users@python.org]
Sent: Friday, March 5, 2021, 00:01 UTC
To: mailman-users@python.org
Subject: [Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

> 
>  Original Message 
> From: Mark Sapiro [mailto:m...@msapiro.net]
> Sent: Thursday, March 4, 2021, 06:04 UTC
> To: mailman-users@python.org
> Subject: [Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment
> 
>> On 3/3/21 3:36 PM, Mark Dale via Mailman-Users wrote:
>>> Hi Listers,
>>>
>>> I've got a client's list to which a plain text email notice is sent
>>> everyday.
>>>
>>> Mailman-Version: 2.1.34
>>> Postfix
>>> Debian 10
>>>
>>> The message contents are fairly similar each day and the text message
>>> renders in email clients just fine and the .mbox file reads fine also.
>>>
>>> However, Pipermail renders each archived message as a ".ksh" attachment.
>>>
>>>
>>> *
>>>
>>> >From noreply at XXX.com  Wed Mar  3 12:06:05 2021
>>> From: noreply at XXX.com (noreply at XXX.com)
>>> Date: Wed, 3 Mar 2021 07:06:05 -0500
>>> Subject: [XXX] XXX Published daily - 26097
>>> Message-ID: <202103031206.123C662K015222@XXX>
>>>
>>> An embedded and charset-unspecified text was scrubbed...
>>> Name: not available
>>> URL:
>>> 
>>>
>>> *
>>>
>>> The message header contains:
>>>
>>> Content-Disposition: inline
>>> MIME-Version: 1.0
>>> X-Mailer: MIME::Lite 3.031 (F2.85; T2.17; A2.21; B3.15; Q3.13)
>>> ...
>>> Content-Type: text/plain; charset="us-ascii"
>>> Content-Transfer-Encoding: 7bit
>>>
>>>
>>> Could anyone point me in the right direction to get the archived
>>> messages to display their content as text in the message body instead of
>>> as an attachment?
>>
>> In order to help with this, I need to see the complete MIME structure of
>> the message. I.e. all the Content-Type: headers including the top level
>> and all the boundaries. This information for a message received from the
>> list is what I want to see as that's what Scrubber sees. Or the message
>> from the archives/private/listname.mbox/listname.mbox would do too.
>>
>> Scrubber should not be scrubbing the 'Content-Type: text/plain;
>> charset="us-ascii"' part with the message 'An embedded and
>> charset-unspecified text was scrubbed...'. Something else is going on here.
>>
>> The .ksh extension comes from the Python library
>> mime_types.guess_all_extensions, which returns the list
>>
>> ['.ksh', '.bat', '.h', '.txt', '.pl', '.c', '.asc', '.text', '.pot',
>> '.brf', '.srt']
>>
>> for text/plain and we arbitrarily pick the first one which is .ksh, but
>> we shouldn't be doing that with a text/plain part with a declared charset.
>>
> 
> ==
> 
> Thanks Mark. I've copied the message from mbox file and pasted below.
> (the client's info is marked as redacted).
> 
> Something I did notice: in the original received email headers I see:
> 
> Content-Disposition: inline
> Content-Type: text/plain; charset="us-ascii"
> Content-Transfer-Encoding: 7bit
> 
> And in the mbox file below (of the same message), I see:
> 
> Content-Disposition: inline
> Content-Type: text/plain
> Content-Transfer-Encoding: quoted-printable
> 
> 
> 
> 
>>From noreply@REDACTED  Thu Mar  4 12:04:42 2021
> Return-Path: 
> X-Original-To: REDACTED@lists.REDACTED
> Delivered-To: REDACTED@lists.REDACTED
> Received: from alln-iport-1.REDACTED (alln-iport-1.REDACTED [173.37.142.88])
>  by mailmanlists.network (Postfix) with ESMTPS id EDCEB1FFCA
>  for ; Thu,  4 Mar 2021 12:04:41 + (UTC)
> X-IPAS-Result: =?us-ascii?q?A0CcAgAZzEBgmJpdJa1iHgEBCxIMghGEL412pSQLAQEBD?=
>  =?us-ascii?q?zQEAQGFBIFFAiU5BQ0CAwEBAQMCAwEBAQEFAQEBAgEGBBQBAQEBAQEBAYZDi?=
>  =?us-ascii?q?TOCUoMIrWIBAQGCJok2gS2BOYsegiQmHIILgUQDgSiMExoEhVWNUAGREpt7h?=
>  =?us-ascii?q?EiGco1JhkWDcp9kC4YsrEqEGoFsIIFZcIM6TxkNVY1jjk0jAQJnAgYKAQEDC?=
>  =?us-ascii?q?YwTAQE?=
> X-IronPort-Anti-Spam-Filtered: true
> X-IronPort-AV: E=Sophos;i="5.81,222,1610409600"; d="scan'208";a="656725958"
> Received: from rcdn-core-3.REDACTED ([173.37.93.154])
>  by alln-iport-1.REDACTED with ESMTP/TLS/DHE-RSA-SEED-SHA;
>  04 Mar 2021 12:04:41 +
> Received: from mail.vrt.REDACTED ([10.83.44.69])
>  by rcdn-core-3.REDACTED (8.15.2/8.15.2) with SMTP id 124C4e83021704
>  for ; Thu, 4 Mar 2021 12:04:40 GMT
> Message-Id: <202103041204.124C4e83021704@rcdn-core-3.REDACTED>
> Received: from localhost.localdomain (sigmanager.vrt.REDACTED
>  [10.7.89.25])
>  by mail.vrt.REDACTED (Postfix) with ESMTP id 7E95E424D5
>  for ; Thu,  4 Mar 2021 12:04:40 + (UTC)
> Content-Disposition: inline
> Content-Type: text/plain
> MIME-Version: 1.0
> X-Mailer: MIME::Lite 3.031 (F2.85; T2.17; A2.21; B3.15; Q3.13)
> Date: Thu, 4 Mar 2021 07:04:40 -0500
> From: noreply@REDACTED
> To: REDACTED@lists.REDACTED
> Content-Transfer-Encoding: quoted-printable
> X-Outbound-SMTP-Client: 10.83.44.69, [10.83.44.69]
> 

[Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

[Mark Dale via Mailman-Users]

 Original Message 
From: Mark Sapiro [mailto:m...@msapiro.net]
Sent: Thursday, March 4, 2021, 06:04 UTC
To: mailman-users@python.org
Subject: [Mailman-Users] Re: Pipermail scrubbing ascii txt to ksh attachment

> On 3/3/21 3:36 PM, Mark Dale via Mailman-Users wrote:
>> Hi Listers,
>>
>> I've got a client's list to which a plain text email notice is sent
>> everyday.
>>
>> Mailman-Version: 2.1.34
>> Postfix
>> Debian 10
>>
>> The message contents are fairly similar each day and the text message
>> renders in email clients just fine and the .mbox file reads fine also.
>>
>> However, Pipermail renders each archived message as a ".ksh" attachment.
>>
>>
>> *
>>
>> >From noreply at XXX.com  Wed Mar  3 12:06:05 2021
>> From: noreply at XXX.com (noreply at XXX.com)
>> Date: Wed, 3 Mar 2021 07:06:05 -0500
>> Subject: [XXX] XXX Published daily - 26097
>> Message-ID: <202103031206.123C662K015222@XXX>
>>
>> An embedded and charset-unspecified text was scrubbed...
>> Name: not available
>> URL:
>> 
>>
>> *
>>
>> The message header contains:
>>
>> Content-Disposition: inline
>> MIME-Version: 1.0
>> X-Mailer: MIME::Lite 3.031 (F2.85; T2.17; A2.21; B3.15; Q3.13)
>> ...
>> Content-Type: text/plain; charset="us-ascii"
>> Content-Transfer-Encoding: 7bit
>>
>>
>> Could anyone point me in the right direction to get the archived
>> messages to display their content as text in the message body instead of
>> as an attachment?
> 
> In order to help with this, I need to see the complete MIME structure of
> the message. I.e. all the Content-Type: headers including the top level
> and all the boundaries. This information for a message received from the
> list is what I want to see as that's what Scrubber sees. Or the message
> from the archives/private/listname.mbox/listname.mbox would do too.
> 
> Scrubber should not be scrubbing the 'Content-Type: text/plain;
> charset="us-ascii"' part with the message 'An embedded and
> charset-unspecified text was scrubbed...'. Something else is going on here.
> 
> The .ksh extension comes from the Python library
> mime_types.guess_all_extensions, which returns the list
> 
> ['.ksh', '.bat', '.h', '.txt', '.pl', '.c', '.asc', '.text', '.pot',
> '.brf', '.srt']
> 
> for text/plain and we arbitrarily pick the first one which is .ksh, but
> we shouldn't be doing that with a text/plain part with a declared charset.
> 

==

Thanks Mark. I've copied the message from mbox file and pasted below.
(the client's info is marked as redacted).

Something I did notice: in the original received email headers I see:

Content-Disposition: inline
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit

And in the mbox file below (of the same message), I see:

Content-Disposition: inline
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable




>From noreply@REDACTED  Thu Mar  4 12:04:42 2021
Return-Path: 
X-Original-To: REDACTED@lists.REDACTED
Delivered-To: REDACTED@lists.REDACTED
Received: from alln-iport-1.REDACTED (alln-iport-1.REDACTED [173.37.142.88])
 by mailmanlists.network (Postfix) with ESMTPS id EDCEB1FFCA
 for ; Thu,  4 Mar 2021 12:04:41 + (UTC)
X-IPAS-Result: =?us-ascii?q?A0CcAgAZzEBgmJpdJa1iHgEBCxIMghGEL412pSQLAQEBD?=
 =?us-ascii?q?zQEAQGFBIFFAiU5BQ0CAwEBAQMCAwEBAQEFAQEBAgEGBBQBAQEBAQEBAYZDi?=
 =?us-ascii?q?TOCUoMIrWIBAQGCJok2gS2BOYsegiQmHIILgUQDgSiMExoEhVWNUAGREpt7h?=
 =?us-ascii?q?EiGco1JhkWDcp9kC4YsrEqEGoFsIIFZcIM6TxkNVY1jjk0jAQJnAgYKAQEDC?=
 =?us-ascii?q?YwTAQE?=
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-AV: E=Sophos;i="5.81,222,1610409600"; d="scan'208";a="656725958"
Received: from rcdn-core-3.REDACTED ([173.37.93.154])
 by alln-iport-1.REDACTED with ESMTP/TLS/DHE-RSA-SEED-SHA;
 04 Mar 2021 12:04:41 +
Received: from mail.vrt.REDACTED ([10.83.44.69])
 by rcdn-core-3.REDACTED (8.15.2/8.15.2) with SMTP id 124C4e83021704
 for ; Thu, 4 Mar 2021 12:04:40 GMT
Message-Id: <202103041204.124C4e83021704@rcdn-core-3.REDACTED>
Received: from localhost.localdomain (sigmanager.vrt.REDACTED
 [10.7.89.25])
 by mail.vrt.REDACTED (Postfix) with ESMTP id 7E95E424D5
 for ; Thu,  4 Mar 2021 12:04:40 + (UTC)
Content-Disposition: inline
Content-Type: text/plain
MIME-Version: 1.0
X-Mailer: MIME::Lite 3.031 (F2.85; T2.17; A2.21; B3.15; Q3.13)
Date: Thu, 4 Mar 2021 07:04:40 -0500
From: noreply@REDACTED
To: REDACTED@lists.REDACTED
Content-Transfer-Encoding: quoted-printable
X-Outbound-SMTP-Client: 10.83.44.69, [10.83.44.69]
X-Outbound-Node: rcdn-core-3.REDACTED
Subject: [REDACTED] Signatures Published daily - 26098
X-BeenThere: REDACTED@lists.REDACTED
X-Mailman-Version: 2.1.34
Precedence: list
List-Id: REDACTED announcements 
List-Unsubscribe: ,
 
List-Archive: 
List-Post: