[Mailman-Users] Re: Cloudmark blacklist

[Stephen J. Turnbull]

Jayson Smith writes:

 > Comcast/Charter (found out about that one Saturday night when
 > trying to reply to a legit individual message) both reject the
 > message as soon as a blocked server connects,

Comcast is really bad for any number of reasons.  Unfortunately, if I
remember correctly they're effectively a monopoly ISP for broadband in
parts of the country.  Never heard of Charter.

 > Microsoft, when they decide you're evil and put you on their
 > internal blacklist, reject after Mail from:.

That's bizarre.  Even though I have no respect for the morals or
technology of Microsoft "security" in the broadest sense, the only
sensible reason for blocking at that point (rather than connect or
HELO) I can think of is that they do a lookup on the SPF record for
the domain in MAIL FROM and block based on "From alignment".  I assume
you've checked and rechecked that, but if not, check your SPF record.

 > I find these rejections quite annoying, because clearly this means
 > their spam analytics software is missing out on a lot of details
 > that could help them make a more informed decision about whether to
 > accept the message.

The quantity of spam that they're handling is mind-boggling.  In 2014,
the head of email security at Yahoo (who is very good; disclaimer, she
gave me a kitten many decades ago :-) reported to the DMARC working
group at IETF that after a different department leaked half a billion
contact lists to spammers (who used them for "recommended by a friend"
spam), they were facing sustained campaigns of more than 1 million
spams per minute.  In that context, finding ways to block on connect
makes sense.

What I don't understand is why they don't use rate-limiting techniques
where possible (ie, if they're not being DOS'ed).  For example, at
first contact, temporary failure for 15 minutes.  Upon retry (which
typically will take 4 hours in most MTAs' default configuration), it's
accepted and if not spam, it's delivered to the recipient(s) and the
source whitelisted.  If it *is* spam, you go back on the greylist with
longer and longer delays as a higher proportion of spam is detected.
If no legit mail is found, eventually you go on the blacklist.

 > But oh no, if your IP is on one of the blacklists we check,

I doubt the folks who provide email as an opt-in service (Gmail,
Microsoft) take RBLs very seriously.  They're in the business of
profiling traffic, and it makes sense and dollars to profile
everybody, customers and non-customers.  That's the only way I can
make sense of the way sometimes Microsoft will magically unblock you
after stonewalling for days or weeks.

The ISPs who provide email because that's what ISPs do aka Comcast I
wouldn't be surprised, though.

 > Go away and don't come back until you've solved your spam problem
 > that probably isn't even your problem. Goodbye!

Unfortunately email addresses aren't portable, although it wouldn't be
hard to make them so.  Sure, many customers would stick with their ISP
mailboxes despite losing mail, but for people willing to invest in
better service the big cost to switching email providers is getting
their correspondents to update contact lists.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Cloudmark blacklist

[Stephen J. Turnbull]

Jayson Smith writes:

 > What I mean is that I'd love to find a good, reliable smarthost I
 > can direct my SMTP server on my VPS to use.

You could try some of the services listed here:
Hosting: https://wiki.list.org/COM/Mailman%20hosting%20services
Consulting: https://wiki.list.org/COM/Mailman%20consulting%20services
They might have a better idea or offer exactly the service you want.

Otherwise, I think you kinda have to move your VPS to the service you
want to use, and on top of the monthlies for running a server they'll
charge you for email volume.  AWS SES for example is 10,000 emails for
$1 billed monthly, and there's a throughput charge as well but that
too is probably negligible unless you're mailing videos.  They do
promise an IP with a clean reputation and they bonk your neighbors
(and you) automatically for sending more than a tiny amount of spam,
so I'd expect it to stay that way.  FWIW 

 > The real problem I'm seeing is that seemingly within the last few
 > years, at least some VPS providers (Linode and Digital Ocean for
 > sure) have started getting entire IP ranges put on blocklists.

This is nothing new.  Effort-minimizing admins have been blocking
whole netblocks for well over a decade.  I think one new aspect is
that non-admins have borrowed the technique of mass-reporting to try
to shut down all aspects of an individual's or organization's Internet
presence.  I wouldn't block at the SMTP CONNECT level based on IP or
domain alone for the reasons you give for running your own smtpd, and
I doubt Google or Microsoft do.  But I know a lot of admins who do.

I don't know what to do about it.  I think my own server at my
university got on Microsoft's bad side once, but it got better fairly
quickly.  I did contact Microsoft but I don't know if it had anything
to do with getting off their blocklist, the only reply I got was a
'bot saying thank you for contacting Microsoft, check this link.  I
don't think they have their best minds working on the problem.
Instead they get customers by being too big to block, is my guess.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@mail-archive.com

[Mailman-Users] Re: Error member gets when sending to one of my lists?

[Stephen J. Turnbull]

Mark Sapiro writes:

 > This issue  may be 
 > relevant.

qmail.org is currently contentless but still owned by Crynwr Software.
So I would guess Russ Nelson[1] has retired from supporting qmail.  If
you're still using qmail I would migrate off it asap (free advice,
worth what you paid for it).


Footnotes: 
[1]  Once an old buddy of mine, but we haven't communicated in ~ 2
decades so   But if he charges $200/hr, the service is worth twice
that.


--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@mail-archive.com

[Mailman-Users] Error member gets when sending to one of my lists?

[Stephen J. Turnbull]

Adam Morris writes:

 > A member is subscribed to two lists I run.
 > 
 > He can post to one but gets the following error when posting to the
 > other list.

 > SMTP error from remote mail server after end of data:

The message should indicate which server rejected it.

This error depends on the message content (specifically, a physical
line longer than 998 bytes, which he surely sees as a nicely formatted
paragraph of normal-length lines).  I can't imagine why it depends on
the list, it's not hitting the list processing part.  As Mark points
out, it *could* be Mailman (more precisely, the imported 3rd party
library aiosmtpd), but would be at the very mechnical "let's MOVE SOME
BYTES!" stage of the process, not dependent on list settings at all.
It's definitely a mail server error, not a list-related error.

 > 550 Maximum line length exceeded (see RFC 5322 2.1.1).

It's very unlikely to be a recent (< 4yo) version of aiosmtpd, which
issues a different error: "500 Line too long (see RFC5321 4.5.3.1.6)".
Apple's Postfix also doesn't issue that message.  Nor does Debian's
Exim4.  The citation to the Message Format RFC 5322 is odd, suggesting
Microsoft or other commercial software (you *could* enforce RFC 5322
in a mail server, but no respectable free software does -- that's the
MUA's job).

If this was your server (and not some intermediate gateway), there
should be a log message for it.  Do you have access to the MTA logs on
your server?

So, it's possible but somewhat unlikely that the smtpds at the Mailman
host are mishandling the email (doing this *right* *at scale* is
hard).  But it's also possible that the member's MUA is busted, and is
using Content-Transfer-Encoding "8bit" (or worse, "7bit") where it
should be using "binary", The difference between binary and 8bit is
exactly passing control characters verbatim, and specifically allowing
the message composing agent to ignore line-length restrictions.  If
you have access to the message that got rejected in "source" or "raw"
form, you could check those things.

Also, some intermediate MTA may be busted and not requesting the
8BITMIME extension to SMTP, which is necessary to get some receivers
to relax the line length limitation.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@mail-archive.com

[Mailman-Users] Everything you need to know about SPF/DKIM but are too mad to ask

[Stephen J. Turnbull]

Dmitri Maziuk writes:
 > On 3/12/24 11:40, Julian H. Stacey wrote:
 > 
 > > I'm interested what independent mailman-users@ think on technical
 > > issues of DKIM/SPF,

Disclaimer:  I'm not an independent user.  I am a Mailman developer, a
participant in the development of some of the most recent
authentication protocols, and a paid or pro bono consultant on Mailman
to three organizations without which the Internet as we know it would
not exist (in some exaggerated sense, but it's true ;-).

 > It's only stopping the small mom-and-pop spammers.

DKIM and SPF are not about stopping spam.  They can't be, all they are
is authentication of sending hosts.  Most sending hosts are multiuser,
so stopping spam has to be done by filtering by recipients.

What these protocols do is provide a way to enable trusted senders to
reliably get their mail through.  As we see from the OP in this
thread, that's aspirational, you can do everything according to the
stated rules and still get blacklisted, but that's what conforming to
protocols can do in theory (and often in practice).  And in fact the
default is to trust (at least to the extent that the recipient reads
your mail to decide based on content whether it's spam instead of
slamming the door on MAIL FROM).

 > And mailman users.

Wrong.  It's *enabling* Mailman users.  If you're using email to
communicate with people who would NOT be using email if it weren't for
Minitel, AOL, Gmail, and Outlook365[1], grow up: you have to take the
bad with the good.

As long as there are legitimate mom-and-pop shops that don't
participate in authentication protocols, the spammers can infiltrate
those mail flows because those legit sources are indistinguishable
from spammers "warming an IP", as big "ethical spammers" like
SalesForce call it.  If you're not participating in these protocols,
you're helping to enable spam.[2]

I'm not saying there aren't (more or less) legitimate reasons for not
participating, at least locally.  For example, the host that I use to
communicate with students doesn't.  I did use the university outgoing
gateway at first, but I had to go to direct mail because they kept
marking my terse homework submission acknowledgement emails as spam, I
think it was mistaking the submission's Message-ID and other non-
verbal data for URLs and profiling codes.  Of course if you go up a
level that's on the university (for one thing, they refused to add SPF
and DKIM records for my subdomain).[3]

But most of the time we can do it without great cost.  Sure, it's an
annoyance, and it's tricky to get set up correctly.  But once you have
your SPF, DKIM, and DMARC records set up, and your certificates lined
up, there's very little maintenance.  The university won't give me a
certificate for my website for some reason, but so what, LetsEncrypt
will, and I don't need a cert that's trusted by people who don't know
me.  (I used self-signed for a while but LetsEncrypt is even easier.)

Right now I'm doing a 2->3 migration for a medium-size organization
that's leaving a coloc host for the cloud, and so they have to give up
their IPs.  Guess what?  SPF and DKIM means their reputation is going
to be quite portable to the new IPs.  Of course reputation at that
level is really only meaningful for recipients at -- you won't believe
this -- those big "oppressive" providers like Google and Microsoft who
can afford massive ML systems to maintain site profiles.  That's not a
benefit you get everyday, but in this situation it's big.

I get the feeling that "I'm not a spammer, why do I have to pay this
cost?" too.  But that's part of being an adult -- you sometimes have
to clean up others' messes.  The SPF-DKIM-DMARC-ARC dance is just not
a very high cost to pay for the vast majority of us, and it's not even
all that expensive to buy in the market (but I'm gonna be damned if I
don't do my own and you probably feel that way too :-).  And it's not
just Google and Microsoft that benefit.  We do too.

If you want to complain about the big freemail and corporate
providers, there are *plenty* of valid complaints.  Complete lack of
transparency, unresponsive service, failure to follow published rules,
imposing high error rates on non-customers and then blaming lost mail
on the sender, etc, etc.  But asking us to do the minimum to
authenticate if we want them to extend trust when our content triggers
a false positive isn't one of them.[4]

Steve


Footnotes: 
[1]  And you are -- the complaint was that Google forces you, but
that's wrong -- the Gmail users on your lists are the assholes for
using Gmail, OK?

[2]  And at scale: at one point in early 2014 Yahoo was receiving
sustained flows of spam over 1 million per minute, according to a
Yahoo admin I personally trust because she gave me a kitten once. :-)
She reported that that campaign didn't even try once Yahoo put a
p=reject DMARC policy in place.

[3]  I do have some sympathy for the postmasters because "it's always
September on the Internet."

[4]  

[Mailman-Users] Cloudmark blacklist

[Stephen J. Turnbull]

Jayson Smith writes:

 > I'm getting really tired of these unexplained blacklistings. Does
 > anyone know of any reliable outgoing Email service providers? 

What do you mean by that?  Gmail for example allows you (or did allow
you 18 months ago) to validate an alternate address through the usual
"can you read this mail and send back a cookie" dance, and use those
validated addresses in From.

Unfortunately, in my experience at least Gmail won't allow you to use
a non-gmail address in From unless you're using their app or browser
client.  Authenticated SMTP to port 587 doesn't cut it for whatever
reason.  The best I could figure out was sending through eg gmail
using From: m...@gmail.com and setting Reply-To.

 > Ideally I want to continue to handle my own incoming Email because
 > I don't want someone else's spam blocking software deciding what
 > Emails I receive.

I don't know of freemail who allows that, unfortunately.  The closest
I know of is Google, as above.
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@mail-archive.com

[Mailman-Users] Re: Outlook blocked again, but strange response

[Stephen J. Turnbull]

Jayson Smith writes:

 > Update: Sometime in the night, my IP was silently removed from
 > Microsoft's block list. I've never had that happen, but all's well
 > that ends well, at least for now.

Thanks for the update.

Yes, that happens, and for those of us trying to support you all, this
lack of transparency on the part of the big email providers is one of
the most annoying aspects.  It leaves us (and you) kind of helpless.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@mail-archive.com

[Mailman-Users] Re: Non-delivery of approved messages

[Stephen J. Turnbull]

Francis Jayakanth writes:

 > Transforming to a hosted service would alleviate the technical
 > burden and ensure the smooth operation of the list. If anyone has
 > recommendations or experiences with reliable and affordable hosted
 > services for Mailman list management, I would greatly appreciate
 > your insights and guidance.

For reasons I don't understand (I guess the vast majority of our
subscribers administer their own hosts?), this kind of request rarely
gets much response.

There is a substantial list of hosting services at
https://wiki.list.org/COM/Mailman%20hosting%20services.
The ones who list prices all look pretty "reasonable" to me, but YMMV.

Of those, EMWD (near the top) put a lot of effort into its offerings
including writing completely new front ends for administration and
archiving.  I like ours better, but theirs are slick and definitely
simpler.  Brian (the founder) put substantial effort in on our lists
helping users directly and was circumspect about mentioning his own
company.  He also produced excellent documentation for setting up a
dedicated Mailman 3 server.  The management has changed since due to
Brian's untimely passing, and they're not as hacking-oriented as he
was, but I've seen nothing that diminishes their reputation for
service.  They have been responsive to our occasional inquiries, but
they're much less present here than before.

IIRC, mailman3.com and mailman4.com are run by the same company.  I'm
pretty the principal is a frequent contributor to Mailman development,
both directly and as a mentor in GSoC (though not a core developer).
I can't say anything about service from experience or reputation here,
and he's less present on the user lists than Brian was.

I would suggest that you start by looking at that list.  (Note that
the order is chronological, oldest listings at the top, and I wouldn't
attach much if any significance to order.)  For your situation as you
describe it, I would suggest looking at those where the focus is
clearly on Mailman first.  Folks who offer generic hosting can easily
offer Mailman, but in that class service is spotty.  Some are very
good, I'm sure (and for that reason their customers don't show up here
to compliment them! :-( ).  Others are not so great for Mailman, some
even say "we just install Mailman and cPanel the rest is up to you".
If you have interest in a few specific providers, you could ask here.

It might be useful to specify you want private answers.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Mails are sent in tranches of 100

[Stephen J. Turnbull]

Andreas Grupp via Mailman-Users writes:

Note that this is the wrong list for Mailman 3.  This list is for
Mailman 2, and you are likely to get inaccurate advice here.  You
should use mailman-us...@mailman.org in the future.  (Don't move this
thread there, I think it's mostly over anyway, and we should keep the
thread together.)

 > No, Postfix on the same machine and is sending several thousand
 > mails in 8 hours. Sending for example 600 mails, coming from a
 > forum in a Learning Management System, is done in one step in only
 > a few seconds.

Postfix is quite capable of permitting a particular user to exceed
limits placed on other users.  Seems unlikely in your case, but keep
it in mind as a last resort.

It's also possible the LMS system is specifying 600 recipients to a
single message in one session.  Mailman can do that, but you have to
turn off the personalization features of Mailman.

 > I can see in the mail log that there are only 100 mails delivered
 > from Mailman3 to Postfix.

Here are the relevant Mailman mail delivery configuration parameters.
(If you want multiple recipients per session, you also have to turn
off personalization for that list.)  Note: I believe that in the
comments quoted below "session" and "transaction" are the same thing,
the set of SMTP commands starting with MAIL FROM and ending with the
period at the end of the DATA command.  These are defaults set in
site-packages/mailman/config/schema.cfg:

# Ceiling on the number of recipients that can be specified in a
# single SMTP transaction.  Set to 0 to submit the entire
# recipient list in one transaction.
max_recipients: 10

# Ceiling on the number of SMTP sessions to perform on a single
# socket connection.  Some MTAs have limits.  Set this to 0 to do
# as many as we like (i.e. your MTA has no limits).  Set this to
# some number great than 0 and Mailman will close the SMTP
# connection and re-open it after this number of consecutive
# sessions.
max_sessions_per_connection: 0

If I understand correctly, the maximum number of recipients that will
be sent out before closing the connection is
max_recipients * max_sessions.  The default settings should allow
Mailman to make one connection and send out all of the recipients for
that message, but if you have large numbers of recipients going to the
same domain (eg, GMail or GMX) and personalization is OFF, you can
speed things up dramatically by raising max_recipients.

Warnings: If personalization is on (for example so you can put
personalized unsubscribe URLs in the footer), this won't help because
that forces session-per-recipient performance anyway.  You should also
be aware that some hosts will decide you're a spammer if you try to
send to all your subscribers at that host at the same time.
Personally I would probably not mess with max_recipients.

# Maximum number of simultaneous subthreads that will be used for
# SMTP delivery.  After the recipients list is chunked according
# to max_recipients, each chunk is handed off to the SMTP server
# by a separate such thread.  If your Python interpreter was not
# built for threads, this feature is disabled.  You can explicitly
# disable it in all cases by setting max_delivery_threads to 0.
max_delivery_threads: 0

These defaults serialize all of the recipients.  If you have
personalization on, this means that there will be one recipient in
each session, and the messages will be be sent one at a time until
done, or the MTA breaks the connection.  In the latter case Mailman
will open a new connection, and repeat the process until done.

You can probably speed things up by bumping up max_delivery_threads
(check that Python supports threading).  However, you probably should
do this gradually so that Mailman doesn't crowd out your other mass-
mailing clients.

Any of these changes should be made in mailman.cfg, which probably is
in /etc/mailman3/.  If these parameters are not set in mailman.cfg,
most likely your MTA is doing the throttling, but you can check
schema.cfg to see if maybe Debian messed with the defaults.

 > The only thing I've changed to the standard Debian 

Unfortunately, Debian frequently lags current releases substantially,
as well as making plenty of changes on your behalf that all too often
are confusing for upstream developers.  The variables above are the
kind of thing that Debian frequently messes with, although I would
expect those changes to be visible in the mailman.cfg config file.

 > configuration was due to error mails coming form Django. So I
 > inserted to /etc/mailman3/mailman-web.py at the end the following
 > lines:

QCluster is irrelevant to post distribution, it only affects
archiving.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/

[Mailman-Users] Attach footer only once?

[Stephen J. Turnbull]

Tim Houseman writes:

 > Is it possible for footers to only be attached once?

They are only attached once. ;-)  The part that's annoying your
subscribers is included by your posters.  A mailing list should not
touch the message body provided by the poster, unless there's a
specific rule against it (such as executable programs and pornographic
images) -- that's too likely to cut important, desired content.

 > [T]he footer just continually stacks up at the bottom of subsequent
 > emails. Is there any way to avoid this scenario?

Get more courteous subscribers! ;-)

Otherwise, not generically, short of plugging in a well-trained LLM.
The problem is that quoting conventions etc differ across mail
clients, as do signature blocks.  If the signature appears below the
quoted footer, it will become quite complex to decide how much to
remove, especially since some "modern" (aka nonconformant) clients
don't conform to the traditional signature inclusion conventions.  On
a legal list, removing any disclaimers included in signatures is not
likely to make posters happy.

Given how obvious and regular the pattern is to the human eye, it's
certainly possible to do a very good job with a particular list and
its specific footer, but I think you'll need to hire a consultant for
that.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@mail-archive.com

[Mailman-Users] Re: Attach footer only once?

[Stephen J. Turnbull]

Carl Zwanzig writes:

 > And unfortunately, solving "people problems" with technology seldom
 > ends well.

I find that using appropriate technology often helps me be a less
problematic people. :-)

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@mail-archive.com

[Mailman-Users] Re: Security features of Mailman

[Stephen J. Turnbull]

rich...@karmannghia.org writes:

 > It's very straight-forward:

C'mon, man, Grandpa knows how to tie his shoes.  The construction of
such an encrypted list not technically terribly complex---as you said
yourself, a SMOC.  The problems are describing *who* is the adversary,
*what* will they do to invade your privacy, and *how* does the
proposed system thwart those threats.  You are completely ignoring
those questions.  And no, "unencrypted mail is a threat" to
"everybody" isn't a serious attempt to address them, given that almost
everyone is using MTAs that support TLS nowadays.

 > Subscribers who want encrypted email include their public key in
 > their subscription details,

What about the needs of *posters*, who are *at least* as important as
subscribers here, who want to keep *their* posts private?  That's why
"encryption-optional" lists make no sense to me, except as a proof of
concept.  And the prescription to greppers to leave their mail folders
unencrypted is not comforting to the authors, either.

I see vanishly small added security in an encryption-optional mailing
list of the kind Juergen described.  As a proof-of-concept, it was a
brave experiment that maybe could have led to something.  But it
didn't.

 > HOWEVER, just becasue ONE email list of a group who realized it was
 > there had that experience says NOT A THING about how many who
 > didn't know would love to have a list where users HAD to use
 > encryption to be on the list!
[...]
 > It's myopic to see just one's own use case and think it applies
 > across the board.

Round 'em up, man.  I listen to the community.  I'm listening to you.

 > Over my long and storied 47 year career in computer science I've long 
 > noted that the vast majority of users:
 > 
 > 1) Don't know what they really want;
 > 2) Don't have a clue what's easy and what's hard, and;
 > 3) Don't hang out on email lists like this one.

So?  I think they *do* know a very large fraction of what they want at
the level of expressing *requirements* (WIBNI ...).  Dealing with
what's easy and what's hard is our problem as developers, not theirs.
With that knowledge, we can help them refine and prioritize their
requirements, and sometimes discover new ones.  Convincing them that
we understand their requirements and know easy vs. hard is also our
problem.  And the lack of like-thinking users on mailing lists like
this one is a problem for advocates (like you?)

 > > But there are substantial technical hurdles to extreme
 > > requirements such as "end-to-end encryption" of list traffic.
 > 
 > That is abjectly false, Juergen proved it, and not only was it NOT
 > difficult 20 years ago, in the 20 years since then what's fairly
 > easily possible has expanded considerably.

Your definition of "end-to-end" is not the one in common use.  A
system where an intermediate node decrypts, then reencrypts and
forwards, is not "end-to-end encrypted" in any usage I've seen before,

It's really not useful to discuss technical issues if you won't at
least use the accepted definitions of such critical terms.  You're
welcome to argue that given the threats you perceive, it's not an
important requirement for an encrypted mailing list.  But given the
ease which which systems are penetrated these days, I disagree for
most purposes I can think of.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Security features of Mailman

[Stephen J. Turnbull]

Juergen Dollinger writes:

 > We tried encrypted lists some years ago. Have a look at
 > http://non-gnu.uvt.nl/mailman-pgp-smime/

Thank you for describing your experience!
The people side is always hard.  I'm not unhopeful though, but it's
going to take work, especially good design.

 > The idea is that there is a key for the list, the server decrypts
 > the E-mails and encrypts it for the recipients who have supplied a
 > key. Worked fine with that old version of Mailman 20 years ago.

That's exactly how I would do it, except you wouldn't receive posts
until you submitted a key.  Having half the copies vulnerable
on-the-wire and on-disk would not fill me with warm fuzzy feelings.
:-) 

I think this could be fairly useful in environments where people are
paranoid enough to leave the mail encrypted on disk.  But even the DV
case that I mentioned -- would it stay encrypted for long if a few of
the abusers discovered its existence?  It would only take one!

 > But even in our quite nerdy environment only about the half of the
 > subscribers submitted a key for the list. (excuses are like 'I want
 > to use grep(1) for fulltext search in my list E-mails')

Today I don't think that excuse would fly, machines are fast enough
that few email bodies would take noticable time to decrypt, and
languages like Python and Perl provide very high quality email
processing libraries.  p-grep-p would be written real fast, and it
would work fast too.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Non-delivery of approved messages

[Stephen J. Turnbull]

Francis Jayakanth via Mailman-Users writes:

 > The list is running on Mailman 2.1.26.

If possible you should upgrade Mailman, currently at version 2.1.39.
Almost all of the releases since 2.1.26 are primarily oriented to
"security" and "reliability" issues.  I don't recally any offhand that
would help alleviate your current problem, but who knows?  And the
changes address genuine vulnerabilities that have been exploited at
many sites in the past.

 > During the last couple of weeks, approved messages are not being
 > delivered to the list members. When I check the Email &
 > collaboration alerts in the Microsoft 365 Admin Portal, I see the
 > following message:

Do you mean that Microsoft provides the email services to your
organization or to a host that your organization uses, and they're not
letting *your* traffic go out to *anyone*?  That's a new one to me.
"Shocked but not surprised", as the saying goes.

Your problem is with Microsoft.  You need to get their help,
especially if they are providing your email services.  They do not
tell us why they block some messages or some users rather than others,
and it often seems completely random.  Here it sounds like the
"suspicious activity" is sending lots of mail to many addresses.  Ie,
you're being throttled *because* you're running a mailing list.  But
you'll have to negotiate with them if that's what's happening.

The generic recommendations are

1.  Get your users off Google, Yahoo, Verizon, and Office365.  Yeah, I
know that's "impossible", but it's the single most effective
method for improving the experience of mailing list subscribers.
2.  If you manage your own mail system, make sure your DNS records for
SPF and DKIM are in order, and that your MTA is configured to sign
messages properly.  If somebody else is responsible for that, ask
them to check and fix any problems.  (This applies to 3 throuhg 7
below, too.)
3.  Implement DMARC on your system.  This doesn't have a standards-
based effect on posters whose email accounts are not on your host,
but it may enhance your site's general reputation.
4.  Implement the ARC (Authenticated Received Chain) protocol.  The
basic idea is that any change your mailing list makes such as
adding list tags to the Subject or an organizational footer to the
body of the mail will break the DKIM signature that attests that a
valid user on the original sending system (the author or "From"
address) sent the mail.  ARC allows you to say (in a way that some
email software understands) "We validated this message, the
signatures were in order, and if it turns out to be spam or
whatever you can blame us.  Here's our signature so you can
believe it."  I don't know offhand if Microsoft implements it or
if they put much weight on it, but it can't hurt.  Google and
Yahoo do put a fair amount of trust in it, I'm told.

The following advice is generally good, but it may not be influencing
your current problem.

5.  Put spam filters on your *outgoing* mail.  Folks are of two minds
about this (if you're primarily a mailing list site and you filter
incoming, you won't catch anything new going out), but sometimes
you can catch things going out that you wouldn't catch coming in.
This is especially true if you use Bayesian or "machine
learning"-based spam filters so you can use experience to tell
them in the context of the mailing list whether it's spam or not.
6.  Check your moderation queues ("held messages") for unusual amounts
of spam -- some of it may be getting through to your subscribers.
7.  Check the subscription queues ("awaiting address confirmation")
for an unusual number or a pattern of particular addresses being
subscribed to a large number of unrelated lists.  Unfortunately,
the confirmation process can be abused by malicious third parties
to send large numbers of address confirmation notices to unrelated
addresses, thus clogging their mailboxes.  Not only is this
annoying and even a denial of service to the victims, but it also
can hurt your site's reputation.

I'm sorry I can't be of more help, but the large providers are far
more sensitive to any spam that gets through than they are to lost
mail, because they can blame lost mail on the sender, while users
hold them responsible for spam.

Regards,
Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Security features of Mailman

[Stephen J. Turnbull]

Hi Richard,

Thanks for your followup.  Although you suggested taking the
conversation private, I want to at least continue for one more post
because it's not clear to me that there's demand for this service that
is enough to justify the likely substantial cost of development.  I
don't know how to implement poster-to-subscriber encryption on the one
hand, and on the other hand most list owners are unable or unwilling
to manage their own hosts, and therefore have to provide the ability
to decrypt list traffic to a third party.

But I could be quite wrong, and it's the members of the Mailman
community who are most likely to provide a critical mass of users to
say "that's just what I've always wanted, and here are the features I
want".  If that critical mass doesn't show up, then I want to use my
time (and resources such as summer interns) on features that our
community *does* want.

The fact that nobody but you has raised their hand to say "I want
this" or "my list owners would really like this" is not a point in its
favor.

However, let me say to anybody who has read this far that although I
have played and continue to play the devil's advocate against
"privacy-preserving mailing lists", I want everybody to understand
that I am very much in favor of privacy.  But there are substantial
technical hurdles to extreme requirements such as "end-to-end
encryption" of list traffic.  On the positive side, reduced-
expectation implementations with encryption on the wire both
directions and in queue files, for example, which would pretty much
eliminate attacks on data-in-motion and somewhat reduce the attack
surface for data-at-rest (eg, in queues on disk on the list host and
possibly even in archives) might be useful in some applications.  I
would be happy to discuss such applications, and the cost of
implementing them in Mailman (ie, how much writing and rewriting of
code).

rich...@karmannghia.org writes:

 > [M]y only real interest here is on the accessibility (privacy)
 > aspect, and list-management software like mailman DOES have
 > something to say about architectural details of implementation for
 > an encrypted email world.
[...]
 > A straight-forward approach might be, for example, that the entire set of 
 > data transferred between two MTAs is encrypted save the destination 
 > system's network address and the destination port.

I believe we already have that in some profiles of IPSec.  But that
has nothing to do with mailing lists, and I'm not real interested in
going into MTA implementation.

 > The MTA decrypts

Then the MTA is an agent-in-the-middle, and an obvious point of attack.

 > It's entirely reasonable to design the architecture so the list
 > management code has optional access to the body of an email,

Now the list manager is an agent-in-the-middle and a point of attack,
too.  Both are very likely unacceptable in the case of a large-scale
hosting service, without a massive effort to design and implement a
key management system that can restrict access to keys to authorized
uses.  Absent such a system, the operator of a mailing list would also
need to be a system administrator with security qualifications to
match perceived threats.

 > All of this and much more is not particularly difficult to
 > accomplish from a technical point of view - it's only a "Small
 > Matter Of Code" as Michael Stonebraker (Turing Award Winner)
 > famously says.

As I pointed out above, a lot of it is already implemented -- but not
in MTAs/MDAs, MUAs, or mailing lists.  All of which have multiple
implementations and those implementations must agree exactly, or
somebody is going to get hacked.

 > From the point of view of a mailing list, if the list itself is
 > functional then there's no issue with traffic analysis insofar as
 > legitimate analysis of list traffic is concerned.

I'm not sure what you mean by "traffic analysis".  I am suggesting
that membership in some encrypted lists will be considered privacy-
relevant, and therefore mail being delivered from what is likely a
single-purpose host to a mailbox infringes privacy of the mailbox's
owner.  Consider for example a list for victims of stalking or
domestic violence.  Knowledge that a target is subscribed to such a
list could trigger violence on the part of the abuser.

 > One would imagine that in a properly architected system designed to
 > encrypt various email headers, the design would account for
 > email-list-pertinent options.

In the mailing list context, the fundamental question of "properly
architected" is "who has which keys and how do they get them?"
Mandatory TLS solves the problem automatically for a single TCP
connection.  This can be generalized to automatic secure key exchange
for all members of a (sufficiently small) synchronous group chat in
the chat software.  I don't see how it can be generalized to a mailing
list which is fundamentally asynchronous, should deal with recipients
whose MTAs almost certainly don't participate in such key 

[Mailman-Users] Re: Security features of Mailman

[Stephen J. Turnbull]

rich...@karmannghia.org writes:

 > ...I would hope that all netizens are fully aware (and obviously
 > not all are) that there is not and cannot be such a thing as "safe
 > environment for email discussions" with email as now practiced and
 > to create it requires a serious overhaul of the way email is
 > conducted.

My original point was I feel perfectly safe here on Mailman lists (of
course I am in a position to get people banned, so I am in fact safer
than the average bear, though I would not mess with a Kodiak).

 > It doesn't have to be this way: email bodies and even the
 > destination username and other parts of email headers COULD be
 > encrypted when enroute via the same mechanisms as we have long used
 > for secured web sites,

True, and in fact many sites implement the enroute part, it's called
mandatory TLS.  I would imagine the proposals to make traffic analysis
more difficult would apply here too.

 > and even end-to-end encryption isn't too difficult to implement,
 > and I'd lay a substantial bet that an open-sourced effort
 > harnessing the ideas of DKIM / SPF / DMARC could easily and simply
 > accomplish this.

I've thought a lot about this, it has been proposed multiple times as
a GSoC project for Mailman, and this is simply not true for mailing
lists as implemented in Mailman.  In particular, it's simply not
possible to achieve end-to-end encryption as a mailing list function.
The list has to have access to the session key to give access to that
key to subscribers, at which point you've been hacker-in-the-middle-d. 
I can imagine applications where you're willing to trust the list,
though, and if there were demand for that, I'd be willing to supervise
a GSoC student who wanted to implement it.

Note that it is certainly possible to have end-to-end encryption of
list email, but it requires that each poster have all the subscriber
keys.  I guess you could marry a keyserver with a mailing list, and if
you want to call that "end-to-end encryption via mailing list" go
ahead, but you still have to solve the problem of getting posters to
keep their keyrings up to date, so I consider that "not a mailing list
function".

And of course you only asked for security of "data in motion", but
then you've got the harder problem of securing data at rest (which
also requires cooperation from either recipients or from their MUAs --
buwhahahahaha!)

 > However, the simple (and for me painful) truth is that The Powers
 > That Be _obviously_ do not want us to have secure
 > communications. Their excuse is fear ("terrorism!") and their more
 > dominant motive is profit. It's truly as simple is that.

It's not that simple though.  While you're gonna need some *serious*
booking up before you can win that substantial bet ;-), it would be
possible (and has been done, cypherpunkery is real!)  The problem is
that we don't want it as bad as the cypherpunks did.  So far we've
been able to resist laws that require backdoors (who knows how many
backdoors are there by bribery or other skullduggery, but it's not
*legal*).  So for some things we can win.  But if we want really
secure mail, as secure as for financial networks (which aren't perfect
but they do OK), we're going to have to pay for it, and the average
bloke isn't interested.  They'd rather be outraged when their secrets
get blabbed and their brother-in-law who actually did the dirty deed
says "wasn't me, was some 400-lb-hacker-in-Mom's-basement".

 > Anyone who thinks their unencrypted emails are in any way secure on
 > the open internet is, unfortunately SADLY mistaken.

This is true.  Security by obscurity works up to a point, but if you
ever get targeted by the FBI you're toast.

 > P.S. PERHAPS someone reading this has the energy and gumption to
 > change this?! I sure hope so! ...I've been using email for 47 years
 > now, I did my part, I tried hard, it's up to younger generations to
 > carry it forward now. But I'll be happy to assist anyone else's
 > efforts on this!

I'm not volunteering for the hacking part, but if somebody eligible
for GSoC wants to propose it, and the mentors like the proposal, I'll
mentor it.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Security features of Mailman

[Stephen J. Turnbull]

Christian Buser via Mailman-Users writes:

 > Please don’t feed the trolls...

I don't have time to visit trolly websites, either.

Lucky me, this time! :) :) :)
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Security features of Mailman

[Stephen J. Turnbull]

Sron Dev writes:

 > Please let me know which platform you'd like me to elaborate on,

Me personally?  None.  I don't have time for theoretical discussions.
Maybe somebody else does, you're welcome to pick a platform and hold
forth (but see below for venue, this is almost certainly not the right
list for that).

If you, or anyone else, has a specific feature request, or requirement
for "ensur[ing] a safe environment for email discussions", I'd be
happy to discuss what we can do about that.  That discussion should
move to mailman-develop...@python.org or mailman-us...@mailman3.org
depending on whether the RFE is fairly clear (-developers) or if it
needs refining by input from other users (-users), as new features are
not going to be added to Mailman 2 (the subject of this list).

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Security features of Mailman

[Stephen J. Turnbull]

sanjay jangam writes:

 >  Can you elaborate on the security features of Mailman and how it
 > ensures a safe environment for email discussions?

No.  The question is poorly posed.  You need to say against what
threats you want to protect your discussions.

I can say this much: Mailman 3 does try to provide good authentication
of users and a consistent model of authorization for access to its
databases (user profiles, list user rosters and configuration).
Mailman 2 does not.

Neither Mailman 2 nor Mailman 3 tries to provide cryptographic
security (encryption or authentication) of list traffic, on the wire
or at rest (ie, in archives).

Other than that, you have to define what you mean by "safe", and then
I can tell you whether Mailman can satisfy that requirement and what
you have to sacrifice to get it.

Steve




--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@mail-archive.com

[Mailman-Users] confirm mails / administrative mails are discarded in mailman3

[Stephen J. Turnbull]

Georg Schröder writes:

 > can someone explain this behaviour to me ?
 > 
 > - I upgraded from 3.3.5

This list is specific to Mailman 2.  Some of the Mailman developers
are reading it, but some don't, and the majority of Mailman 3 users
don't.  For broadest audience you should post Mailman 3 questions to
mailman-us...@mailman3.org.

 > to 3.3.9 recently, before that it was working

How did you upgrade?  If it was via a distro package you should ask
them too.

 > ==> mm/var/logs/mailman.log <==
 > Jan 02 12:35:18 2024 (27084) 
 > <787c5824ca33956449ba541547e2c...@posteo.de> Precedence: list message 
 > discarded by: testlist-dev1-request@x

Check the "vette" log and see if there's a reason there,

 > also when i do a subscription in the web-interface and uncheck the "Pre 
 > Confirm" checkbox, there is no confirmation request sent out.
 > welcome messages are sent out with this, also invitations )

Are any confirm messages being received for any subscription or
address confirmations?  If it's only for your domain, check your spam
filters.  They sometimes identify the one-time key as a spammer
device.

I don't know of any reason why Mailman would change in these ways,
I have several times experienced nondelivery issues when other
components of the mail system where upgraded at the same time.  Could
be Mailman, but nothing I've seen before.
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Emails not being sent, but showing up in archives?

[Stephen J. Turnbull]

Seeds of Success writes:

 > For the last two months or so, the emails that I send out on the
 > mailman list do not get sent out to the subscribers, but do show up
 > on the archived threads. I am not very mailman savvy, and have
 > recently taken over admin roles from someone else. Any guidance on
 > how I might address the issue would be appreciated!

1.  Have you tried restarting Mailman?
2.  If you did, and that didn't work, did you try rebooting?
3.  Is there anything in any of the queue directories
($var_dir/queue/*)?  $var_dir is often /var/lib/mailman, but other
directories are used.
If they are in .../shunt or in .../bad, there's something complex
going on, but if they're in the other queue directories restarting
Mailman should clear them out.
4.  Is your MTA running (if other mail is going out, it is)?
5.  Is mail piling up in the MTA queue (most mail systems have a
"mailq" command to check this)?
6.  Is it happening at a specific recipient domain?  Gmail (and maybe
other Google sites) are famous for suddenly deciding to eat all
your list mail without telling you.  They're not the only ones who
do it, just particularly famous.
7.  If you have a smarthost (outgoing mail gateway) configured, check
the same things as for 4 and 5.

There are other things to check but that should keep you busy for a
while.
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Issues with a DMARC record leading to message being shunted

[Stephen J. Turnbull]

Ralf Hildebrandt via Mailman-Users writes:

 > The DMARC record for somenet.org is
 > "v=DMARC1;p=none;pct=100;rua=mailto:postmas...@somenet.org;mailto:postmas...@somenet.org;ri=3600;fo=1;;
 > which is syntactically incorrect. The extra
 > ";mailto:postmas...@somenet.org; is wrong, I guess the tag "ruf="
 > is missing here!

Seems very likely.

 > It should probably read 
 > "v=DMARC1;p=none;pct=100;rua=mailto:postmas...@somenet.org;ruf=mailto:postmas...@somenet.org;ri=3600;fo=1;;
 > 
 > So, should mm3 somehow "catch" this error somehow?

Yes.  The parser is in the authheaders package, a 3rd-party package on
PyPI.  I'll get in touch with the maintainers, but given the season I
wouldn't bet on a release soon.  Of course you may get lucky with an
upgrade if you're not up-to-date (v0.15.3).  Upgrade should be safe,
the maintainers are reliable authors in this field.

The rest of this post is for RFC nerds. :-)

 > I wonder how a syntactically incorrect DMARC record should be
 > handled at all

Per RFC 7489, Sec. 6.3:

A DMARC policy record MUST comply with the formal specification found
in Section 6.4 in that the "v" and "p" tags MUST be present and MUST
appear in that order.  Unknown tags MUST be ignored.  Syntax errors
in the remainder of the record SHOULD be discarded in favor of
default values (if any) or ignored outright.

I'm pretty sure the best strategy is to divide the string on ";",
check that the first component is exactly "v=DMARC1", the second is a
"p=..." specification.  If either of those fails, the whole record
should be ignored (but the receiver can go ahead and do DMARC
processing and make its own decisions based on the result).

After that, any component that is not of the form "tag=value" for a
tag that is defined in the RFC and a value that is syntactically valid
for the tag should be ignored but parsing continues for the remaining
components..

There is no default for ruf, so IMO you must ignore it:

ruf:  ... If not provided, Mail Receivers MUST NOT generate
failure reports.

Although the evidence for ruf=mailto:postmas...@somenet.org is really
strong, given it's the same URL as rua and fo=1; best effort might
apply.

 > (ignore?

The whole policy?  Definitely not, unless the "v" and "p" tags are
missing or syntactically incorrect.  Of course it's optional for
receivers to participate, but if you participate, best effort is
indicated (Sec. 6):

A Mail Receiver implementing the DMARC mechanism SHOULD make a
best-effort attempt to adhere to the Domain Owner's published DMARC
policy when a message fails the DMARC test.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: spamming

[Stephen J. Turnbull]

Jim Dory writes:

 > I've started getting these spamming attacks again so thought I
 > would dive into trying this recaptcha. I got the keys for V2
 > recaptcha from google and put the 2 lines at the bottom of the
 > mm_cfg.py with proper keys from google. Spelling double
 > checked. After saving the file, I can't log into the web interface
 > of mailman - I get a Bad Request error page. I commented out the
 > RECAPTCHA_*_* lines and could then access the admin web pages
 > again.

There's a lot missing here.

1.  What version of what operating system are you using?
Ubuntu and Debian are likely to require some hoop-jumping to get
the needed software installed.
2.  What version of Python are you using?
3.  What version of Mailman are you using?
If it's recent enough, the listinfo.* pages will include a tag
"" which does all the heavy lifting for you.
4.  How did you install Mailman?  Preinstalled on a cPanel host, from
the OS, from source in a virtual environment, other from source?

 > web admin pages. What would I add and to which files? I don't see
 > list_info under  /usr/local/cpanel/3rdparty/mailman/Mailman/ .

Try `find /usr/local/cpanel/3rdparty/mailman -name 'listinfo.*'`
and you should see a bunch of them.  Most likely you are only
interested in
/usr/local/cpanel/3rdparty/mailman/templates/en/listinfo.html
and maybe the .txt version of that file if it exists, but if you offer
other languages to your users, you may need to deal with the
$TWO_LETTER_LANGUAGE_CODE/listinfo.* versions for those languages.


--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Updated Drupal user mailman register module (announcement)

[Stephen J. Turnbull]

Russell Clemings writes:

 > Because I needed it for a project, I've just updated the "user mailman
 > register" module for Drupal 9/10.

Thank you for the work and for letting us know about it!

Yes, IWBNI "somebody" would do the Mailman 3 port, and if you're
thinking about it, feel free to get in touch with us (and are you able
to consult on such a project even though you can't work on it at the
moment, Russell?)

Since that would be Mailman 3, the appropriate venue for discussion of
such a project is mailman-develop...@python.org, not this list.
Reply-To set.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] "This is an ex-Python"

[Stephen J. Turnbull]

Hi all,

I had occasion today to try to duplicate a Python 2 application on a
fresh install of Debian 12 "Bookworm".  It was a bit of a lift, and I
realized just how deprecated Python 2 is.  Properly speaking, Python 2
"is no more", "has ceased to be", "bereft of life, it rests in peace",
and adding insult to injury "this is an ex-parrot".

Specifically, there's no Python 2 .deb, and I'm not even sure Python 2
branches are still in the GitHub repo (probably, but it was easier to
grab the tarball).  Ditto, no .deb for OpenSSL 1.1.1w.  There's no
pip or ensure_pip, and get-pip.py no longer supports Python 2.  Cloned
the pip repo, but then setup.py doesn't work without setuptools.
Cloned that one too, fortunately it has a bootstrapper so that built
and installed, and then so did pip.  Then it turned out I only needed
one non-stdlib package, dnspython -- which doesn't support Python 2
anymore, so built that one from git, too.  I have to wonder if pip is
usable at all, since you'll need to specify the most recent version of
each package that still supports Python 2, assuming PyPI keeps them
around.

So ... make a copy of your Ubuntu 16 distribution disks and your
current site-packages, you're gonna need them if you ever need to
migrate your Mailman 2 to a new host.

In case you're wondering, I kept careful notes of the process as I got
it working, including archive and repo URLs and release tags to
checkout.  I'll post it to the wiki at some point.

Of course, you can avoid all this by migrating to Mailman 3!  It's
less work. :-)

Regards,

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Excessive or fatal bounces issue

[Stephen J. Turnbull]

Florin Pasăre writes:

 > I would like to ask you where the system

What do you mean by "the system"?  Mailman?  The mail servers (these
are the programs that exchange mail between hosts on the Internet)?
Your mail client (the program you use to read and send mail)?
"Something" between the author's fingers and your eyes?

 > gets the names you see in the "sent by" field,

What "sent by" field?  Do you mean something you see in your mail
client?  Maybe the From field?  (There is no standard "sent by" field
for email,)

 > because sometimes the name doesn't make it clear who is sending the
 > email and we would like to know if it is something we can control
 > or change.

The mail standard RFC 5322 specifies a "From" field which contains the
address of the author and optionally a display name for them.  This is
set by the author's mail client, and according to the standard should
not be changed by anyone else.  However, (1) mail clients such as
Outlook and Apple Mail frequently change, interpret, or add random
stuff to the From field when they display the author's identity, and
(2) there is an anti-phishing protocol called DMARC according to which
a sending system can request rejection of mail claiming to be from
that system unless a valid digital signature is present.  Since some
things that a mailing list does (like adding unsubscription information
as legally required in many places) break the signature, Mailman has
an option to change the From field from "Some User "
to "Some User via This List ".  This DMARC
mitigation is only used if the list owner explicitly configures it.

 > Another question is where the system gets the name of the person who
 > appears in the "reply to" field.

According to RFC 5322, that field also should only be set by the
author of the original message.  However, nonconforming behavior is
common, and under certain circumstances Mailman will set that field:

1.  By default, Mailman will not touch that field, but will pass it
through if it exists.
2.  When configured to do so, Mailman will add a fixed address
(usually the list's posting address) that the list owner
specifies to Reply-To.
3.  Alternatively, when Mailman is configured to change the From
address to mitigate rejections at subscribers' systems due to the
DMARC protocol, Mailman will add the original From address to
Reply-To.

I think in the case of setting Reply-To to an specific address, the
list owner may specifically ask Mailman to remove other addresses.  I
don't think that any mail servers (such as Postfix or Exchange) ever
touch Reply-To.

 > I've already seen wrong/different than what was expected names
 > twice using Mail on Macbook.  The message is sent, for example, by
 > Mario Rossi, and the system adds his email address to the cc.

I don't recall for sure if there are circumstances where Mailman
changes Cc, but I'm pretty sure it never does.  I don't think that any
mail servers (such as Postfix or Exchange) ever touch Cc, only the
originating mail client does.

 > The problem is the reply-to field, because there one sees the name
 > of, for example, Claudio Bianchi, which has nothing to do with this
 > message and so I don't understand why his name appears above the
 > general address of the mailing list.

What do you mean by "above"?  Please quote exactly what is on screen.
(Sorry, this list doesn't permit screenshots.)

 > With MS Outlook instead what one sees in the sent-by field is the
 > following: list name list-boun...@domain.ext on behalf of XXX via list name
 > l...@domain.ext.

By "sent-by field" do you mean the "From" field?

That sort of looks like something Mailman might produce, but not
exactly (which means Mailman did *not* produce it).  I believe in the
case of DMARC reformatting Mailman produces the "XXX via list name
" portion, but it will *definitely* not put
"list-bounces" anywhere in From.  Check that you are quoting exactly.

If so, I think that is Outlook hallucinating.  Outlook definitely
displays garbage under some conditions, and I think I remember this
particular problem being an Outlook misfeature.  Specifically, it
grabs the content of the "Sender" field and puts it before "on behalf
of".

 > Is it possible to have l...@domain.ext instead of
 > list-boun...@domain.ext?

The  is the address Mailman puts in the
"Sender" field (normally not displayed by mail clients).  If Sender is
present in the mail header, mail servers send various administrative
messages to Sender rather than From or Reply-To, including
non-delivery messages.  You do not want to change the "Sender" field
to the list-post address, because all those administrative messages
would then go to the list, and many would likely be distributed to the
subscribers.  You do not want to remove the "Sender" field, because
without it such messages would go to post authors but they probably
cannot do anything useful with them (and sometimes there are many for
just one post).

 > If you have any hints 

[Mailman-Users] Re: Unable to post messages to a Mailman 2 List

[Stephen J. Turnbull]

Odhiambo Washington writes:

 > You can use the address rewriting facility of your MTA, but is that even
 > necessary if the old domain name is gone?

"Gone, but not forgotten."
That's some network team if they can erase people's address books!

The tradeoff is immediate convenience for posters, vs. maintaining
the alias forever (and perhaps losing it unexpectedly in a cleanup of
"unused and unnecessary cruft" ;-).

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Hide list address?

[Stephen J. Turnbull]

Odhiambo Washington writes:
 > On Mon, Nov 13, 2023 at 4:24?PM ta515eeo0o2--- via Mailman-Users 
 >  wrote:

 > > "I want to change the configuration of the mailing list so that all
 > > messages are addressed directly to the recipient, and the mailing list name
 > > does not appear anywhere in the headers."
 > > I dont find this options in Mailman 2.1.37.
 > 
 > It is there under Non-Digest options. You have to login to the Admin
 > interface to find it.

The option is probably "full personalization" but I forget exactly,
look for "personalization" in the page Odhiambo mentions.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Problems with mailman installation

[Stephen J. Turnbull]

Nils writes:

 > I've checked [for logs] in both locations. I even created a
 > writeable syslog file in chroot to be sure. Neither had any
 > information.

It's been a decade since I looked into the wrapper code, but I suspect
that the reason for the odd phrasing about "log will be written" is
due to the wrapper not knowing where to write logs, and passing them
to the Python code to handle.  If the problem was that there was no
python binary, the log didn't get handled and wrtten.  Just guessing.

Glad something I wrote gave you the clue you needed!

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Problems with mailman installation

[Stephen J. Turnbull]

Nils writes:

 > vhost configuration:

This is irrelevant at this point, I think.  Apache is calling the cgi
so you're past that.

 > Here is the significant part of the fstab file that provides the 
 > necessary directories in the chroot environment:

You're missing the bin directory and Python's files.  Are they
correctly specified in the chroot?

 > I have checked that all script files are accessible. There is no
 > related entry in either the syslog, the apache log files or the
 > mailman log files.

They should be in the chroot's /home/www/var/log hierarchy (except
maybe syslog if that's being logged to a pipe or socket).  Is that
where you're looking?

 > I am not able to find more details about this error.

I don't see an fstab entry for /var/log.

 > The Mailman CGI wrapper encountered a fatal error. This entry is
 > being stored in your syslog:
 >
 > No such file or directory

Assuming you just omitted correct fstab entries for /usr/bin,
/usr/lib, etc, because the basic chroot configuration gets that right,
I wonder how you installed the new Mailman.  Did you run apt or dpkg
in the chroot, or did you tell them to install to /home/www instead of
to /?  Try grepping the CGIs in /home/www/usr/lib/mailman/cgi-bin for
"/usr/", and also run "strings wrapper | grep /usr/" on the wrapper to
see if the paths look sane (specifically you don't want any /home/www
in them).

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] how to change email address

[Stephen J. Turnbull]

Teijo writes:
 > Hello,
 > 
 > 
 > I am unsure how to change my email address where messages of this list 
 > are emailed.

Visit https://mail.python.org/mailman3/lists/mailman-users.python.org/
and log in ("sign in" at the upper right, using any of your registered
email addresses as the user, or some social account login).  Then
visit your profile (the small downward pointing triangle at the
personal avatar at the upper right).  If you have not yet registered
the new address, do so using the "link another address" link at the
bottom of the other emails list.  You may need to validate this
address to subscribe.  Then use the "manage lists" item at the upper
right (to the left of the search box) to find this mailing list and
change your address there.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Question regarding message-ids

[Stephen J. Turnbull]

Ralf Hildebrandt via Mailman-Users writes:

 > I'm sure I can hack something up in Postfix, but is there an easy
 > way (tm)?

Get rid of Exchange. :-)

This is not something Mailman should implement in my opinion.  The
author decided to send a single message to multiple addresses.
Evidently that author considers it to be the same message regardless
of which list delivers it.  I also don't think it makes for a healthy
community based on the points below.  It may work fine in some special
cases, but it seems likely to cause a lot of confusion.

Now, as Richard Damon points out, whether two messages should have the
same Message-ID is not a matter of identical content, it's whether
users consider them "the same".  Since a mailing list is an
intermediary, that is, it removes a message from the Internet mail
system, processes it, and then reinjects it, the mailing list can
claim to be the originator of the distributed post in some sense
(though obviously not in the sense of copyright law!)  So if you want
to make this decision for all subscribers, you can do that.

However, here are some points to consider.
- Are the recipients who subscribe to both lists unanimous in their
  desire to duplicate messages in this way?  If not, are the
  individuals who want distinct messages "more important"?
- Unless you are very clear about this change, some recipients may
  think the authors are sending multiple copies, and ask them to
  desist.
- Anybody who uses reply-all will also have their posts duplicated in
  the same way.  This will create a weird proliferation of threads,
  which I doubt will display sanely in most mail clients.
- It will very likely cause more than usual redundant posts because
  even people who are subscribed to both will be missing most of the
  potentially relevant replies.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Problem with outlook.com

[Stephen J. Turnbull]

pau.bai...@csuc.cat writes:

 > We have a problem with some Outlook users who on the list return
 > this error:

This is an Outlook problem.  With luck somebody here may have an
answer, but we are not Outlook experts.  You should go to Outlook
channels for help.

Here's my best guess:
The error message is completely garbled (or is that just the way
Outlook rolls?), but it looks like there is data encoded in
"eucgb2312-cn".  I've seen a lot of Chinese over the years, but I
don't think I've ever seen that encoding.  So my guess is that some of
your users have Outlook installations that know what that is, and some
don't.  (I would bet that it's just an alias for what almost everyone
calls "gb2312", IIRC that's 8-bit EUC-formatted by default, although
there are 16-bit and 7-bit versions as well.)

The easiest thing is to tell your users who are sending in that
encoding to cut it out (getting admins to fix mail software is really
hard, but most mail clients have flexible settings for preferred
encodings).  Use "gb2312" itself, or "gbk" or "gb18030" (superset
encodings of "gb2312"), or "UTF-8" (a different encoding), and I bet
the problem evaporates.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Mailman 3 moderator login page as mailman 2.1 one

[Stephen J. Turnbull]

Mark Sapiro writes:

 > First the moderator needs to sign up for an account in the web UI
 > at a URL like https://example.com/accounts/signup/. Then when
 > logged in, the moderator can visit a URL like
 > https://example.com/mailman3/lists/ to handle requests.

For most people it should just work -- give it a try.  There are a few
more points to be careful of, that you might want to review if it
doesn't "just work".

1.  They must use an email address that was listed as a moderator or
owner of the list they want to moderate.  If they used that email
address to subscribe to any lists on the old site, their old
subscriber password should work.  The old moderator password will
*not* work.

2.  If they were listed as moderator but never subscribed so they
don't have an existing user password, they have to choose a new
password, but it doesn't matter what password they choose for the
new account.  I suggest not using the old moderator password even
though it's probably easy to remember.

3.  If they were not listed as one of those roles, they need the owner
or site administrator to give them that role.

4.  I've seen cases where people were listed as *moderator* but given
the *owner* password so they could mass subscribe or unsubscribe
other people.  In that case, the moderator will no longer be able
to mass (un)subscribe in Mailman3.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Excessive or fatal bounces issue

[Stephen J. Turnbull]

Florin Pasăre writes:

 > Thank you for your response. I know we should upgrade, but because
 > of company policies this is not really an option at the moment,

Like Carl and Mark, I strongly recommend upgrading to most recent
Mailman 2.  This is straightforward, and during the process you can
get round-the-clock support here because everyone has done it several
times.  You don't need to get the attention of developers, community
support is excellent.

You do not have to worry about losing archives from a Mailman 2
upgrade, because the archival format is just the mbox that your MTA
produces for you.  The web pages are a presentation that can be
rebuilt (at some cost in time) at any time.  In fact, the most recent
month's pages are rebuilt from mboxes every day by a cron job.  Even
the URLs of individual messages remain the same (unless you edit the
mbox by removing or reordering messages).  (If you use an alternative
to the bundled "Pipermail" archiver, you still probably have all the
mboxes somewhere in the system, but you should check that.)

 > at least not an upgrade to Mailman 3.

That is a project that will require some planning in any case.  You
can preserve the mbox files (and even leave the old Mailman 2 archive
CGIs running), and in that sense you won't lose archives.  However,
the version of the Python email package in Python 3 does not cope well
with some of the weird things you find in Mailman 2, and we have not
yet learned how to catch and repair all of them at import time.  Also,
importing messages into HyperKitty changes all the message URLs
(completely different format based on hashed Message-IDs rather than
serial number).  So you probably should think in terms of a man-week
for planning, and a man-week or so for execution, anyway.

I recommend you start "softening up" the CIO for a migration in a year
or two.  Eventually it's going to become painful to maintain Python 2
applications because even the oldest LTS distributions don't provide
old system libraries needed by Python 2, or because QA or CISO says
you can't use them any more because they're too old/vulnerable.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Excessive or fatal bounces issue

[Stephen J. Turnbull]

Florin Pasăre writes:

 > Hello,
 > I've been having some issues lately. We are using Mailman 2.1.15.

Mailman 2.1.15 was released in 2012, ie, long before the AOL/Yahoo
"contact list leak" fiasco. This means your lists are unable to
dealwith anti-phishing and anti-spam measures adopted by many sites
(and specifically AOL and Yahoo and their successor operators).

You should upgrade to the most recent Mailman 2 for now, and strongly
consider upgrade to Mailman 3 in the near future.  (Both Mailman 2 and
Python 2 are long since "end of life", and increasingly vulnerable to
attack as they are not being actively developed.)  Make sure you check
the upgrade notes for any special procedures and requirements for
supporting software.

Recent Mailman provides a large number of important new features.
Most important, many recipient sites participate in the DMARC and ARC
protocols.  The important aspect of DMARC is that a site can declare
that emails with "From" address in that domain MUST have a valid DKIM
signature from that site (called "From alignment"), otherwise the
recipient should reject or discard the email.  Because mailing lists
frequently make changes such as appending footers to the body and
prepending list tags to Subject which invalidate DKIM signatures, From
alignment will fail on all mailing lists messages.  I cannot be sure
without more information whether your users are running into this
problem, but it's usually pretty obvious even to non-experts, because
mail from a person at site X starts bouncing for everybody at several
sites.

Recent Mailman provides two options to deal with this.  The first is
to rewrite the address in From from 'John Doe '
to something like 'John Doe via Your List '
for all posts passing through the list.  The second is to do the same
rewriting, but only for sending domains which publish a "please
reject" policy.  These are both per-list options.

It also provides a number of other security improvement, in particular
protection against cross-site scripting.

The ARC protocol deals with the From alignment problem in a different
way.  Each participating mail gateway validates the DKIM signature,
adds information about the result to the mail, and digitally signs
that information.  So this basically amounts to each gateway
testifying that "everything was OK when it got to me, I made some
changes and signed them, check that."  Of course this depends on the
final recipient trusting intermediary who makes changes, but in most
cases that's only your list.

Mailman 2 doesn't provide ARC, but ARC is best implemented in the
MTA.  Mailman 3 does have an implementation, but we still recommend
doing it in the border MTAs as defined by the ARC RFC.

Note that the two DMARC "From" modifications and ARC are all
substitutes for each other, with differing advantages and
disadvantages.  You can get both DMARC features just by upgrading
Mailman 2, which you should do anyway to improve the overall security
of the Internet.  To get ARC, you need to install additional software
to your MTAs (or upgrade all the way to Mailman 3).

 > in the last months the number of bounces we get has increased. We
 > receive these bounce notifications informing us that the given
 > *subscription has been disabled* due to *excessive or fatal bounces
 > *(up to around 30 bounce notifications triggered by a single email
 > sent to one of the listservs).

[Disclaimer: "Listserv" is the trademark of a commercial mailing list
manager owned by L-Soft, and should not be used to refer to Mailman
installations.]

I'm not sure why you would experience an uptick more recently than say
2014-2016 when many mail servers adopted stricter policies toward
malicious mail.

 > Every time we get such notifications, we untick the bounce under
 > “nomail [reason]" in the admin management site for each of the
 > email addresses that haven been disabled by the system.

If these are people who are regularly reading their email, this is a
somewhat dangerous policy.  They're just going to get disabled again,
and perhaps unsubscribed.  In the meantime, they're losing mail.  You
really need to investigate why these bounces are happening.  By far
most delivery failures these days are due to recipient site policy,
not equipment or software failures.  Occasionally inattentive users
exceed mailbox allocations, but that's not so common in these days of
freemail providers with default 10GB allocations.

What more can you do?

First, get list owners to check for "Delivery Status Notifications"
(DSNs).  These often explain in more or less plain language why the
mail was rejected.  Often it's because it was considered spam or
phishing (the less plain language is "administratively prohibited").
In that case it's very likely your posts will get caught frequently.

Second check both the Mailman logs (usually in some directory like
/var/lib/mailman/logs) and the MTA logs.  MTA logs are usually in
directories like /var/log/$MTA_NAME, but also may be in

[Mailman-Users] Re: Where's the installation directions / source, etc, please?

[Stephen J. Turnbull]

Barry S. Finkel writes:

 > And I wanted to get support from Mark and this list, instead of
 > from Debian.  So, I figured out how to create a package from the
 > Mailman source.  This was on an older version of Mailman, but I
 > assume that my technique should work with the latest Mailman 2
 > source.  Contact me personally for details.  It is not complicated.

I second this method if you're not interested in keeping the source
tree around.  It keeps your system tidy, and I'm pretty sure most
distros (specifically Debian does) support a configuration to look in
a local package archive for packages before going out to the main
distro repos.  So do it each time you upgrade and you always have a
way to back out to the previous working system.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Python 2.7.15, etc, vs Python3...

[Stephen J. Turnbull]

rich...@karmannghia.org writes:

 > (Maybe! How do we know they won't abandon Python3 like they did
 > Python2?

They supported Python 2 for most of a decade after the release of
Python 3.  Not only does that bode well for longterm Python 3 support,
there also will not be another break like Python 3 vs. Python 2.
Nobody has the stamina to accept that much abuse again.  There once
was talk of going from Python 3.9 to Python 4.0, but that would have
been just Python 3.10 by another name, just as backward compatible.

 > I mean, whatever happened to Python1?!

It gracefully evolved into Python 2.  There were a couple of small
compatibility breaks (introduction of true Boolean values was one, I
think), which justified a few years with both Python 1 (with the
Unicode type bolted on in v1.6) and Python 2 (with the compatibility
breaks and a number of new syntaxes, and a more thorough integration
of Unicode) being supported and distributed at the same time.

 > Why isn't it just "Python"?)

To let people know that there are definitely backward
incompatibilities.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Where's the installation directions / source, etc, please?

[Stephen J. Turnbull]

Steven Jones writes:

 > I feel the same way, hence still running Mailman2 (on RHEL8). It is
 > simple and low CPU hit, however Red Hat stops it support in May
 > 2024.

That's fine with us.  Mailman 2 is pretty bulletproof and low-
maintenance from our point of view too.

 > Containers are really useful where done well but I tried 2 or
 > times to get mailman3 going on RHEL9 with podman and even docker
 > and failed.

Not sure what containers have to do with anything, to be honest.  In
any case, we don't really support containers AIUI.  Abhilash provides
multiple containers in a configuration that's convenient for him to
distribute, but the container environment isn't something we support,
nor can we.  A lot of people have difficulty configuring the network
with multiple containers.

I would recommend configuring everything (except perhaps the database)
in a single "host" (hardware, VM, or container), unless you're willing
to take on all that complexity.

 > I think Debian12 does mailman3? worth a go if so.  In my case I am
 > not allowed to run an unsupported OS and app.

Current Debian is pretty close to most recent release (maybe at this
point it is the most recent release).  But as you say, if you need a
supported OS, you're probably going to end up with a pretty old
version of Mailman.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] moderation queue and member list management weirdness

[Stephen J. Turnbull]

Hoover Chan writes:

 > A moderated mailing list where I see items in the moderation queue
 > but nothing happens after I select the actions to be taken (defer,
 > approve, delete) and then execute.
 > 
 > For the same list, I can see the first page of the member list but
 > can't search for members.

The first thing to do is look in the Mailman logs, which are typically
in places like /var/log/mailman or /var/lib/mailman/logs.  Then look
in the webserver logs.  Because of the variety of webservers, those
are even harder to be sure where they'd be located, but on modern
systems with Apache they're normally in /var/log/apache2.

Pro tip: since these are apparently 100% replicable, open a couple of
shell windows, do tail -f on the relevant logs, and do the problem
operations.  Then you can watch them in real time.

Tell us what you see (give us exact log entries and error messages in
the web UI).  If you see nothing, tell us that, too.  Feel free to
replace your IPs, domain names, and list names with generic ones.  I
recommend IPs starting with "10." (those are not publicly visible),
domain names ending in "example.com" (specially registered for exactly
this purpose), and list names like "list-1".  If multiple IPs,
domains, or lists are involved, use different substitutes for each.
(For domains, at least example.net and example.org are also registered
for this kind of use.)

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Can Mailman 2 run on Debian 12?

[Stephen J. Turnbull]

Thomas Gramstad writes:

 > We are running quite a few mailing lists here, so it would be a
 > rather big job changing them all to Mailman 3.

At least with Postfix and Exim4, there is experience running Mailman 2
and Mailman 3 in parallel.

I recommend you consider installing Mailman 3 and gradually migrating
to that installation.  Although Mailman 2.1 and Python 2.7 are both
pretty bulletproof, supporting libraries are continuously being
upgraded.  I'm particularly thinking of OpenSSL which is planning
large changes for its next major version.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Mailman 2.1.15 doesn't allow admin changes on private lists

[Stephen J. Turnbull]

Richard Damon writes:

 > One configuration error that I remember being able to cause this
 > sort of issue is if the list is configured with a http:// address,
 > but the server automatically forwards to https:// then it can lose
 > the data for the submission.

Interesting idea.  But I don't think the effect would be so narrow.  I
haven't worked with Mailman 2 in a while but IIRC the base URL is
configured once, and everything else is derived from that.  Of course
there's the perennial issue where somebody changes from HTTP to HTTPS
and doesn't run fixurl, but that affects *everything* that requires a
login, doesn't it?

Anyway, it would be easy to check if Charles has access to mm_cfg.py.
(I don't think there's anything in config.pck that affects the URL
scheme.)

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Mailman 2.1.15 doesn't allow admin changes on private lists

[Stephen J. Turnbull]

Mark Sapiro writes:

 > Actually, the first 2.1 release was in December, 2002. See
 > https://wiki.list.org/DOC/7%20Mailman%20history

You would 
You have to admit the early 2.1 release history is extremely
confusing.  I'm pretty sure I chose the tag that was exactly "2.1".

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Mailman 2.1.15 doesn't allow admin changes on private lists

[Stephen J. Turnbull]

Charles Buckley writes:

 > I experimented with this a bit, and found that I could eliminate
 > the footer on my public (browsable) list on the same server. So I
 > tried converting my other private (non-browsable) list to be
 > browsable, at which point I could eliminate the footer, and then
 > switch the list back to being non-browsable.
 > 
 > But once I tried to implement the workaround on the non-browsable
 > list I wanted to change, I got the same defective behaviour when
 > trying to switch the list to be browsable

Did you test on the second private list *without* changing the
"private" flag?  If not, my guess is that the private flag is a red
herring, and that there is some other issue with the recalcitrant list
that causes you to get bounced back to the login page.

Please check the Mailman and webserver logs to see if there is
evidence of errors there.  With luck there will be a Python traceback
from an exception.  If you're using Apache as the webserver,
tracebacks are usually in the error.log, and there may be a 5xx status
in the access.log (I bet not though since you get served the login
page rather than a Server Error page, and that makes me somewhat
pessimistic about finding a traceback in error.log).

 > I saw a report of this behaviour on this mailing list from the year
 > 2000.

If you have an URL for this post, or a timestamp, or even a precise
date, it might be helpful.  I can't find it.

 > It is still going on now in 2023. One would think that some
 > information on how to workaround this bug would have been found
 > between now and back then. 

I rather doubt it's the same bug (but it's worth comparing).  Mailman
2.0 was in beta in 2000, and pretty much anything from mail composed
by badly written Japanese MUAs to mail composed by the even less
conformant Windows 2000 Outlook betas could crash it.  Mailman 2.1 was
released in 2006 with a *lot* of attention to input validation and
exception handling, although more on the email side than the web UI
side.

 > Note that, when I am able to successfully change a setting, I am
 > never sent back to the list admin login page.

That's expected.  My guess is that some content, probably an invisible
control character in a text field in the form (I've seen ^T mentioned
more than once, don't ask me why), is causing the form parser to raise
an exception, which who-knows-why gets caught by the not-logged-in
handler.  (My guesses are close to correct about 20% of the time.
Good enough to look there first, but don't bet your car. ;-)

I think all the browsers you mention have developer modes or plugins.
Mailman pages don't have horribly complicated DOMs, so if you want to
go through either the DOM or the page source for the form and see if
you can spot some weird character in one of the fields (likely, but
not certainly, the footer you're trying to change), you might have
some luck.  Also, "_" (underscore) may be a "weird character" --
Mailman 3's list importer complains about footers that contain it.
(Who knows why, I don't think it's weird, but Mailman 3 does kvetch.)

Regards,
Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Retrieve extra copy of Mailman bounces- report message?

[Stephen J. Turnbull]

Thomas Gramstad writes:

 > My question is: If I accidentally delete that report message from
 > Mailman, is there a way I can retrieve or send myself a copy of
 > it?

You can have multiple owners or moderators for the list.  Why not just
add a mailbox-of-record that you don't actually read except in the
rare case you deleted a report you wanted to respond to or quote from?
Maybe make a note to delete old mail once a month or so.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: Size limit leads to rejection instead of moderation

[Stephen J. Turnbull]

Mark Sapiro writes:

 > The size limit, General Options -> max_message_size will only
 > result in a message being held. These messages are being rejected
 > for some other reason. Check Mailman's vette log and/or the
 > rejected message for the reason.

If that doesn't reveal the reason, also check your MTA logs and
configuration.  MTAs are usually configured to reject or discard huge
messages (I think Postfix defaults to 50MB), and that would happen
before getting to Mailman.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: posts building up in /var/spool/mailman/in

[Stephen J. Turnbull]

Good to know!

and thank you for the followup, we do appreciate it!

ssaini writes:
 > 
 > > ```
 > > If urllib2.urlopen throws an exception, you need to figure out why. 
 > > Does
 > > ```
 > > wget https://publicsuffix.org/list/public_suffix_list.dat
 > > ```
 > > retrieve the data?
 > 
 > After posting the message to this list I checked if the mailman server 
 > could resolve https://publicsuffix.org and it failed thus for our 
 > incident it was related to a firewall rule that allowed outbound 443 
 > traffic being revoked which caused the delay in delivering and build up 
 > of posts. After the rule was added back the spooled mailman posts in the 
 > "in" directory were delivered within minutes.
 > --
 > Mailman-Users mailing list -- mailman-users@python.org
 > To unsubscribe send an email to mailman-users-le...@python.org
 > https://mail.python.org/mailman3/lists/mailman-users.python.org/
 > Mailman FAQ: http://wiki.list.org/x/AgA3
 > Security Policy: http://wiki.list.org/x/QIA9
 > Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
 > https://mail.python.org/archives/list/mailman-users@python.org/
 > Member address: turnbull.stephen...@u.tsukuba.ac.jp
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Re: migrations questions

[Stephen J. Turnbull]

Mark Sapiro writes:

 > You need to build a working Mailman 3 installation on the new
 > server. We recommend following
 > https://docs.mailman3.org/en/latest/install/virtualenv.html
 > for that. 

That is certainly the method I would use, and have several times
used.  (Caveat I'm also a Mailman core developer, so, that's not
really an independent opinion.)

If you are a Debian person and are building a new host from the ground
up, you might find Brian Carpenter's HOWTO
https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10
of use.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/
Member address: arch...@jab.org

[Mailman-Users] Question: If the list name and a member address are known, can foreign mails be channeled into the list?

[Stephen J. Turnbull]

Thomas F. Holz writes:

 > If I know the address of a list member and the address of the mailing 
 > list, I seem to be allowed to write in the list in his place.
 > Is this correct?

Yes, as far as Mailman 2 goes.  Mailman 2 doesn't know anything about
a user except their address.  Mailman 3 knows a little bit more, but
Mailman doesn't know how to authenticate posters by digital signatures
(and you probably don't want to put your subscribers through that
pain, either).

 > 1)---
 > First, I can fake the sender address. If the original sender address and 
 > mail with the forgery are sent from the same domain, then this is not 
 > prevented by the MTA (SPF/DKIM check), is it?

Not by standard MTAs, which only make the appropriate check if the
sending domain has set a restrictive DMARC policy.  But you should be
able to create an MTA or spam filter rule that checks for from
alignment yourself.

 > With freemailers like gmail, web.de, gmx etc. this doesn't seem so 
 > impossible to me (i.e. that listmember and bad guy write from the same 
 > domain).

That won't work from gmail.  Gmail will only allow you to send From an
address if you can prove you own it, either by using it to log in to
Gmail, or by reading a one-time token from that mailbox and sending it
back to Gmail.  I can't speak for the other freemailers, but I imagine
they work the same.  And if you send it from somewhere else, it won't
have Gmail's DKIM, so from alignment will fail.

 > If I write to the mailing list from a valid address (which is NOT a 
 > member of the mailing list), and specify a "return-to" in the header 
 > with a listmember's address, then that gets waved through to my mailing 
 > list as well. My mailman lists here seem to ignore the "From" address 
 > completely then.

That is configurable on a sitewide basis.  Add the SENDER_HEADERS
variable to mm_cfg.py, and change it to ('from') or ('from', None).

# Membership tests for posting purposes are usually performed by looking at a
# set of headers, passing the test if any of their values match a member of
# the list.  Headers are checked in the order given in this variable.  The
# value None means use the From_ (envelope sender) header.  Field names are
# case insensitive.
SENDER_HEADERS = ('from', None, 'reply-to', 'sender')

 > Have I understood this correctly?

Not 100%, but basically so.

 > And if this is as described, how can I prevent this?

1.  In practice, as long as you do normal content-based spam
filtering, this seems to mostly be a theoretical problem even if
you do nothing special about checking senders.  Maybe you (or your
users) have nastier than usual enemies though, you have to decide
that.
2.  For a little more security and transparency, remove reply-to and
sender from SENDER_HEADERS.  This will inconvenience some user
occasionally, but it should be rare in most user populations.  It
won't stop spoofing, but it will be easy to see it and the victims
will complain.  This may do the trick depending on what the goal
of the spoof is (and if the spoofer is a a bot).
3.  For maximum security with little inconvenience to users, have your
MTA check for From alignment.  You can either reject on that basis
(which will inconvenience some users substantially, I suspect) or
you can have the MTA add a header to the message, and have Mailman
hold the mail for human moderation if alignment fails.
It would also be possible to have Mailman do this but it's more
efficient to have the MTA do it.

I believe some users in the past have mentioned 3rd-party patches to
check user's digital signatures, but that's quite compute-intensive,
and requires that you teach your users to sign their own email.  I'm
pretty sure they won't like that. ;-)

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] post confirmations

[Stephen J. Turnbull]

Bernie Cosell writes:

 > This with mailman 2.1.39.   And this in response to Gmail's utterly foolish 
 > and 
 > obnoxious "feature" of throwing away an incoming email that appears to be 
 > "from yourself"

I agree it's obnoxious, but it's actually dup suppression, not
throwing away mail from self.

 > I regularly get inquiries about "how come my submission didn't get posted".  
 > Is it 
 > possible to generate some sort of "your message has been posted"  
 > confirmation 
 > reply?

I forget what it looks like in user view, but in "Membership
Management > Membership List (the default in that category), there's a
column "ack" for "Acknowledge".  I don't recall whether it
acknowledges *receipt* of the post or *distribution* of the post, but
for your purpose it probably doesn't matter.

Presumably it's more explicit in the user view.  If you prefer to turn
it on for all the gmail users, you can use a withlist script to set
it.  Or you can just let the users who want it set it themselves.

Steve


--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Line breaks in monthly reminder emails

[Stephen J. Turnbull]

Mark Sapiro writes:

 > If you have the ability to patch Mailman's cron/mailpasswds, this will 
 > do it.

Possibly more flexible (but harder to implement and dependent on user
MUAs) would be to use format=flowed in Content-Type.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: manage.py not found in virtual environment

[Stephen J. Turnbull]

Martin Lorenz writes:

 > >> `python manage.py hyperkitty_import -l stamm ...`
 > btw. this is, what it says in the docs: 
 > https://docs.mailman3.org/en/latest/migration.html

I assume you're just pip'ing everything from PyPI, without any version
constraints on the command line?  If yes, good, that's what I need to
know.  If not, what versions of Mailman 3 applications are you
installing, from what source?

I suspect those docs predate the creation of mailman-web.  Instead of
running "python manage.py hyperkitty_import ...", make sure your venv
is activated and run "mailman-web hyperkitty_import ...".  You still
use the "mailman" command to import list configurations (such as
subscriber lists and their profiles) from Mailman 2 to Mailman 3, but
you use "mailman-web" for importing list archives (past messages).

To test if you have your settings.py in the right place, make sure
your venv is activated and run "mailman-web help | less".  If
everything is in the right places that will produce a (long) list of
commands that you can use, including hyperkitty_import.

I don't think you actually need settings_local.py since we introduced
mailman-web.

If you are following the virtualenv install instructions, you need to
put mailman.cfg and settings.py in /etc/mailman3 (or you can set
MAILMAN_WEB_CONFIG to point to the right settings.py).  This is also
where you should usually put configuration files for WSGI providers
like gunicorn or uwsgi, and possibly some webserver-related files.

Steve


--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] manage.py not found in virtual environment

[Stephen J. Turnbull]

Martin Lorenz writes:

 > (venv) mailman@arda:/etc/exim4$ python manage.py hyperkitty_import -l 
 > stammesleit...@list.poc.im 
 > /data/var/lib/mailman/archives/private/stammesleitung.mbox/stammesleitung.mbox
 >  

Executive summary: you have to specify an absolute or relative path
to manage.py.

Here's why.  When invoked as "python manage.py ..." it is treated as
an ordinary file argument (that happens to be a Python script), and is
not searched for on Python's sys.path or on the shell's PATH.

When invoked as "python -m manage ..." it is treated as a module, and
searched for on sys.path.  But don't do this, there are differences
between "python foo.py" and "python -m foo", and manage.py is not
designed to be invoked with "-m".  Also, the same Django installation
may support a variety of applications each of which has its own
manage.py, and there's no reason to suppose the one that's first on
sys.path is the one you want unless you are already cd'ed to
manage.py's directory.  For example, before the creation of Mailman
web, both HyperKitty and Postorius had their own manage.py.  In fact,
in some applications manage.py may not be on sys.path at all (unless
it's in the current working directory).

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Hevy resource footprint forces backdrop to mailman 2.1

[Stephen J. Turnbull]

Martin Lorenz writes:

 > so I had to switch to a new provider. The new v-server is
 > debian-based.  the release is bullseye which does still provide
 > python2 but lacks a few packages that are necessery to install
 > mailman2.1 I would have to manually install dependencies via pip
 > instead of apt.

As long as you have necessary access to install, that should not be a
problem.

 > The problem occurs whenever mail ist sent to a migrated list.
 > All mails sent to the lists I migrated from my old 2.1 instance (23
 > lists in total) are swallowed and never get delivered.

That sounds a lot more like MTA configuration problems or network
configuration problems than resource usage.

 > I have not found out where they are.

IIRC Debian puts queues in /var/lib/mailman3/queue/.  Look in ../shunt
in particular (that queue swallows posts that for some reason Mailman
is unable to process, and you have to do something to fix the problem
-- there's no automatic retry for those).

> There are no errors in the logs.

Which logs are you looking at specifically?


> 1 was only because I quickly needed a working solution - which I did
> not get.

Unfortunately, I don't know of a quick working solution from any OS
distro.  All the distros either provide very old versions, or we have
reports of difficulties configuring them.  The closest thing to a
turnkey solution are Abhilash's docker containers, but even there it's
non-trivial.  My recommendation (FWIW, now that you've already gotten
started it may not make sense to start over from scratch) is that you
get yourself a nice stable distro with at least Python 3.8 (better 3.9
or 3.10) installed, browse both the "venv" installation described in
our docs at https://docs.mailman3.org/en/latest/install/virtualenv.html
and Brian Carpenter's Debian installation guide at
https://wiki.list.org/DOC/Howto_Install_Mailman3_On_Debian10.  Then
choose the guide you find more compatible with your thinking, and
follow that.

 > thank you for the help you offer.
 > The time I can invest on the subject is rather limited for the coming week.
 > from Oct. 14 on I can dive into it :)

OK, no hurry.  We'll be here when you're ready.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Hevy resource footprint forces backdrop to mailman 2.1

[Stephen J. Turnbull]

Martin Lorenz writes:

 > I am afraid, mailman3 needs far too much system resources for this
 > virtual server.

This whole story doesn't make much sense to me.  You switched to a new
virtual server because Mailman 2 isn't supported but imply that
Mailman 3 is, but you can't get support from your vendor?  Do you mean
that *Python* 2 is not supported but *Python* 3 is?

Then, as far as I can tell on my hosts, Mailman 3 does not use a lot
more resources than Mailman 2.1 to process email, because it's
basically the same processing model.  By "doing anything," do you mean
posting to lists?  Migrating lists?  Creating and managing lists with
Postorius?  Migrating archives?  Accessing archives and posting via
HyperKitty?

Python 2 is *unsupported* and likely to have security issues (in
particular, it probably allows SSL/TLS connections now considered
insecure).  Mailman 2 is also end-of-life, and may have security
issues (although Mark continues to accept patches and even fix them
occasionally, he keeps promising to stop "soon").  You're asking for
endless hurt if you get into trouble with Mailman 2.  (I don't think
it's likely except maybe a botnet will take your virtual server over
for Bitcoin mining or DDoS attacks, but if it happens it will be bad.)

Like Mark, I recommend that you at least let us try to help you to
figure out why the virtual server fails to run Mailman 3.  Are you
sure the new virtual server will handle the load of Mailman 2?  Maybe
it's just too small for the job of handling mailing lists.

If you're not willing to upgrade, you're going to need to install
Python 2, and a few packages.  You'll need to get support from your
virtual server provider and/or the OS vendor for that, we can't help
you.  It's possible that they're EOL at the OS vendor as well.  (I was
looking to install Python 3.8 on my Debian system to support a user
who's stuck on 3.8 for corporate QA reasons, and surprise! it's not on
their principal mirrors any more.  There are Linux distros that never
supported Python 2, they were on Python 3 from the start.)  Python 2 is
deeply embedded in some ecosystems, so most likely your OS vender
still delivers the most recent Python 2.7.  But you can be sure Python
2-compatible packages are bitrotting in most cases, including those
that support Mailman.

Regards,
Steve


--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Welcome E-mail Reply To

[Stephen J. Turnbull]

Omri Kalinsky writes:

 >      Problem #1:
 >      One of my friends (who isn't tech savvy and doesn't read 
 > directions) was my guinea pig and successfully joined. The problem is he 
 > tried to send his first e-mail to the list by simply replying to the 
 > welcome e-mail, so he e-mailed my-list-requ...@mailmanlists.org and not 
 > my-l...@mailmanlists.org. Wouldn't surprise me if others do the same 
 > thing. Is there any option to change the address the welcome e-mail 
 > comes from?

No, there is no such option.

This situation embarrasses or annoys one subscriber.  If you arranged
for the reply to go to the list, as he expected, it would go out to
the entire list, under a confusing "Welcome to the list" subject, and
would possibly induce a chain of "why am I getting 'welcome to the
list' messages now?" replies.  Mark Dale's suggestion of adding the
"how to post" message at the top is the best idea I know of.  Most
modern mail clients will turn the address into a clickable link, as
well.

 >      The other has to do with the replying and the "nodupes" feature. So 
 > first, I am trying to keep the list as collaborative as possible and 
 > have people replying to messages to the whole list and not to the 
 > original sender. first_strip_reply_to is on, reply_goes_to_list is set 
 > to "This list" and from_is_list is set to "Munge From".
[...]
 >      I couldn't find any combination of first_strip_reply_to, 
 > reply_goes_to_list and from_is_list to make this problem go away

There isn't one in Mailman, except anonymous list.  The problem is
that anyone can pretend to be Bob simply by setting their display name
to Bob.  Then the mailing list strips the address which might be
completely unrelated to Bob, and only an email expert can detect the
spoof.  We judged this to be a bigger problem, as it violates the
usual expectations of subscribers.[1]

To get the behavior you want from Mailman, you'll need to patch the
code.  As Mailman 2 is end of life we won't be patching and releasing
such a feature ourselves.

 > and keep everyone on the list regardless of reply or reply all.

Forum software is a better choice for this kind of channel.

Steve


Footnotes: 
[1]  We don't consider it an important problem on anonymous lists
because the presumption of and need for anonymity overcomes this
problem.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Welcome E-mail Reply To

[Stephen J. Turnbull]

Omri Kalinsky writes:

 >      So I just recently purchased a mailman 2.1.39 hosted list from 
 > mailmanlists.org.

Feel free to ask questions here, but we are volunteers and have
no connection that I know of to mailmanlists.org.  If you're paying
for support, you should get quicker support from the vendor.

I will take a look at the rest of the thread and if I have comments
I'll make them there.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Search box for archives

[Stephen J. Turnbull]

Odhiambo Washington writes:

 > 1. I am getting too many messages from cron which I am unable to address
 > because I am not a perl programmer:

I'm not, either.

 > Useless use of greediness modifier '?' in regex; marked by <-- HERE in
 > m/^\w+:{1,1}? <-- HERE / at /usr/local/share/namazu/filter/mp3.pl line 155.
 > 
 > I wonder if doing away with the "?" on that regex is sufficient. I have
 > done that, but I am unsure how it affects the indexing.

That's a very strange RE.  It appears say "you should match at least
one ':' and at most one ':' and if there is more than one way to match
that many ':' you should choose the shortest one that allows the whole
pattern to match".  A normal person would just write ':' instead of
':{1,1}?' but this is Perl code so ;-)

 > Where should I place these two files - archtoc.html and archtocnombox.html
 > - and how do I now integrate them in the whole setup to get the
 > searchbox?

I haven't done this is a very long time.  I assume those two files go
in the template directory, and probably you need to put a link to them
in each template where you a search function.  But if you want a
search box in the thread page, for example, you're going to need to
extract the search form from one of those search pages and add it to
the thread template, the date template, and the author template.


--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Search box for archives

[Stephen J. Turnbull]

Odhiambo Washington writes:

 > What is the simplest way to add a searchbox for the archives?

If you're talking about about Mailman 2's Pipermail, you need to
install and configure a separate application.  I think HT:Dig is most
popular, but I've only used Namazu, which worked well for me.  There's
a detailed FAQ on it: https://wiki.list.org/x/4030514, which says
there are maintained branches on LaunchPad with patches to integrate
HT:Dig.  That may be easiest.  Mark will know more when he gets back.

Another formerly somewhat popular approach is to substitute MHonArc
for Pipermail, although that software is quite old now, and I'm not
sure about its security status.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: AT Blocking (was AOL list member not receiving list traffic)

[Stephen J. Turnbull]

Michael Reeder -- Hygeia MS writes:

 > > Is there possibly something somebody could consider spammy in your
 > > content, such as frequent or repeated announcements of new services or
 > > invitations to events?  Do members ever report the list traffic being
 > > marked as (potential) spam?  Are these discussion lists among the
 > > members, or is it all announcements from your side?

 > Few frequent or repeated announcements or event invitations (we don't 
 > allow paid event ads).  I suppose the list headers and footers 
 > themselves could look too frequent.

I guess, but .signatures and legal disclaimers are very common.  I
would think that triggering on headers and footers would get Very
Important Stuff from vendors and customers spam-trapped.

 > There is lots of ongoing discussion amongst members -- not an 
 > announcement list.

I would think that build up a list's credibility.

 > We are now requiring them to get non-AT addresses.  A shame
 > really.

That really is.

 > Mailman 3 and HyperKitty may be an option if we port away from Dream 
 > Host management of GNU Mailman where we don't have command line access.  
 > That said, Dream Host is responsive to concerns so I do not intend to 
 > throw them under the bus for having some restrictions.

A lot of people seem to like Dream Host.  They eventually have to move
to Mailman 3 unless they want to support both Mailman 2 and Python 2,
though.

 > like Google Groups that I suspect is not given so much guff simply 
 > because they are Google.

I have to say that I learned to have a lot of respect for the senior
technical staff at Google and Yahoo! during the DMARC and ARC design
processes.  They really did know what they were doing.  I don't have
an active Yahoo! account, but Google does quite a good job of trapping
spam and tracking trends in spam.  Yes, they're that big, but they're
also good at what they do.

Anyway, let us know if we can help.  I don't know anything about Dream
Host's dashboard etc, but sometimes we can help with advice on
settings that seem to work for our Mailman lists and for other admins
who report their status occasionally.
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: AT Blocking (was AOL list member not receiving list traffic)

[Stephen J. Turnbull]

Michael Reeder -- Hygeia MS writes:

 > AT has now started blocking the IP address of the multiple 
 > auto-forwarding email addresses I set up to get around them blocking the 
 > IP address of my GNU Mailman list.

I have to conclude AT wants to get rid of their email provision
activity.

Is there possibly something somebody could consider spammy in your
content, such as frequent or repeated announcements of new services or
invitations to events?  Do members ever report the list traffic being
marked as (potential) spam?  Are these discussion lists among the
members, or is it all announcements from your side?

 > 37 list members at risk unless they change their email addresses.

AT hates them, so I recommend they get an alternative address for
list mail and eventually get off AT altogether.  It's not going to
get better, I imagine.  But most likely they won't change until they
get sued for nonpayment of bills invoiced to their AT address. :-/

 > We started 20+ years ago with a different list program and hosting 
 > provider, so we have lots of legacy members suddenly getting cut off.

Do you keep archives?  If so, an RSS feed might be the way to
communicate to them in somewhat less than real time.  Similarly,
moving to Mailman 3 + HyperKitty allows use of the archives in a way
quite similar to dedicated web forum software like Discourse.
Finally (though I hate to recommend it) you could move to forum
software or Google Groups.

I'm sorry to be putting all the work on you, but I don't think there's
anything Mailman can do.  It seems AT just doesn't want your list
traffic.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: List Archives

[Stephen J. Turnbull]

David Andrews writes:

 > Thanks everyone. This all sounds reasonable except it also identifies 
 > another problem we are having. There is no good search method in 
 > place.

The only way to resolve it in Mailman is to upgrade to Mailman 3, and
use HyperKitty.  As far as I know you can import a Mailman 2 mbox into
HyperKitty without installing the rest of Mailman 3.  If you decide to
go that way, don't use the default indexer Whoosh, it's tragically
slow.  Xapian is highly recommended, and other indexers also seem to
work.  I think I've heard of ElasticSearch installations, and one
friend used freeWAIS -- that's mostly a joke, freeWAIS is unsupported
for decades, but librarians/archivists might feel some nostalgia.

 > Also the organizations librarian wants the archives for historic and 
 > other reasons, so leaving them here won't satisfy that.

As Mark says, there's a $LIST.mbox file at the top of the list archive
subtree for the list that contains all of the posts, which most all
open source mail clients will happily treat as a mail folder, and I
would hope commercial clients do as well.  Just download that to
somewhere the mail client will recognize it as a folder, then you/they
can use the client's capabilities to organize it.



--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] List Archives

[Stephen J. Turnbull]

David Andrews writes:

 > I want to delete some lists, but would like to save the archives. 
 > Pipermail saves by month, as I have it configured. Is there a way to 
 > obtain an entire archive at once?  Are there tools to convert the 
 > MBOX format to something non-technical people can deal with easily?

Exactly what is your use case?  What do you mean by "obtain"?  What do
you mean by "non-technical people deal"?

I suspect "obtain" means something you don't need to do, and
"non-technical people deal" means "click to read".  If so, the answer
is basically "delete the list, don't delete the archive".  That's
easiest for the users since they just go to the same place.  Best of
all the site admin doesn't have to do much of anything besides delete
the list configuration file, and possibly edit the MTA config.[1] Note
that this can create backscatter (mail rejection notices to third
parties0, but that's no different from any other non-existent address
at your server.

The only problem is if the archives are "members only", because
deleting the list would also delete their credentials. Since in an
important sense the membership database *is* the list, in that case,
you don't want to delete the list, you want to disable delivery and
access to the administrative interfaces.  There are several ways to do
those things, and some vary according to the MTA.  They have subtly
different consequences for what happens if somebody tries to post to
the list or subscribe to it.  The one you're most likely to care about
is "the post disappears into a black hole and the member is
distressed," but some of them can create backscatter in the case of
spammers.

Footnotes: 
[1]  In the usual configuration for Exim4, Exim checks for
lists/$NAME/config.pck, and if it's there, routes mail for $NAME to
Mailman.  Nothing to do once you delete config.pck.  Postfix OTOH
usually needs you to run a script to update the aliases.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] AOL list member not receiving list traffic

[Stephen J. Turnbull]

Jayson Smith writes:

 > Is AOL known to silently discard mail they think is spam for some
 > reason?

I hope someone with actual experience will speak up, but my take is
that it's entirely possible.  (Footnotes are of historical interest,
but not directly relevant to solutions.)

The last time I had any insight into AOL was 2014, during the DMARC
development process.  The freemail providers lobbied to make their use
case eligible.[1]  Gmail didn't have an issue AFAIK, but both AOL and
Yahoo! were groaning from a mind-boggling flood of spam.  Gmail and
Yahoo! techs were very competent and helpful, and contributed a lot of
useful statistics and protocol ideas.  OTOH, the AOL representatives
clearly were dramatically under-resourced, and basically just pleading
for relief.

AOL was later acquired by Yahoo, which is now 90% owned by private
equity (ie, may be presumed completely unethical) and 10% by Verizon
(one of the most irresponsible ISPs).  I really doubt they invest in
best-practice email services :-(.  On the other hand, the extremely
competent tech from Yahoo is still there, managing all Yahoo/AOL email
services (AFAIK that means all freemail services provided by Verizon).
*sigh* *I* dn't knooow... :-(

 > I replied to her message from the same server and she did receive
 > that reply, so they haven't outright blocked my IP or
 > something. Even if I could contact someone who knows what they're
 > doing at AOL, there are no error logs for me to show.

The big freemail providers are deliberately opaque about their
operations, so the following is generic advice -- I don't have
evidence that it will help.  But it's good advice! ;-)

First, you should check that your DNS records for DKIM and SPF are
up-to-date, and outgoing mail is being signed.  If you haven't done
this recently, you should do it anyway.  Over the years, stuff
happens, as they say. :-)

"Friends don't let friends use AOL."  I understand the pain of moving
your email to a new address, so I completely respect anyone's decision
to stick to their current one.  That's the easiest solution from your
point of view, so I mention it.  Gmail has historically been the
easiest of the big (opaque) providers to work with because they
conform to current best practices and don't negotiate anything else. :-þ

The third suggestion is most burdensome for you: set up ARC
processing.  The Authenticated Received Chain protocol creates a chain
of custody, where each domain that alters the message in ways that
invalidate signatures testifies that in the received message the
sending IP was an authorized sender for the domain and/or the DKIM
signature validated.  This is good enough for most recipients, so it
should help with AOL if broken DKIM signatures or failed SPF
authorization are the problem.  I trust the OpenArc implementation
https://github.com/trusteddomainproject/OpenARC because I've worked
with Murray Kucherawy since the 2014 DMARC travesty.  I'm pretty sure
it's not so difficult on a single host, but if you're working with an
email provider with a complex MX system, they may balk.  (Of course
you may already have ARC if you're working with a major hosting
service.)

I'm not in a position to use ARC on my own host and have never needed
it, but I'll help as much as I can if you have problems with setup.
Mark will be back online in mid-September, I think he has some
experience.

HTH

Regards,
Steve

Footnotes: 
[1]  The original idea of DMARC was to protect "transactional mail
flows", ie, sensitive direct business mail such as conversations
between a bank and an account holder.  Freemail providers were out of
scope, because of mailing lists and other such usage.  Then in
2013-2014 there was a huge increase in spam flows of the particularly
pernicious "referred by a friend" kind based on theft of huge numbers
of contact lists from Yahoo and AOL.  The big 3 freemail providers
(with Gmail) got references to transactional flows purged from the
drafts, and in April 2014, Yahoo and AOL proceeded to protect their
"From" domain with p=reject.  Yahoo's representative claimed that
several tests showed that this would stop literally millions of spam
mails per *minute* during spam campaigns.  This is not inconceivable,
given that there were already botnets with millions of bots at that
time: multicast a spam to 100,000 waiting bots each with a list of 10
victims, there's your million in well under 1 minute.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: B.S. With Google Yellow Boxes and GNU Mailman

[Stephen J. Turnbull]

Michael Reeder -- Hygeia MS writes:

 > *For now -- problem solved!**

Good news!

 > I will take a look at Mark's DMARC mitigations below also.

You may also want to ask Dreamhost if they can enable the ARC protocol
for your host.  This protocol allows your host to testify which
authentication tests passed on the way in, in particular DMARC's "From
alignment".  This means that the host takes responsibility for the
changes in the message (such as adding a list name tag in Subject or a
footer explaining how to access list resources, which break DKIM
signatures).

The difference between using ARC and the DMARC mitigations Mark
mentions are

1.  ARC allows your list to leave From as is, while the Munge_From
options change From to point to your list, instead.  "Munge From"
may confuse some subscribers (or their filtering and sorting
software), although that's usually not a problem.

2.  ARC requires that the final recipient participate in the protocol.
Most of the largest freemail sites support it.  On the other hand,
"Munge From" allows your site to authenticate itself, since it
DKIM signs and From is the same domain as the signature.

Which is better depends on the tradeoff between some inconvenience for
subscribers who want to reply to author only when From is munged, and
the risk of having sites that don't participate in ARC bouncing your
traffic.

If you aren't getting DMARC bounces already, I would suppose ARC would
be good insurance against some (but not all) DMARC bounces in the
future.  If you are, you might want to go straight to Munge From
rather than try ARC and hope it fixes them for all current and future
recipient hosts.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Change in email routing

[Stephen J. Turnbull]

Francis Jayakanth via Mailman-Users writes:

 > I'm told that there are per minute and per hour restrictions of 30
 > and 1800 emails respectively (inbound and outbound) on o365.

I'm not sure what "limit of 30 emails/minute" means.  In the below, I
am going to assume it means "addresses to be delivered."  The other
meaning I could imagine would be "connections", which would make it
much easier to comply (as long as you have a few "giant" destinations
like Gmail and Yahoo).

 > How can the said restrictions be complied with in Mailman?

There is no facility for this in Mailman itself.  Mailman does
maintain queues, but their purpose is only to ensure that messages are
processed by each function in order and do not get lost while waiting
for processing.  It contains no logic for "fair queuing" or
"throttling" for individual outgoing messages.  It just sends them all
to the MTA (Postfix), with popular domains getting multiple addressees
and only one message body.  The only restriction implemented in
Mailman is the maximum number of addressees per message.  That is
maybe you have 1500 Gmail addresses, then you could limit to 25
addressees per SMTP transaction, to allow 5 other emails to get
through every minute.

Normally I would recommend using Postfix to do the throttling you need
(see the various "recipient_limit" and "rate_delay" parameters in
postconf(5)), but given this requirement:

 > One of our lists has close to 6k members.

you are in a bad place no matter how you look at it unless you can
throttle the *incoming* posts to 4-6 per day, spaced at least 3 and
probably 4 hours apart.  Once one post is in the queue, I don't think
there is any way to guarantee it will be sent to all addressees before
the next post starts to be sent.  So unless you can guarantee posts
spaced out in time, you could end up in a situation where 1/4 of the
list gets the post, then you wait until the hour, but before that
another post sneaks in and it gets delivered to the same 1/4 of the
list.  That is as far as I know an MTA goes through the recipient
domains in a deterministic order, and will start over on the domains
that have already had post #1 delivered, by delivering post #2 to
them.  And of course processing just one post for this list is going
to make it difficult for anything else to get delivered until it's
done.

Of course the whole time this is going on, you have to keep all of the
queued posts on disk, one copy plus the address list per domain.  For
that reason it would be nice if Mailman handled the queuing for you,
but it doesn't.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Change in email routing

[Stephen J. Turnbull]

Francis Jayakanth via Mailman-Users writes:

 > Hi, I'm administering and moderating a list with Mailman version
 > 2.1.20.

This is extremely old.  If it works, that's fine, but you're missing
19 releases worth of security fixes, including some quite nasty and
easily exploitable ones (like cross-site scripting).

 > I need to make the following changes with immediate effect:
 > 
 >   1.  Switch to the Postfix email router from Sendmail,
 >   2.  Use Office 365 with authentication as the relay host
 > 
 > I have stopped the Sendmail service and configured Postfix for
 > email transactions.

What does "configure for email transactions" mean?  We really can't
help unless you're quite precise about these things.  Where does the
O365 relay host sit?  Between Mailman and Postfix, or between Postfix
and the Internet?

 > Things are not working as they used to while using Sendmail. I have
 > specifically noticed that the following Mailman functions are not
 > working after the change over:
 >
 >   1.  Online membership registrations are not receiving email
 >   communication about the membership being held for approval.
 >   2.  The moderator does not receive communication about new online
 >   registrations. So, new online registrations are not
 >   happening.
 >   3.  Adding new subscriptions through the command line works, but
 >   neither the subscriber nor the moderator receives any
 >   communication. Ex. ./add_members -r new -w y -a y listname

Is O365 involved in transmitting these notifications?  If so, is the
sending agent (Mailman or Postfix) configured to use the
authentication credentials?  What do the logs say?  Both Mailman and
the MTAs should be keeping logs of all outgoing messages.

 >   4.  No confirmation email is received after posting a message by
 >   a subscribed member.

Does the post go out to the subscribers?  Are you sure this
notification is enabled?  (In my experience members hate it with a
passion).  Again, this should be logged by Mailman and both MTAs.

 >   5.  The  email address of the approved messages is changing
 >   to the authentication email id, but the name remains that of
 >   the member making the post.

Is it possible that this issue is affecting receipt of notifications
because recipients have not whitelisted the authenticated address, and
they're being discarded or quarantined as spam?  Is the authenticated
address in the same domain as Mailman, and if not, do you have a DMARC
policy of p=reject or p=quarantine?

That sounds like something that O365 is doing.  Mailman should not
touch the From email address unless you have configured one of the
following:

- from is list (normally used to work around DMARC issues)
- anonymous list
- full personalization

Stock Mailman will not use connection creditials to modify the
message; it only becomes aware of them at the point it actually
connects to the MTA.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Managing Lists Remotely

[Stephen J. Turnbull]

Daniel Krause via Mailman-Users writes:

 > We're looking at options to manage our mailman3 lists programmatically from
 > a saas platform we offer.
 > 
 > The rest api seems like the way to go, but almost everything I read
 > about it says do not expose this publicly.

Use a dedicated encrypted tunnel from the saas platform to the list
host.  As long as that goes directly to Mailman's REST port, I don't
see why the management would have a problem.  Unless you're sharing
the Mailman instance---but if that's a problem I don't think they'd
give you REST access either.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Flooded with signup requests

[Stephen J. Turnbull]

Max writes:

 > I would have expected that Mailman shows the user/bot a message saying: 
 > Hey simon1...@gmail.com, you already have a pending signup request, 
 > please be patient while the moderators are reviewing your request.

I don't think that information is readily available to the
subscription request handler.  The database Mailman 2 uses is simple
and foolproof for its own purposes: each request is processed
immediately, any emails etc are sent, and the request ends up in a
file with a unique name ordered by time of receipt.

It wouldn't be hard to keep an auxiliary database with request
addresses in it so we could easily check for duplicates, but it
doesn't simplify things very much because both moderators and
subscribers lose mail, either by mistake or because they have a strict
filter.  So we'll just send another confirmation response anyway
because there's a good chance that they didn't get the first.

Obviously that doesn't apply to the kind of attack your site is
facing, but it's very unusual for a list to be attacked this way in my
experience.

 > Instead all moderators get the same signup request email from the same 
 > email address every minute for hours.

In 'General Options' set admin_immed_notify to No, and only one mail
will be sent per day.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Flooded with signup requests

[Stephen J. Turnbull]

Max writes:

 > Hi, I'm admin for multiple lists and I am getting flooded with fake 
 > signup requests.

As Mark says, stopping email signups is probably a good idea.

I wonder if the gmail and yahoo signups are genuine accounts there.
If not, checking From alignment (ie, the same domain that is in From
has DKIM signed the message) should allow you to filter them.  That
could be done with code that's already in Mailman, but it would
require some additional coding.

 > It also seems that the same address is trying to sign up multiple
 > times and this results in multiple mails to the admins/moderators.

I believe that there is an option to send moderation mail once a day.
If you need more rapid response than that, presumably you have
designated folks doing the work, have them check every N hours, and
disable the mail notifications entirely.  Have them ban the address
once and then discard the rest.  It's possible that another one could
come in in the seconds between polling for moderation requests and
sending the ban, but that should be fairly rare.

 > How can I at least stop the bots from repeated signups with the
 > same email?

This is the Internet; you can't stop them.  The best Mailman can do
for you is put the address on the ban list.

If you think you're being targeted by a specific botnet, you might be
able to analyze source IPs and ban them from talking to your hosts at
all using a firewall.

This list ... OK, none of them are GMail or Yahoo! any more, but
Yikes! you've gone scorched-earth!

 > ^.*@aol.com$
 > ^.*@qq.com
 > ^.*@yandex.ru$
 > ^.*@mail.ru$
 > ^.*@sbcglobal.net$
 > ^.*@msn.com$
 > ^.*@163.com$LOL.  Wish *I* could ban them.
 > ^.*@verizon.net$
 > ^.*@comcast.net$


More proactively, we'd all like to use HIMARS, but I think Ukraine
needs them more.  Firewall and ban list is the best we have at this
point.  N.B. We don't know everything and we may have missed something
we can do.  If you have any bright ideas, perhaps we can implement
them (but they won't be added to Mailman 2, sorry).

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Malicious Third-Party Unsubscription Requests

[Stephen J. Turnbull]

Karl Semich writes:

 > I'm experiencing mailbombing via spoofed unsubscription and
 > subscription requests from mailman lists. Mostly unsubscription
 > requests from a mailman 2.1 list.

This is an unfortunate consequence of the fundamental design of using
access to a mailbox as proof of identity.

There is really no way lists can defend you from these attacks without
shutting down all "open subscription" lists.  (Analysis below.)

Your best defense is to set a mail filter which discards or
quarantines mail which matches either of the following regular
expressions in "From":

.*-confirm(\+[A-Fa-f0-9]*)?@# Used by Mailman 2.1
^mailman@   # Used by Mailman 3.x

Unfortunately you will need to explicitly permit those addresses
temporarily if you wish to subscribe to or unsubscribe from a Mailman
list in the future.  You can however restrict permission to the
specific host (and for mailman 2, list) whose subscriptions you wish
to manage.

Addresses matching the following regexp are used in "your post is held
for moderation" messages (which can be triggered by posts apparently
from you) on many lists:

.*-bounces@

However, such addresses are also used to send notifications that your
mail was undeliverable, and to check for deliverability.  I would not
discard mail from this class of address until used in a clear pattern
of attacks.  In most cases the moderator's response will be to ban
your address from posting, so it will only work once for the attacker
on a given list.

I'm not sure if addresses matching any of the following can be
triggered to send abusive confirmations:

.*-owner@
.*-request@

I would not filter them unless they are used in a clear pattern of
attacks.

 > Is there any configuration option I can ask the list administrator to
 > enable, to require my password to be provided before the confirmations
 > are sent to me?

As Mark says, no.

 > If this configuration option does not exist yet, could anybody advise
 > what sourcefiles would need modification so as to contribute it as a
 > feature addition?

As Mark says, Mailman 2 is EOL, so we aren't accepting such
contributions.

For Mailman 3, the only approach I can think of, which probably works
for your use case, would be to require DMARC from alignment on
requests via email, and multiple authentication methods (a "social
auth" or a list of one-time recovery keys) on web requests so
confirmation messages would not be needed, except for the first time a
mailbox is linked to a user account.

Such code might be accepted, and an option to disable email
administration of subscriptions might be (but with DMARC checking it's
probably not necessary).  Third party authentication is already
available if sites enable it, and generation of recovery keys could be
added, I suppose.  Code to allow the member to disallow password
resets when no password is set and third party authentication is in
effect might be accepted.  For Mailman 3 I'd start by looking at

mailman/src/mailman/commands/eml_membership.py
mailman/src/mailman/model/subscriptions.py
mailman/src/mailman/model/user.py

Steve


--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: HTTPD

[Stephen J. Turnbull]

Software Info writes:

 > Just a little update. I just ran
 > # obhttpd -d -vvv -f obhttpd.conf and
 > # slowcgi -d -p /
 > to see if I could get anything that made sense show up on the screen
 > and the first error I saw was:  slowcgi: execve
 > /usr/local/mailman/cgi-bin/: Permission denied
 > This is strange because slowcgi runs as www, obhttpd runs as www and
 > www is the owner of the cgi-bin directory. Not sure what I am missing
 > here.

What are the permissions on the cgi-bin directory?  Specifically, you
need "x" on that directory.  I don't know specifically about OpenBSD,
but on macOS "Big Sur" and on a recent missing "x" means you can't
search the directory.  In that case open(2) fails with

 [EACCES]   Search permission is denied for a component of
the path prefix.

and you'd get a 500 from the httpd.  The other possibility might be
that the cgi itself is setuid, and its owner and group don't have
permission to search.  (I don't know if that can actually happen, just
a WAG to cover all bases I can think of.)

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] HTTPD

[Stephen J. Turnbull]

Stephen J. Turnbull writes:

 > I hope that helps, if not, more information about your configuration

Also check your logs for the httpd and for Mailman.  With a 500, it's
likely that Mailman isn't logging much, but it's worth checking.
Typically there will be a traceback in the httpd log.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] HTTPD

[Stephen J. Turnbull]

Software Info writes:

 > Strangely, I am still getting error 500. Internal Server Error. Has
 > anyone gotten this pair working together? Any help would be
 > appreciated.

"Have you got this working" questions should probably be addressed to
OpenBSD lists.  Offhand, I can't recall anyone here mentioning
installing on OpenBSD.

The Quernus page says this:

OpenBSD's https server tries to chroot itself to /var/www in order
to limit the potential damage an exploit could do. Alas, mailman
is quite tricky to get running in a chroot environment.

(I suspect this just means it's a PITA to install a whole Python under
/var/www, but there may be other issues as well.)

As this whole VM will be exclusively running this mailman server
and nothing else, I decided to forego the chroot side of things
and get the httpd server to chroot to /

I can think of two possibilities based on that.  (1) You didn't change
the chroot from /var/www to /.  In that case, Mailman's CGIs won't
find Python, and you get a 500.  (2) You're running an HTTPS-only
server, and you either haven't configured Mailman's URLs to https, or
didn't run .../mailman/bin/fixurl when you configured.  I think  that
should probably give a 404 or maybe can't connect, but it might give
you a 500 under some conditions.

I hope that helps, if not, more information about your configuration
(it's best if you don't tell us about the configuration, and instead
you send us the relevant files, redacting any information you consider
sensitive such as passwords, account names, domain names, IP
addresses, and so on.  Please substitute a consistent identifier for
each redacted item so that we can check that items that appear in
multiple places are consistent.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Problem with outlook.com

[Stephen J. Turnbull]

saran...@intracom-telecom.com writes:

 > Do you know when this change will be deployed in Mailman3 or
 > Python3? do i need to issue a bug report in order to speed up
 > things?

This is the "missing hyphen in iso8859-7" issue, right?  To answer the
question literally, at present we don't want to put this in Mailman
since the suggested code change is in Python.

A bug report to Mailman is unlikely to speed things up, since we want
this in Python, not in Mailman (we'd have to vendor the stdlib email
package).  If you file a bug with Python, I don't know whether this
would be picked up for Python 3.11 (already in beta), but since it's
arguably a bug (cite IANA registration, where the standard MIME
charset name is "ISO-8859-7", case-insensitive) it might be.  In that
case there would be a release in early October of this year.  Since
it's a bugfix it might be picked up for backport to any versions still
in maintenance.

If not, it would likely be in Python 3.12, scheduled for release in
early October of 2023.

Looking a little deeper at the ISO 8859 series of encodings in
cpython/Lib/encodings, *all* of them have a line of the form
"name='iso8859-$PARTNO'", not just iso8859_7.py.  So I don't
understand why this error hasn't happen for every ISO 8859 encoding a
million times a day since 2000 or whenever outlook.com came online.
They're not a small provider.

I will take another look at this but it may take some time.
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: How to wrap text in archived messages

[Stephen J. Turnbull]

David Andrews writes:

 >. Search worked, and users liked it, but I had another 
 > problem with cPanel, don't even remember what. The cPanel folks said 
 > they would not give me technical support as long as I used HTDIG,

Condolences.  It's not clear to me that that is any of their business
(unless you need tech support for htdig itself, in which case I can't
really get upset with them, YMMV), though.

 > so now I have no search, a problem with a system with over 300 lists.

I don't know what your situation is (eg, private lists), but for your
public or private-but-not-sensitive lists, mail-archive.com[1] might
be an option.  Somewhat inconvenient to have to switch back and forth
for full archives (they strip images and most other attachments)
vs. search capability, but better than nothing.

Unfortunately, it's a hobby site, so I doubt they provide any service
at all for private lists.  At least their FAQ is of extremely high
quality, I think you can probably trust them to be quite up to the
task for you, if you have lists with minimal requirements for privacy,
and don't need attachments stored and archived there.

Steve

Footnotes: 
[1]  mailarchive.com seems to live in Poland, and otherwise I couldn't
read their page. :^)

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: How to wrap text in archived messages

[Stephen J. Turnbull]

Mark Sapiro writes:
 > On 5/25/22 19:32, Mark Dale via Mailman-Users wrote:

 > > Before I go down this rabbit hole: was there any particular
 > > reason (back in the day) that Pipermail was favoured (and
 > > implemented) over MHonArc.
 > 
 > Mailman was initially implemented by John Viega in the mid 1990s to 
 > manage a mailing list for fans of the Dave Mathews Band. I don't know 
 > why pipermail was chosen, but MHonArc was fairly new at that time and 
 > pipermail was probably more mature.

I doubt that Hypermail (the original code) was more mature than
MHonArc, but it was written in Python and so considered more
appropriate for bundling with Mailman.  "Batteries included" has
always been a goal for Python, and Mailman inherited it.  MHonArc is a
Perl script, written in an idiosyncratic style (so I was told by a
Perlmonger), and I myself would have to punt on most maintenance
questions for MHonArc despite admining lists using it for a decade.

I can't speak for other members of the team, they may be more capable
with Perl than I am.  But to me, this would be the main issue with
using MHonArc -- you'd likely be dependent on somebody other than us
if you need help with it.  (Probably you could get some help on this
list, though.)

As far as I can recall, Pipermail never been especially recommended
over 3rd party solutions like MHonArc or external services like
mailarchive.com, it's just easy to use because it's guaranteed to be
there, we provide support for it, and it turned out to be "good
enough" for an awful lot of lists.

Regards,
Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Mailman3 - from Whoosh to Xapian

[Stephen J. Turnbull]

Kimmo L. writes:

 > import xapian
 > ModuleNotFoundError: No module named 'xapian'

I'm pretty sure this occurs when the Xapian application and its
libraries are not installed.  xapian-haystack is an adapter from C++
(or maybe C) to Python, it is not a complete installation of the
Xapian package.  See Requirements at

https://pypi.org/project/xapian-haystack/

Have you installed Xapian itself?  If you have and still no luck, let
us know.  I know that many sites have successfully installed Xapian as
the indexer for their Mailman instance.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Problem with outlook.com

[Stephen J. Turnbull]

saran...@intracom-telecom.com writes:

 > /usr/lib/python2.7/encodings/iso8859_7.py, in def getregentry(),
 > there is a line: name='iso8859-7' which if it is changed to:
 > name='iso-8859-7' then the encoding is sent correctly and the
 > emails are received by the Exchange server. Nevertheless, i'm
 > wondering if there is a better solution than changing the python's
 > file, so any further thoughts are welcome.

No, there is no better solution for Mailman 2.  The handling of email
headers is done by a library supplied by Python, and that form of the
charset name is in all of the ISO 8859 unibyte encodings.  In Python
3/Mailman 3 we may be able to change this, but both Python 2 and
Mailman 2 are end-of-life so this change won't be made there.

In theory you could also change it in Mailman, but it's risky because
you'd need to check for it and then substitute this form for the
original everywhere it needs to be done, and I can imagine several
pathways where it might be present.  If you're lucky it might be
abstracted into a single function that only needs to be changed in one
place, but what if not?

"iso-8859-7" with the dash is the preferred form (see
https://www.iana.org/assignments/character-sets/character-sets.xhtml)
so it "should not" cause problems to make this change locally.  But
that's not something we'd want to bet on for other folks' installations.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: How to wrap text in archived messages

[Stephen J. Turnbull]

Mark Dale via Mailman-Users writes:

 > And of course, any such "nl2br" equivalent will do exactly the same
 > as wrapping with P tags -- with everything left aligned.

Right.

I'm not sure that we couldn't do better nowadays with libraries that
will handle the same DOM that browsers do, but it certainly wasn't
possible in 1994.  And even with those libraries it would require a
complete rearchitecture of the archiver.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] How to wrap text in archived messages

[Stephen J. Turnbull]

Mark Dale via Mailman-Users writes:

 > I'm looking for a way to wrap lines in archived messages.

Executive summary: There's not really a good way to do this.  It's
extremely complicated, *especially* in email (as opposed to most
"normal" text) because of quoting conventions in email.

 > With zero understanding of Python my attempts to implement this
 > have failed so far and I may well be barking up the wrong tree
 > completely. Any clues or pointers gratefully received.

It's not your lack of Python, it's that reliably reformatting email
for different formats of email is a *very* hard problem in natural
language processing, and requires some knowledge of message user agent
internals.  And that's why Pipermail punts by just wrapping the whole
thing in a PRE element.  Works for Mutt users (= Unix email elders).

Gory details follow (because I think it's an interesting problem!)

 > Looking at the HTML page source -- in both cases (wrapped and
 > unwrapped) I see the message content is enclosed by PRE tags.

Right.  PRE is not very pretty as HTML goes, but it works OK for all
RFC-conforming text/plain email.  I assume that that in fact this
comes from text/plain parts created by the author's MUA, because the
agents that we use to transform a text/html part to text/plain will
format to a reasonable width such as 72 characters.

 > And the lines in that block that seem responsible for the PRE tags are ...
 > 
 > lines.insert(0, '')
 > lines.append('')
 > 
 > My question is: Can those PRE tags be removed and replaced with
 > something equivalent to PHP's "nl2br" (which inserts a line break
 > BR in place of new line entries)?

No, because there *are no* newlines to break those very long lines.
These MUAs use newline to mean "paragraph break", not "line break".

You might get a better result in these messages by removing the "PRE"
tags, and wrapping each line with "...", but that's a real
hack, and almost certain to make RFC-conforming email look quite ugly,
because every line becomes a paragraph, and you'll lose all
indentation.  Eg, in the code blocks you posted, all the lines will
end up flush left.  If your members are posting code or poetry, or
using indented block quotations etc, they're likely to be extremely
unhappy with the result.

Python's standard library does have a textwrap module, but I'm not at
all sure it's suitable for this.  If you know that the long lines of a
message are actually paragraphs, you can use something like

from textwrap import wrap
# work backward because wrapping changes indicies of later lines
for i in range(len(lines) - 1, -1, -1):
# NDT = detect_prefix(lines[i])
lines[i:i+1] = wrap(lines[i], initial_indent=NDT, subsequent_indent=NDT)

If a line is indented or has a quoting prefix, you have to detect that
for yourself and set NDT to that prefix.  Something like

import re
prefix_re = re.compile('[ >]*')
def detect_prefix(line):
m = prefix_re.match(line)
return m.group(0)

should capture most indentation and quoting prefixes, but there are
other conventions.

Whether you use P elements or the textwrap module, it's probably a
good idea to find out how long the long lines are, and what percentage
of the message they are, and avoid trying to wrap a message that looks
like it "mostly" has lines of reasonable length.  If you don't, and
your target is the old "typewriter standard" width of 66, and somebody
using an RFC-conforming MUA just prefers 72, you'll reformat their
mail into alternating lines of about 60 characters and 10 characters.
Yuck ...

Which of the above would work better for you depends a lot on the
typical content of your list.  But issues with quoting and indentation
are likely to have you tearing your hair out.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Double opt-in, Question about adding people to a Mailman list

[Stephen J. Turnbull]

Mark Sapiro writes:
 > On 5/22/22 00:17, Jayson Smith wrote:

 > > I run a Mailman 2 list for an organization of writers with disabilities. 
 > > Recently our president has become concerned that some people wanting to 
 > > join the group may not be responding to the standard Mailman 
 > > subscription confirmation message

@Jayson Is this especially a problem for people with disabilities, as
compared to new subscribers in general?

In fact, I expect the answer is "no".  But I think it's worth trying
to improve this in Mailman 3 for the general population, too, and if
we can improve this in a more accessible way I would like to be aware
of it.

 > By default, confirmation requests are sent with From: and Subject: like
 > ```
 > From: listname-requ...@example.com
 > Subject: confirm+the_hex_token
 > ```
 > If you, or the installation sets
 > ```
 > VERP_CONFIRMATIONS = Yes
 > ```
 > in mm_cfg.py, they will be sent like
 > ```
 > From: listname-confirm+the_hex_token

@Mark This is "From: listname-confirm+the_hex_to...@example.com",
right?  I'm not sure that's much better, especially in Jayson's
situation where the email address and the organization are hard to
associate with each other.

 > Not really. Person C can still send email to person B spoofing person A. 
 > In your scenario, upon receiving email allegedly from person A, person B 
 > would need to respond to person A asking for confirmation and receive 
 > confirmation from person A before adding person A to the list.

Note that the point of this multipart handshake is that email itself
is insecure; it is rather easy to fake authorship of an email message
well enough to get past someone who is not well-versed in email
arcana.  It is much harder to fake the ability to read from a mailbox.

So it's really not possible to omit the "send token" and "receive
confirmation" steps if you want to be sure the person who requests a
subscription has the right to request people send stuff to the
mailbox.

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: group mismatch

[Stephen J. Turnbull]

Christian Buser via Mailman-Users writes:

 > You know that in a mailing list everyone should help if he can and
 > not only consume?

Technically correct, but please don't be so antagonistic.  

 > So then, please unsubscribe here.

What you do with your own filters is your business, but for the
mailing list this is outright wrong.

Mailing lists are easy to abuse, as we all know.  We put a lot of
effort into minimizing the damage and closing as many vulnerabilities
as we can, but the bad folks are out there developing new ones 9-5
every day, and some of them can't be closed without destroying
Mailman's main feature: the convenient discoverable interface for
managing list mail flows.  I hope that every Mailman 2 admin is
subscribing and continues to subscribe to this list, so that we can
help them serve the subscribers, and when necessary address
vulnerabilities and help them upgrade.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: group mismatch

[Stephen J. Turnbull]

Lucio Chiappetti writes:
 > On Sun, 15 May 2022, Jon Baron wrote:
 > 
 > > I am trying to use spamassassin by running everything through
 > > /etc/procmail,
 > 
 > Sorru, I do not understand what procmail and spamassassin, intended to 
 > process INCOMING mail, have to do with mailman which is SENDING OUT
 > mail.

I assumed the OP knows procmail fairly well, doesn't understand
milters (or whatever the equivalent is for $MTA), and is using a
pipeline like

sendmail | procmail | spamassassin && mailman

since the error message implied that mailman was started by procmail.
procmail may not be the tool of choice these days, but it should work.

Note that the error message mentions postfix several times; I'm not
sure that a sendmail cf is of much use to the OP.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: [ext] How do I send an email to all list admins?

[Stephen J. Turnbull]

Bruce  Johnson via Mailman-Users writes:

 > Ralf Hildebrandt writes:
 > 
 > How do I send an email to all list admins?
 > 
 > I have a script I wrote (or rather I’m pretty sure I have one of
 > Mark’s scripts that  I modified :-) that gets the name and address
 > of every list on our system and exports a tab-delimited file with
 > the admin and moderator emails as separate lists .

Interesting ... so this *is* a common need, I guess I haven't heard
about it either due to inattention  or
perhaps it's just that people with a site of such scale have the
chops to go straight to bin/withlist.

I'm still not sure this is needed/a good idea, but I created a
wishlist issue for it.  Comments welcome, especially on the default
configuration details for these lists.  Eg should list admins be
allowed to post, how about moderators? or should it be pure announce
for the site admin?  I think it should be archived, any problems with
that?  How are these related to the mailman@site list?

https://gitlab.com/mailman/mailman/-/issues/1007

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] How do I send an email to all list admins?

[Stephen J. Turnbull]

Ralf Hildebrandt writes:

 > How do I send an email to all list admins?

AFAIK, one by one, or you could make a mailing list for them.  I guess
you could save them one message by putting your announcement in the
welcome message.

I think the original idea of the mail...@list.example.net list was to
put the admins there.  You'd have to do it by hand, and pretty much
nobody seems to do that.  I don't recall anybody asking for this
before, although I'm pretty sure it occasionally comes up.

In Mailman 3 we could make a feature of this, but Mailman 2 is EOL, so
it won't happen there.  I'm not sure this is worth it since you'd need
to configure all the options anyway (I'm not sure there are obvious
defaults for a lot of them), so getting the list of lists is
the least time-consuming part of the task.

In Mailman 3, perhaps we could make it easier to get the list of lists
(or maybe the site admin can already get it in the top listinfo page).

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] The Hotmail complaint saga continues

[Stephen J. Turnbull]

Jayson Smith writes:

 > 1. Is Linode, my VPS provider, also receiving these complaints? If so, 
 > I'm surprised they haven't at least sent me a notice telling me about them.

Seems unlikely, both on general principles and since you haven't heard
from them.

Have you confirmed that these notifications are really from Hotmail
staff (or an automated process there)?  Although other things equal
it's unlikely, the most likely possibility (the subscriber complained)
is denied by the subscriber so it could theoretically be some third
party trying to mess with you or the subscriber.

Also, have you confirmed that it's actually the specific messages from
you or your list that were cited?  Is it possible that some spammer is
spoofing your return address, and the subscriber is legitimately
complaining about mail that to the email provider appear to be from
you but to the subscriber are from somebody else?  Is it possible your
Linode VPS has been hacked, or the MTA is an open relay, and is being
used to send spam?  If you've confirmed that the complaints are citing
mail you know you sent, these possibilities don't apply, and they're
fairly unlikely anyway given that it's only one subscriber that's
having the issue.

 > 2. Does anyone know if having received these complaints might cause 
 > Microsoft to be more likely to add my IP to their infamous block list? 

More likely, yes, but how much more likely, you'd have to get an
answer from Microsoft.  I don't know anybody who has gotten anything
useful out of them, though.  Mistaken additions to block lists
anywhere seem to quite random for good actors, and the blockers are
rarely willing to explain what the problem was, or how to avoid it.

You could try explaining the situation to staff@hotmail, and get the
subscriber to do so too.  But I wouldn't expect too much.

You could also look up what their mitigation strategies, if any, are.
Some providers have services you can sign up to which provide more
information about complaints, and guidelines on how to keep your
list(s) in good standing with the providers.  I don't know about
Microsoft/Hotmail.

There are also some general rules:

1.  Check that your server is not in any of the reputable RBLs.  I
don't have a list offhand.
(A couple of RBLs are known to shake down sites by putting them in
bad actor lists and then asking for money for a "service" to help
clean your reputation.  The big providers like Hotmail know who
they are and ignore them, you probably can ignore them too.)
2.  Make sure that your DNS has correct configurations for SPF and
DKIM.  Make sure that your signing keys are correct.  It may halp
a little to do DMARC, too, even if you just set a policy of
p=none.  If your lists accept posts from off-host, you likely
would benefit from configuring ARC as well.
3.  Everything that goes *in* to your MTA *except from Mailman* should
be spam- and virus-filtered.  That should include mail generated
on the Mailman host.  Then you can safely forward all mail from
Mailman.  (Filtering mail *from* Mailman on the way out is very
expensive unless you have very few subscribing domains.)
4.  If your lists are open subscription and of moderate size, and you
don't require moderator approval of subscriptions, we recommend
setting all new subscribers to "hold for moderation".  If the
first post is acceptable, then set them to default processing or
accept.  For many lists that nearly eliminates "drive-by" spams
and other unwanted posts.  YMMV of course.

 > 3. I haven't ever received such complaints before this situation started 
 > on March 24. Is Microsoft noted for somehow generating spurious 
 > complaints like this?

Not that I've heard of.  Maybe somebody else has.  The only case I
know of like that was AOL, which put the "Report Spam" button next to
the "Delete Message" button.  Perhaps Hotmail has a similar issue, but
I haven't heard of it.  You could ask the subscriber if it's possible
she missed and hit the wrong button, but since she's already denied
doing that, you'll have to be careful about it.

I'm sorry not to be of more help, but given that big providers like
Microsoft are never willing to engage in situations like this, there's
little information in the public domain.  I'm afraid you're going to
have to work it out with the subscriber.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: ARC protocol in Mailman 2?

[Stephen J. Turnbull]

Sorry for the top-post, my regular laptop is busted so I am using GMail.

Executive Summary: Either removing "sv" from the "mode" line in
openarc.conf or removing that line entirely should make OpenARC behave
correctly for you.

Gory details (sorry for verbosity):

I took a look at the OpenARC man pages and sample configuration file.  It
seems to me that Sendmail+OpenARC should work fine in your single-host
configuration, although I guess that you're in a VM, and that might affect
the network configuration as perceived by the MTA+milter.  I don't think it
*should* but I'm not familiar with Linode VPSes.

There are two optional configuration files that may be set in openarc.conf:
InternalHosts, which contains a list of those hosts where the milter trusts
an incoming message from those hosts to be authentic, and PeerHosts, which
lists those hosts where the milter is not invoked at all.  (I guess it's
invoked by the MTA but does nothing.)  My guess is that the "peer" in
PeerHosts does not mean peers of this host, but rather those hosts that
communicate with peers in other administrative (untrusted) domains.  You
should not have a PeerHosts files (or it should be empty), and your
InternalHosts file can probably not be specified, but perhaps it should be
specified and contain all aliases for the domain that might be in use (IP4
and IP6 addresses, domain names, maybe localhost).

The other possibility is that you have mode specified improperly, but I
think that no mode spec or empty mode spec should do what you want.  I am
guessing that you have "mode sv" set.  This may explain why you have full
sets of ARC headers incoming and outgoing, and they break.  According to
the man page, if neither s nor v is set, external connections will be
verified but not signed, and internal host connections (ie, from Mailman)
will be signed but not verified (ie, trusted to have the appropriate
Authentication-Results header).

If changing mode doesn't help, it may help if you can send the complete
headers from a message after it has passed through your Linode host so I
can see the complete set.

On Sun, Apr 10, 2022 at 9:06 PM Jayson Smith 
wrote:

> Hi,
>
> To answer your main question, everything is on one Linode VPS, there's
> no networking or separate servers/hosts involved, everything's on one box.
>
> Jayson
>
> On 4/10/2022 4:41 AM, Stephen J. Turnbull wrote:
> > Jayson Smith writes:
> >
> >   > I've recently been playing with the OpenARC milter for Sendmail.
> >
> > IIRC, OpenARC is the sample implementation by the ARC developers.  It
> > should be robust.  Mailman uses a different implementation based on
> > Python.  (You should use an MTA-based implementation if it works
> > correctly for all the usual reasons: performance, more correct
> > behavior especially for SPF.)
> >
> >   > I have it running, and it seems to be working properly, except for
> >   > one thing. When a message is sent to one of my Mailman 2 lists,
> >   > OpenARC adds an ARC set to the incoming message before it ever hits
> >   > Mailman.
> >
> > It's been a while since I read the RFC, but AIUI, adding the full set
> > is incorrect behavior.  An ARC processor should add only
> > ARC-Authentication-Results on the way in to the AD (administrative
> > domain), then add any DKIM stuff, the ARC-Signature, and the ARC-Seal
> > on the way out of the AD.
> >
> > The fact that it adds the full set suggests that it thinks that
> > Mailman is outside of the AD.
> >
> >   > Then the message hits Mailman,
> >
> > Is Mailman running on the same host as Sendmail?  Is it the same host
> > running the same instance of Sendmail on the way in and the on the way
> > out?
> >
> >   > and on the way out, OpenARC adds another ARC set to the message,
> >   > this one indicating the ARC validation failed. Now, if I understand
> >   > the RFC correctly, any ARC-aware MTA that sees this failure is
> >   > going to treat the entire ARC chain as though it never existed
> >   > since the most recent ARC set indicated validation failure.
> >
> > That is not quite correct.  An ARC-capable MTA will treat the ARC
> > chain as though it begins at Mailman's outgoing MTA.  If Mailman's MTA
> > has a good reputation, that may help with some filters.  It will not
> > help with DMARC, though.
> >
> >   > If this is the case, then the whole exercise is pointless.
> >
> >   > Now some questions. OpenARC has a configuration option to treat
> certain
> >   > hosts as trusted, and the Man page indicates that if no hosts are
> listed
> >   > there, localhost is automatically added. If this is true, I don't

[Mailman-Users] ARC protocol in Mailman 2?

[Stephen J. Turnbull]

Jayson Smith writes:

 > I've recently been playing with the OpenARC milter for Sendmail.

IIRC, OpenARC is the sample implementation by the ARC developers.  It
should be robust.  Mailman uses a different implementation based on
Python.  (You should use an MTA-based implementation if it works
correctly for all the usual reasons: performance, more correct
behavior especially for SPF.)

 > I have it running, and it seems to be working properly, except for
 > one thing. When a message is sent to one of my Mailman 2 lists,
 > OpenARC adds an ARC set to the incoming message before it ever hits
 > Mailman.

It's been a while since I read the RFC, but AIUI, adding the full set
is incorrect behavior.  An ARC processor should add only
ARC-Authentication-Results on the way in to the AD (administrative
domain), then add any DKIM stuff, the ARC-Signature, and the ARC-Seal
on the way out of the AD.

The fact that it adds the full set suggests that it thinks that
Mailman is outside of the AD.  

 > Then the message hits Mailman,

Is Mailman running on the same host as Sendmail?  Is it the same host
running the same instance of Sendmail on the way in and the on the way
out?

 > and on the way out, OpenARC adds another ARC set to the message,
 > this one indicating the ARC validation failed. Now, if I understand
 > the RFC correctly, any ARC-aware MTA that sees this failure is
 > going to treat the entire ARC chain as though it never existed
 > since the most recent ARC set indicated validation failure.

That is not quite correct.  An ARC-capable MTA will treat the ARC
chain as though it begins at Mailman's outgoing MTA.  If Mailman's MTA
has a good reputation, that may help with some filters.  It will not
help with DMARC, though.

 > If this is the case, then the whole exercise is pointless.

 > Now some questions. OpenARC has a configuration option to treat certain 
 > hosts as trusted, and the Man page indicates that if no hosts are listed 
 > there, localhost is automatically added. If this is true, I don't know 
 > why OpenARC is processing messages on the way out of Mailman, since that 
 > should be a localhost to localhost connection.

It's processing on the way out because what ARC establishes is a chain
of custody:

1.  I checked it on the way in (ARC-Authentication-Results = A-A-R).
2.  I watched it all the way to the outgoing MTA, and only processes I
trust touched it.  They may have changed it (invalidating the
author's DKIM signature) but I assure you my processes didn't
change anything you care about (specifically From if you're doing
DMARC).  If you trust me, you can trust the authenticity of From.
3.  Here's my ARC-Signature (= A-Sig) on the final state (which I
swear is authentic to the author's intent) just before I put it
back on the wire, and here's an ARC-Seal to bind up both the A-A-R
and A-Sig so you can trust them.

If you don't do A-Sig and A-Seal on the way out, the chain of custody
is broken if changes are made to the message (such as mailing lists
typically do) because the incoming signatures (author's DKIM and all
intermediate A-Sigs, as well as your incoming A-Sig!!) are all broken.

 > Is OpenARC the best Milter to use with Sendmail for this purpose? 
 > Is there something else I'm doing wrong or overlooking?

I don't see why not.  I'm pretty sure that's what most of the IETF
Working Group used, although Google and Yahoo! probably used their own
code.

I would guess there's some issue with OpenARC configuration, or with
your network configuration (this might include DNS! does Mailman live
on a different domain aliased to that host?), that makes OpenARC think
Mailman leaves the AD.

If this doesn't provide the necessary hints, let me know.  I don't
have time to study up on OpenARC today, but if needed I'll try to get
to it in the next couple of days.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Step by step guide to installing Mailman

[Stephen J. Turnbull]

Odhiambo Washington writes:
 > On Mon, Apr 4, 2022 at 7:52 PM Carl Zwanzig  wrote:

 > > Please also note that for some uses and for some people, MM2 is probably
 > > still the better choice-

 > Where did I go wrong in my advise to the OP?

You didn't. :-)

 > I actually gave him all the options. I did not tell him to NOT install MM2
 > :-)

You did point out that it's EOL, and that is likely to affect many
site's decision whether to install a product.  Carl is simply
explaining why Mailman 2 may still be a good choice.

Steve


--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Reply-to addresses

[Stephen J. Turnbull]

Christian via Mailman-Users writes:
 > Hi Michael   
 > 
 > I don’t think you have a problem here. The "From:"-address will probably 
 > look like this:
 > 
 > "Firstname Lastname usern...@domain.com" 

This isn't true.  "domain.com" is replaced by "---" because it's
believed that having more than one address in any part of From
(including the display name and comments) is a spam indicator at many
recipients.

 > This means that the original sender’s address is in the "realname"
 > (or "comment")-part of the address field. The sender appears still
 > to be  and replies should go there -
 > unless your list is set to deliver replies only to the original
 > sender (as it is here in the Mailman list).  

It's more complicated than that.  We try very hard to ensure there is
a machine-replyable address for the author.  So when either of the
"munge From" options is set, the author's address will be added to
Reply-To.  The only case where this is not done is for an anonymous
list, but that's probably extremely unattractive.

 > > After years of working fine, we'd recently been having bouncing 
 > > problems specifically with gmail addresses.

I'm surprised you have problems specifically with Gmail.  Their DMARC
policy is still p=none, so DMARC is not why they're bouncing.  Lack of
SPF seems more likely.  AOL and Yahoo! are much more comon sources of
problems.

 > > I changed from_is_list to Munge From but now the reply to headers
 > > have both the list address and the original sender's address. So
 > > I assume the original sender gets sent two copies of responses,
 > > one from the list and the other directly to them.

That is correct.

 > > I'm hoping I can fix this easily.

Getting rid of the poster address in Reply-To requires patching the
code.  We are not going to change this in Mailman 2, for sure.  The
general sense of the developers is that it's a bad idea to completely
remove the poster's address, and they can work around with no-dupes,
so we probably will not change it in Mailman 3 either.

Depending on the size of the list, it may be easy.  Go into the
Membership Management | Membership List section, and set the no-dupes
attribute on for everybody.  If this would be annoying to impossible
because the list is very large, Mark may have script for this but your
provider would have to run it.

This is not a proper fix, because it's implemented by not sending the
post to subscribers whose addresses are already in To or CC.  There's
no way for Mailman to be sure they actually get it (although they
almost always do), and of course they lose any decorations (such as
useful links in the footer).

You can partially mitigate the problem by setting from_is_list to
"No", and instead setting dmarc_moderation_action to "Munge From".
Any users from DMARC p=none domains will not suffer from duplicates.
However, AOL, Yahoo!, and several other major email providers set
p=reject, and users at those providers will suffer from duplicates
unless they set no-dupes.

If you have access to a DNS query utility, you can find out whether
common subscriber domains have a p=reject or p=quarantine policy by
prefixing the domain with "_dmarc." and querying the TXT record.  For
example, with the popular 'host' program for gmail.com addresses, the
query looks like this:

% host -t TXT _dmarc.gmail.com
_dmarc.gmail.com descriptive text "v=DMARC1; p=none; sp=quarantine; 
rua=mailto:mailauth-repo...@google.com;
% 

You're looking for the "p=" attribute.  The "sp=" attribute isn't
relevant.  On Un*x hosts, you may also have the nslookup or dig
utilities.  The query template suggested above is not 100% reliable
because of variations in email addressing to the same domain, but it's
probably 95% reliable
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Handling non-members in Cgi/options.py

[Stephen J. Turnbull]

Mark Sapiro writes:
 > On 2/22/22 10:00, Mark Sapiro wrote:
 > > On 2/22/22 05:56, Stephen J. Turnbull wrote:

 > >> I think in both cases you can return the login page with the address
 > >> filled in.
 > > 
 > > You are correct and it's trivial to do. I will fix it.
 > 
 > Fixed at 
 > https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/revision/1888

Nice!  That makes a rough day a little bit nicer. :-)

Steve
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Handling non-members in Cgi/options.py

[Stephen J. Turnbull]

Mark Sapiro writes:

 > There is still a subtle difference in that if the address given is
 > a member, the login page asks only for a password, but if it's not
 > a member login page asks for both and address and a password, but I
 > think that's the best that can be done.

I think in both cases you can return the login page with the address
filled in.  Not sure if this would be easy to do in the code, but I
think this would satisfy both the "minimum effort for user" criterion
and the "don't reveal subscription status" criterion, unless I
misunderstand the scenario.

Steve


--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: illegal BOM

[Stephen J. Turnbull]

Executive summary:

- There is a BOM in the X-Ham-Report header field.
- There is reason to believe that it, and not just any non-ASCII,
  triggered this rejection.
- Disabling the X-Ham-Report field (and possibly an X-Spam-Report
  field) seems to be the best option.

Christian via Mailman-Users writes in an earlier message:

 > Diagnostic-Code: smtp; 550 Headers contain illegal byte order mark (BOM)

and now:

 > Hello Mark Sapiro. On Thu, 3 Feb 2022 14:31:11 -0800, you wrote:
 > 
 > >>> X-Ham-Report: Spam detection software, running on the system
 > >>> "crift.digimouse.eu", has NOT identified this incoming email as
 > >>> spam.  The original message has been attached to this so you can view
 > >>> it or label similar future email.  If you have any questions, see
 > >>> root\@localhost for details. Content preview:  systemerweiterungen,
 > >>> benutzer, dein account, startobjekte: ist da noch was drin? Jean-Luc
 > >>> Aeby CH-4052 Basel > Am 03.02.2022 um 09:05 schrieb Max Röthlisberger

In the line above there is a SMALL LATIN LETTER O WITH UMLAUT (U+00F6)
which gets no complaint.

 > >>> Mus : > > Guten Morgen zusammen > > Mein MacBook Pro,

In this line, immediately before "Guten Morgen" there is a ZERO-WIDTH
NO-BREAK SPACE (ZWNBSP, U+FEFF) aka "byte order mark" or BOM.  I'm
satisfied that the error above really is complaining about the ZWNBSP,
and not random non-ASCII.  I conclude that the spam milter used a
proper content transfer encoding for the X-Ham-Report header field.

ZWNBSP is now deprecated in favor of WORD JOINER (WJ, U+2060), but
conforming implementations should support both with identical
semantics, except as the first character where ZWNBSP has BOM
semantics and WJ is just a PITA.

 > >>> OS 10.11.6 sucht zu Hause nach einem Neustart 4 - 5 > mal im Heimnetz
 > >>> den ? [...]  Content analysis details:   (-0.0 points, 4.0 required)

 > > This is the only header in the message that looks suspicious. I 
 > > suspect the `?` characters are actually non-ascii characters in an 
 > > unencoded header and that's the problem. I think whatever is adding 

I suspect it's not unencoded, since it's very specific about the BOM,
and the BOM is not the first non-ASCII character in that field.  I
don't think this is a non-ASCII problem, I believe it's BOM-specific.

It appears to be the first character in body of the message quoted,
and ends up in the middle of the body of the message rejected.  I
guess the original source is a broken MUA that delegates editing the
body to an editor that prepends a BOM to all Unicode files (probably
including UTF-8, which is severely deprecated).  Then it copies that
file including BOM into the message after the CRLFCRLF that separates
the header from the body.

This really doesn't hurt anybody because of the way mail is parsed.
IMO the real culprit here is the excessively strict MTAs that are
apparently decoding that header field and examining it for merely
deprecated features of Unicode, and rejecting on that basis.  But
you're not going to get that fixed at other people's sites.

 > > this header (SpamExperts ?) is the root of the problem. If this can 
 > > be configured to not add that X-Ham-Report: header, that may solve 
 > > the issue.

 > I’ll contact the provider whether it is possible to switch off the
 > spam detection software for our lists.

You probably don't want to do that, though.  Even if you trust your
posters, there's no reason to suppose one couldn't get hacked.

 > > Or, you could patch 
 > > https://bazaar.launchpad.net/~mailman-coders/mailman/2.1/view/head:/Mailman/Handlers/Cleanse.py#L62
 > >  
 > > and add
 > > ```
 > > del msg['X-Ham-Report']
 > > ```
 > > to have Mailman remove it. That may help.

I recommend this instead.  I guess that in the case of spam there
might also be an X-Spam-Report header field.  Depending on under what
circumstances you block Spam, you may want to disable that as well.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Listservers currently hosted with EMWD - urgent request for assistance

[Stephen J. Turnbull]

Note: Reply-To set to mailman-us...@mailman3.org.  Please check that
any replies are addressed to that list, and not mailman-users@python.org.

Jonathan,

FWIW, I haven't heard anything.  I have 3 students submitting MS/PhD
theses in less than (checks watch) 18 hours, but after that I'll see
if I can find out anything.  I wouldn't bet on it, though, as I don't
think anyone here has a connection to the Carpenter family or whoever
is left of the EMWD staff.  I assume you have checked the emwd.com
website?

 > We have our listservers hosted with EMWD which stopped working on
 > 23 December.

What does "stopped working" mean?  I assume that list traffic is no
longer being forwarded to subscribers, but perhaps other services are
working.  It may help if you can answer the following questions.

1.  Are you using the GNU Mailman-supplied Postorius/HyperKitty
combination for administration and archiving, or EMWD's
proprietary "Harmony" (Affinity/Empathy) platform?
2.  Can you see archives (if any)?
3.  Can you log in to the administrative interface?
4.  Is mail to the lists being returned to sender, or just
disappearing into the void?
5.  You mention an automated response in a later message.  Is that to
an email you sent, or from a web-based issue-tracking system?
What does it say about the current status of EMWD services, if
anything?

Regards,
Steve

 > We have since found out the devastating news that
 > Brian Carpenter passed away last year. Brian was really helpful
 > when we moved our listservers from a legacy system to hosting on
 > the EMWD platform. My sincere condolences to his family and
 > friends. 
 > 
 > As we can no longer get a response from EMWD about our listserver issue I 
 > was wondering if any one else in this community could help.
 > 
 > Many thanks.
 > 
 > Jonathan
 > 
 > Jonathan Ashby
 > ICT Strategic Communications Project Manager
 > Londonwide LMCs and Londonwide Enterprise Ltd
 > Working days: Tuesday and Wednesday
 > Direct dial: 020 3818 6228
 > Mobile: 07768 109601
 > Fax: 020 7383 7442
 > Email: jonathan.as...@lmc.org.uk
 > Web: www.lmc.org.uk
 > Twitter: @LondonwideLMCs
 > 
 > ​
 > ​
 > ​
 > Visit www.lmc.org.uk/coronavirus-covid-19 for the latest official resources 
 > and guidance on covid-19.
 > ​
 > ​Remember that you can use WhatsApp and FaceTime for online consultations as 
 > well as the usual online tools.
 > ​ Think of the environment. Do you really need to print this email?
 >  This email and any files transmitted with it are confidential and intended 
 > solely for the use of the individual or entity to whom they are addressed. 
 > If you have received this email in error please accept our apologies and 
 > notify the sender.
 > ​
 >  The registered and office address for Londonwide Local Medical Committees 
 > Limited is: Tavistock House South, Tavistock Square, London WC1H 9LG. 
 > Registered in England No: 6391298.
 >  
 > ​Londonwide Enterprise Limited is a wholly owned subsidiary of Londonwide 
 > Local Medical Committees Limited. Londonwide Enterprise Limited is 
 > registered at: Tavistock House South, Tavistock Square, London WC1H 9LG. 
 > Registered in England No. 6990874. Londonwide Enterprise Limited is 
 > registered as a Company Limited by Shares. VAT no: 130 1454 66.
 >  
 > ​Londonwide Local Medical Committees Limited and Londonwide Enterprise 
 > Limited do not provide legal or financial advice and thereby excludes all 
 > liability howsoever arising in circumstances where any individual, person or 
 > entity has suffered any loss or damage arising from the use of information 
 > provided by Londonwide Local Medical Committees Limited and Londonwide 
 > Enterprise Limited in circumstances where professional legal or financial 
 > advice ought reasonably to have been obtained.
 >  
 > ​Londonwide Local Medical Committees Limited and Londonwide Enterprise 
 > Limited provide guidance and support to GPs and practice teams in the 
 > Londonwide area. Additionally Londonwide Local Medical Committees Limited 
 > provides representation to GPs and practice teams in the Londonwide area. 
 > Londonwide Local Medical Committees Limited and Londonwide Enterprise 
 > Limited strongly advises individuals or practices to obtain independent 
 > legal or financial advice.
 > ​
 > --
 > Mailman-Users mailing list -- mailman-users@python.org
 > To unsubscribe send an email to mailman-users-le...@python.org
 > https://mail.python.org/mailman3/lists/mailman-users.python.org/
 > Mailman FAQ: http://wiki.list.org/x/AgA3
 > Security Policy: http://wiki.list.org/x/QIA9
 > Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
 > https://mail.python.org/archives/list/mailman-users@python.org/
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to 

[Mailman-Users] Re: Charset chaos

[Stephen J. Turnbull]

Mark Sapiro writes:

 > Mailman 2.1 is almost 20 years old and predates the common use of
 > unicode and utf-8. That's why the German message catalog is
 > iso-8859-1 encoded. What you did below is the appropriate fix.

Speaking to Johannes and those in his situation:

To be honest, in the past we could have done for you what you did for
yourself, but that would surely cause the opposite problem for a lot
of existing systems, not to forget annoying the translators.  For many
years, our European users were perfectly happy with iso-8859-1, and
many continued to use that by default in their text (or at least were
happy to do so for the relatively few changes they tended to make in
footers, list descriptions, and the like).  Nowadays of course most
systems default to UTF-8, there's little reason to ever use anything
different, and so you unfortunately ended up with a system with mixed
charsets (I wouldn't go so far as "chaos," I live in Japan and know
chaos in charsets intimately ;-).

Until Mailman 2 went EOL from a development point of view a few years
ago, we (well, me, but I expect the other devs will agree) didn't
think it's worth the chaos for existing Mailman 2 installations that
haven't been reconfigured in a decade or so that would certainly ensue
from our decision to go "all UTF-8".  Sorry about being so inertial
about it, but I think it still was the right way to go.  Now of course
Mailman 2 is EOL, so it makes sense to support you individually (OK,
you supported yourself, and we just say "well done" ;-), but not to
make changes to Mailman 2.

Sincere regards,
Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Should CSRF check disregard case of addresses?

[Stephen J. Turnbull]

Bill Cole writes:

 > > So this is potentially very complicated.
 > 
 > Case-squashing domain parts? Not complicated. Simple.

This is true if you are talking about following the Internet's rules.
I wasn't; I was talking about equivalencing identity tokens that
happen to look like email addresses.

There is, of course, some constraint on Mailman's behavior in that it
actually uses those tokens as email addresses to confirm identity by
sending email to them.

 > Also simple: NEVER try to interpret or canonicalize local-parts that 
 > exist in someone else's domain.

As Mark points out, that horse left the barn decades ago.

In this case apparently that is "user-friendly" (it seems that only
the domain differs in case), but if some site is in fact case
insensitive for local parts, the CSRF check will throw a false
positive in a situation similar to the OP, but where local parts
differ in some way insignificant for that domain.

 > You cannot programmatically determine whether 2 different
 > local-parts are equivalent unless you run the delivery system for
 > them.

Yup.  Which means making Mailman behave "nicely" from the user's point
of view is complicated in the situation in the OP.

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: Should CSRF check disregard case of addresses?

[Stephen J. Turnbull]

Mailman-admin writes:
 > Am 13.12.21 um 12:09 schrieb Sebastian Hagedorn:

 > > Nov 24 19:33:24 2021 (117276) Form for user x...@smail.uni-koeln.de
 > > submitted with CSRF token issued for x...@smail.uni-koeln.de.
 > > 
 > > The only difference is in the case of the email address. I’m no expert
 > > on CSRF attacks, but to me it seems as though the comparison should
 > > perhaps disregard differences in case only?
 > 
 > As local part of an email address can be case sensitive,

This is true, but

 > this should only be case insensitive for the domain part.

this part depends on exactly how these addresses are generated.  In
fact, the definition of "equivalent" for the local part is entirely up
to the site.  If the site policy is to make local parts case
insensitive, then the addresses are equivalent in that sense.

On the other hand, whether they should be equivalent for CSRF
validation is another question.  Since the CSRF validation is supposed
to be entirely transparent to the user, I would (naively) expect that
the strings representing the same address in the request should be
identical.  We'd need to figure out why the case of the address is
changing, and whether that could be an attack.

Also, some providers equivalent many more local parts.  For example,
there is the "+" notation separating the real mailbox from an
extension token, and IIRC, Google ignores punctuation in local parts.

So this is potentially very complicated.

Steve

--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: {Spam?} in subject lines.

[Stephen J. Turnbull]

Barry S. Finkel writes:

 > Mailman has no method for changing the Subject: line.

As you see above, Mailman sure does have a method for changing the
Subject field. :-)

The problem is that stock Mailman has no idea whether something is
spam or not, so neither adding nor removing spam tags makes sense, and
it does not try.  If you add Handler to get that information, then it
can handle the subject line, too.

But this is better done in the MTA for many reasons.

Steve



--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/

[Mailman-Users] Re: setting up mailman lists from the command line

[Stephen J. Turnbull]

Mark Sapiro writes:

 > But I'm concerned about what you are trying to do when you say "I'm 
 > thinking I'll script a way for the users to do this." These commands 
 > should only be used by admins. Are you trying to enable users to create 
 > lists? I don't think that's a good idea. I would think that what you 
 > want is for you to set up an announce only list for each sensor and let 
 > the users subscribe themselves to the list's of interest. See 
 > https://wiki.list.org/x/4030685 for info on setting up announce only lists.

A possible alternative would be to create a single SensorsByDave list,
and a list *topic* for each sensor.  Then the users can decide what
they want to see but only have one subscription to manage.

I agree with Mark that announce only makes sense here, based on what
you've written.
--
Mailman-Users mailing list -- mailman-users@python.org
To unsubscribe send an email to mailman-users-le...@python.org
https://mail.python.org/mailman3/lists/mailman-users.python.org/
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: https://www.mail-archive.com/mailman-users@python.org/
https://mail.python.org/archives/list/mailman-users@python.org/