Re: [Mailman-Users] Config dump? - WAS Re: speed up mailman

[Hank van Cleef]

The esteemed Ruben Safir has said:
> 
> On 02/28/2016 03:57 PM, Tanstaafl wrote:
> > On 2/28/2016 1:19 PM, Stephen J. Turnbull  wrote:
> >> If that doesn't make it obvious what you need to do, you might want to
> >> tell us something about your configuration and use case.  What version
> >> of Mailman?  Did you install as a package from a distribution or from
> >> source?  If from a distribution, which one?
> > 
> > Been meaning to ask...
> > 
> > Would it be difficult to add a command similar to postfix's postconf -n
> > that would dump the currently used config?
> > 
> When I read this, it just seems to me that you guys don't know how the 
> software works.  I posted logs to postfix and what they are asking is 
> frankly impossible to acquire.  This is a live system running a lot 
> of email in /var/spool/mail and the delays show up when the system 
> is under use.  I can't just restart it and expect the delays to 
> show up and I can't get answers to basic questions such as
> 
> When the email comes to mailman, where does it go.  How does the MTA know 
> to pick up the mail.  It seems to process mail in sweeps, rather than in 
> real time when mail arrives.
> 


I'm going to chime in here, as a Mailman user, not as a list advisor.
Mark and Steve know perfectly well how the software works.  So do a good
many others who are reading this list.  I don't expect them to give you
a 3-semester-hour tutorial on E-mail basics.  They have asked you a few
fundamental questions, which I don't see being answered.  To wit:

a. What is your computer hardware?
b. What operating system are you running?
c. What is your current DNS setup?  (I presume a couple of ISP servers
not on your site).
d. Where did you get your Mailman package?  Prebuilt package for your
O/S?  Build and install from Mailman source?  
e. What are the contents of your mm_cfg.py file?  
f. What's the volume of your outgoing mail?  That's number of
messages/hr, average message size, number of users receiving mail.
And how much other mail is your MTA handling along with Mailman mail?
g. What is your connection to the internet backbone?  And what is your
upload speed?


>From looking at your Postfix logs, it looks as though you have a major
DNS problem.  I suggest that you install bind and set up a caching DNS
server on the same box that is running Mailman.  That will stop
dependence on external servers, which aren't set up to handle the floods
of requests that Mailman can generate.  Not only is that a matter of
common courtesy to outside sysadmins, any request flooding is going to
be handled much more quickly on your own box.  

I'm not familiar with Postfix and its logs, as I run Sendmail.  However,
I can tell you that reading the MTA logs carefully to see if initial
attempts to connect are failing, and asking "why," if they are, can go a
long way toward solving delay problems.  Remember that MTA's queue
connection failures for retry later, and those retries happen on 15
minute intervals in Sendmail default installations.  Mailman has logging
for its side of delivery to the MTA, as Mark has told you.  But I'll
suggest looking at MTA performance first.  Keep in mind that while
batching mail with a large RCPT list is efficient for some of the large
target sites, those sites often simply defer back with 4.xx responses
unless you cut down the number of RCPT's.  

If you haven't got a clear understanding of how SMTP E-mail works, there
are plenty of sources for study and learning.  One thing SMTP is not, is
"instantaneous," or even quick.  After all, it was developed for
slow-speed telephone lines a long time ago.  

Hank


--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] sendmail and mailman mail0-wrapper

[Hank van Cleef]

The esteemed Bruce Harrison has said:
 
 
 Installing a new version of mailman on a new virtual machine (debian 
 4.6.3-14), and Mailman is complaining of a group mismatch error on the 
 mail-wrapper.  System is executing as mail and mailman wants it executed as 
 mailman group.
 
 Looked for the smrsh config in sendmail, but this version on Debian seems not 
 to use that.
 Also can't get sendmail to accept remote connections.  Localhost works fine.
 Telnet to port 25 on host works fine.  Remote telnet gets 554 
 mailman2.xxx.xxx ESMTP not accepting messages
 
 Looked at various pages on the web for sendmail and mailman, but not finding 
 much.
 
 I'm convinced it's a setting in sendmail,but... no idea what it is.
 
 Any sendmail experts out there?
 
I'll respond to this because I don't see any other responses on the
Mailman mail list, and I use sendmail with Mailman on a Solaris 10
system.  I've used sendmail ever since there was a sendmail, but would
hardly call myself an expert.  I use sendmail because the Solaris
distributions have a good implementation as a part of the general
distribution.  

First off, let me say that MTA's in general are supported using
MTA-specific resources.  

Also, and more importantly, you have to have whatever MTA you are
using already set up to send and receive mail from MUA clients before
thinking about integrating Mailman as an additional MUA.  I don't know
what's available for sendmail on Debian, what the quality of any
prepackaged distribution is (or isn't).  I do know that smrsh is part
of the sendmail package, so if you don't have it somewhere on your
system, you probably haven't got a good sendmail to work with.  In
that case.  It's open source, so you can compile your own, which I'd
recommend unless you've got a known-good distribution available.  

For sendmail support, the distribution website has a lot of
information, and there is a big README included in the package.  Most
administrators who do sendmail have a copy of the O'Reilly Sendmail
book, commonly called the Bat Book.  You'll need that.  There is
also a comp.mail/sendmail Usenet group, which is reasonably active.

If you are limited to prepackaged distributions, I'd strongly suggest
using Postfix on a Debian system.  

Hank
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] building a new mailman server

[Hank van Cleef]

The esteemed Mark Sapiro has said:
 
 On 03/26/2014 02:19 PM, Bruce Harrison wrote:
  
  We are currently running an old mailman instance and am planning to bring 
  up the newest version of mailman and migrate to it.  Current system uses 
  sendmail.  Since I now have a chance to change things, what is the best 
  mail server to run on my mailman box?  It will be talking to an exchange 
  2010 frontend server (current mailman system is doing that now).  I can 
  install sendmail or go with postfix or ???
  This is running on a debian box.
 
 
 Disclaimer: I am not an MTA expert. This is all personal opinion based
 on possibly limited experience.
 
 I would go with either Postfix or Exim. Both integrate well with
 Mailman; http://www.exim.org/howto/mailman21.html and
 http://www.list.org/mailman-install/node12.html.
 
 I have used both in Mailman environments, but Exim only in a Mailman
 development environment, Postfix both in development and on a production
 server handling both Mailman and other mail.
 
 I have little experience with sendmail, and while it is possible to
 hijack Mailman's Postfix integration to use with sendmail, on the whole
 it seems 'clunkier' to configure, and I see no reason to chose sendmail
 over Postfix.
 
I'm not an MTA expert, either.  However, I've used sendmail ever
since there was a sendmail.  My systems are Solaris, which comes with
a good implementation of sendmail, and my installations are all
configured to do what I want them to do with regular e-mail.  

I found adding Mailman to my existing setups to be quite simple.
You'll need to manually add the Mailman pipes to the aliases, using
the Mailman utility that gives you the list, but with the vi editor,
it's a simply copy-and-paste.  The only issue I had with integrating
Mailman with sendmail was getting the Masquerades set up properly to
host a different domain on my systems.  however, that is covered in 
the O'Reilly Bat Book, which you should have.  It is not a Mailman
issue, and proper configuration is done through the M4 macro files.  
I haven't had to tamper with the sendmail.cf file separately.

I think that if you're familiar with sendmail, and have a satisfactory
setup, do your upgrade, and integrate it with your current sendmail.
I'm aware that most of the people posting to this list are using other
MTA's on Linux systems, so there's little support here for a sendmail
setup.  

For reference, my links between Mailman and sendmail are through
localhost to a local sendmail, and I also have a caching DNS server,
all on the same box.  

For me, switching to another MTA would mean a lot of work learning how
to integrate all of the spam filtering and other things I already have
integrated and working in sendmail.  

Hank
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Question about Mailman hosting service.

[Hank van Cleef]

The esteemed Joe has said:
 
 Hello, everyone.
 
 In my attempt to find a Mailman hosting service I have received a message 
 from one IT specialist advising me to avoid hosting services that utilize a 
 cPanel. According to this specialist this is a bad arrangement and one that 
 doesn't provide me full control of my lists. Since I am not an IT specialist 
 I don't know what a cPanel is or how this can be a problem. Can any of you 
 enlighten me ? What is a cPanel, why can this be a problem and how would it 
 not allow me full control of my lists ?
 
 In addition, what problems should I expect to encounter in hosting my lists 
 with an outside Mailman hosting service ?
 
Joe, I'm going to comment on meeting your needs from my own direct
experience.  Just by way of introduction, let me say that I don't
consider myself an IT specialist, though there are times when I've
had to do until a real one comes along.  

Instead, I'm a hardware engineer who first worked with digital systems
in the 1950's, and learned software on the side.  You've heard of the
DEC PDP-1. I worked on that team.  Fast forward 43 years until I
retired at 67 in 2001.  Most of that time, I worked on teams who
didn't go to school to learn that stuff, because we were too busy
inventing it.  (Later some of those schools hired us to teach what
we'd invented).  

A couple of years after I retired (and moved into a small Rocky Mt.
ranch community), the local dialup service developed wireless radio
link services, and when they learned that I was a potential customer
and had Sun servers with Solaris available (and the skills to use
them), I was part of their development.  My setup here is as my own
ISP using them as my upstream feed.  About a year after we got that
up and running, a mail list I was on fell apart because of
sociological conflict.  I agreed to set up Mailman needed resources
to host his domain name, and received the needed data from the last
host site, which was also Mailman.   I downloaded the Mailman and
Python sources, built them, and installed Mailman using the existing
Solaris sendmail and apache builds in the Solaris distribution.  
Some of the Mailman web pages got customized for sociological reasons,
which meant that I had to maintain a non-standard configuration.

Our assumption at the time was that the list would operate for 3-5
years and sail off into the sunset.  That's not what happened.  We
solved the sociological problems, and at eight years, the list was 
more active than ever.  It was time to for me to get out of the hosting
sideline, and find a commercial provider.  Before starting that
search, I did a stock build of Mailman, uncustomized, and moved my
lists to that, to see if there was any more need for the
customization.  There was none, and we found a commercial service that
was running stock Mailman with the HtDig search engine, which we
needed.  Sent them a dump of the subscriber base, the configuration
files we were using, and many gigabytes of mbox files, and they've run
the list for a good three years now.  I'll identify them in private
e-mail if you wish.

What was my actual need for full access to the installation?  So far
as Mailman itself was concerned, virtually none was the answer.  It
was convenient to be able to read the log files, but that generally
was to monitor activity involving Sendmail and its control files.  

I don't know about cPanel itself.  But I think you want to consider
what your objectives are.  I think you are saying you want to set up a
mail list, but don't want to try to turn into a systems administration
wizard in the process.  My experience is this: you don't need to,
if you're hosted on a good system with competent sysadmins.  

Hank
--
Mailman-Users mailing list Mailman-Users@python.org
https://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
https://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Personalized header based on Gender.

[Hank van Cleef]

The esteemed Patrick McEvoy has said:
 
 Hello,
 
 I have turned on Full Personalization so I can have the individual's
 email address in the To field instead of the list email address.  I am
 adding a personalized header to the list as well and would like to know
 if there is any way to personalize the header based on gender.  For
 example Dear Mr for males and Dear Mrs females.
 
 Thanks for your help,
 
 Patrick
 
If you try to do this, you're going to get into a nasty thicket of
political correctness issues.  As Mark suggests, you're best off
using Dear [name] with no attempt at honorifics.

You're looking at sociological issues here, not technological.  

Hank
--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Personalized header based on Gender.

[Hank van Cleef]

The esteemed Patrick McEvoy has said:
 
 This is a multi-part message in MIME format.
 --020104040702070705060401
 Content-Type: text/plain; charset=ISO-8859-1
 Content-Transfer-Encoding: 7bit
 
 To clarify, in some languages other than English dear has gender such as
 Spanish,  Estimado for males and Estimada for females.
 
 Thanks,
 
 Patrick
 
Yes, French (which I deal with) has the same issue.  And while
other-language cultures don't seem to have as much of a political
correctness problem as US American, it still exists to the extent that
I consider it wise to devise a gender-neutral greeting, even if not
the commonly-used one in the culture.  

Hank
--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Problems regenerating archives from mbox files

[Hank van Cleef]

I'm trying to rebuild the Pipermail archives for a list.  The archive
base runs from 1998 to present, about 25,000 mails/year.  The mbox
files are segmented by year.  What documentation there is for the
bin/arch utility appears to be in its --help printout and in the
Python code.  

Thus far, it appears that we've gotten a complete build, but have
problems with year 2002.  About 250 posts have their headers archived
properly, but without the message text.  That text is archived as 
no subject on the date arch was run.  I've done a quick check, just
to idenfity the problem period, and so see what's in the mbox files
for the affected posts.  The posts themselves look correct, and I 
haven't spotted (yet) something in common such as one MUA being used
for the original posts.  Anyway, research proceeds.  I'm using the elm
mailreader as a check and diagnostic program on the assumption that if
elm will read the mail, arch should be able to.  Also using vi to
examine the files.

Qhuestion number one, of course, is why is arch behaving this way?
This is affectingn one year's mbox file's builds.  That mbox (year
2002) was generated under Mailman, and I think it was an early 2.x rev
that was replaced over New Year's 2003.  

Question: Does the --wipe option delete all the archives, or does it
see what period the selected mbox covers?  I've assumed it's all
archives, so doing an incremental rebuild will require storing
anything previously build elsewhere?

Question: Is there a way to do an incremental rebuild and have it
replace already-built archives for a given time period?  Or is that a
manual rm job?

Other things being equal, my plan of attack right now is to set up 
an empty archive directory, do incremental builds in it, then move
each increment to a backup directory, clean out the build directory,
and do the next increment.  When all increments are built clean, I
can then move the backup to production.  If there's a better way,
I'm all ears.

Hank

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Unexplained unsubscribes of yahoo.com email

[Hank van Cleef]

The esteemed Mark Sapiro has said:
 
 Very persistent as in delaying for more than 5 days, or is your MTA's
 retry period shorter than that?
 
 I'm sure you know what's happening in your case; maybe I'm just lucky,
 but I think my Yahoo deferrals/retries always succeed within way less
 than 24 hours.
 
 rant
 I think Yahoo is wrong in this. If the deferrals are a form of
 greylisting, there's no point in deferring more than once. One delayed
 retry proves the sending MTA is going to retry and subsequent
 deferrals are pointless. On the other hand, if they're never going to
 accept the mail, they should reject it outright with a 5xx. The only
 result from deferring multiple times and then accepting is to cause
 problems for their users resulting from delayed mail and to waste
 network resources with all the unnecessary retries.
 /rant
 
I recognize this pattern in dealing with several of the large
recipient sites.  Can't recall now, but I think yahoo, hotmail, and
verizon all pulled this stunt, and what's happening is not obvious to
the Mailman admin who does not have access to sender site logs.
I'm using sendmail as MTA, so this scenario is what happens with
sendmail.

Rather than bounce the mail with a 5.x.x DSN the recipient site defers
with a 4.x.x DSN.  Mailman, as an MUA, receives notices sent to
listname-bounces that the mail is being deferred, but unless Mailman
is set to deliver those messages to the admin, they vanish.  

The MTA has a timeout (sendmail default is 5 days, I reset to 3) after
which it discards mail from the queue and sends a bounce message,
which gets processed.  After receiving bounces for N days, Mailman 
sets the address nomail-by-bounce.  It becomes obvious to the Mailman
admin when the bounce processor sends the bounce notices.  Generally,
the bounce reminders also get blocked by deferral, so Mailman
unsubscribes the acount after the waiting period.

To put the frosting on the cake, the recipient doesn't get the you've
been unsubscribed message, either.

In a couple of cases, the bounce messages sent to the sendmail log 
will start having http links in the message.  Only the site
administrator who is reading the MTA logs gets to see that, and it's
sporadic---not every deferral message has that included.  The link is
generally to a URL for getting the sender site whitelisted with the
big mail operation.

Needless to say, the whole thing is pretty invisible to both a Mailman
administrator and the recipients.  In the meantime, the MTA queues get
filled up, the addresses get retried at regular intervals.  Consider 5
days of deferrals before the first bounces to Mailman, three more for
Mailman to set the user nomail, then three more weeks to the
unsubscribe.  And of course, the days of deferrals and retries make
the mail list look like a spam generator.

We're set up here to discard unrecognized bounce messages (that
address is a prime spam target), but to get bounce and unsubscribe
notices.  

Hank

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Forcing users to enter real name in listinfo page subscribe

[Hank van Cleef]

I'm changing one of my mail lists in preparation for moving it to
another host which will be running Mailman with unmodified Python code
underlying.  One of my changes is to replace a custom listinfo page
which routes subscription requests outside Mailman to the standard
Mailman confirm-and-approve form.  We will be able to run with a
customized listinfo.html page.

The template listinfo.html line for entering the applicant's real name
is:
td bgcolor=#dd width=55%Your name (optional):/td

We would like to make that mandatory.  I can remove the (optional)
from the HTML, but is there a way in the HTML to make Mailman reject a
blank response?  

Hank
--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Formatting incoming mail in Mailman

[Hank van Cleef]

I just sent a message to my list from a webmail account, and got an
empty text body in what was sent out from the list.  Sent a second
message to a user account which I read with elm to see what's in 
the message body.  

This is all of what comes through sendmail into the mbox:



DIV style=font-family:Arial, sans-serif; font-size:10pt;FONT
size=2SPAN
 style=font-family: Arial,sans-serif;send a line or two of text to
see what t
his sob sends.BR/SPAN/FONT/DIV

Note that it's HTML, but with no mime type indicator of any sort.

I have Mailman content filtering for this list set to filter the
content (yes), remove attachments that don't match standard mime
types, collapse alternatives, convert html to text, and discard 
messages meeting the filtering rules.

Up to now, we've used demime in the sendmail post alias pipe
to demunge non-plaintext posts, but I'd like to switch to Mailman
internal demunging.  

The above webmail formatting is the default new subscribers to that
ISP get.  We need to able to handle posts from total novices who 
send such stuff.   

Current Mailman is 2.1.9.  

What do we do?

Hank
--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Formatting incoming mail in Mailman

[Hank van Cleef]

The esteemed Mark Sapiro has said:
 
 Hank van Cleef wrote:
 
 I just sent a message to my list from a webmail account, and got an
 empty text body in what was sent out from the list.  Sent a second
 message to a user account which I read with elm to see what's in 
 the message body.  
 
 This is all of what comes through sendmail into the mbox:
 
 
 
 DIV style=font-family:Arial, sans-serif; font-size:10pt;FONT
 size=2SPAN
  style=font-family: Arial,sans-serif;send a line or two of text to
 see what t
 his sob sends.BR/SPAN/FONT/DIV
 
 
 Really? No headers at all? Or is this just some elm view of the message
 body?
 
 Look at the mbox with vi and look at everything from the From 
 separator through the end of the message.
 
Here's the whole message from the spool file:

From vancl...@wyoming.com Sat Feb 13 07:57:07 2010
Return-Path: vancl...@wyoming.com
Received: from omta0109.mta.everyone.net (imta-38.everyone.net [216.200.145.38])
by julie.lostwells.net (8.13.8+Sun/8.13.8) with ESMTP id o1DEv6c8020571
for vancl...@lostwells.net; Sat, 13 Feb 2010 07:57:06 -0700 (MST)
Received: from dm0104.mta.everyone.net (sj1-slb03-gw2 [172.16.1.96])
by omta0109.mta.everyone.net (Postfix) with ESMTP id 33D71288402
for vancl...@lostwells.net; Sat, 13 Feb 2010 06:57:06 -0800 (PST)
X-Eon-Dm: dm0104
Received: by resin18.mta.everyone.net (EON-PICKUP)
id resin18.4b721e04.106b6; Sat, 13 Feb 2010 06:57:06 -0800
MIME-Version: 1.0
Content-Type: text/html; charset=UTF-8
Message-Id: 20100213065706.b5c70...@resin18.mta.everyone.net
Date: Sat, 13 Feb 2010 06:57:06 -0800
From: vancl...@wyoming.com
To: vancl...@lostwells.net
Subject: See what's in this mail
X-Eon-Sig: AQLk58tLdr3CQFXGBwEB,10e73ece43724ae87d84f3c7c2c3384c
X-Originating-Ip: 216.67.170.236
Content-Length: 199

DIV style=font-family:Arial, sans-serif; font-size:10pt;FONT 
size=2SPAN style=font-family: Arial,sans-serif;send a line or two of 
text to see what this sob sends.BR/SPAN/FONT/DIV

(EOM implicit here)
 
 Note that it's HTML, but with no mime type indicator of any sort.
 
 I have Mailman content filtering for this list set to filter the
 content (yes), remove attachments that don't match standard mime
 types, collapse alternatives, convert html to text, and discard 
 messages meeting the filtering rules.
 
 
 Presumably you are using pass_mime_types to remove the 'non-matching'
 types. What's in pass_mime_types?

(copied from the options page):
mixed
alternative
text/plain
text/html


 
 What is your HTML_TO_PLAINTEXT_COMMAND setting? Is this the problem
 described at
 http://mail.python.org/pipermail/mailman-users/2003-January/025373.html?
 
This may be part of the problem, as the line in Defaults.py has not
been overridden to point to where I have a lynx on the system.

I've added to mm_cfg.py the line:
HTML_TO_PLAIN_TEXT_COMMAND = '/usr/local/bin/lynx -dump %(filename)s'

which is the Defaults.py line change to point to the correct
directory.  

Verification: 
julie:vancleef:$ which lynx
/usr/local/bin/lynx

julie:vancleef:$ lynx -version
Lynx Version 2.8.5rel.1 (04 Feb 2004)
Built on solaris2.9 Dec 20 2006 19:54:22

julie:vancleef:$ ls -l /usr/local/bin/lynx
-rwxr-xr-x   1 root other1892964 Dec 20  2006
/usr/local/bin/lynx

Checking logs, I don't get a grep hit on lynx in either syslog or 
the Mailman logs.  Checking mailman/logs/error, I have a line
Feb 13 07:52:07 2010 (296) HTML-text/plain error: 256
This matches the timestamp on the incoming webmail post.

That build of lynx isn't used anywhere else on my system.  Are there
any other install considerations needed?  The e-mail you refenced
flagged lynx as a potential problem, but seems to be primarily about
a Linux distro and /tmp.  

I've restarted Mailman with the new line in mm_cfg.py, and will
retest.

Hank

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Mailman

[Hank van Cleef]

The esteemed Mike Hughes has said:
 
 I have a mailing list that has been running under Mailman. Another  
 fellow has been hosting the mailing list for me. I would like to begin  
 doing so myself. I have a computer available for the task. What Linux  
 version will work best? What will I need to do to set up mailman?
 
 
 Mike  Hughes - Minister  
 Wilmington church of Christ
 

I think you need answers to two questions here, and the first is more
significant than the second.

1.  Are you prepared to set up what is essentially an in-house ISP
connected directly to the Internet backbone?

2. Can you do Unix/Linux system administration from the command line?

As to which Linux system is best, I'd say that there are several that
are, in the end, about equal.  As I recall, several of them offer an
already-compiled mailman package which you can install.  You will also
need a Python, a Mail Transfer Agent (Postfix, Qmail, sendmail), an 
Apache WEB server, and should have at least a local caching DNS server 
(bind) in addition to Mailman.  

Services you will need from an upstream internet backbone provider
are:
1.  At least one fixed IP with an unfiltered feed both ways for at
least ports 25 (SMTP mail) and 80 (Web).  
2.  Registered domain name with DNS A, MX, and PTR records for your
IP.  Your upstream feed will have to set up the PTR record.  

You'll need to set up local security.  A bare as-installed Linux/Unix 
system on the Internet backbone is like parking a new Mercedes in
Chicago, unlocked, with the keys in it.  Easiest solution is to use a
hardware firewall between your Linux box and the backbone feed.  A
major factor in your setup will be dealing with spam mail, which is
primarily done in firewall and MTA configuration. 

You'll also need to consider hardware availability and reliability,
and failure recovery. That means having backup resources.  

Now, to give you a ferinstance, I run an independent Mailman server as
a post-retirement hobby site from a bedroom in my house.  The
primary server is a Sun SPARC running Solaris, connected to the
backbone through a Fortigate 60 firewall.  MTA is sendmail, Web server
is Apache and I have a caching DNS on the same box, using program
resourced that come in the Solaris distribution.  My Python and
Mailman installation are local builds from (locally modified) source.  
Traffic is about 150 e-mails/day to 1500 users.  I've been online 
with this setup for about three years.  

For backup, I take daily incremental backups onto tape, and have a
second Sun box with the same setup which gets refreshed weekly, so
that it can be brought online in minutes if the primary server fails.

My upstream feed provider is a small local outfit.  It's a plus for
them to have a pro-adminstered Solaris site on their net, so getting 
IP's, PTR records, and an unfiltered feed was pretty straightforward
and quick.  Your Mileage May Vary on that one.  I did have to pass
what was essentially an employment interview on my administration
skills.

I figure on spending about an hour a day reading logs, looking for
trouble, doing list administration, etc.  That's ongoing overhead; any
time spent upgrading hardware or software, diagnosing and solving
problems, etc. is additional.  

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Users] Mailman 2.1.12 and htDig integration patches

[Hank van Cleef]

I'm wondering what is the situation with Mailman 2.1.12 integration
with the htDig search engine.  In the past, I've relied on Richard
Barrett's excellent support at openinfo.co.uk and have a 2.1.11 build
with his patches.  However, I don't see anything listed there for
2.1.12.  

While I can work through the exercises needed to get 2.1.12 to work
with htDig, I thought I'd ask first.  Do we need a different patch set
for the indexer and the archiver integration?

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Users] Trapping a post with a blank Subject:

[Hank van Cleef]

How can I catch a post sent to the list that has nothing in the
Subject: field?

It is the first post in a thread that I need to catch.  Mailman adds
(no subject) before distributing the post, and I can trap all the
responses.  But evidently that is added after the incoming post
headers are scanned.

I've tried several strings in the spam filter, but I don't get
reliable results.  I don't know what various MUA's actually put in
that field, but a simple \n seems to provoke a lot of false trapping
on Windows browsers that are actually sending subject line contents.

Hank


--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Users] How to config sendmail on FC5

[Hank van Cleef]

The esteemed Daniel.Li has said:
 
 Well, actually, I mean how to configure mailman to send mails by
 sendmail.
 And my sendmail works find.
 
I run Mailman on Solaris 10 with sendmail 8.13.4, so will comment as a
Mailman-sendmail admin.

The setup is quite simple.  Make sure that your sendmail will work
properly with a simple MUA (such as elm or mutt) installed on the same
box as Mailman.  

The one major difference between a simple MUA and Mailman is the need
to set up the aliases to pipe incoming mail to Mailman.  These look
like:

mailman:  |/usr/local/mailman/mail/mailman post mailman
mailman-admin:|/usr/local/mailman/mail/mailman admin mailman
mailman-bounces:  |/usr/local/mailman/mail/mailman bounces mailman
mailman-confirm:  |/usr/local/mailman/mail/mailman confirm mailman
mailman-join: |/usr/local/mailman/mail/mailman join mailman
mailman-leave:|/usr/local/mailman/mail/mailman leave mailman
mailman-owner:|/usr/local/mailman/mail/mailman owner mailman
mailman-request:  |/usr/local/mailman/mail/mailman request mailman
mailman-subscribe:|/usr/local/mailman/mail/mailman subscribe mailman
mailman-unsubscribe:  |/usr/local/mailman/mail/mailman unsubscribe mailman

I believe Mark has already covered this.  You'll need a set of these
for each list you create.  

If you running your Mailman as a virtual host for another domain,
you'll need to add that domain name into local-host-names.

Also set up sendmail to masquerade sender headers properly.  The only
lines in my sendmail .mc files to handle Mailman for a virtual host
are:

MASQUERADE_AS(`mainname.net')dnl
FEATURE(`masquerade_entire_domain')dnl
FEATURE(`limited_masquerade')dnl
LOCAL_DOMAIN(`mainname.net virtualname.com')dnl
MASQUERADE_DOMAIN(`mainname.net')dnl

If you want to use smrsh (recommended), follow the instructions in the
Mailman installation guide.  You'll have to enable smrsh in sendmail
with a line in main.mc.

FEATURE(smrsh, /usr/lib/smrsh)dnl

Except for the need for the alias pipes, Mailman looks like just
another MUA to sendmail.  I've included the above sendmail items for
your convenience, but all of them are covered in the sendmail
documentation and/or the O'Reilly Sendmail book, often called the
Bat Book.  

I'll reiterate that, for the most part, if your sendmail installation
works properly with a simple user MUA installed on the same machine,
Mailman should work properly.  Use the simple MUA to debug any
sendmail problems.  

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Users] Linux Preferred?

[Hank van Cleef]

The esteemed Brad Knowles has said:
 
 on 1/27/09 9:07 AM, Tim Ferguson said:
 
  Is there a preferred flavor of Linux that Mailman seems to work and
  install the best with?  I'm trying Ubuntu, but the apt-get feature is
  getting hung up on trying to find the repositories and can't, so,
  until I get that fixed I just thought I'd ask.
 
 If you're installing from our source, then most any recent version of 
 Linux should work fine.  If you're installing from someone else's binary 
 packages, then you are dependent on the person doing the packaging -- 
 they have to keep up with our releases, back-port patches, etc
 
 
 On python.org, where we host all the official mailman-* mailing lists, 
 Debian works just fine.  But we run the version from our own source, and 
 not the binary packaged version.
 
I have to put in another plug for building Mailman from sources vs.
doing a prepackaged install built by somebody else.   I have just
built and installed Mailman 2.1.11 and 2.1.12rc1 on Solaris 10
following the installation guide at

http://www.gnu.org/software/mailman/mailman-install/index.html

If you've got an appropriate Python installed and have a compiler on
the build machine, actually configuring, building, and installing the
Mailman application is the easiest part of the job.  The remainder are
very straightforward systems administration tasks to get the mailman
account set up properly and get correct permissions on everything.  

We also built Mailman 2.1.9 on a Debian machine a couple of years ago.
Another guy did the work and had problems because he did not follow
the Installation Guide completely.  I was asked to audit and correct
things, which involved no more than correcting group name and
permissions as outlined in the Installation Guide.  

I have always been an advocate for building from source rather than
using prepackaged as the default choice for add-on programs.  Only
when I trust a pre-built package (as with the sendmail and bind in the
Solaris distribution) do I use a prepackaged version.

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Users] Gathering additional iniformation at signup

[Hank van Cleef]

The esteemed Mark Sapiro has said:
 
 Walter wrote:
 
 I have just started using Mailman and wish to add fields (user title, 
 organization) that must be filled out upon subscription and available to the 
 list moderator for approval.  I can easily change the subscription form, 
 adding these fields, but where do I go to modify the approval process to 
 display them?  I looked thru the archives and I'm guessing that I have to 
 customize something in the admin pages, but where do I begin?  Thanks.
 
 
 This question is probably more appropriate for the
 mailman-develop...@python.org list, but ...
 
 This kind of information is kept in a UserDesc instance during the
 subscribe process. This class is defined in Mailman/UserDesc.py and
 you would probably need to augment its methods to account for your new
 items.
 
 This class is imported in all the following modules which would at
 least need to be reviewed.
 
 Mailman/Cgi/admin.py
 Mailman/Cgi/confirm.py
 Mailman/Cgi/subscribe.py
 Mailman/Commands/cmd_subscribe.py
 Mailman/ListAdmin.py
 Mailman/MailList.py
 bin/sync_members
 tests/test_membership.py
 
 This kind of information is also kept in the list's request.pck file
 for subscriptions waiting confirmation or approval. Thus, the
 SUBSCRIPTION request record is affected and this potentially affects
 the following modules.
 
 Mailman/Cgi/admindb.py
 Mailman/Cgi/confirm.py
 Mailman/ListAdmin.py
 bin/update
 cron/checkdbs
 
 In other words, this is not a simple change.
 
I'm another Mailman admin who needs a totally non-standard
subscription mechanism.  Currently, all of the Mailman subscription
stuff from the listinfo.html page is disabled with a custom page with
a link out to a site-specific form which gets mailed back to the list
admins.  New subscribers get added manually through the mass
subscription page.  

I see a couple of other people who need a similar mechanism.  While
our method works for us, with our low turnover, the underlying Python
doesn't like our changes, but things work well enough anyway.  I've
given some thought to redoing things so that a custom page can link
out to a non-Mailman process, then return the mail address and real
name to Mailman as an approved subscription for Mailman processing.

As you point out, Mark, this is probably a topic for
mailman-developers.  Thus far I've assumed that any work I did here
would be strictly site-specific.  But if there is a real need for 
a customized subscription mechanism beyond what is currently in
Mailman, we might consider something better than local band-aids.

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

[Mailman-Users] Mailman installation on Solaris 10

[Hank van Cleef]

I'm beginning to replace the Mailman 2.1.9 installation on Solaris 9
Sparc, which has served us very well for the last two years.   The new
target machines are Sun Ultra 60's, one running Solaris 10-u4 (8/07)
and the other, Solaris 10-u6 (10/08).  All that is on either machine
is a fresh install of Solaris (full install).

While I've been through the drill of building and installing both
Python and Mailman on Solaris before, I have a few obsrvations and
questions.

I went through the Mailman web page to find a download site.  Going to
Sourceforge offers only 2.12.rc1 unless you click stable on the
first screen.  I was unable to download either 2.1.11 or 2.1.12 from
Sourceforge.  That site links to Superb Hosting, which appears to be
unresponsive.  I had to get the software from the alternative GNU
site.

Both of the Solaris installs include Python 2.4.4.  Whether the
Solaris build is something resembling a full build or not, I don't
know, but will soon find out unless someone has checked this out
before me.  In the past, getting anything resembling a full build of
Python on Solaris has been a labor of love (porting the build
scripts), and I'd prefer not to have to build and install another
version of Python.  

I see Mark Sapiro's mail about incompatibility between 2.1.12rc1 and
Python 2.4.  

My production installations have to have an archive search (not part
of Mailman).  In the past, we've used htDig 3.1.6, which is decidedly
long in tooth, particularly as it requires gcc/g++ 2.95.3 to build
(will not build with gcc 3 or above).  For one installation, I'll
filch the already-built binaries.  

However, I'll reopen the question of a better (and currently-maintained) 
search engine.  I'm aware of a mail list discussing using Mhonarc and
Mnogosearch, but nothing since the last posts to that site (early
2006).   Is there something newer and better, or is doing the porting
work needed to use Mnogosearch the best alternative available?  One
issue with the current 2.1.9 installation is performance while
searching archives with htDig.  We currently have 5gb of archives, and
archive searches are very popular with our listers. 

I have picked up the patches for htDig-Mailman integration for 2.1.11
and assume I'm on my own for 2.1.12, for the present.  

Unless somebody tells me otherwise, I'll assume that I'm the first
installing 2.1.12 on Solaris 10.  That will go on the later (current
rev) machine.  Worth noting that on Solaris 10 10/08, it appears that
sun has included the Studio 12 development system, with the c compiler
in /usr/bin/cc.  While our old nemesis, the BSD stub /usr/bin/cc is
still there, a PATH that has /usr/bin before /usr/ucb should find a
working cc. 

The config.pck files currently on the production system are for 2.1.9.
If I build 2.1.11 and 2.1.12 as fresh builds and installs (no upgrade)
and later copy my 2.1.9 config.pck files into them, will Mailman
detect and correct the configuration?  

Any notes, comments, advice greatly appreciated.

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Users] Multi list management ..

[Hank van Cleef]

The esteemed Khalil Abbas has said:
 
 
 Dear Brad,
  
 first of all, godaddy has a very very strict policy against spamming, you can 
 read about it in their website.. if they noticed the least of bounces they 
 would terminate your whole server .. besides, my list is OPT-IN and I never 
 use the mass subscribe feature unless I want to move people from one list to 
 another ..
  
 second, what I meant was: each subscriber likes the service so much that he 
 sends it to his friends, and his friends like it so much that most of them 
 subscribe.. for the service is very cool, a single line of quote that is 
 never hard to read with great wisdom and benefit ..  Third, you should think 
 better of people, religious messages is not about bugging other people it's 
 about teaching them not to disturb other people ..Thanks..
 GOD FORGIVES..
  
Khalil Abbas:

The more you protest in mails to this list, the more you come across
as a spammer, pure and simple.  

Your original description:

the problem is, everyday I have to MASS-REMOVE the list of addresses
that wants to cancel on all the small lists .. as well as have to 
MASS-SUBSCRIBE the list of new subscriptions on all of the small lists 
which is getting really exhausting especially that my list is growing 
very fast (sometimes 5000+ subscribers a day) and I have to open a new 
small list every like 50 or 60 new thousand subscribers ..

Mass-subscribe isn't opt-in by any stretch of the imagination.  And
having to mass-remove those who protest simply acknowledges that
people don't like to be spammed.  Beyond which, many users are taught
that responding to spammers will simply validate their e-mail and
produce more spam.  5000+ new subscribers a day says to me loud and
clear that you are harvesting e-mail addresses from somewhere, or
several somewheres.  That's junk mail quantities.  I live in a 9900
square mile county where every e-mail address in the county would be
subscribed in 2-3 days at that rate.  All of New York City might take a 
couple of weeks.  A demographic of 250K qualified buyers comes
across as an order of magnitude (i.e. 10 times) larger than anything
I'd expect without clear-cut proof.

Add to that the subject matter you are mailing.  I am an old bull
(mid-70's), and if there are is a red flag that gets the old bull's
attention, it's religious mailings of any type.  Politics rates a
distant second.  
 
Don't bother talking to me about thinking better of people, forgiving
gods, etc. when your description is of religious messages is not
about bugging other people it's about teaching them not to disturb 
other people, and your methodology is to mass e-mail them with these
mailings.

In short, the more you post, the less I'm inclined to believe that you
are running anything other than a spam operation.  I may not know
everything, but as the late Red Skelton said, I calls them the way I
sees them.

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Users] Digest options - list configuration

[Hank van Cleef]

The esteemed Brad Knowles has said:
 
 On 8/7/08, Mark Sapiro wrote:
 
   After some off list back and forth, Alan determined that the problem was
   that the mailman user had /bin/false as a login shell. Once that was
   changed, the script worked as expected.
 
 Was this on a Solaris box?  I've had problems with them lately not 
 allowing cron jobs to run under a specific userid, even if they were 
 in the /etc/cron.allow list, unless the user had a valid password 
 hash and a valid shell.
 
 One of the more screwed-up things I've seen from a Solaris box 
 lately, I tell you
 
Nothing particularly screwed up about Solaris as a platform for
mailman.  When setting up the mailman account, set up the account as a
normal account, then use vipw to edit /etc/passwd and /etc/shadow. 
Just delete specification of any shell from the passwd line.

Shadow line should look like:
mailman:NP:::

The letters NP in the password hash field are important.  But there
are already plenty of examples in the file for you to follow.

When setting up the crontab, and changing it, su from root to mailman,
and follow the Solaris instructions.  They are in the man pages.  

You don't need to fuss with cron.allow and cron.deny to get cron to
process the mailman crontab.  

Just because:
Solaris isn't Linux, and sendmail isn't Postfix, isn't a reason to
call either one of them screwed up.  The Mailman build-install
instructions work just fine on Solaris/sendmail, if you pay attention
to doing things the Solaris way, and treat Mailman as just another 
sendmail user on an existing and functioning sendmail installation.

julie:vancleef:$ uptime
  9:54am  up 194 day(s), 17:24,  3 users,  load average: 0.00, 0.01,
0.02

It would be longer than that if we hadn't had a power outage shutown
last winter.

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Users] Feature Request: Selective Mass Subscription

[Hank van Cleef]

The esteemed Brad Knowles has said:
 
 Personally, I would be happy to see the mass subscribe feature go away 
 completely from the web interface of Mailman, or at least disabled by 
 default.  But then I've been fighting spam since the time I was the Sr. 
 Internet Mail Administrator at AOL in '95, and this is a subject that I feel 
 very strongly about.
 
I am going to go on record as very strongly OPPOSED to removal of the
mass subscribe feature.  We used it as the only method a new user
can subscribe to the list.  

Our method for new subscribers is to direct them to a web page with a
form they fill out.  On completion, that form is mailed by a cgi
script to the moderators.  After moderator review, we manually add
those we approve to Mailman through the mass subscribe interface.

The two mail subscription ports (-join, -subscribe) are disabled.  
Some time ago I did a check to see how much spam and how many bogus
attempts to subscribe those ports got.  Answer: plenty, and it was
quite clear that there were several attempts to evade our checks by
listers we had removed for list violations.  

I don't know what you did at AOL, but I'm here to tell you that AOL in
general was one big headache right from the get-go, and the source of
many disguised attempts to thwart our subscription policies.  We're
down to two AOL subscribers who are very well known to the moderators.

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Users] Mailman 2.1.11 final has been released.

[Hank van Cleef]

The esteemed Mark Sapiro has said:
 
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 The release notes and NEWS file for Mailman 2.1.11 contains the
 following innocuous looking item.
 
 ~- Improved bounce loop detection and handling in BounceRunner.py.
 
 This actually first appeared in 2.1.11rc2. It turns out this had an
 unintended consequence, but I actually think it is a good thing.
 
 The unintended consequence is that bounces of password reminders will
 also now go to the site list owner whereas before they were probably
 just ignored or processed as unrecognized bounces to the site list.
 
 Most of these bounces will probably be for dead addresses where the user
 disabled delivery and forgot about the list and the address died and the
 password reminders have been bouncing for a long time.
 
 In the longer term, plain text passwords and reminders are going away,
 but in the short term, the site list owner may get a lot of bounced
 password reminders (possibly a whole lot in a large site) on the first
 of the month following installation of this release.
 
 I think the best way to deal with these is to remove the dead addresses
 from the lists. Once this is done, the number of bounces on an ongoing
 basis should be small.
 
Mark, I've snipped your message, but it brings up something that is a
problem for us, and maybe hijacking your note with a feature
request.

We have a pattern of having list subscribers set themselves nomail,
and sometime later, close out the ISP account that was used to receive
mail.  We would like to be able to cull out these moved/left no
address subscribers.  

Evidently, 2.1.11 will allow me to more-or-less identify those
accounts by telling me which accounts bounced the monthly reminder
e-mail.  It is not too great a task to look at perhaps ten accounts
per month and correlate nomail by user with the bounce message.

Over time, dead accounts add up.  Last time I did something about
this, I set all the nomail accounts to receive mail, and the bounce
processor killed off close to half the accounts.  This was after eight
years of running.  The problem with that was that I had a hundred or
so users who were still active but who did not want to receive list
mail.  (Many were following the list through reading the archives).

I'm also concerned about this proposed scheme to discontinue montnly
mailings.  It most definitely needs some mechanism whereby a user can
reset or re-obtain their password without moderator intervention.  Our
user base is just plain not password-savvy, and I'm concerned about
the increase in moderator workload if the recovery method is similar
to password resetting by root (the Unix method).

Hank

--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Users] Apache,

[Hank van Cleef]

The esteemed Mark Sapiro has said:
 
 Vidiot wrote:
 
 Thanks for pointing this out.  I'll go away now and compile the source
 when I get Sol10 working.  Speaking of that, do you suggest using GNU or
 Sun Studio to compile the package?
 
 
 There are others on this list that can speak to Solaris installation
 about which I know almost nothing.
 
 The results of
 http://www.google.com/search?q=site%3Apython.org+inurl%3Amailman+solaris+korean
 may interest you.
 

Installing Python and Mailman on Solaris is pretty straightforward.  I
have done several builds and installs on Solaris 9 and 10, using both
gcc and the Sun compiler.  The installs run with the Sun versions of
sendmail and bind included in the Solaris distribution.   

The choice of compilers primarily affects the Python build.  There is
very little C-language code in Mailman, as it is primarily a Python
application.  I am prejudiced toward using the Sun development system
in general, but haven't developed any solid rationale for using it in
preference to gcc for Python.  You will get a usable, but incomplete,
Python build using either compiler, and I have yet to do a Python
build that will pass all its tests.  As I recall, the Python that
builds without trying to fix any of the Python build script problems for
Solaris is adequate to support the Mailman application.

The PATH environment is critical, and you will have to set this up
properly with a .profile script in the root directory (for using root
to build) and in the /etc/profile script used by users.  My root PATH
reads:
julie:root:# echo $PATH
/usr/sbin:/sbin:/usr/bin:/usr/dt/bin:/usr/openwin/bin:/etc:/opt/SUNWspro/bin:/usr/ccs/bin:/usr/local/bin:/usr/sfw/bin:/opt/sfw/bin:/usr/ucb

In particular, make sure the Mailman configure script finds your local
Python build, not the (older) version included in the Solaris
distribution.  This is the root of the Korean Codec problem that
Mark provided a google search link to.  Also make sure that the configure 
scripts find the cc or gcc that you intend to use.  Renaming the
/usr/ucb/cc stub to cc.bsd will help, but Solaris 10 includes a
version of gcc as well, which you may or may not want to use.  
If you want to use the Sun devsys, you'll have to force the configure
scripts not to use gcc with command line arguments.

I create and use the /usr/local directories.  Some System V purists
object to this.  Any objections I've seen are overcome by using a
separate partition for /usr/local.   

In general, follow the Mailman build and installation guide.  When
setting up the mailman account, I find it easiest to use the standard
useradd and then use vipw to configure the passwd and shadow files
correctly.  The shadow file line should read:
mailman:NP:::
Don't disable the account, as this will prevent cron from running
the cron scripts.  

The only gimmick specific to running Mailman with sendmail is to
include the alias pipes for Mailman in /etc/mail/aliases (and run
newaliases after changing the file).  

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Users] Apache,

[Hank van Cleef]

The esteemed Brad Knowles has said:
 
 Vidiot wrote:
 
  Thanks for the info, but the last time I looked, aliases was in /etc,
  not /etc/mail.
 
 Depends on your platform.  For Solaris, the aliases have been in 
 /etc/mail/aliases for a very, very long time.
 
I gave the location of the actual aliases file in /etc/mail for
Solaris.  /etc/aliases is provided, but is a symlink to the file in
/etc/mail.

One real difference between Solaris 9 and Solaris 10 is that the
sendmail M4 file setup has been moved from /lib/mail to /etc/mail/cf.
In practise, I copy the distribution M4 files to /usr/local/mail and 
work from that copy.  

I will note, and think that Brad and Mark will agree, that given the
predominance of Linux and Postfix on Mailman hosts, there probably is
very little interest among list subscribers for discussion of general
Solaris and sendmail issues.  Both systems are well-supported on
Usenet news groups and sponsoring web sites.  

Hank


--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: http://wiki.list.org/x/QIA9

Re: [Mailman-Users] Bounce processing questions (feature, or bug?)

[Hank van Cleef]

The esteemed Barry Finkel has said:
 
 Hank van Cleef [EMAIL PROTECTED] wrote, in part,
 
 Basically, mail to verizon.net addresses failed and bounced for a 24
 hour period on the 12th.  Service appeared to have been restored on
 the 13th.  Again, yesterday (the 18th), Verizon quit receiving mail
 again.  The log message being shown by sendmail is of the the form:
 
 Jun 19 10:49:00 julie sendmail[16973]: [ID 801593 mail.info]
 m5JGmxNk016971: to=[EMAIL PROTECTED],
 ctladdr=[EMAIL PROTECTED] (101/10), delay=00:00:00,
 xdelay=00:00:00, mailer=esmtp, pri=121428, relay=relay.verizon.net.
 [206.46.232.11], dsn=5.0.0, stat=Service unavailable
 
 For the four days June 13-17, mail went verizon.net normally,
 dsn-2.0.0, stat=Sent 
 
 Two Verizon users have contacted me and told me that they had received
 no mail from our list since the 11th; that they had contacted Verizon
 support, and had been told that Verizon was not blocking or dumping
 our list mails.  
 
 It seems to me that 
 
  stat=Service unavailable
 
 means that the inbound mailer is not available at the moment.
 This should result in a 400-level SMTP retryable reject, not a
 500-level hard reject.  The statement
 
  Verizon was not blocking or dumping our list mails
 
 is technically a correct statement; Verizon is not accepting mail.
 
 Another thought comes to mind - when this happens, is Verizon rejecting
 all mail, or is that mailer selectively not accepting mail?  If
 Verizon is selectively rejecting mail, then I would expect a different
 message than
 
  dsn=5.0.0 stat=Service unavailable.
 
Since I'm the original poster, I should perhaps clarify a few things.
My real question, which Mark answered (thank you Mark) was about an
unwanted side effect from the Mailman bounce processor.  

When I made my original post, I decided to summarize my problems with
verizon.net, and develop my rationale for resetting the bounce
processing parameters as a diagnostic tool.  Given the recent thread
on problems with Hotmail, and other discussions about ISP's
spam-blocking Mailman mail list mails over time, I felt that including
some of the details could be informative to the larger audience of
Mailman-users readers.

First off, list mail deliveries to verizon.net (and yahoo.com) addresses
have always been slow.  My daily log-reading has led me to expect a
great deal of mail requeuing from those two sites on a daily basis,
and sporadic random bounces.  So I'm not psyched up to pay a lot of
attention to increases in problem activity from those two sites.

Bounces with dsn=5.0.0 from verizon aren't news to me. I felt that the
complete blackout on the 12th was very suspicious, but when service
resumed on the 13th, none of the affected verizon.net list members had
hit the bounce trap, so I did not have any of the bounce messages
being sent back to my site.

I had several offlist communications with one verizon.net subscriber
who did not receive mail after June 11.  I sent him the log messages I
had from the 12th (bounces) and the 13th (normal delivery) and told
him to work the problem with Verizon from his end.  His report back to
me was that Verizon had told him they were not purposely blocking our
list mail nor dumping it after receiving it.  

When Verizon reinstated the block on the 18th, an offlist mail to this
user bounced, which was the first hard evidence I had that we were
dealing with a purposeful and specific blacklist action by Verizon.  I
forwarded the bounce message to this user through another (unblocked)
IP to the user and told him to work the problem from his end with the
hard evidence I'd furnished.  

I reset the bounce score so that all of the active verizon.net would
bounce-disable almost immediately, which they did.  I forwarded
several of the bounce messages to other verizon.net users, and am
aware that several of them used them in dealing with the Verizon
administrators.  

I did go to the web site listed in the bounce messages and fill out
the form with my site information.  And I did get a response from
Verizon that they would remove the block.  The block was removed
around noon on the 20th, and for the moment, mail is now going to the
verizon.net listers and being delivered.  

I'm documenting this to the Mailman-users list in hopes that it is
useful and informative information for other administrators
encountering similar problems with user ISP's.  I've been through this
drill before with other ISP's.  Misinformation seems to be a standard
problem in communications between ISP's and their users, and giving
the users the log information from the sending site clears the air in
a hurry.  I make it clear to the affected users that if reliable mail
service cannot be restored, they will have to move to a site that will
accept list e-mail, but go no further than that.  

As Mark has mentioned, the dsn=5.0.0 is legitimate, but it certainly
is not very informative in a case like this.  

Hank

[Mailman-Users] Bounce processing questions (feature, or bug?)

[Hank van Cleef]

The background scenario is that I have been experiencing the now you
see it/now you don't big-consumer-internet mail service spam
blocking.  This week it is Verizon.net.  

Basically, mail to verizon.net addresses failed and bounced for a 24
hour period on the 12th.  Service appeared to have been restored on
the 13th.  Again, yesterday (the 18th), Verizon quit receiving mail
again.  The log message being shown by sendmail is of the the form:

Jun 19 10:49:00 julie sendmail[16973]: [ID 801593 mail.info]
m5JGmxNk016971: to=[EMAIL PROTECTED],
ctladdr=[EMAIL PROTECTED] (101/10), delay=00:00:00,
xdelay=00:00:00, mailer=esmtp, pri=121428, relay=relay.verizon.net.
[206.46.232.11], dsn=5.0.0, stat=Service unavailable

For the four days June 13-17, mail went verizon.net normally,
dsn-2.0.0, stat=Sent 

Two Verizon users have contacted me and told me that they had received
no mail from our list since the 11th; that they had contacted Verizon
support, and had been told that Verizon was not blocking or dumping
our list mails.  Both users are sufficiently knowledgeable to have
looked for shunting to a spam or bulk file.  I wrote a private
e-mail to one user giving log details for him to use.  That e-mail
bounced back sky high, and the bounce message made clear that Verizon
has a spam block on my IP.  The log message was another dsn=5.0.0
stat=Service unavailable.

I decided to change the bounce processing so that if this Service
unavailable bouncing continued into a second day, the affected
Verizon users would get bounce-trapped, and I'd see the triggering
bounce messages.  Accordingly I changed:

Bounce score from 3.0 to 2.0
Discard period from 2 to 4 days
Number of warnings from 3 to 2
Notification interval left at 7 days

I've just double checked to be sure that the numbers in the window are
2.0, 4, 2, and 7.

This did exactly what I wanted, and I've got about thirty bounce
messages showing 5.7.1 security bounces, some of which I've forwarded
(via another IP) to affected Verizon users for their use in dealing
with the ISP.

However, I've also gotten about fifty other bounce disables on the 2.0
score.  Some investigation shows that affected users had accumulated
two bounces several months ago, but none since.  From what I see in
the bounce logs still in my rotation, it appears that the bounce
processor did not sense that these bounces were very stale (over 90
days, when I've specified 4), but simply operated on my reducing the
score from 3.0 to 2.0.  A couple of users have forwarded back the
bounce e-mail they received and have noted the last bounce dates in
March-April. 

Mailman version is 2.1.9, local build with Python 2.4.3 on Solaris 9.

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] Why am I being ignored???

[Hank van Cleef]

The esteemed Terrence Brannon has said:
 
 I posted this 
 http://www.mail-archive.com/mailman-users%40python.org/msg49373.html
 and basically the whole list is having other discussions like I never 
 even posted...
 
Ignored?  I may have missed something but the one lively discussion
I've seen here in the past few hours (you posted at 17:37 CDT
yesterday and are following up at 06:10) has been about spam handling,
which is a topic that every sysadmin knows about and deals with no
matter what mail programs they are running.  And I think that most of
the discussion has been between sysadmins.   

I will qualify my comments by stating that I am neither a Mailman nor
a Python developer.  I'm just another sysadmin running Mailman 2.1.9
and Python 2.4.3, locally compiled, on a Solaris 9 system, who is
building up a 2.1.10 Solaris 10 upgrade.  

Aside from the fact that Debian/etch only tells me you are running
Linux, which is off my turf, I don't see some basic information that 
is probably needed to help you.  What version of Mailman?  What
version of Python?  Are these local compiles from source, or are they
prepackaged versions downloaded from elsewhere (rpm files or whatever
Debian uses for precompiled packages)?  If prepackaged, where did they
come from?  Is this a new installation, or one that has been running
for a while?

If I read you correctly, you've got a Python module running in some
sort of a loop.  What's the name of the module?  

Have you checked basics, such as whether your Qrunners are all
running?  Have you tried stopping and restarting Mailman.  Are you
continuing to try to analyze and debug your problems  (Mailman, MTA,
Apache log reading, determining what's in the files that are not being
processed, etc.)?  Have you trolled through the Mailman FAQ for
clues?  Or are you just sitting there waiting for somebody---anybody---
to give you the magic answer. (Sorry, I don't have that).

I don't know where the Mailman developers are this fine Memorial Day
Weekend Saturday morning.  Maybe out playing golf.  They are
volunteers, donating their time, and that last thing I expect is to
have them jump when somebody says frog.  In the meantime, I'd
suggest that you gather up the basic configuration information,
provide it here, and continue your own debugging efforts.

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] GNU Mailmain 2.1.9: Filtering Messages based on

[Hank van Cleef]

The esteemed Steve has said:
 
 I'm admiring a GNU Mailman email list that is running on version 2.1.9
 on a server that I do not have access too.
 
 Is it possible through the use of the admin interface to filter out
 messages based on key words in the body of an email?
 
 I have some banned ex-subscribers who periodically resubscribe under
 new email addresses to post URLs to rant pages about my list.I
 would rather not moderate all new incoming subscribers.   I don't want
 to slow legitimate users down.  The trolls often bury the URLs at the
 end of long messages making it easy to miss.
 
I'll comment on this request from an administrator's perspective.  If
there is a software solution to the problem of rogues subscribing to a
list and using it to disrupt things, I do not know what that would be.
Rogues are very good at finding a software trap pattern and changing
their tactics to bypass it.  

Historically, the main list I administer was set up on a listserv
system with subscription by invitation only in 1998.  It was
purposely set up to get away from another list that was dominated by
trolls, spammers, and flamers.  We migrated the list to another
system, using Mailman, in 2001.  Over the years we have endured
several concerted attempts by those who made themselves unwelcome and
who were unsubscribed, to resubscribe under false pretenses and do a
hit-and-run attack on our list.  For several years, there was another
mail list, hosted elsewhere, populated by our enemies, but I think
that has more-or-less fallen by the wayside.   

What has worked for us is a two-fold administrative solution.  
1.  We do not do software subscription through the Mailman interface.
All subscriptions are manually-processed.  New subscribers are
required to fill out a form that requests real name, address,
telephone number, educational level, profession, and birth year and
mailed to the moderators through a cgi-bin script.  The moderators
review these and manually subscribe those who pass through the bulk
subscription Mailman resource.

2.  All new subscriptions are moderated.  Once we see a few clean
posts from new subscribers, we take them off moderation.  Virtually
all of the rogues who have managed to get through the subscription
screening tip their hand quite quickly.  Some offlist e-mail between a
moderator and an offender either gets an apology or a flame.   

Which leads to:

3. Mail lists post messages from humans that are intended to be read
by other humans.  What is acceptable behavior is strictly human in
nature.  Sociology, rather than technology, is what will set and
maintain standards.  It is very important to have moderators who have
good skills at guiding group behavior.  And, yes, there is moderator
workload involved in making this work.  Once a list's focal point and
demographics are clearly established, almost all issues involving
listmember conduct are sociological.

 I would prefer not to use procmail or software other than GNU Mailman.
  As I wrote the list is made available to me freely on a server that I
 do not have access to.  I don't want to bother my admin more than I
 have to.
 
Our use of various filter programs is oriented toward reducing spam
volume.  We do not attempt to use it to detect rogue posts.

 Is it possible to set up GNU Mailman 2.1.9 to filter on URLs or other
 keywords in the body of an email?
 
What it is possible to do is one thing.  What is practical and
effective is another.  I've worked in the field of sociology of
computer-mediated communications for about thirty years now, and have
coded up no end of good ideas that more-or-less failed in their
objective of reducing human workload.  

Mailman is rich in resources to handle basics, such as reformatting
funny-format MUA text into readable text, stripping off attachments
(with or without virii), not allowing postings from non-registered
people, and the like.  But it does not attempt to use technology to
regulate human behavior.  My experience suggests that if one is going
to run a clean mail list, one has to define clean---in human
sociological terms.  And it is going to take humans to set and enforce
those standards.  Ten years of experience with our list has also made
clear that clean is a moving target.  

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] installing mailman and sendmail

[Hank van Cleef]

The esteemed Shams Fantar has said:
 
 No, I didn't say that (because sendmail.cf exists), I just said that
 with grep smrsh /etc/mail/sendmail.cf, I'm getting nothing with grep,
 so, smrsh isn't in this file.
 
 
  
  I think you need to make certain that sendmail is set up correctly and
  can process mail before going forward with trying to troubleshoot mailman.
 
 Yes, I think it's just a problem with sendmail (or with mailman).
 
  
  PS : I'm running debian lenny/sid.
  

I'm going to jump in here as I run Mailman with sendmail on a Solaris
system.  I can't comment on pecuiliarities of a Linux precompiled
distribution.

The Mailman installation manual goes through the steps of enabling 
a link to smrsh.
http://www.gnu.org/software/mailman/mailman-install/node32.html

Additionally, you'll need to enable the sendmail smrsh capability.
You need to do this by adding a line in the main.mc file for sendmail
and rerunning the M4 process to recreate the sendmail.cf file.
The line to add is:

FEATURE(smrsh, /usr/lib/smrsh)dnl

This assumes that the smrsh executable is in /usr/lib.

If you are trying to run with a precompiled sendmail, you'll need to
configure main.mc for a variety of things that generally aren't in
default installations; the line above in in addition to selecting
things like relay control, masquerading, access_db, virtusertable etc. etc.  

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] Email response very slow using Mailman and sendmail

[Hank van Cleef]

The esteemed Patil, Manjiri S has said:
 
 
 Hi Hank and mark , 
 
(snipping for brevity)

 I do see that SENDAMAIL is trying to send messages only to bunch of
 people which varies from 10 to 50 at time. Also how do I tune up
 SENDMAIL with number of recipients to process at one time?
 
 
 Apr 17 09:30:30 smgpub sendmail[27208]: m3HD1x7v027207:
 to=[EMAIL PROTECTED], delay=00:28:31, xdelay=00:28:31, mailer=esmtp 
 9147447, relay=relay.bu.edu. [128.197.27.198], dsn=5.1.1, stat=User
 unknown
 
 Apr 17 10:00:02 smgpub sendmail[27208]: m3HD1x7v027207:
 to=[EMAIL PROTECTED],[EMAIL PROTECTED],[EMAIL PROTECTED],.@
 ,[EMAIL PROTECTED],[EMAIL PROTECTED] [more], delay=00:58:03, 
 xdelay=00:58:03,
 mailer=esmtp, pri=9147447, relay.bu.edu. [128.197.27.198], dsn=2.0.0,
 stat=Sent (m3HD1x78004079 Message accepted for delivery)
 
 Apr 17 10:00:02 smgpub sendmail[27208]: m3HD1x7v027207:
 to=[EMAIL PROTECTED] [more], delay=00:58:03, xdelay=00:58:03,
 mailer=esmtp, pri=
 7, relay=relay.bu.edu. [128.197.27.198], dsn=2.0.0, stat=Sent
 (m3HD1x78004079 Message accepted for delivery)
 

I've snipped out a lot of you post for brevity, but it does seem clear
from what you've provided that your issues are primarily sendmail
issues, not Mailman.  

I'll make the assumption that your listmember base is all, or almost
all, at bu.edu, and that it is primarily students (plus faculty).  

You did not show the log message showing the outgoing message from
listname-bounces (Mailman) that give the size and nrcpts.  This is
what is being sent by Mailman as MUA to sendmail as MTA.  Also, please
don't snip log messages when posting them.  

I see a couple of things you can probably change to get radical
improvement fairly quickly.

1. If your outgoing (from listname-bounces) log message shows
nrcpts=50, or some other big number, put a limit on what Mailman is
allowed to send.  In the $MMROOT/mailman/Mailman/mm_cfg.py file, put
in (or modify an existing) line to read:

SMTP_MAX_RCPTS = 5

Then restart the Mailman Qrunner daemons.  

Reason:  sendmail will terminate sending of multiple recipient mails
on the first failure.  The remaining messages are deferred for retry.
Your user base is probably high-turnover (students entering and
leaving), so a lot of your delays are peeling the onion retries.

2. It appears that your sendmail outgoing daemon is being started with
a retry time of 1 hour.  Do a ps -ef |grep sendmail and look for lines
that read

 /usr/lib/sendmail -bd -q1h

The -q argument is the retry interval.  Find where sendmail is being
started and arrange to restart it with -q15m (15 minutes).  

Sendmail is generally started by a boot-time rc file, so you'll need
to find that file on your system (Solaris 9 is /etc/init.d/sendmail)
and adjust the startup appropriately.  

Reason:  Four retries per hour vs. 1 per hour should be obvious. Don't
cut the time down so much that sendmail retries while still processing
messages from the last retries.  

3.  Use the mailq utility to watch queue processing.  

4.  Any bounces with a DSN of 5.x.x should be going back to Mailman
and shown in the $MMROOT/mailman/logs/bounce log.  Check that you are
getting them.  Beyond that, you can set your Mailman bounce processing
from the Mailman List Administration screens.  I run with 3.0 (three
consecutive days), and 3 7-day notification intervals.  If you need to
clean up your listmember data base because of account turnover, I'd
suggest cutting that down to 2.0 bounces and 2 4-day notifications or
something similar.  

You should familiarize yourself with sendmail, and generally be on top
of any peculiar characteristics of your user base and the sites you
deliver mail to.  You should have the O'Reilly Sendmail (the bat
book) and the O'Reilly sendmail 8.13 supplement.  Also, the Usenet
news group comp.mail.sendmail, the sendmail README (in the
distribution), and the sendmail web site, which has a huge (and
valuable) FAQ.

In general, the Mailman developers don't work with sendmail on a daily
basis.  I'm jumping in here as a Mailman user site manager who is
currently working delivery problems to yahoo.com listmembers, so I am
using the same resources you need to use to tune your system.  

Hank

--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] Email response very slow using Mailman and sendmail

[Hank van Cleef]

The esteemed Patil, Manjiri S has said:
 
 Hi, 
 
 We have internal Mailing List server running Mailman 2.1.9 and SENDMAIL
 8.13.8. Whenever someone sent email to particular mailing list
 ([EMAIL PROTECTED])consisting app 300 users recipients receive
 it after several hours. E.g see the following Email I sent on Apr14th at
 3:19PM that came in at 6:09 PM and some received at 7:09 PM. I don't
 understand why there is so much delay when sending mail to 300 users
 mailing list. I clearly see in Outlook headers that the mail is sitting
 on Mailman for approximately 2hours . 
 
 
 Here is the corresponding SENDMAIL Log.
 
  
 Apr 14 15:19:42 smgpub sendmail[32663]: m3EJJgII032663:
 from=[EMAIL PROTECTED], size=2721, class=0, nrcpt s=1,
 msgid=[EMAIL PROTECTED]
 , proto=ESMTP, daemon=MTA,  relay=relay1.bu.edu [128.197.27.99]
 
 Apr 14 15:19:42 smgpub sendmail[32664]: m3EJJgII032663:
 to=|/usr/lib/mailman/mail/mailman post everyone,
 ctladdr=[EMAIL PROTECTED] (8/0), delay=00:00:00,
 xdelay=00:00:00, mailer=prog, pri=32911 , dsn=2.0.0, stat=Sent
 
 Apr 14 15:19:50 smgpub sendmail[32667]: m3EJJk3b032666:
 to=|/usr/lib/mailman/mail/mailman post everyon e,
 ctladdr=[EMAIL PROTECTED] (8/0), delay=00:00:00,
 xdelay=00:00:00, mailer=prog, pri=32342 , dsn=2.0.0, stat=Sent
 
  
You aren't looking at the complete picture in the sendmail logs here.

You should have:

1. The incoming message to the list---I assume that your first log
message is this.
2. Transmission by sendmail to the Mailman post address.  Both your
second and third log entries shown above appear to be this, but for
two different messages.   

Note the m3EJJgII032663: identifier in your first two messages.  These
match, so are the same message.

When Mailman sends outgoing mail, you should see messages with
from=everyone-bounces
Pick up the identifier on that message and find log entries that match
it.  Here is a ferinstane of a message to a yahoo.com lister that is
being deferred by yahoo.  


1.  Mailman sends message to sendmail for delivery to yahoo.com

Apr 16 10:30:52 julie sendmail[14680]: [ID 801593 mail.info] m3GGR9as014680: fro
m=[EMAIL PROTECTED], size=15494, class=-30, nrcpts=1, msgid=[EMAIL 
PROTECTED], proto=ESMTP, daemon=MTA-v
4, relay=localhost [127.0.0.1]

2. First attempt to deliver:

Apr 16 10:34:38 julie sendmail[15894]: [ID 801593 mail.info] m3GGR9as014680: 
to=[EMAIL PROTECTED], delay=00:03:46, xdelay=00:03:46, mailer=esmtp, 
pri=189494, relay=g.mx.mail.yahoo.com. [209.191.88.239], dsn=4.0.0, 
stat=Deferred 

2. Second attempt to deliver:

Apr 16 11:18:26 julie sendmail[19203]: [ID 801593 mail.info] m3GGR9as014680: 
to=[EMAIL PROTECTED], delay=00:47:34, xdelay=00:00:00,mailer=esmtp, 
pri=279494, relay=b.mx.mail.yahoo.com., dsn=4.0.0, stat=Deferred

3. Nth attempt to deliver:

Apr 16 13:17:51 julie sendmail[27648]: [ID 801593 mail.info] m3GGR9as014680: 
to=[EMAIL PROTECTED], delay=02:46:59, xdelay=00:00:00,mailer=esmtp, 
pri=459494, relay=b.mx.mail.yahoo.com., dsn=4.0.0, stat=Deferred

Note that what keys these messages together is the ident string
m3GGR9as014680:

You'll note that the last delay is delay=02:46:59

This is a yahoo.com problem which we can't solve at the sending end.
Obviously, the listmember won't get the mail message until yahoo
agrees to accept it.  

I've cited a problem delivery that I am working at this end and
trying to get yahoo.com to do something about.  Look for stat=Deferred
in your system.  In any event you need to examine the outgoing mail
messages, as well as the incoming.  There are several hundred log
lines between these attempts, so you will have to search on the ident
string to find the affected messages.

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] Changing mm_cfg.py for existing lists

[Hank van Cleef]

The esteemed Jussi Hirvi has said:
 
 Hi, list,
 
 The problem I'm trying to solve is that Hotmail (and probably msn.com) do
 not accept our Mailman emails - this seems to be a new phenomenon. The error
 code is too many recipients. From other sources I assume that Hotmail now
 accept sonly max 9 recipients per one request, while Mailman's default
 maximum is 500.
 
 I try to solve this by adding
 SMTP_MAX_RCPTS = 9
 to my mm_cfg.py.
 
 After Mailman restart, this seems to have no effect on existing lists - like
 I expected.
 
I think you will have to set the limit lower than 9.  My list runs
with 
SMTP_MAX_RCPTS = 5

It has been a couple of years since I experimented with this, and the
problem I was dealing with was that verizon.net would either accept
mails immediately or defer them until the MTA expired them.  

With a limit of 5 recipients, I get good delivery to all of the big
mail sites except yahoo.com.  They will defer even single-recipient
list priority mails during prime time, but deliver them within 24 hours.  

Read your MTA logs in addition to reading the bounce messages.  I have
had problems where the bounce message said one thing but the MTA log
said something quite different.

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] Bug in Mailman version 2.1.8

[Hank van Cleef]

The esteemed Ki Song has said:
  
  One major error I see is that there is no more disk space on the server.
  How can I remedy this situation?
  
  Buy a bigger disk. :-(
  
  Seriously, this is an OS question, not a Mailman question.
  -- 
  Mark Sapiro [EMAIL PROTECTED]The highway is for gamblers,
 
 The weird thing is that until this Friday, there was approximately 70-80%
 free space on the 75GB partition.
 
 I know this is not a mailman question, but how should I go about looking for
 files that may have suddenly taken up all that disk space.
 
 Any help would be greatly appreciated!
 
I'd suggest using a few of the handy Unix tools that have been around
since ATT was inventing it.

df and du on appropriate directories and with appropriate flags.
ls -ltr will show you what's recent in a hurry.
find -size -newer -type f to look for recent big files.  
fine -name core to find core files.
Where are your /tmp files located and what's in them?
top will find you processes that may be running out of control.

I doubt that suddenly filling up a 75GB partition has anything to do
with Mailman.  You've got basic system administration issues.  
Has someone hacked your system and installed a rootkit?

Hank
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

[Mailman-Users] Google gmail problem

[Hank van Cleef]

I'm posting this to the mailman-users list in hopes that I can get
some input that will either explain or resolve a problem I am having
with gmail users.

Their complaint is that they do not see a copy of their posts to the
list reflected back to them.  

The options profile for these users is typically nodupes and plain
All other flags (including the ack and not metoo) flags are clear. 

My system is configured with personalization enabled, and the
individual messages sent out are personalized.  

Basic configuration is Mailman 2.1.9, Solaris 5.9, sendmail
8.13.8+Sun.  

Sendmail logs are quite clear that the user messages are being sent
back to the users and received and acknowledged by the google mail
servers (dsn=2.0.0, stat=Sent).  

I'm a bit hampered by knowing absolutely nothing about gmail.  I've
got one user who is quite adamant that the problem is at my end.

   True with emails from this list, but not with other lists I 
am subscribed to.  Gmail shows both sent one and the copy coming 
back at me. (from other lists, which are not identified).

Does anybody have any experience with this problem?  Is there a
solution, and if so, what?

Hank

--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] Mailman log rotation

[Hank van Cleef]

The esteemed Mark Sapiro has said:
 
 Hank van Cleef wrote:
 
 I want to put the Mailman logs into the Solaris logadm.conf file 
 for automatic cron rotation.
 
 This moves the old logfiles to logfiles.0 and touches new logfiles.
 I'm assuming that the qrunners have to be restarted.  Anything else?
 
 
 Qrunners don't need to be restarted, just log files reopened
 
   /path/to/bin/mailmanctl reopen
 
 That should be sufficient.
 
Thanks much, Mark.  I used the mailmanctl reopen in the
/etc/logadm.conf file (with the full paths to everything---mandatory for
cron jobs) and let cron do its thing.

This morning, the logs are rotated and writing happily.

Hank

-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

[Mailman-Users] Mailman log rotation

[Hank van Cleef]

I want to put the Mailman logs into the Solaris logadm.conf file 
for automatic cron rotation.

This moves the old logfiles to logfiles.0 and touches new logfiles.
I'm assuming that the qrunners have to be restarted.  Anything else?

Hank
-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] mailman install problem

[Hank van Cleef]

The esteemed Piniella, David A has said:
 
 Hi,
 
 I'm trying to install mailman 2.1.9 on a Solaris 10 x86 machine and I'm
 running into problems with make install.
 I have no cc installed, I am using GCC 3.4.5 which I've installed from
 blastwave.org; python is 2.3.5. 
 
 I've searched the archives of the list and hit up google to no avail,
 any suggestion(s) or hints would be most appreciated, I'm ready to scrap
 this install. I don't know if it's something funky in Sol10's make or
 maybe mailman's expecting gcc to be cc (see unable to execute cc: No
 such file or directory \ error: command 'cc' failed with exit status 1
 -- I did NOT specify the --with-gcc=no option during the configure).

(snippity snippity)
 
 /opt/csw/lib/python2.3/distutils/dist.py:213: UserWarning: 'licence'
 distribution option is deprecated; use 'license'
   warnings.warn(msg)
 unable to execute cc: No such file or directory
 error: command 'cc' failed with exit status 1
 make[1]: *** [install-packages] Error 1
 make[1]: Leaving directory `/usr/local/mailman/misc'
 *** Error code 2

You're probably seeing what's a well-known problem with Solaris using
GNU-type configure and Makefile generation.  Easy to fix.

The cc that some makefile is finding is probably /usr/ucb/cc in the 
Solaris install. That is a stub that does nothing.  Rename it to cc.bsd.

On a Solaris site, you have to set up PATH as a post-install operation.
Most Solaris admins have a post-install script for doing this.  

Do a check, using the which command.  which gcc should point to the path
where the gcc you want to use is located, and do a version check.   

Then do a which cc.  Unless you have Studio 11 installed, you should get
a not found return.  The Studio 11 compiler resources are in
/opt/SUNWspro/bin if they are installed.  

You may have to locate the offending Makefile or rerun configure to
recover, but try make install first, after assuring that nothing can
find /usr/ucb/cc.

Hank

Hank
-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] mailman install problem

[Hank van Cleef]

The esteemed Piniella, David A has said:
 
 I have actually already renamed the /usr/ucb/cc (to cc_old) and modified my 
 path (specifically for the install):
 
 bash-3.00# which gcc
 /opt/csw/gcc3/bin/gcc
 
 bash-3.00# which cc
 no cc in /usr/local/sbin /opt/csw/sbin /opt/sfw/sbin /opt/sbin /usr/sbin 
 /sbin /usr/local/bin /usr/bin /opt/bin /bin /usr/openwin/bin /opt/csw/bin 
 /opt/sfw/bin /usr/apache/bin /opt/csw/gcc3/bin /usr/ccs/bin
 
 bash-3.00#
 
 I'm still getting the same unable to execute cc error. I really don't 
 understand it; would there be any cached info?

Hokay! Here's the deal.  I have just walked this question through with a
junk Mailman 2.1.9 tree.  This tree was originally build using Studio 11
cc.  I took the pointer to that out of my PATH, ran a make clean, and 
ran configure.  It crashed at what appears to be the same point yours
did.  

Then I ran a make distclean, and reran configure.  That ran to
completion properly.  Followed that up with a make, although that
doesn't compile the stuff in the misc directory.  However, checking that
makefile it has CC=gcc.

I'll suggest that you start over with a make distclean and a new
configure.  

Hank

-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] mailman install problem - Solaris

[Hank van Cleef]

The esteemed Piniella, David A has said:
 
 
 
 OK, for giggles, I updated python to 2.5 from 2.3.5, and now configure does 
 this:
 
 checking for group name mailman... Traceback (most recent call last):
   File conftest.py, line 1, in module
 import grp
 ImportError: ld.so.1: python: fatal: libgcc_s.so.1: open failed: No such file 
 or directory
 cat: cannot open conftest.out
 configure: error:
 * No mailman group found!
 * Your system must have a mailman group defined
 * (usually in your /etc/group file).  Please see the INSTALL
 * file for details.
 bash-3.00# cat /etc/group | grep mailman
 mailman::100:mailman
 bash-3.00#
 
 ...what?

David, at this point I'm going to bow out and let Mark do the talking.
The only real contribution I had to make was to flag the /usr/bin/cc
issue in Solaris, which is is something that bites almost everyone
sooner or later.   

Hank

-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] htdig authorization problem (fixed)

[Hank van Cleef]

I'll sum up the problem I was encountering, and post the fix.  I note of
thanks to Mark, who worked with me offlist to get a diagnosis.   As with
many things, what was wrong was blatantly obvious---after the fact.

Symptoms:  On a new Mailman installation, with the htdig patch and htdig
program installed, validation failed when attempting to look at search
returns or to run a second modified search.  Entering a value user name
or a valid user password in the validation form would allow viewing a
returned page, but had to be repeated for every page viewed.  

Diagnosis:  The URL in the htdig data base was different from that 
in Mailman.   

Solution:  
1.  Run blow_away_htdig against the affected list.
2.  Post a single posting to the affected list.
3.  Manually run the nightly_htdig utility that is in the mailman
crontab.  

What is significant is the answer to how did we get into this jam in
the first place?

This installation was set up on a new host site, to host a mail list 
that had run for years on another host site.   Included in moving the
list host was a domain name registration used as a virtual domain name.

For test, the new list was set up for test at the new host site configured 
[EMAIL PROTECTED],net.  The old host continued to run in parallel as
[EMAIL PROTECTED] until it was clear that the new host was ready
to take over for the old one.  A part of the new site test configuration
included setting up list archives with htdig.  

The transfer was accomplished by moving the domain registration DNS IP's
from the old site to the new.  Access to the new site's LAN is via a
router NAT table.  Except for setting up the MTA (sendmail) to handle 
mail for for alisasdomain.com, the new host treated the new name as a
synonym for the old.

Several days later, the site administrator ran 
bin/withlist -l -r fix_url listname
This was considered a cosmetic change, and a quick check showed all of
the web pages associated with the list as working properly with the 
alias name in place of the site domain name.  

Not so.  This is what broke the htdig access.  Since htdig is an add-on
to Mailman, the existing data bases have to be recreated through the
steps outlined above.  

Once again, my thanks to Mark Sapiro for following this through with me.

Hank

-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

[Mailman-Users] htdig authorization problem

[Hank van Cleef]

I have mailman 2.1.9 with the htdig patch installed.  Set up to do local
search.   

I rebuilt the archives from the list mbox file a couple of weeks ago,
and htdig would search on it, deliver pages, and they could be viewed.

This week (several days later), to view a post requires entering 
a password before viewing.  Trying to pull up a second page gets an
error message.  This affects using the list admin password as well as
user passwords.  To view any hit, a user account name or the admin
pasword (not both) has to be entered in a validation screen.  

The mailman error log reports
htsearch for list: mercedes, cause: auth, detail: -10-

Browsers have cookies enabled (mine certainly does).  Where do I look?
Permissions on the private directory were 2770; I reset them to 2774.
All of the files in private/listname are owner/group both mailman,
permissions on files are 666, on directories, 2775.

Where do I go from here?  

Hank

-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] htdig authorization problem

[Hank van Cleef]

The esteemed Mark Sapiro has said:
 
 
 But was the archive 'private' or 'public' (admin Archiving options)
 before the problem started?
 
We're going back to Jan. 11 (when this started), and I'm trying to
remember what I changed then.  I suspect that I ran check_perms around
then an fixed what it griped about, and the permissions on the private
directory was one such.  Let's assume this was the case (I can easily
check, but I don't think setting the directory permission open really
proves anything.  Nonetheless, I should do it to see that it does clear
the fault).
 
 The scenario plays like this:
 
 On cold entry (no cookies in the browser) go to the archives page from
 the main page.  User validation screen pops up.  Enter username and
 password, and you get to the pipermail archives.  You can now read those
 all day without needing to revalidate.
 
 
 Good.
 
 
 Now, set up an htdig search, which returns a few pages of hits.  Click
 on one of those hits, and the validation screen pops up again.
 Validate, and the message is displayed.  Go back and click on another
 hit.  The validation screen pops up again, and displays the page after
 another validation.  
 
 Trying to display the second page makes the 
 
 
 I assume you mean the second page of hits.
 
Yes.  Htdig will do the search and display a page of returns.  Anything
beyond that requires revalidation.
 
 
 It seems clear to me that the htdig process is not able to find the 
 validation
 cookie.
 
 
 I don't think that is the issue. It seems to me that the htdig process
 finds the cookie the first time, but then removes it. This explains
 why you need to revalidate when you try to visit the message, and it
 explains why you get mmsearch.py's authorization error when you try to
 get the second page of hits.

Yes, it's the second time in.  Doing a second search on top of the first
(i.e. changing the htdig search return display and telling it to search
again) gets the error. 
 
I have been reading the code (and Python in a Nutshell).  

mmsearch.py has 
if mlist.archive_private:
if not mlist.WebAuthenticate((mm_cfg.AuthUser,
 mm_cfg.AuthListModerator,
 mm_cfg.AuthListAdmin,
 mm_cfg.AuthSiteAdmin),
'', ''):
raise _search_exception(listname, 'auth', '-10-')

What I'm seeing is consistent with the if not condition.  And if this
is the second pass through this, rather than the first, then it implies
that something in the code following is involved.  I've included a cut
and paste of exactly what is in the mmsearch.py, and will be doing some
creative code-reading to see if this call differs from other calls to
the same function.  

I've located the WebAuthenticate() function in SecurityManager.py and
note that it is used in several places, all of which behave as
expected.
 
One question comes to mind: why are we revalidating here?  I'll assume
without having researched it that the entry to the pipermail archives
has its own validation, and in my configuration, you can't get to the
archive search except from that post-validated page.  

Theoretically, I suppose I could work around for the moment by simply
commenting out that check, but since something's getting trashed
somwhere, think it better to pursue debugging the problem.  

 If this were compiled code and not Python, I'd dbx the thing,
 but don't know enough about Python to know how to use the debugging
 resources.  If there's some sort of crash and traceback message, I don't
 know where to look for that either.  
 
I'm busy reading the O'Reilly python book on debugging to see where I
might get some hooks into this.
 
 There wouldn't be a traceback in this case.
 
No, I can see that the code has an exit.
 
 It seems that htdig per se is working fine. It returns the hits. I
 think the problem is in mmsearch.py. It checks to be sure it is
 authorized for the private archive, and it is. It does the search and
 displays the results and somehow in this latter step the cookie is
 removed.
 
I'll guess that you're right about the htdig installation.  However some
more testing is showing is not that the cookie is being removed, but
that we've got something that's nulling something.

Running with the admin password set, I go from the admin area to the
main list page, then to the archives, where I can read the pipermail
archives repeatedly.

Do an htdig search, get the first page of hits back, and try to look at
one.  Anything entered in either the username or password field clears
the block.  One character in either field suffices, and it does not have to be
alphanumeric.   

After having gone throught this exercise several times, I can go back to
the admin pages without having to revalidate---the cookie is still set.

Hank

-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL

[Mailman-Users] htdig authorization problem

[Hank van Cleef]

I just succeeded in blowing away my spooled mail, so no longer have
what was in this thread (yes, I can look at the mailman-users archives).
Thankyew Thunderbird, which linked to my local spool instead of the 
external one it was supposed to.  

Getting back to the salient points:

If the htdig access ever actually worked in the private directory, 
I can't confirm reliably.  Inputs from my listers aren't sufficiently 
accurate for me to work with (this would be the third case where some of 
them thought it was working and it wasn't).  The error log indicates
failures from the time that the archives and htdig data base were built,
and are in sufficient quantity that I believe that this problem has
existed since I brought them on line.  

I have checked the problem with a Firefox 1.5.0.9, a late Mozilla build
and Netscape 4.78) ancient, but it's the Solaris 9 default).  

The Apache is 1.13.34 (Solaris 9 build).  There is a later version in 
the Solaris 9 patch cluster, but I am not going to run patching on this
box until I can finish building the Mailman installation on another box
and switch over.  I doubt that this is an Apache problem. 

The httpd.conf on this site is right out of the Mailman install guide.
I am not using https, and all of the mailman stuff is in one directory
tree without symbolic links.  Default URL in mm_cfg.py is for the 
DNS domain being used, and I ran withlist/fix_url to point to that
domain before going on-line.  

Running with Firefox, I cleared the cache and all cookies, then retested
as a user, going from the listinfo page.  This URL is:
http://www.mercedeslist.com/mailman/listinfo/mercedes

The archives page is:
http://www.mercedeslist.com/mailman/private/mercedes/

As I have said, that directory tree is all in one place on one
filesystem.  

Running as a user, after having assured that Firefox is cold (cache
cleared, one cookie present), I can get an article to display through
htdig by filling in either the account name or the password.  Both are
not needed.  However, for the user account, the name or password has to
be correct.  If one of the two fields is correct, bogus information in
the other does not prevent access.  However, in true user state, one
field has to be correct.  

The cookie held by the browser does not go away, and does not appear to
be altered.  It is valid for authorization to go to the options field
without having to revalidate.

Hank

-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

Re: [Mailman-Users] htdig authorization problem

[Hank van Cleef]

The esteemed Mark Sapiro has said:
 
 Hank van Cleef wrote: 
 
 Running with Firefox, I cleared the cache and all cookies, then retested
 as a user, going from the listinfo page.  This URL is:
 http://www.mercedeslist.com/mailman/listinfo/mercedes
 
 The archives page is:
 http://www.mercedeslist.com/mailman/private/mercedes/
 
 
 Right, but when you get the archive index page with the search box,
 what is the POST URL for the search form, and when you get the results
 page, what are the URLs for the message links and what is the URL for
 the next page of results. It is these latter URLs that seem to lose
 the cookie data.
 
Taken from the apache access_log.  Notes interspered:

192.168.1.99 - - [19/Jan/2007:20:36:03 -0700] GET / HTTP/1.1 200 4807
192.168.1.99 - - [19/Jan/2007:20:36:03 -0700] GET
/welcome_files/welcome.gif HTTP/1.1 200 533
(Entry point)
192.168.1.99 - - [19/Jan/2007:20:40:18 -0700] GET
/mailman/listinfo/mercedes HTTP/1.1 200 6097
192.168.1.99 - - [19/Jan/2007:20:40:32 -0700] GET
/mailman/private/mercedes/ HTTP/1.1 200 1493
192.168.1.99 - - [19/Jan/2007:20:40:57 -0700] POST
/mailman/private/mercedes/ HTTP/1.1 200 136912

(Look at a pipermail archive)
192.168.1.99 - - [19/Jan/2007:20:41:31 -0700] GET
/mailman/private/mercedes/Week-of-Mon-20061106/date.html HTTP/1.1 200 46660
192.168.1.99 - - [19/Jan/2007:20:41:37 -0700] GET
/mailman/private/mercedes/Week-of-Mon-20061106/164538.html HTTP/1.1 200 2466
192.168.1.99 - - [19/Jan/2007:20:42:08 -0700] POST

(do a search that produces several pages of hits)
/mailman/mmsearch/mercedes H TTP/1.1 200 5922
192.168.1.99 - - [19/Jan/2007:20:42:08 -0700] GET /htdig//button5.gif
HTTP/1.1 200 780
192.168.1.99 - - [19/Jan/2007:20:42:08 -0700] GET /htdig//button4.gif
HTTP/1.1 200 786
192.168.1.99 - - [19/Jan/2007:20:42:08 -0700] GET /htdig//button6.gif
HTTP/1.1 200 791

(Try to look at one)
192.168.1.12 - - [19/Jan/2007:20:42:45 -0700] GET
/mailman/htdig/mercedes/Week- of-Mon-20060227/145334.html HTTP/1.1 200 1580
(revalidate with password only)
192.168.1.99 - - [19/Jan/2007:20:43:37 -0700] POST
/mailman/htdig/mercedes/Week -of-Mon-20060227/145334.html HTTP/1.1 200 4335
192.168.1.99 - - [19/Jan/2007:20:43:39 -0700] GET /mailman/htdig/mercedes/Week-
of-Mon-20060227/145335.html HTTP/1.1 200 4018

(Try to look at another)
192.168.1.12 - - [19/Jan/2007:20:44:12 -0700] GET
/mailman/htdig/mercedes/Week- of-Mon-20030623/041974.html HTTP/1.1 200 1580
(revalidate with password only)
192.168.1.99 - - [19/Jan/2007:20:44:19 -0700] POST
/mailman/htdig/mercedes/Week -of-Mon-20030623/041974.html HTTP/1.1 200 4009
192.168.1.99 - - [19/Jan/2007:20:44:21 -0700] GET
/mailman/htdig/mercedes/Week- of-Mon-20030623/041897.html HTTP/1.1 200 2395

(Try to view second page)
192.168.1.12 - - [19/Jan/2007:20:44:34 -0700] GET
/mailman/mmsearch/mercedes?config=mercedes;method=and;format=short;sort=score;words=penoff;page=2
HTTP/1.1 2 00 1336
192.168.1.12 - - [19/Jan/2007:20:45:17 -0700] GET
/mailman/htdig/mercedes/Week- of-Mon-20040308/073188.html HTTP/1.1 200 1580
192.168.1.99 - - [19/Jan/2007:20:45:23 -0700] POST
/mailman/htdig/mercedes/Week -of-Mon-20040308/073188.html HTTP/1.1 200 3364
192.168.1.99 - - [19/Jan/2007:20:45:25 -0700] GET
/mailman/htdig/mercedes/Week- of-Mon-20040308/073191.html HTTP/1.1 200 2593

(Note: none of the above displayed.  Instead, I got the auth
error page for htdig)

(Now, go over to the options page for the account without having
to revalidate)
192.168.1.99 - - [19/Jan/2007:20:45:53 -0700] GET /icons/mm-icon.png
HTTP/1.1 200 281
192.168.1.99 - - [19/Jan/2007:20:45:54 -0700] POST
/mailman/options/mercedes HT TP/1.1 200 14726
192.168.1.99 - - [19/Jan/2007:20:46:14 -0700] GET /favicon.ico
HTTP/1.1 404 28 3

The error log has entries for favicon.ico, which I don't have.  No other
errors.

Hank
-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

[Mailman-Users] getting correct alias domain in message headers

[Hank van Cleef]

I've brought a fresh installation of Mailman 2.1.9 on line on 
an IP that has been assigned to mydomain.net for some time.
However, I want to appear as otherdomain.com.  The DNS records
for otherdomain.com have been changed to point to my IP, with A records
to otherdomain.com and www.otherdomain.com, and an MX record to point to
a mydomain.net A record.

What I did to handle mail was add othedomain.com to the sendmail
cw file (local-host-names).  

What I see in the headers in a post to the list is:

X-BeenThere: [EMAIL PROTECTED]
X-Mailman-Version: 2.1.9
Precedence: list
Reply-To: The Mylist Discussion List [EMAIL PROTECTED]
Sender: [EMAIL PROTECTED]
Errors-To: [EMAIL PROTECTED]

How do I get the correct names (otherdomain.com) in the last three
header lines?  

I've added to mm_cfg.py

DEFAULT_EMAIL_HOST = 'otherdomain.com'
DEFAULT_URL_HOST = 'www.otherdomain.com'
add_virtualhost(DEFAULT_URL_HOST, DEFAULT_EMAIL_HOST)

(actually, tried the add_virtualhost before the redefined defaults
before trying it after).  The list URL is correct (withlist/fix_url)
but the mail headers aren't.  

How to fix this?  Do I need to restart the qrunners?

Hank




-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

[Mailman-Users] Options page bombing out on a security violation

[Hank van Cleef]

I'm attaching a user's description of something about 20 of my users are
reporting on a fresh install of Mailman 2.1.9  I've edited the user 
identity, and am including configuration info following the forwarded
mail.  I'm baffled, because I cannot get this problem to happen from 
any of the systems at my site, and some users (unfortunately, all of 
the computer-literate types who might be able to help) aren't getting it
either.  

 
 I too am suffering from the
 
 High security alert!!!
 You are not permitted to download the file [EMAIL PROTECTED].
 
 URL = http://bronze.lostwells.net/mailman/options/mercedes/[EMAIL PROTECTED]
 
 when trying to switch off the mail
 
 path followed is as follows:-
 
 
 
 1, http://www.lostwells.net/mailman/listinfo/mercedes
 
 2, click on unsubscribe or edit options button
 
 3, http://bronze.lostwells.net/mailman/options/mercedes
 
 4, sign in with normal login details, the page refreshes with all the
 parameters viewable
 
 5, disable mail delivery and click submit my changes
 
 6, High Security message at the url
 http://bronze.lostwells.net/mailman/options/mercedes/[EMAIL PROTECTED]
 
 ===

Configuration details: Solaris 9 on Sun Ultra 10
Python 2.5
Mailman 2.1.9

Since I am taking over hosting for a list that was running on a Mailman
2.1.4 installation, I built a local 2.1.4 tree, created the list, then
copied the old site config.pck over the newly-created one.  Then built
2.1.9 and upgraded the 2.1.4 tree.

# cd /usr/local/mailman
# ./bin/check_perms
No problems found

Hank


-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
1986 420SEL A stranger in paradise (Fremont Co. Wyoming)
1986 GMC 1500 6.2 diesel pickup Seen one, seen them all
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp

[Mailman-Users] Using demime with Mailman

[Hank van Cleef]

I'm in the process of setting up a system to host Mailman lists
which will replace another system at another site.  The other site has
sent me a copy of their /usr/lib/mailman tree, so I can examine their
configuration and copy list-specific files, and have a Beta-test
version running.

Major problem with all of the funny-formats included in mails these
days.  Mails from specific users are just being passed through with 
no reformatting.  On the old site, it's clear that Demime is being
used.  However, I can find nothing in the old Mailman tree indicating
that it's installed.  

Hank

For reference:
Mailman 2.1.9
Python 2.5
Solaris 8 9/05 on Sun Ultra 10 (Sparc system)
Sun Studio 11 devsys used to compile.

Old configuration:  Mailman-2.1.4

Migration was to build a new 2.1.4 tree, create the list, than move
the old list's config.pck to the new one.  Then install 2.1.9 as an
upgrade to the 2.1.4 tree.  
-- 
Hank van Cleef ([EMAIL PROTECTED], [EMAIL PROTECTED])
--
Mailman-Users mailing list
Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://www.python.org/cgi-bin/faqw-mm.py
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Security Policy: 
http://www.python.org/cgi-bin/faqw-mm.py?req=showamp;file=faq01.027.htp