Re: [Mailman-Users] Integration with external search engine

2010-12-17 Thread Mark Sapiro 
On 12/17/2010 4:55 AM, Lukáš Vlček wrote:
> 
> I am looking at a best practice way how to integrate mailman with external
> search engine. I found the following Wiki page [1] which contains a link to
> Ext_Arch.py template which is brainchild of Mark Sapiro and Cedric Jeanneret
> [2]. Cerdic was after indexing emails using Xapian and his implementation of
> the Ext_Arch.py can be found here [3]. This all looks very promising but I
> have a few questions/concerns:
> 
> To me it seems that the PUBLIC_EXTERNAL_ARCHIVER and
> PRIVATE_EXTERNAL_ARCHIVER commands (which are both set in mm_cfg.py) are
> executed only when a new message arrives, that means it is not executed when
> bin/arch is executed. This means that if there has been running some mail
> list on mailman for a few years now and now I would like to allow searching
> its content via new external search engine (like Xapian) it is simply no
> enough to add external archiver and restart mailman because this would index
> only newly added messages. Am I right?


Yes, you are right. The design intent of external archivers is that they
provide a hook to use an external process for both archiving and
searching of the external archive. External archivers were never
intended to be used to index the built-in pipermail archive. Thus, the
Ext_Arch.py template is just a kludge which is admittedly incomplete in
this respect.


> How can I then have reindexed old content from that mail list into Xapian as
> well? bin/arch  does not do that as it does not execute external
> archivers. Moreover, running bin/arch can change URLs of individual public
> emails (re-numbering) and that is probably unacceptable. So is there any way
> how to iterate over existing emails, parse them and get an existing URL
> value for them? (Such information could be then used to re-index old content
> into external search server without need to run bin/arch).


find /path/to/archives/private/LISTNAME \
 | egrep "[0-9]{6}.html" \
 | sed "s;.*archives/private;http://www.example.com/pipermail;";

with the obvious modification will get the URLs. Will that be enough?

> 
> [1]
> http://wiki.list.org/display/DOC/4.87+How+do+I+invoke+some+process+on+messages+as+they+are+added+to+the+pipermail+archive
> [2] http://www.mail-archive.com/mailman-users@python.org/msg56679.html
> [3]
> https://bugs.launchpad.net/mailman/+bug/531942/+attachment/1199211/+files/archive-and-index.py

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] User unknown in virtual mailbox table

2010-12-17 Thread Mark Sapiro 
On 12/16/2010 7:16 AM, Umarzuki Mochlis wrote:
> 
> I created a list from the admin interface, not from CLI. I administator of
> that list do not get the list creation notification & I found these messages
> in /var/log/mailman/smtp-failure
> 
> Dec 16 13:49:13 2010 (26989) All recipients refused:
> {'umarz...@mydomain.com.my': (550, '5.1.0 :
> Sender
> address rejected: User unknown in virtual mailbox table')}, msgid:
> 
> Dec 16 13:49:13 2010 (26989) delivery to umarz...@mydomain.com.my failed
> with
> code 550: 5.1.0 : Sender address rejected:
> User
> unknown in virtual mailbox table
> 
> # cat /var/log/maillog | grep test02
> Dec 16 13:49:13 hpj-mail postfix/smtpd[30059]: NOQUEUE: reject: RCPT from
> localhost.localdomain[127.0.0.1]: 550 5.1.0 > :
> Sender address rejected: User unknown in virtual mailbox table;
> from= to=
> proto=ESMTP
> helo=
> 
> This installation in on centos 5.5
> 
> What should I do next so I can send and receive email to this list (test02)?


You did not receive the list created notice because your Postfix is
doing sender verification and You have not (properly) configured
Mailman/Postfix configuration for automatic generation of list aliases
and virtual alias maps at list creation time, so the virtual_alias_maps
entry for test02-boun...@mydomain.com.my did not exist at the time the
mail was sent.

See , in particular
subsections 6.1.1 and 6.1.2.

Also see the first several posts in the thread at

for a discussion of another aspect of this issue.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Notification set to no, but still received unsubscription message while member still there

2010-12-17 Thread Mark Sapiro 
On 12/16/2010 10:06 AM, MiaoKe wrote:
>  
> As the admin of one list, I received the following message recently:
>  
> From: mailman-boun...@
> Sender: mailman-boun...@
> To: mailinglistname-ow...@
> Subject: mailinglistname unsubscribe notification
> Sent: Dec 16, 2010 9:00 AM
> 
> mi...@.. has been removed from Councilor.
>  
> But I checked the member database of that mailingilist, this mi...@.. is 
> still there...
>  
> Please give some suggestions on how to resolve this!


Examine the Received: headers of the message you received to determine
where it originated. It could have come from an old version of the list
on a prior host or from a test or backup installation on the current host.

It is also possible the user unsubscribed by accident and then
resubscribed before you checked, although that doesn't explain why you
got the unsubscribe notification. If you have access, check Mailman's
'subscribe' log.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] email delay for 2 days

2010-12-17 Thread Mark Sapiro 
On 12/15/2010 8:10 PM, KAIWEN wrote:
> 
> i configure ubuntu 9.04 + mailman 2.1.12 + postfix 2.5.5. it works fine, but
> there is a delay for sending emails from last week. i checked the mail log
> file, it shows the mail is sent, but the recipient only can receive the
> email 2 days later, any advise on solving this problem?


If by mail log, you mean the Postfix log, the mail has left this server
and been delivered to the relay or recipient MX at the time of the log
entry. If the recipient didn't receive it for two days, you need to
examine the Received: headers of the recipient's message to find where
the delay occurred.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] owner alias bounces

2010-12-17 Thread Mark Sapiro 
On 12/15/2010 1:40 AM, Florian Effenberger wrote:
> 
> interestingly, a withlist -l -r fix_url -a seems to have cured the
> problem. Of course, copy-pasting it out of my documentation without
> thinking led to the hostnames for 50% of the lists being wrong, which
> I manually corrected in the list interface, but now things seem to
> work. Maybe it also has something to do with the nightly cronjobs,
> resetting the bounce counter?
> 
> Strange...


The only thing fix_url would "fix" is the domain of the sender of the
mail and the domain of the list-ow...@domain recipient (much but not all
mail to owners/moderators is sent first to the list-ow...@... address
and only then resent to the actual owner/moderator addresses. Possibly
this domain was not valid or didn't have a -owner alias.


> Anyone knows an easier solution when migration another host to our
> mailing list server while adding another hostname?


The procedure for adding a host name is

1) Put the appropriate add_virtualhost() in mm_cfg.py.
2) If Postfix is to generate virtual alias maps for this domain, add it
to POSTFIX_STYLE_VIRTUAL_DOMAINS in mm_cfg.py.
3) To move any lists to that domain, run fix_url with the -u option on
the list(s)

I don't think there's any 'easier' solution.

Note, if you don't care about the web domain, you can set the host_name
on the list's General Options page instead of step 3.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Integration with external search engine

2010-12-17 Thread Lukáš Vlček 
Forgot to mention Mailman version: 2.1.13

On Fri, Dec 17, 2010 at 1:55 PM, Lukáš Vlček  wrote:

> Hi,
>
> Short version - I have two questions:
> ==
>
> 1) How to setup external archiver so that the email content gets indexed by
> external search engine
> 2) How to (re)index existing content from mail list by external search
> engine
>
> Longer version:
> ==
>
> I am looking at a best practice way how to integrate mailman with external
> search engine. I found the following Wiki page [1] which contains a link to
> Ext_Arch.py template which is brainchild of Mark Sapiro and Cedric Jeanneret
> [2]. Cerdic was after indexing emails using Xapian and his implementation of
> the Ext_Arch.py can be found here [3]. This all looks very promising but I
> have a few questions/concerns:
>
> To me it seems that the PUBLIC_EXTERNAL_ARCHIVER and
> PRIVATE_EXTERNAL_ARCHIVER commands (which are both set in mm_cfg.py) are
> executed only when a new message arrives, that means it is not executed when
> bin/arch is executed. This means that if there has been running some mail
> list on mailman for a few years now and now I would like to allow searching
> its content via new external search engine (like Xapian) it is simply no
> enough to add external archiver and restart mailman because this would index
> only newly added messages. Am I right?
>
> How can I then have reindexed old content from that mail list into Xapian
> as well? bin/arch  does not do that as it does not execute
> external archivers. Moreover, running bin/arch can change URLs of individual
> public emails (re-numbering) and that is probably unacceptable. So is there
> any way how to iterate over existing emails, parse them and get an existing
> URL value for them? (Such information could be then used to re-index old
> content into external search server without need to run bin/arch).
>
> Thanks,
> Lukas Vlcek
>
> [1]
> http://wiki.list.org/display/DOC/4.87+How+do+I+invoke+some+process+on+messages+as+they+are+added+to+the+pipermail+archive
> [2] http://www.mail-archive.com/mailman-users@python.org/msg56679.html
> [3]
> https://bugs.launchpad.net/mailman/+bug/531942/+attachment/1199211/+files/archive-and-index.py
>
--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

[Mailman-Users] Integration with external search engine

2010-12-17 Thread Lukáš Vlček 
Hi,

Short version - I have two questions:
==

1) How to setup external archiver so that the email content gets indexed by
external search engine
2) How to (re)index existing content from mail list by external search
engine

Longer version:
==

I am looking at a best practice way how to integrate mailman with external
search engine. I found the following Wiki page [1] which contains a link to
Ext_Arch.py template which is brainchild of Mark Sapiro and Cedric Jeanneret
[2]. Cerdic was after indexing emails using Xapian and his implementation of
the Ext_Arch.py can be found here [3]. This all looks very promising but I
have a few questions/concerns:

To me it seems that the PUBLIC_EXTERNAL_ARCHIVER and
PRIVATE_EXTERNAL_ARCHIVER commands (which are both set in mm_cfg.py) are
executed only when a new message arrives, that means it is not executed when
bin/arch is executed. This means that if there has been running some mail
list on mailman for a few years now and now I would like to allow searching
its content via new external search engine (like Xapian) it is simply no
enough to add external archiver and restart mailman because this would index
only newly added messages. Am I right?

How can I then have reindexed old content from that mail list into Xapian as
well? bin/arch  does not do that as it does not execute external
archivers. Moreover, running bin/arch can change URLs of individual public
emails (re-numbering) and that is probably unacceptable. So is there any way
how to iterate over existing emails, parse them and get an existing URL
value for them? (Such information could be then used to re-index old content
into external search server without need to run bin/arch).

Thanks,
Lukas Vlcek

[1]
http://wiki.list.org/display/DOC/4.87+How+do+I+invoke+some+process+on+messages+as+they+are+added+to+the+pipermail+archive
[2] http://www.mail-archive.com/mailman-users@python.org/msg56679.html
[3]
https://bugs.launchpad.net/mailman/+bug/531942/+attachment/1199211/+files/archive-and-index.py
--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] How to send messages out to only a portion of themembers?

2010-12-17 Thread Alan McConnell 
On Tue, Dec 14, 2010 at 09:40:15PM -0800, Mark Sapiro wrote:
> Alan McConnell wrote:
> 
> > I have an idea, but not the Mailman/Python skills to implement
> > it.  It is as follows:
> >Every month, create(automatically if possible) a new temporary  mailman
> >E-list(*).   Copy(automatically if possible) the members -- and data --
> >of the members to be reminded -- to this new temporary List.  Blast out
> >the warning.(Blast out a second time if deemed necessary).  Destroy
> >this temporary List.
> >(*)  If the regular List is called AbracaDabra, the new temporary
> >List can be called AbracaDabra-Remind, thus alerting AbracaDabra
> >members that here is an E-mail related to AbracaDabra, and should
> >be paid attention to.
> >
> > Is there anything in the above suggestion that is impossible?
> > If not, how much effort -- Python writing, etc -- would be
> > needed?  It would seem to me that such a capability would be
> > very useful for all kinds of groups and a multitude of purposes.
> 
> 
> I would do the following:
> 
> Create an announcement list named AbracaDabra-Remind following the
> guidelines in the FAQ at . This list will
> be permanent, but its membership will change. It should probably have 
> Privacy options... -> Subscription rules -> advertised = No and
> Archiving Options -> archive = No.
Thanks for the reply, Mark.  I've taken a look at the documentation,
and there is no mention there, nor probably can there be, of
my situation.   Namely:  I am the admin of a couple of E-lists,
run under mailman, but I am not the "owner".  These are E-lists
set up at my requests(one is free, the other I pay for) at two
different sites.  So, even if I could write python(not one of 
my languages unfortunately), could I write commands from
either my home machine or from my shell accounts at one of these
big sites, that could do what I wanted?

I think that in order to implement my suggestion(I don't say
"solve my problem", since I can get along without this capability)
it would have to be a part of the mailman package.  But maybe
mailman developers aren't interested in this capability?  I
have run the idea up the flagpole, but there have been minimal
salutes so far.

But I am certainly grateful for your thought and your response, Mark.

Best wishes of the Season to all!

Alan

-- 
Alan McConnell :  http://patriot.net/users/alan
It is difficult to get a man to understand something when 
his income depends on his not understanding it.
--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Stripping sig files

2010-12-17 Thread McNutt Jr, William R 
That was kinda my take on it, too.

Bill

-Original Message-
From: mailman-users-bounces+mcnutt=utk@python.org 
[mailto:mailman-users-bounces+mcnutt=utk@python.org] On Behalf Of Mark 
Sapiro

If I were you, I would moderate the offending users and simply reject their 
posts until they learn.

-- 
Mark Sapiro The highway is for gamblers,
San Francisco Bay Area, Californiabetter use your sense - B. Dylan

--
Mailman-Users mailing list Mailman-Users@python.org 
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3 Security Policy: 
http://wiki.list.org/x/QIA9 Searchable Archives: 
http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/mcnutt%40utk.edu


--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Stripping sig files

2010-12-17 Thread Christopher X. Candreva 
On Fri, 17 Dec 2010, Paul Tomblin wrote:

> On Thu, Dec 16, 2010 at 9:24 AM, bruce clark  wrote:
> > My problem: the users on my lists are waging a low-level war with their sig
> > files. I do not allow commercial posting so they have started creating sig
> > files with their commercial messages inside them.
> >
> 
> You're not going to solve a social problem with technology.  If people
> are violating the terms of the list, boot them off until they agree to
> follow the terms of the list.

Or, moderate the list amd reject any posts with such a sig, explaining why.


==
Chris Candreva  -- ch...@westnet.com -- (914) 948-3162
WestNet Internet Services of Westchester
http://www.westnet.com/
--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org

Re: [Mailman-Users] Stripping sig files

2010-12-17 Thread Paul Tomblin 
On Thu, Dec 16, 2010 at 9:24 AM, bruce clark  wrote:
> My problem: the users on my lists are waging a low-level war with their sig
> files. I do not allow commercial posting so they have started creating sig
> files with their commercial messages inside them.
>

You're not going to solve a social problem with technology.  If people
are violating the terms of the list, boot them off until they agree to
follow the terms of the list.

-- 
http://www.linkedin.com/in/paultomblin
http://careers.stackoverflow.com/ptomblin
--
Mailman-Users mailing list Mailman-Users@python.org
http://mail.python.org/mailman/listinfo/mailman-users
Mailman FAQ: http://wiki.list.org/x/AgA3
Security Policy: http://wiki.list.org/x/QIA9
Searchable Archives: http://www.mail-archive.com/mailman-users%40python.org/
Unsubscribe: 
http://mail.python.org/mailman/options/mailman-users/archive%40jab.org