Re: FastMail.FM Patchset Updated

2007-11-21 Thread Bron Gondwana

On Wed, 21 Nov 2007 07:40:21 +0100 (CET), "Simon Matter" <[EMAIL PROTECTED]> 
said:

> Hi Bron,
> 
> Did you consider this one
> http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3006 in the patch above?
> From a quick look it seems both patches conflict, is #3006 obsolete now?

You're right, they probably do conflict and Ken has applied 3006.  I should
have checked on that.  Hmm...

Actually, my patch doesn't even fix #3006.  It does sort of suck to be working
from a stable past point rather than the latest CVS actually when I do these
patches - though it's nice from a stability point of view.

OK - Ken, would you like me to re-build this patch on top of current CVS, or
would you like to do it yourself?  My patch has the replication bugfixes which
are still worth doing, but Simon's logic is actually correct for the test
while mine isn't (though I renamed the function he's using...)

Bron.
-- 
  Bron Gondwana
  [EMAIL PROTECTED]


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: LARGE single-system Cyrus installs?

2007-11-21 Thread Ian G Batten


On 20 Nov 07, at 1756, David Lang wrote:



however a fsync on a journaled filesystem just means the data needs  
to be
written to the journal, it doesn't mean that the journal needs to  
be flushed to

disk.

on ext3 if you have data=journaled then your data is in the journal  
as well and
all that the system needs to do on a fsync is to write things to  
the journal (a

nice sequential write),


Assuming the journal is on a distinct device and the distinct device  
can take the load.  It isn't on ZFS, although work is in progress.   
One of the many benefits of the sadly underrated Solaris Disksuite  
product was the metatrans devices, which at least permitted metadata  
updates to go to a distinct device.  When the UFS logging code went  
into core Solaris (the ON integration) that facility was dropped,  
sadly.  My Pillar NFS server does data logging to distinct disk  
groups, but mostly --- like such boxes tend to do --- relies on 12GB  
of RAM and a battery.  A sequential write is only of benefit if the  
head is in the right place and the platter is at the right rotational  
position and the write is well-matched to the transfer rate of the  
spindle: if the spindle is doing large sequential writes while also  
servicing reads and writes elsewhere, or can't keep up with writing  
tracks flat out, the problems increase.


for cyrus you should have the same sort of requirements that you  
would have for
a database server, including the fact that without a battery-backed  
disk cache
(or solid state drive) to handle your updates, you end up being  
throttled by
your disk rotation rate (you can only do a single fsync write per  
rotation, and
that good only if you don't have to seek), RAID 5/6 arrays are even  
worse, as
almost all systems will require a read of the entire stripe before  
writing a

single block (and it's parity block) back out, and since the stripe is
frequently larger then the OS readahead, the OS throws much of the  
data away

immediatly.

if we can identify the files that are the bottlenecks it would be very
interesting to see the result of puttng them on a solid-state drive.


I've split the meta-data out into separate partitions.  The meta data  
is stored in ZFS filesystems in a pool which is a RAID 0+1 4 disk  
group with SAS drives, the message data is coming out of the lowest  
QoS on my Pillar.  A ten second fsstat on VM operations shows that by  
request (this measures filesystem activity, not the implied disk  
activity) it's the meta partitions taking the pounding (ten second  
sample):


  map addmap delmap getpag putpag pagio
0  0  0 45  0 0 /var/imap
   11 11 11 17  0 0 /var/imap/meta-partition-1
  290290290463  5 0 /var/imap/meta-partition-2
  139139139183  3 0 /var/imap/meta-partition-3
   66 66 66106 10 0 /var/imap/meta-partition-7
  347347342454 16 0 /var/imap/meta-partition-8
   57 57 57 65  5 0 /var/imap/meta-partition-9
4  4  8  4  0 0 /var/imap/partition-1
   11 11 22 14  0 0 /var/imap/partition-2
1  1  2  1  0 0 /var/imap/partition-3
6  6 12 49 10 0 /var/imap/partition-7
   15 15 28457  0 0 /var/imap/partition-8
1  1  2  2  0 0 /var/imap/partition-9

Similarly, by non-VM operation:

 new  name   name  attr  attr lookup rddir  read read  write write
 file remov  chng   get   setops   ops   ops bytes   ops bytes
0 0 0 2.26K 0  6.15K 0 0 045 1.22K / 
var/imap
0 0 0   356 0707 0 0 0 6 3.03K / 
var/imap/meta-partition-1
3 0 3   596 0902 0 6  135K90  305K / 
var/imap/meta-partition-2
0 0 0   621 0  1.08K 0 0 0 3 1.51K / 
var/imap/meta-partition-3
3 0 3 1.04K 0  1.70K 0 6  149K36  650K / 
var/imap/meta-partition-7
0 0 0 2.28K 0  4.24K 0 0 0 7 1.87K / 
var/imap/meta-partition-8
0 0 018 0 32 0 0 0 2   176 / 
var/imap/meta-partition-9
2 2 222 0 30 0 1 2.37K 2 7.13K / 
var/imap/partition-1
3 41284 0157 0 1   677 3 7.51K / 
var/imap/partition-2
1 1 1 1.27K 0  2.16K 0 0 0 1 3.75K / 
var/imap/partition-3
2 2 435 0 56 0 1 3.97K36  279K / 
var/imap/partition-7
1 2 1   256 0514 0 0 0 1 3.75K / 
var/imap/partition-8
0 0 0 0 0  0 0 0 0 0 0 / 
var/imap/partition-9



And looking at the real IO load, ten seconds of zpool (for the meta  
data and /var/imap_


 capacity operationsbandwidth
pool   used  avail   read  write   rea

Re: FastMail.FM Patchset Updated

2007-11-21 Thread Ken Murchison
Bron Gondwana wrote:
> On Wed, 21 Nov 2007 07:40:21 +0100 (CET), "Simon Matter" <[EMAIL PROTECTED]> 
> said:
> 
>> Hi Bron,
>>
>> Did you consider this one
>> http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3006 in the patch above?
>> From a quick look it seems both patches conflict, is #3006 obsolete now?
> 
> You're right, they probably do conflict and Ken has applied 3006.  I should
> have checked on that.  Hmm...
> 
> Actually, my patch doesn't even fix #3006.  It does sort of suck to be working
> from a stable past point rather than the latest CVS actually when I do these
> patches - though it's nice from a stability point of view.
> 
> OK - Ken, would you like me to re-build this patch on top of current CVS, or
> would you like to do it yourself?  My patch has the replication bugfixes which
> are still worth doing, but Simon's logic is actually correct for the test
> while mine isn't (though I renamed the function he's using...)

If you have the time.  I'm off banging on other things at the moment.

-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: FastMail.FM Patchset Updated

2007-11-21 Thread Ken Murchison
Bron Gondwana wrote:
> As usual you can get the patches here:
> 
> http://cyrus.brong.fastmail.fm/
> 
> 
> I've been busy with Cyrus _again_ - so much for my theory
> that I was taking a break.
> 
> OK - here's what's new.
> 
> * http://cyrus.brong.fastmail.fm/patches/cyrus-skiplist-bugfixes-2.3.10.diff
>   http://cyrus.brong.fastmail.fm/patches/cyrus-skiplist-robustify-2.3.10.diff
> 
>   Skiplist issues - there were two things that could be found
>   in recovery that actually bit us during the whole "restart
>   every single store with the new skiplist code" project the
>   other day.  ADD where the record already existed and DELETE
>   where it didn't.  The later also had an obnoxious bug where
>   it would instead delete _the_alphabetically_NEXT_record_
>   silently.  Ooops.
> 
>   I rolled these two into my bugfix and robustify patches, not
>   realising Ken had already applied the previous copies upstream.
>   Ken - do you want me to break this out as a separate patch on
>   top of the others?

Yes, if you wouldn't mind.


-- 
Kenneth Murchison
Systems Programmer
Project Cyrus Developer/Maintainer
Carnegie Mellon University

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


how to limit pop/imap login password attempts

2007-11-21 Thread Martin Kraus
Hi,
  I've been trying to figure out, how to limit login attempts for cyrus
pop/imap daemons. I'm trying to prevent brute-force password guessing. I'm
using cyrus sasl with /etc/sasldb2 user database, which also authenticates
postfix users. I'd like to solve this problem through sasl so I won't have to
figure the same for postfix or keep different passwords for mailboxes and
smtp. Is there any mechanism to do this through sasl or do I have to try doing
it through a firewall? 

I'm running debian etch system. If imap and pop do not allow multiple login
attempts within a single session, I could try to work around this problem
using iptables with the recent module but it's like scratching your left ear
with your right hand around the back of your head.

thanks for any pointers
Martin Kraus

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: Bad problems with 2.3.10 on FreeBSD

2007-11-21 Thread Per olof Ljungmark
Rich Wales wrote:
> I recently upgraded my system to 2.3.10.  My main Cyrus server runs
> FreeBSD 6.2, replicating to a second server running Ubuntu 7.10.  I
> had decided to upgrade now from 2.3.9 to 2.3.10 because of replication
> "bailing out" errors in 2.3.9 that are reportedly fixed in 2.3.10.
> 
> At first, I was having lots of seg-fault errors, but I added the
> getgrouplist() patch to auth_unix.c on my FreeBSD system, and that
> seemed to take care of the core-dumps.
> 
> However, I started seeing other problems on the FreeBSD box -- such
> as "ctl_cyrusdb -c" checkpoint jobs hanging, and other mail delivery
> attempts hanging (or timing out) and piling up in my Postfix queue.
> I had to restart Cyrus frequently on the FreeBSD system in order to
> clear the logjams.
> 
> I finally gave up and went back to 2.3.9 on both servers.  Given the
> instability of 2.3.10 on my main server, I felt I had no choice.
> 
> Any ideas what might have been happening?  If I can't figure out how
> to get 2.3.10 running stably, I assume I'm going to either have to
> stick with 2.3.9 (and figure out some other way around the replication
> bailout errors until 2.3.11 comes out), or else move my main Cyrus
> operations onto some other platform (maybe another Ubuntu box).
> 

Did you check anything with the FBSD port maintainer? There may be 
patches available that cannot be commited due to the freeze.

--per

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Creating mailboxes with cyradm - useraccount with a 'dot'

2007-11-21 Thread Kuhlemann, Stefan
Hi mailinglist.

( newbie qestion ahead ;-) )

At the moment I am experimenting with cyrus, running on fedora 7. 

Problem:
Creating mailboxes with cyradm works fine until I want to create a
mailbox for a account like 'peter.testaccount':

In cyradm:

> cm user.peter.testaccount
> createmailbox: Permission denied

While it seems to be logical to me, that mailboxnames with a dot are not
working out-of-the-box because of the way cyrus organizes it's mailboxes
(user..), it would be nice, if sombody could give
me a hint - how to cope with useraccounts with a 'dot' in them and the
corresponding cyrus IMAP mailboxes.

Thanks for your help in advance...

Stefan



Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: how to limit pop/imap login password attempts

2007-11-21 Thread Dan White
Martin Kraus wrote:
> Hi,
>   I've been trying to figure out, how to limit login attempts for cyrus
> pop/imap daemons. I'm trying to prevent brute-force password guessing. I'm
> using cyrus sasl with /etc/sasldb2 user database, which also authenticates
> postfix users. I'd like to solve this problem through sasl so I won't have to
> figure the same for postfix or keep different passwords for mailboxes and
> smtp. Is there any mechanism to do this through sasl or do I have to try doing
> it through a firewall? 
> 
> I'm running debian etch system. If imap and pop do not allow multiple login
> attempts within a single session, I could try to work around this problem
> using iptables with the recent module but it's like scratching your left ear
> with your right hand around the back of your head.

Hi Martin,

A couple of ideas come to mind. You could force the use of the 
'NODICT' security flag, or force the use of mechanism which 
support it. see:

http://www.sendmail.org/~ca/email/cyrus2/mechanisms.html

Using those mechanisms would probably require a change in the way 
your users authenticate.

Another idea, and this isn't really an approach that will work 
today, is to use the ldapdb auxprop plugin to store your 
passwords, and make use of the openldap ppolicy module to enforce 
  password policy.

This doesn't really work, because openldap ppolicy does not (yet) 
enforce password policy when sasl bind (which ldapdb uses) is in 
use. It only support simple bind. I haven't actually looked at 
OpenLDAP 2.4.x yet to see if it's supported.

A modification to the ldapdb plugin could probably be made to 
perform a simple bind just after the step where it retrieves the 
userPassword attribute.

- Dan

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: Creating mailboxes with cyradm - useraccount with a 'dot'

2007-11-21 Thread Michael Menge

Hi,

If you use unixhierarchysep: 0 in you /etc/imapd.conf (which is default)
the . is used as hierarchyseperator.
So in your case user.peter.testaccount would be the Folder testaccount  
of the user peter.


There are two way use ^ to replace .

cm user.peter^testaccount

Or use unixhierarchysep: 1 so / is used as hierarchyseperator.

cm user/peter.testaccount

Regards

Michael

Quoting "Kuhlemann, Stefan" <[EMAIL PROTECTED]>:


Hi mailinglist.

( newbie qestion ahead ;-) )

At the moment I am experimenting with cyrus, running on fedora 7.

Problem:
Creating mailboxes with cyradm works fine until I want to create a
mailbox for a account like 'peter.testaccount':

In cyradm:


cm user.peter.testaccount
createmailbox: Permission denied


While it seems to be logical to me, that mailboxnames with a dot are not
working out-of-the-box because of the way cyrus organizes it's mailboxes
(user..), it would be nice, if sombody could give
me a hint - how to cope with useraccounts with a 'dot' in them and the
corresponding cyrus IMAP mailboxes.

Thanks for your help in advance...

Stefan



Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html






M.Menge Tel.: (49) 7071/29-70316
Universitaet Tuebingen  Fax.: (49) 7071/29-5912
Zentrum fuer Datenverarbeitung  mail:  
[EMAIL PROTECTED]

Waechterstrasse 76
72074 Tuebingen


smime.p7s
Description: S/MIME krytographische Unterschrift

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: how to limit pop/imap login password attempts

2007-11-21 Thread Dan White
Dan White wrote:
> A modification to the ldapdb plugin could probably be made to perform a 
> simple bind just after the step where it retrieves the userPassword 
> attribute.

On second thought, that doesn't actually work. The auxprop plugin 
doesn't know anything about the password, or sasl exchange, that 
the users performs, only what the correct userPassword is...

- Dan

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


AW: Creating mailboxes with cyradm - useraccount with a 'dot'

2007-11-21 Thread Kuhlemann, Stefan
> -Ursprüngliche Nachricht-
> Von: [EMAIL PROTECTED] 
> [mailto:[EMAIL PROTECTED] Im Auftrag 
> von Michael Menge
> Gesendet: Mittwoch, 21. November 2007 17:07
> An: info-cyrus@lists.andrew.cmu.edu
> Betreff: Re: Creating mailboxes with cyradm - useraccount with a 'dot'

[...]

> 
> There are two way use ^ to replace .
> 
> cm user.peter^testaccount
> 
> Or use unixhierarchysep: 1 so / is used as hierarchyseperator.
> 
> cm user/peter.testaccount
> 
> Regards
> 
>  Michael
> 

[...]


Thanks...

I will use 'unixhierarchysep' :-)

Btw. - are there any drawbacks coming with 'unixhierarchysep: 1' ? Not that I 
am running into problems later on? (...so that I will miss some very usefull 
fuctions)

Stefan

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: pop3d 100% cpu

2007-11-21 Thread Rafael Mahecha


I have... the problem was actually with our sendmail it is supposed to
have a message size limit... for the maximum size of message that sendmail
will transfer for it turns out that during one of the updates, that option
was not set... so by default it is unlimited, so when a user wanted to send a
170MB home video from her outlook, the server was sitting there trying to
transfer it...

The problem was fixed by removing the mailq and
resetting the option in sendmail.


I guess that for the pop,
your server received a large message and one(or more) of the user(s) is trying
to download that to their computer?

I think cyrus has a max_size
option ... or may be lmtp... which tells it to fail if it finds a message
larger than that
-- 
--
--
Rafael Mahecha
E-mail
Administrator
Office of Information Management
Jackson State
University

JSU e-Center
1230 Raymond Road
Jackson, MS
39204

[EMAIL PROTECTED]
(601)-979-1783
http://www.jsums.edu

On Tue, November 20, 2007 3:51 pm, Faust
wrote:
> 
> 
> About 5 to 10 minutes after starting
cyrus_imapd the process pop3d will take
> 100% cpu usage and the
system will freece.
> 
> Anyone else has seen this problem?
> 
> I am using fedora 8 with the following packages;
>

> cyrus-sasl-plain-2.1.22-6
>
cyrus-imapd-utils-2.3.9-7.fc7
> cyrus-sasl-lib-2.1.22-6
>
cyrus-sasl-md5-2.1.22-6
> cyrus-imapd-perl-2.3.9-7.fc7
>
cyrus-imapd-2.3.9-7.fc7
> cyrus-sasl-2.1.22-6
> 
> 
> Faust
> 
> 
> Cyrus Home Page:
http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ:
http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info:
http://asg.web.cmu.edu/cyrus/mailing-list.html
>

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html

Re: LARGE single-system Cyrus installs?

2007-11-21 Thread David Lang
On Wed, 21 Nov 2007, Ian G Batten wrote:

>> however a fsync on a journal ed filesystem just means the data needs to be
>> written to the journal, it doesn't mean that the journal needs to be 
>> flushed to
>> disk.
>> 
>> on ext3 if you have data=journal ed then your data is in the journal as well 
>> and
>> all that the system needs to do on a fsync is to write things to the 
>> journal (a
>> nice sequential write),
>
> Assuming the journal is on a distinct device and the distinct device can take 
> the load.  It isn't on ZFS, although work is in progress.

I was responding to the comments about ext3 and other journal ed filesystems as 
alternatives to zfs and the claim that doing a fsync on one of them required 
flushing the entire journal. sorry if I wasn't clear enough about this.

David Lang


Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


prefork mupdate slaves

2007-11-21 Thread Andrew Morgan
With the following service entry on a traditional murder frontend 
(v2.3.10):

   mupdate   cmd="/usr/local/cyrus/bin/mupdate" listen="3905" prefork=1

I notice that 2 mupdate processes are spawned at startup:

Nov 21 14:43:43 cyrus-fe3 mupdate[6935]: successful mupdate connection to 
cyrus-mm.onid.oregonstate.edu
Nov 21 14:43:43 cyrus-fe3 mupdate[6935]: unready for connections
Nov 21 14:43:43 cyrus-fe3 mupdate[6935]: synchronizing mailbox list with master 
mupdate server
Nov 21 14:43:43 cyrus-fe3 mupdate[6931]: successful mupdate connection to 
cyrus-mm.onid.oregonstate.edu
Nov 21 14:43:43 cyrus-fe3 mupdate[6931]: unready for connections
Nov 21 14:43:43 cyrus-fe3 mupdate[6931]: synchronizing mailbox list with master 
mupdate server
Nov 21 14:43:48 cyrus-fe3 mupdate[6935]: mailbox list synchronization complete
Nov 21 14:43:52 cyrus-fe3 mupdate[6931]: mailbox list synchronization complete

And those 2 mupdate processes remain running.

If I set prefork=0, then no mupdate processes are ever spawned, as far as 
I can tell.

On my old v2.2.13 murder frontends, prefork=1 only causes 1 mupdate 
process to be spawned.

Is this a bug?  Should I be concerned that 2 mupdate slaves are running?

Thanks,
Andy

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


mixing versions in murder

2007-11-21 Thread Andrew Morgan
Are there any DOs or DON'Ts when mixing different versions of Cyrus in a 
traditional murder?  Specifically, I have a murder at version 2.2.13, and 
I'd like to upgrade to version 2.3.10.  Can I upgrade the frontends first? 
Do I need to upgrade all of them at once?

Thanks,
Andy

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


reconstruct -u or -U

2007-11-21 Thread Andrew Morgan
The changelog says:

   Added -u and -U options to reconstruct -- courtesy of David Carter.

But I can't find those options listed in the manpage or the built-in help 
of reconstruct.  What do those options do?

Andy

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: how to limit pop/imap login password attempts

2007-11-21 Thread Alain Spineux
On Nov 21, 2007 2:27 PM, Martin Kraus <[EMAIL PROTECTED]> wrote:
> Hi,
>   I've been trying to figure out, how to limit login attempts for cyrus
> pop/imap daemons. I'm trying to prevent brute-force password guessing.

You can try to use nginx as a proxy imap, pop and smtp protocol. (and
HTTP of course).
The goal is to have the same frontend for multiple pop/imap server and redirect
any connection to the good one depending on the username.

You need to provide a small application that depending the username
will give the
address of the server where the imap/pop account is stored. Some perl
and php sample
are on the nginx wiki

Here is a simple I wrote in python

#!/bin/env python

import sys, BaseHTTPServer

class NginxAuth(BaseHTTPServer.BaseHTTPRequestHandler):

def do_GET(self):
print 'GET', self.client_address, self.path, self.headers

user=self.headers["Auth-User"]
password=self.headers["Auth-Pass"]
protocol=self.headers["Auth-Protocol"]

self.send_response(200, 'OK')
self.send_header('Auth-Status', 'OK')
self.send_header('Auth-Server', '127.0.0.1')
self.send_header('Auth-Port', '143')
self.end_headers()

server=BaseHTTPServer.HTTPServer(('127.0.0.1',8081), NginxAuth)
server.serve_forever()

Here I redirect all connection to my unique server 127.0.0.1 without
doing any check on the user/password (Auth-Status='OK')
If the password was wrong, then the imap server will reject the
connection anyway.
But you can keep a log of all connections with a timestamp and reject
the connection
if the password is changing too often in a small amount of time.


Dont forget to share your experiences if you get some success.

> I'm
> using cyrus sasl with /etc/sasldb2 user database, which also authenticates
> postfix users. I'd like to solve this problem through sasl so I won't have to
> figure the same for postfix or keep different passwords for mailboxes and
> smtp. Is there any mechanism to do this through sasl or do I have to try doing
> it through a firewall?
>
> I'm running debian etch system. If imap and pop do not allow multiple login
> attempts within a single session, I could try to work around this problem
> using iptables with the recent module but it's like scratching your left ear
> with your right hand around the back of your head.
>
> thanks for any pointers
> Martin Kraus
> 
> Cyrus Home Page: http://cyrusimap.web.cmu.edu/
> Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
> List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
>



-- 
Alain Spineux
aspineux gmail com
May the sources be with you

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: LARGE single-system Cyrus installs?

2007-11-21 Thread Andrew McNamara
>>> Would'nt it be nice to have a configuration option to completely  
>>> turn off fsync() in Cyrus? If you want, with a BIG WARNING in the  
>>> doc stating NOT TO USE IT unless you know what you doing. :)
>>
>> Its already in imapd.conf(8):
>>
>> skiplist_unsafe
>
>Well shiver me timbers!  Ya'll rock.

Note, however, that fsync() still serves a purpose with data journalling - 
without it, your application writes may be sitting in buffer cache,
and may be arbitrarily re-ordered before hitting the disk. The fsync()
should not return until the blocks have reached the disk (or journal),
and thus forms a syncronisation point (which is critical for maintaining
sanity of on-disk data structures like skiplists).

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: prefork mupdate slaves

2007-11-21 Thread Gary Mills
On Wed, Nov 21, 2007 at 02:58:11PM -0800, Andrew Morgan wrote:
> With the following service entry on a traditional murder frontend 
> (v2.3.10):
> 
>mupdate   cmd="/usr/local/cyrus/bin/mupdate" listen="3905" prefork=1
> 
> I notice that 2 mupdate processes are spawned at startup:
> 
> Nov 21 14:43:43 cyrus-fe3 mupdate[6935]: successful mupdate connection to 
> cyrus-mm.onid.oregonstate.edu
> Nov 21 14:43:43 cyrus-fe3 mupdate[6935]: unready for connections
> Nov 21 14:43:43 cyrus-fe3 mupdate[6935]: synchronizing mailbox list with 
> master mupdate server
> Nov 21 14:43:43 cyrus-fe3 mupdate[6931]: successful mupdate connection to 
> cyrus-mm.onid.oregonstate.edu
> Nov 21 14:43:43 cyrus-fe3 mupdate[6931]: unready for connections
> Nov 21 14:43:43 cyrus-fe3 mupdate[6931]: synchronizing mailbox list with 
> master mupdate server
> Nov 21 14:43:48 cyrus-fe3 mupdate[6935]: mailbox list synchronization complete
> Nov 21 14:43:52 cyrus-fe3 mupdate[6931]: mailbox list synchronization complete
> 
> And those 2 mupdate processes remain running.
> 
> If I set prefork=0, then no mupdate processes are ever spawned, as far as 
> I can tell.
> 
> On my old v2.2.13 murder frontends, prefork=1 only causes 1 mupdate 
> process to be spawned.
> 
> Is this a bug?  Should I be concerned that 2 mupdate slaves are running?

I have two on the murder front end as well:

 UID   PID  PPID  CSTIME TTY  TIME CMD
   cyrus   765   708  0   Sep 24 ?   332:49 mupdate
   cyrus   768   708  0   Sep 24 ?1:35 mupdate

On the murder back end, I actually have three, although one terminates
almost immediately with this error:

  bind: /imap/mupdate/conf/socket/mupdate.target: Address already in use
  bind failed
  service mupdate pid 1595 in READY state: terminated abnormally

I also have `prefork=1' in both places.  This is with cyrus-imapd-2.3.8.

-- 
-Gary Mills--Unix Support--U of M Academic Computing and Networking-

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: reconstruct -u or -U

2007-11-21 Thread Bron Gondwana
On Wed, Nov 21, 2007 at 03:06:39PM -0800, Andrew Morgan wrote:
> The changelog says:
> 
>Added -u and -U options to reconstruct -- courtesy of David Carter.
> 
> But I can't find those options listed in the manpage or the built-in help 
> of reconstruct.  What do those options do?

The changelog needs to be updated.  They got renamed as -g and -G.
They either wipe or calculate the message GUID (globally unique
identifier, we hope!) which is the sha1 of the message contents
now.

Bron.

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Murder in replicated mode

2007-11-21 Thread Diego Woitasen
Hi!
I'm trying to setup murder in replicated mode. My schema is:

-Two servers + one shared storage
-Redhat Cluster Suite (RHEL 5.1) with GFS2 working.
-Cyrus 2.3.10 in both servers working.
-spool and sieve directories on GFS
-config dir on local filesystems.

I have configured Cyrus in replicated mode but when I create an
account in the master server, it isn't replicated unless Cyrus
is restart in either node. It isnt't an authentication problem,
when I create an account with cyradmin mupdate do nothing.

My config files:

Master imapd.conf

configdirectory: /var/lib/cyrus
partition-default: /var/cyrus/spool
admins: cyrus
sievedir: /var/cyrus/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: auxprop
sasl_mech_list: PLAIN
tls_cert_file: /var/cyrus/etc/cyrus-cert.pem
tls_key_file: /var/cyrus/etc/cyrus-key.pem
tlscache_db: skiplist
autocreatequota: 100
allowplaintext: 1
defaultdomain: xcoop.coop

mupdate_config: replicated

-Master cyrus.conf-


START {
# do not delete this entry!
recover cmd="ctl_cyrusdb -r"

# this is only necessary if using idled for IMAP IDLE
#idled  cmd="idled"
}

# UNIX sockets start with a slash and are put into /var/imap/socket
SERVICES {
# add or remove based on preferences
imapcmd="imapd" listen="imap" prefork=0
imaps   cmd="imapd -s" listen="imaps" prefork=0
pop3cmd="pop3d" listen="pop3" prefork=0
pop3s   cmd="pop3d -s" listen="pop3s" prefork=0
sieve   cmd="timsieved" listen="sieve" prefork=0
mupdate cmd="mupdate -m" listen=3905 prefork=1

# these are only necessary if receiving/exporting usenet via NNTP
#nntp   cmd="nntpd" listen="nntp" prefork=0
#nntps  cmd="nntpd -s" listen="nntps" prefork=0

# at least one LMTP is required for delivery
#lmtp   cmd="lmtpd" listen="lmtp" prefork=0
lmtpunixcmd="lmtpd" listen="/var/imap/socket/lmtp" prefork=0

# this is required if using notifications
#notify cmd="notifyd" listen="/var/imap/socket/notify" proto="udp" 
prefork=1
}

EVENTS {
# this is required
checkpoint  cmd="ctl_cyrusdb -c" period=30

# this is only necessary if using duplicate delivery suppression,
# Sieve or NNTP
delprunecmd="cyr_expire -E 3" at=0400

# this is only necessary if caching TLS sessions
tlsprunecmd="tls_prune" at=0400
}

Slave imapd.conf---


configdirectory: /var/lib/cyrus
partition-default: /var/cyrus/spool
admins: cyrus
sievedir: /var/cyrus/sieve
sendmail: /usr/sbin/sendmail
hashimapspool: true
sasl_pwcheck_method: auxprop
sasl_mech_list: PLAIN
tls_cert_file: /var/cyrus/etc/cyrus-cert.pem
tls_key_file: /var/cyrus/etc/cyrus-key.pem
tlscache_db: skiplist
autocreatequota: 100
allowplaintext: 1
defaultdomain: xcoop.coop

mupdate_server: rh-cluster1
mupdate_username: cyrus
mupdate_authname: cyrus
mupdate_password: 123456
mupdate_config: replicated

Slave cyrus.conf-


START {
# do not delete this entry!
recover cmd="ctl_cyrusdb -r"

# this is only necessary if using idled for IMAP IDLE
#idled  cmd="idled"
}

# UNIX sockets start with a slash and are put into /var/imap/socket
SERVICES {
# add or remove based on preferences
imapcmd="imapd" listen="imap" prefork=0
imaps   cmd="imapd -s" listen="imaps" prefork=0
pop3cmd="pop3d" listen="pop3" prefork=0
pop3s   cmd="pop3d -s" listen="pop3s" prefork=0
sieve   cmd="timsieved" listen="sieve" prefork=0
mupdate cmd="mupdate" listen=3905 prefork=1

# these are only necessary if receiving/exporting usenet via NNTP
#nntp   cmd="nntpd" listen="nntp" prefork=0
#nntps  cmd="nntpd -s" listen="nntps" prefork=0

# at least one LMTP is required for delivery
#lmtp   cmd="lmtpd" listen="lmtp" prefork=0
lmtpunixcmd="lmtpd" listen="/var/imap/socket/lmtp" prefork=0

# this is required if using notifications
#notify cmd="notifyd" listen="/var/imap/socket/notify" proto="udp" 
prefork=1
}

EVENTS {
# this is required
checkpoint  cmd="ctl_cyrusdb -c" period=30

# this is only necessary if using duplicate delivery suppression,
# Sieve or NNTP
delprunecmd="cyr_expire -E 3" at=0400

# this is only necessary if caching TLS sessions
tlsprunecmd="tls_prune" at=0400
}


Regards!

-- 

--
Diego Woitasen

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki

Re: Murder in replicated mode

2007-11-21 Thread Bron Gondwana
On Wed, Nov 21, 2007 at 09:41:15PM -0300, Diego Woitasen wrote:
> Hi!
>   I'm trying to setup murder in replicated mode. My schema is:
>   
>   -Two servers + one shared storage
>   -Redhat Cluster Suite (RHEL 5.1) with GFS2 working.
>   -Cyrus 2.3.10 in both servers working.
>   -spool and sieve directories on GFS
>   -config dir on local filesystems.
> 
>   I have configured Cyrus in replicated mode but when I create an
>   account in the master server, it isn't replicated unless Cyrus
>   is restart in either node. It isnt't an authentication problem,
>   when I create an account with cyradmin mupdate do nothing.

>From memory you may have to actually deliver a message to one of
the user's folders before it will replicate.  At least that's what
the docs suggested.  Of course we always LMTP deliver a message
right at creation time, so we never bothered to check this.

Bron.

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: FastMail.FM Patchset Updated

2007-11-21 Thread Bron Gondwana
On Wed, Nov 21, 2007 at 06:37:17AM -0500, Ken Murchison wrote:
> Bron Gondwana wrote:
>> On Wed, 21 Nov 2007 07:40:21 +0100 (CET), "Simon Matter" 
>> <[EMAIL PROTECTED]> said:
>>> Hi Bron,
>>>
>>> Did you consider this one
>>> http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3006 in the patch above?
>>> From a quick look it seems both patches conflict, is #3006 obsolete now?
>> You're right, they probably do conflict and Ken has applied 3006.  I 
>> should
>> have checked on that.  Hmm...
>> Actually, my patch doesn't even fix #3006.  It does sort of suck to be 
>> working
>> from a stable past point rather than the latest CVS actually when I do 
>> these
>> patches - though it's nice from a stability point of view.
>> OK - Ken, would you like me to re-build this patch on top of current CVS, 
>> or
>> would you like to do it yourself?  My patch has the replication bugfixes 
>> which
>> are still worth doing, but Simon's logic is actually correct for the test
>> while mine isn't (though I renamed the function he's using...)
>
> If you have the time.  I'm off banging on other things at the moment.

Attached are tidied up versions of both my earlier patches on top of
what you've already got in CVS.  They apply fine on top of current
CVS trunk:

[EMAIL PROTECTED]:/work/cvs/cyrus-imapd# patch -p1 < 
/work/cyrus-imapd-2.3.10/patches/cyrus-skiplist-newfixes-2.3.10.diff
patching file lib/cyrusdb_skiplist.c
Hunk #1 succeeded at 2113 (offset -1 lines).
Hunk #2 succeeded at 2140 (offset -1 lines).
Hunk #3 succeeded at 2166 (offset -1 lines).
Hunk #4 succeeded at 2206 (offset -1 lines).
[EMAIL PROTECTED]:/work/cvs/cyrus-imapd# patch -p1 < 
/work/cyrus-imapd-2.3.10/patches/cyrus-deletemode-userfix-2.3.10.diff
patching file imap/mboxname.c
patching file imap/mboxlist.c
patching file imap/mboxlist.h
patching file imap/mboxname.h
patching file imap/imapd.c
Hunk #1 succeeded at 5006 (offset 2 lines).
Hunk #2 succeeded at 5098 (offset 2 lines).


Bron.

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: FastMail.FM Patchset Updated (new patches)

2007-11-21 Thread Bron Gondwana
On Wed, Nov 21, 2007 at 06:37:17AM -0500, Ken Murchison wrote:
> Bron Gondwana wrote:
>> On Wed, 21 Nov 2007 07:40:21 +0100 (CET), "Simon Matter" 
>> <[EMAIL PROTECTED]> said:
>>> Hi Bron,
>>>
>>> Did you consider this one
>>> http://bugzilla.andrew.cmu.edu/show_bug.cgi?id=3006 in the patch above?
>>> From a quick look it seems both patches conflict, is #3006 obsolete now?
>> You're right, they probably do conflict and Ken has applied 3006.  I 
>> should
>> have checked on that.  Hmm...
>> Actually, my patch doesn't even fix #3006.  It does sort of suck to be 
>> working
>> from a stable past point rather than the latest CVS actually when I do 
>> these
>> patches - though it's nice from a stability point of view.
>> OK - Ken, would you like me to re-build this patch on top of current CVS, 
>> or
>> would you like to do it yourself?  My patch has the replication bugfixes 
>> which
>> are still worth doing, but Simon's logic is actually correct for the test
>> while mine isn't (though I renamed the function he's using...)
>
> If you have the time.  I'm off banging on other things at the moment.

I guess I should attach them!

Bron.
Fix Delayed Delete replication

Candidate for upstream - now respun on top of
   Simon Matter's allowusermoves fixes from CVS

Deleting user.foo was broken on replicas thanks to
me choosing a bad method for making it work.

Also, it was leaving foo.TIMESTAMP.sub files because
it considered that to be a valid "user".  Oops.

This patch takes a different approach (on top of the
delayed delete code already in Cyrus 2.3.10):

1) mboxname_isusermailbox() no longer returns true for
   DELETED.user.foo.TIMESTAMP

2) The "can I rename this" code checks for the target
   name being either another username or something in the
   DELETED. namespace.
2a) mboxname_isdeletedmailbox() rather than the somewhat
   (IMHO) misplaced mboxlist_in_deletedhierarchy().

3) if "forceuser" is passed (only done by sync_server),
   then all ACL checks are completely bypassed, so the
   rename will always succeed on the replica.

Index: cyrus-imapd-2.3.10/imap/mboxname.c
===
--- cyrus-imapd-2.3.10.orig/imap/mboxname.c	2007-11-22 01:22:08.0 -0500
+++ cyrus-imapd-2.3.10/imap/mboxname.c	2007-11-22 01:29:17.0 -0500
@@ -600,35 +600,47 @@
 {
 const char *p;
 const char *start = name;
-const char *deletedprefix = config_getstring(IMAPOPT_DELETEDPREFIX);
-size_t len = strlen(deletedprefix);
-int isdel = 0;
 
 /* step past the domain part */
 if (config_virtdomains && (p = strchr(start, '!')))
 	start = p + 1;
 
-/* step past any deletedprefix */
-if (mboxlist_delayed_delete_isenabled() && strlen(start) > len+1 &&
-	!strncmp(start, deletedprefix, len) && start[len] == '.')  {
-	start += len + 1;
-	isdel = 1; /* there's an add'l sep + hextimestamp on isdel folders */
-}
-
 /* starts with "user." AND
  * we don't care if it's an inbox OR
- * there's no dots after the username OR 
- * it's deleted and there's only one more dot
+ * there's no dots after the username 
  */
-if (!strncmp(start, "user.", 5) &&
-	(!isinbox || !strchr(start+5, '.') ||
-	 (isdel && (p = strchr(start+5, '.')) && !strchr(p+1, '.'
-	return (char*) start+5; /* could have trailing bits if isinbox+isdel */
+if (!strncmp(start, "user.", 5) && (!isinbox || !strchr(start+5, '.')))
+	return (char*) start+5;
 else
 	return NULL;
 }
 
 /*
+ * If (internal) mailbox 'name' is a DELETED mailbox
+ * returns boolean
+ */
+int mboxname_isdeletedmailbox(const char *name)
+{
+static const char *deletedprefix = NULL;
+static int deletedprefix_len = 0;
+int domainlen = 0;
+char *p;
+
+if (!mboxlist_delayed_delete_isenabled()) return(0);
+
+if (!deletedprefix) {
+	deletedprefix = config_getstring(IMAPOPT_DELETEDPREFIX);
+  deletedprefix_len = strlen(deletedprefix);
+}
+
+if (config_virtdomains && (p = strchr(name, '!')))
+	domainlen = p - name + 1;
+
+return ((!strncmp(name + domainlen, deletedprefix, deletedprefix_len) &&
+	 name[domainlen + deletedprefix_len] == '.') ? 1 : 0);
+}
+
+/*
  * Translate (internal) inboxname into corresponding userid.
  */
 char *mboxname_inbox_touserid(const char *inboxname)
Index: cyrus-imapd-2.3.10/imap/mboxlist.c
===
--- cyrus-imapd-2.3.10.orig/imap/mboxlist.c	2007-11-22 01:29:03.0 -0500
+++ cyrus-imapd-2.3.10/imap/mboxlist.c	2007-11-22 01:32:08.0 -0500
@@ -1223,8 +1223,7 @@
 	isusermbox = 1;
 	} else if ((config_getswitch(IMAPOPT_ALLOWUSERMOVES) &&
 		mboxname_isusermailbox(newname, 1)) ||
-		   mboxlist_in_deletedhierarchy(oldname) ||
-		   mboxlist_in_deletedhierarchy(newname)) {
+		   mboxname_isdeletedmailbox(newname)) {
 	/* Special case of renaming a user */
 	access = cyrus_acl_myrights(auth_state, oldacl);
 	if (!(access & ACL_D

Re: Murder in replicated mode

2007-11-21 Thread Janne Peltonen
On Thu, Nov 22, 2007 at 12:50:47PM +1100, Bron Gondwana wrote:
> On Wed, Nov 21, 2007 at 09:41:15PM -0300, Diego Woitasen wrote:
> > Hi!
> > I'm trying to setup murder in replicated mode. My schema is:
> > 
> > -Two servers + one shared storage
> > -Redhat Cluster Suite (RHEL 5.1) with GFS2 working.
> > -Cyrus 2.3.10 in both servers working.
> > -spool and sieve directories on GFS
> > -config dir on local filesystems.
> > 
> > I have configured Cyrus in replicated mode but when I create an
> > account in the master server, it isn't replicated unless Cyrus
> > is restart in either node. It isnt't an authentication problem,
> > when I create an account with cyradmin mupdate do nothing.
> 
> >From memory you may have to actually deliver a message to one of
> the user's folders before it will replicate.  At least that's what
> the docs suggested.  Of course we always LMTP deliver a message
> right at creation time, so we never bothered to check this.

Yes, that's the way it goes.

When I setup my system (migrated from an ages-old cyrus system using
imapsync), I had first imapsync'd the primary system from the old
system, before setting up the replica. So, when the replica was up and
running, I did a sync_client -u for every user and a sync_client -m for
every bulletin board in the primary system... took a couple of days :)

(I had the replica running during following imapsyncs and the final one,
so of course the changes got propagated automatically then.)


--Janne Peltonen
Imap admin
Univ of Helsinki
-- 
Janne Peltonen <[EMAIL PROTECTED]>

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html


Re: LARGE single-system Cyrus installs?

2007-11-21 Thread Pascal Gienger
Vincent Fox <[EMAIL PROTECTED]> wrote:

> This thought has occurred to me:
>
> ZFS prefers reads over writes in it's scheduling.
>
> I think you can see where I'm going with this.  My WAG is something
> related to Pascal's, namely latency.  What if my write requests to
> mailboxes.db
> or deliver.db start getting stacked up, due to the favoritism shown to
> reads?

I got substantial benefits from setting compression=on and recordsize=32K 
on the filesystem where deliver.db resides. After talking with our SAN 
staff it showed up that storage was our problem - it has problems with 
concurrent write and read calls, the system won't answer read requests if 
the write channel is "full". I don't know whether it is a firmwire issue or 
a non-capability of the storage system.

Lowering ZFS' recordsize and activating compression on that partition cut 
down i/o rate and things are going normal here again.

Thanks to all who helped!

Pascal

PS: The mirror resilvering problem was a misconfiguration of a brocade 
switch... Sometimes you don't see the forest due to so many trees (german 
proverb, "Man sieht den Wald vor lauter Bäumen nicht")...

Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html