Follow-up: different APPEND behavior v2.1.15 / v2.2.6
On Wed, 14 Jul 2004, Andrew J Caird wrote: > Hello all, > Upon further testing, this behavior seems to have surfaced between > v2.2.3 and v2.2.4 and is also in v2.2.5 and v2.2.6. > > The changes listed between v2.2.3 and v2.2.4 aren't too extensive; the > only thing that looks like it could have impacted this behavior is: > o All incoming messages are now staged to disk before locking the >destination mailbox (locks are no longer held during a network >read). > > If it seems reasonable that this could be the problem, can someone point > me in the right direction to try to track this down? This issue is fixed with two patches provided with great speed and kindness by Ken (thanks again). You need to first apply: https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/imap/imapd.c.diff?r1=text&tr1=1.473&r2=text&tr2=1.476&f=c then apply: https://bugzilla.andrew.cmu.edu/cgi-bin/cvsweb.cgi/src/cyrus/imap/imapd.c.diff?r1=text&tr1=1.476&r2=text&tr2=1.480&f=c -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
More info: different APPEND behavior v2.1.15 / v2.2.6
Hello all, Upon further testing, this behavior seems to have surfaced between v2.2.3 and v2.2.4 and is also in v2.2.5 and v2.2.6. The changes listed between v2.2.3 and v2.2.4 aren't too extensive; the only thing that looks like it could have impacted this behavior is: o All incoming messages are now staged to disk before locking the destination mailbox (locks are no longer held during a network read). If it seems reasonable that this could be the problem, can someone point me in the right direction to try to track this down? Thanks a lot. -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
different APPEND behavior v2.1.15 / v2.2.6
Hello all,
I'm trying to use UW's mailutil to migrate mbox files to Cyrus. When
migrating to v2.1.15 this worked perfectly; read/unread/answered states
were all transitioned from the mboxes to Cyrus. When migrating to v2.2.6,
the states are not correct; in fact, the state of all of the messages in
the newly migrated folder seem to be set to the state of the last message
migrated.
For example, I have an mbox file called "/var/spool/mail/acaird" and a
Cyrus v2.2.6 server named after the star "azah". /var/spool/mail/acaird
has 7 messages in it; messages 1, 3, and 7 have been Seen, and message 7
has been Answered. Using mailutil like:
mailutil copy -debug /var/spool/mail/acaird \
"{azah/novalidate-cert/authuser=admin/user=acaird}testfolder-1"
creates a new folder called "testfolder-1" on the Cyrus server and copies
the 7 messages into it. On the v2.2.6 server, the client reports all of
the messages as having the state "Seen, Answered".
The output from mailutil and the telemetry logs after the login steps are
below. It appears as though the state information is being correctly read
by mailutil and correctly received by Cyrus, but isn't being written to
the seen database correctly. Performing this same operation on a Cyrus
v2.1.15 server produces the same results in the telemetry logs, but when
the mailbox is opened, the states are correct.
The differences between the two Cyrus servers, other than version, are:
v2.1.16 v2.2.6
altnamespace: no yes
seendb format: flatskiplist
Is this a known issue, or is there anything else I can do to gather
information to track this down?
Thanks for any help.
--
Andrew Caird
Output from mailutil, after the login steps:
---
0005 Create testfolder-1
0005 OK Completed
0006 APPEND testfolder-1 (\Seen) "14-Jul-2004 08:36:49 -0400" {637}
+ go ahead
() "14-Jul-2004 08:37:08 -0400" {637}
+ go ahead
(\Seen) "14-Jul-2004 08:37:19 -0400" {637}
+ go ahead
() "14-Jul-2004 08:48:46 -0400" {635}
+ go ahead
() "14-Jul-2004 08:49:02 -0400" {641}
+ go ahead
() "14-Jul-2004 08:49:13 -0400" {641}
+ go ahead
(\Seen \Answered) "14-Jul-2004 08:49:27 -0400" {641}
+ go ahead
0006 OK [APPENDUID 1089815290 1:7] Completed
0007 LOGOUT
* BYE LOGOUT received
0007 OK Completed
---
The output from the telemetry logs on the Cyrus side for the same session
(with the message bodies and login steps removed for brevity) is:
---
<1089815290<0005 Create testfolder-1
>1089815290>0005 OK Completed
<1089815290<0006 APPEND testfolder-1 (\Seen) "14-Jul-2004 08:36:49
-0400" {637}
>1089815290>+ go ahead
<1089815290<
<1089815290< () "14-Jul-2004 08:37:08 -0400" {637}
>1089815290>+ go ahead
<1089815290<
<1089815290< (\Seen) "14-Jul-2004 08:37:19 -0400" {637}
>1089815290>+ go ahead
<1089815290<
<1089815290< () "14-Jul-2004 08:48:46 -0400" {635}
>1089815290>+ go ahead
<1089815290<
<1089815290< () "14-Jul-2004 08:49:02 -0400" {641}
>1089815290>+ go ahead
<1089815290<
<1089815290< () "14-Jul-2004 08:49:13 -0400" {641}
>1089815290>+ go ahead
<1089815290<
<1089815290< (\Seen \Answered) "14-Jul-2004 08:49:27 -0400" {641}
>1089815290>+ go ahead
<1089815290<
>1089815290>0006 OK [APPENDUID 1089815290 1:7] Completed
<1089815290<0007 LOGOUT
>1089815290>* BYE LOGOUT received
---
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Importing mbox to Cyrus
On Sun, 4 Jul 2004, Allen Unueco wrote:
> I have just installed Cyrus IMAP for the first time, everything went
> very smoothly and I was up and running any major problems.
>
> I'm moving from a mbox based mail system to Cyrus and I'm having some
> difficulty moving the old mail.
>
> I've found a few mbox2imap scripts and but none of them work completely.
> The one from http://teyandei.net/projects worked the best, but on the
> two large mbox files I've tried both exited out with an error during
> imap.append() claiming the email had a bad header. I wish I could get
> some more detailed info regarding the 'bad header' but thats all I get.
>
> I'm starting the think that the mbox, some which are quite old, may have been
> muddied up by email clients over the years.
>
> I've seen headers (Status, X-Status,X-Keywords,X-UID,X-Sender-Ip) in the
> middle of emails and other strange things. Mozilla/Thunderbird were
> totally happy with these mbox files and I'm not sure how some of these
> things were introduced.
>
> One of my thoughts was to clean the mbox files. I was working on some
> scripts to remove the 'funny' things I've found. And was just wondering
> if perhaps someone else has gone through this same process and might
> know of some good tools to use.
>
> Has anyone had these problems before? What is the best way to bring
> legacy email into Cyrus? Anyone know of some good mbox repair/cleaning
> tools?
Allen,
In attempting a similar process, I'm using the "formail" program from
the procmail distribution to clean up mailboxes when I get that error.
The syntax is pretty simple:
mv mbox mbox.bad ; /usr/local/bin/formail -ds < mbox.bad > mbox
Then I use mailutil from the pine distribution like:
mailutil copy|append /path/to/username.mboxfile \
{cyrus-server/novalidate-cert/authuser=admin/user=user}user.username
I'm sure it's not fool proof, but it has done an OK job for me so far.
Good luck.
--
Andrew
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mailutil permission denied when setting flags
Hello all:
For the record, this turns out to be a mailutil problem, as far as I can tell.
It appears as though mailutil successfully copies the messages from oldServer
to newServer, but then tries to set the flags on the messages on oldServer,
when it should be doing it on newServer, it just gets mixed up about which
server to talk to. Fortunately, it has the sense to open oldServer read only,
and that's where the "Permission Denied" message comes from. Unfortunately,
the mailutil code is less than transparent, so I haven't yet found where the
mix-up happens.
I'll be asking on the pine mailing list, and digging through the mailutil code
to try to make this work.
By the way, my reasons for using mailutil for migrations are:
1. It lets me authenticate as admin to move a user's mailbox, so I don't have to
know any passwords other than admin.
2. I can move mailboxes one by one, which I don't think I can do by going to
"copy data, upgrade databases" method described in the documentation.
If anyone knows of an existing tool that does what I like about mailutil, but
works, I'd certainly be happy to hear about it.
Thank you.
--
Andrew Caird
Quoting Andrew J Caird <[EMAIL PROTECTED]>:
> Hello all,
> I'm migrating from two servers, one Cyrus 2.1.15 and one UW IMAP, to a
> single Cyrus 2.2.6 server.
>
> Using mailutil, I am able to migrate from the UW IMAP server to the
> 2.2.6 server with great success - all of the state is kept, and all of the
> messages make it across.
>
> However, when trying to migrate from 2.1.15 to 2.2.6 using mailutil I
> lose the state because mailutil can't seem to write to the seen
> database and mailutil tells me only:
> 002d STORE 22 +Flags (\Seen)
> 002d NO Permission denied
> Permission denied
>() "29-Jun-2004 16:55:20 -0400" {4638}
> + go ahead
> 002e STORE 22 +Flags (\Seen)
> 002e NO Permission denied
> My mailutil command is below. This happens when going from INBOX to
> INBOX, or to any other folder. Note also that I'm migrating from a Cyrus
> server altnamespace:no (implied) to one with altnamespace:yes and
> hashimapspool:1.
>
> There are no errors logged on the Cyrus side and telemetry doesn't seem to
> shed any light on things (which seems odd, but I really can't find
> anything).
>
> The fact that migration from a UWIMAP server works fine implies that the
> problem is not strictly on the side of the 2.2.6 Cyrus server, but it
> seems to be in the combination of mailutil, 2.1.15, and 2.2.6.
>
> I know this isn't a mailutil/pine list, but does anyone have any light to
> shed on this?
>
> Thanks a lot.
> --
> Andrew Caird
>
>
> mailutil append -verbose -debug \
> {pollux/novalidate-cert/authuser=admin/user=acaird}user.acaird \
> {joker/novalidate-cert/authuser=admin/user=acaird}
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
mailutil permission denied when setting flags
Hello all,
I'm migrating from two servers, one Cyrus 2.1.15 and one UW IMAP, to a
single Cyrus 2.2.6 server.
Using mailutil, I am able to migrate from the UW IMAP server to the
2.2.6 server with great success - all of the state is kept, and all of the
messages make it across.
However, when trying to migrate from 2.1.15 to 2.2.6 using mailutil I
lose the state because mailutil can't seem to write to the seen
database and mailutil tells me only:
002d STORE 22 +Flags (\Seen)
002d NO Permission denied
Permission denied
() "29-Jun-2004 16:55:20 -0400" {4638}
+ go ahead
002e STORE 22 +Flags (\Seen)
002e NO Permission denied
My mailutil command is below. This happens when going from INBOX to
INBOX, or to any other folder. Note also that I'm migrating from a Cyrus
server altnamespace:no (implied) to one with altnamespace:yes and
hashimapspool:1.
There are no errors logged on the Cyrus side and telemetry doesn't seem to
shed any light on things (which seems odd, but I really can't find
anything).
The fact that migration from a UWIMAP server works fine implies that the
problem is not strictly on the side of the 2.2.6 Cyrus server, but it
seems to be in the combination of mailutil, 2.1.15, and 2.2.6.
I know this isn't a mailutil/pine list, but does anyone have any light to
shed on this?
Thanks a lot.
--
Andrew Caird
mailutil append -verbose -debug \
{pollux/novalidate-cert/authuser=admin/user=acaird}user.acaird \
{joker/novalidate-cert/authuser=admin/user=acaird}
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: how to use saslauthd AND sasldb
On Mon, 28 Jun 2004, Jukka Salmi wrote: > Hello, > > is it possible to configure Cyrus IMAPd (2.2.6, Cyrus SASL 2.1.18) to > use both saslauthd and sasldb? > > The problem I want to solve: some users are stored in Kerberos, some are > in a sasldb. I'd like Cyrus IMAPd to contact saslauthd (for plaintext > authentication with the kerberos service), and, if this fails, to try to > authenticate the user against the sasldb - or vice versa. > > Using saslauthd _or_ sasldb (via auxprop plugin) works fine; however, I > don't know to make PLAIN logins work with both mechanisms - in case > that's possible at all. Jukka, I think you can do this by listing multiple methods after the "sasl_pwcheck_method" option in imapd.conf, for example: sasl_pwcheck_method: saslauthd auxprop I do this to authenticate users via LDAP, but authenticate the admin user locally via sasldb. If I'm doing this wrong, let me know! :) -- Andrew --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: newbie question
Quoting Dudi Goldenberg <[EMAIL PROTECTED]>: > Hello list, > > I need to provide a solution to one of my customers. > > They expect around 6000-8000 emails a day, no big issue here, postfix > will handle this part. > > There are only 10 users accounts on this site, but all 10 accounts are > simultaneously active on all 10 machines. > > Some reading on Cyrus site made Cyrus IMAP my choice due to its multiple > access abilities per mailbox. > > Now, not being familiar with Cyrus at all, I wonder what am I facing > here time wise, complexity, reliability etc. > > Any comments, pros/cons, recommendations are welcomed. Dudi, You might want to look at this information on CMU's installation: http://acs-wiki.andrew.cmu.edu/twiki/bin/view/Cyrus/CarnegieMellonCyrusInstallation Cyrus IMAP is more complicated than UWIMAP, for example, but it does many things UWIMAP can't do - multiple access being one of them. And it's not terribly complicated, allow some time for testing, and know that when you've got it sorted out, it will be very reliable. 32,000+ CMU Tartans can't be wrong, can they? Not knowing what the real requirements you have are, or your general experience level with this sort of thing, I can't begin to guess how much time you'll invest. You'll probably be able to get a test-bed up and going in less than a week. Test until your happy, and to prove to yourself that the software does what you want and need. Hope this helps, and good luck. -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: [Fwd: cyrus pop]
Quoting Yann Gabriel Carlier <[EMAIL PROTECTED]>: > I just installed fedora 2 and I want to have it runing POP and sendmail... > > Do I nead to create acounts? where? > > [EMAIL PROTECTED] web-cyradm-0.5.4]# cyradm > cyradm> cm user.emma > createmailbox: no connection to server > cyradm> Yann, You do need to create accounts, or at least link mailboxes with accounts drawn from somewhere else (PAM, LDAP, MySQL, etc.). You've almost got the mailbox creation part right, but you need to tell cyradm which server you want it to operate on. To do that, run it like: [EMAIL PROTECTED] web-cyradm-0.5.4]# cyradm localhost cyradm> auth admin [enter password] cyradm> cm user.emma Hope this gets you started. -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mass permissions change
On Thu, 10 Jun 2004, Felix Cuello wrote: > On Thu, Jun 10, 2004 at 06:06:14PM -0500, Albert Chin wrote: > > Why not use the Perl module and whip up a small script? > > I'm just asking if there is an application to do that, just before > write the PERL script :). > What do you think is the best way to do the script?, using > IMAP::Admin? > > thanks, > > Felix Felix, Cyrus::IMAP::Admin has lots of admin type things, but for a simple client, which is sort of what you want, you might look at Mail::IMAPClient on CPAN. It's pretty much strictly a client, and it's pretty straightforward in my limited experience. -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Follow-up: client configuration - slightly off-topic
On Wed, 2 Jun 2004, Andrew J Caird wrote: > Hello all, > We are planning on migrating a UW IMAP server to Cyrus IMAP (2.2.5), and > I have some questions about client configuration; not exactly a Cyrus > questions, but perhaps someone here has dealt with this before. > > 1. Apple's Mail.app doesn't see the users' folders until the "IMAP > Prefix Path" is set in the configuration. We'll migrate about 150 > people who use this client, and even if we give instructions before > hand, this will generate 200 calls to the helpdesk on the Monday > after we do this. Is there any way of having the server share this > information with this client, since the server certainly knows my > "IMAP Prefix Path"? Or should this simply work and I've done > something wrong? > > 2. Netscape does not show the folders below Inbox until either the user > subscribes or the user unchecks the box saying "Show only subscribed > folders". We don't use shared folders, so there is no harm in > unchecking that box in our environment, but is there a way to > automatically subscribe people to all of their folders? > > Outlook, of course, grabs everything it can, which, in this case, is what > we want. After posting this, I got very helpful replies from Ludovic Marcotte, Andrew Morgan, and Denis V. Suhanov. The solution to number 1 is to use altnamespace=yes in the imapd.conf file, which makes the folders appear at the same level as the inbox. The solution to number 2 isn't as elegent, but one way is to write a script that enumerates all of each user's folders, and subscribes to them all. Because Cyrus (wisely) lets the administrative user operate on all of the users' mailboxes, this isn't as hard as it might otherwise be. I hope this helps someone else out, and thanks a lot to everyone who replied! -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: client configuration - slightly off-topic
On Wed, 2 Jun 2004, Kent L. Nasveschuk wrote: > I don't have an answer for 1 or 2, but I would be interested in knowing > about auto-subscribing to the users folders. We are planning to do a > large installation this summer and with my limited experience with > Cyrus-IMAP, we will have the same problem. I have the autocreate inbox > patch that works but there still has to be some user intervention unless > someone as an answer to question 2. Kent, I did get the following good suggestion in response to that question. I haven't written this yet, but I probably will. If I manage to get it done, I'll share with the list. If anyone else happens to have this utility laying around and is willing to share, please let Kent and me know. Thanks Ludo and everyone else who replied! -- Andrew Caird On Wed, 2 Jun 2004, Ludovic Marcotte wrote: > [snip] > > 2. Netscape does not show the folders below Inbox until either the > > user subscribes or the user unchecks the box saying "Show only > > subscribed folders". We don't use shared folders, so there is no > > harm in unchecking that box in our environment, but is there a > > way to automatically subscribe people to all of their folders? > > A small perl script which uses IMAPClient should do it. Use the PLAIN > SASL authentication mechanism and proxy the connection from the Cyrus's > admin to the normal user. Then, list all mailboxes and subscribe to > them. --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
client configuration - slightly off-topic
Hello all, We are planning on migrating a UW IMAP server to Cyrus IMAP (2.2.5), and I have some questions about client configuration; not exactly a Cyrus questions, but perhaps someone here has dealt with this before. 1. Apple's Mail.app doesn't see the users' folders until the "IMAP Prefix Path" is set in the configuration. We'll migrate about 150 people who use this client, and even if we give instructions before hand, this will generate 200 calls to the helpdesk on the Monday after we do this. Is there any way of having the server share this information with this client, since the server certainly knows my "IMAP Prefix Path"? Or should this simply work and I've done something wrong? 2. Netscape does not show the folders below Inbox until either the user subscribes or the user unchecks the box saying "Show only subscribed folders". We don't use shared folders, so there is no harm in unchecking that box in our environment, but is there a way to automatically subscribe people to all of their folders? Outlook, of course, grabs everything it can, which, in this case, is what we want. Thanks for any insights. -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Please help, Perl Module problem.
On Fri, 28 May 2004, Kent L. Nasveschuk wrote: > This is what I added but I get the same db_version error :( I have yet > to get this to run, very frustrating. It obviously works people use it. > I'm so close on this... Yes, when you said "path problem", I assumed that it was the same as mine. It isn't. I do remember you posting from a while ago, and I'm afraid that I can't help, as I don't use Berkeley DBs at all for Cyrus IMAP. -- Andrew --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Please help, Perl Module problem.
On Fri, 28 May 2004, Kent Nasveschuk wrote:
> Hello,
> You had a perl path problem back in April in cyradm (Cyrus IMAP) that it
> seems many people had. I was just wondering if you solved it and exactly
> what you did to solve it. I have Cyrus 2.2.4 installed and working on a
> test machine with exception of cyradm. Any help would be appreciated,
> I'm really beating my head against the wall on this one.
Kent,
If your problem is truely only a path problem, you might want to try
what I do, which is adding:
-I/usr/local/cyrus/lib/perl5/site_perl/5.8.3/sun4-solaris to the two
"exec perl" lines at the top of cyradm (see below). Of course, if you
aren't using solaris or Perl v5.8.3, the last two components of the path
will be different for you.
Hope this helps.
--
Andrew Caird
-
Now the top of cyradm looks like:
case "x$BASH_VERSION" in
x) exec perl -MCyrus::IMAP::Shell
-I/usr/local/cyrus/lib/perl5/site_perl/5.8.3/sun4-solaris -e shell -- ${1+"$@"} ;;
*) exec perl -MCyrus::IMAP::Shell
-I/usr/local/cyrus/lib/perl5/site_perl/5.8.3/sun4-solaris -e shell -- "$@" ;;
esac
echo "$0: how did I get here?" >&2
exit 1
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
to hash or not to hash?
Is there any reason not to use: hashimapspool: true Are there guidelines on when it is an advantage or a disadvantage to use it? I've looked through the docs and the list archives and haven't seen anything either way. Thanks. -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Moving mailboxes som one server to another?
On Fri, 21 May 2004, [ISO-8859-1] Bjørn T Johansen wrote:
> Is there another way of doing this than to copy the mailboxes and
> running reconstruct on every mailbox and every subfolder? Or to put it
> simple, does anyone have a list on how to best do this?
Bjørn,
I think the generally accepted way of doing this is using IMAP to move
the mailboxes with mailutil or some other such program. Lots of people
have done this other ways, but it seems that they often end up either
losing information, or investing a lot of energy in custom/local scripts
to make the move work.
Search the archives (see the URL below) for "migrate" and you'll have
days worth of reading on this subject. In the end, download the pine
source of University of Washington and compile the "mailutil" program that
comes with it. An approximate command line is:
mailutil copy {oldserver/novalidate-cert/authuser=admin/user=username} \
{newserver/novalidate-cert/authuser=admin/user=username}
Good luck.
--
Andrew Caird
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Imapd Experiences
On Fri, 21 May 2004, Simon Matter wrote: > > On Thu, 20 May 2004, Wil Cooley wrote: > > > >> > Okay, that's fine. From what I know you can only look up the defaults > >> > in the sources. However, until you configure pther db's in imapd.conf, > >> > you are always using the defaults which should be fine. IIRC the > >> > mailboxes db is skiplist that way, which is the most import db. > >> > >> With later 2.1.14 and probably 2.2 you can use the 'version' command > >> with cyradm to find that stuff out. > > > > Uh oh, I don't see any mention of any DB types. What have I done wrong > > now? > > Nothing wrong here. version does not show DB types in 2.2.x. Thanks Simon - I feel better. I guess that's why Wil said "probably". Too bad, it would be nice to see the DB types... -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus Imapd Experiences
On Thu, 20 May 2004, Wil Cooley wrote: > > Okay, that's fine. From what I know you can only look up the defaults > > in the sources. However, until you configure pther db's in imapd.conf, > > you are always using the defaults which should be fine. IIRC the > > mailboxes db is skiplist that way, which is the most import db. > > With later 2.1.14 and probably 2.2 you can use the 'version' command > with cyradm to find that stuff out. Uh oh, I don't see any mention of any DB types. What have I done wrong now? localhost> version name : Cyrus IMAPD version: v2.2.4 2004/05/14 20:59:12 vendor : Project Cyrus support-url: http://asg.web.cmu.edu/cyrus os : SunOS os-version : 5.9 environment: Built w/Cyrus SASL 2.1.18 Running w/Cyrus SASL 2.1.18 Built w/OpenSSL 0.9.7d 17 Mar 2004 Running w/OpenSSL 0.9.7d 17 Mar 2004 CMU Sieve 2.2 mmap = shared lock = fcntl nonblock = fcntl auth = unix idle = poll -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: cyradm timeout?
On Wed, 19 May 2004, Bret Hammond wrote:
> On Wed, 19 May 2004, Andrew J Caird wrote:
>
> > I'm installing Cyrus v2.2.3 on Solaris 9 and I've been through the
> > weirdness with imclient.a (see
> > http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&searchterm=imclient&msg=28484)
> > and now I can connect to Cyrus IMAP servers (is there a better
> > solution to this?). However, in my only test so far: connecting to a
> > Cyrus v2.1.15 server, cyradm appears to timeout after about 5 seconds.
> >
> > Is this normal? I couldn't find any configuration options in either
> > the cyradm client (Cyrus/IMAP/Shell.pm) or in the server
> > configuration. It doesn't matter if I type commands or not, it
> > appears to be about a 5 second timeout and the server logs it as a
> > "normal" disconnect.
>
> Andrew,
>
> You might consider grabbing the 2.2.4 code (released yesterday),
> as it seems to have fixed the imclient problem along with some other
> issues. We are testing Cyrus v2.2.4 on Solaris 9, and noticed a problem
> with cyradm related to the getaddrinfo that comes with the SMCperl pkg.
> After removing the perl package, and recompiling perl from source our
> problems with cyradm went away. We're able to use cyradm to connect to
> both a 2.2.4 and 2.0.12 server.
Bret,
Thanks for the tip about SMCperl - compiling my own perl seems to have
fixed the problem with imclient and not being able to connect to any
servers, and there was no need to patch imclient.c. Too bad about
SMCperl, that's a lot handier than compiling perl from source.
The problem with the mysterious "5 second timeout" ("truss -d" showed it
to be exactly 5 seconds) seems to have been shell related. Running it
under tcsh seemed to fix it, although I (unscientifically) changed a few
things at once (edited cyradm and removed the space in the #!/bin/sh
line between "#" and "/", started a tcsh from my bash shell, then exited
back to bash), and now can't get back to the old behavior.
I hope these notes help someone else.
--
Andrew Caird
---
Cyrus Home Page: http://asg.web.cmu.edu/cyrus
Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
cyradm timeout?
I'm installing Cyrus v2.2.3 on Solaris 9 and I've been through the weirdness with imclient.a (see http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&searchterm=imclient&msg=28484) and now I can connect to Cyrus IMAP servers (is there a better solution to this?). However, in my only test so far: connecting to a Cyrus v2.1.15 server, cyradm appears to timeout after about 5 seconds. Is this normal? I couldn't find any configuration options in either the cyradm client (Cyrus/IMAP/Shell.pm) or in the server configuration. It doesn't matter if I type commands or not, it appears to be about a 5 second timeout and the server logs it as a "normal" disconnect. Thank you. -- Andrew Caird --- Cyrus Home Page: http://asg.web.cmu.edu/cyrus Cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Running reconstruct as cyrus user
On Fri, 2 Apr 2004, Robert Fitzpatrick wrote: > I found a script that supposedly will sa-learn Cyrus IMAP mailboxes for > spam and ham. The script wants to run the reconstruct command as the > cyrus user, but 'su' seems to have a problem with the way it is setup: > > su - -s /bin/sh cyrus -c "/usr/local/cyrus/bin/reconstruct spam" > > Aside from the fact that 'su' has a problem with the syntax, if I take > out the offending options of the shell and '-s', it will give me a > directory error, of course, because the port install on my FreeBSD-5.2.1 > box setup the user with nologin and nonexistant home directory: > > esmtp# su - cyrus -c "/usr/local/cyrus/bin/reconstruct notspam" > su: no directory > > Can someone offer advice on how to get this command the script is trying > to execute working? Maybe I don't need to be using the 'cyrus' user to > do this? Robert, If you don't use the "-" in "su - " it won't try to find the environment of the account you're specifying (cyrus, in this case), so you don't need to have a home directory. Of course, you need to make sure that the environment where you run that has what you need. I believe that you do need to use the cyrus user, just try it like: su cyrus -c "/usr/local/cyrus/bin/reconstruct notspam" Good luck. -- andrew caird --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Remote User's SMTP relay authorization
Quoting John Gibson <[EMAIL PROTECTED]>: > Roaming ( my def.) : Users who access the internet from multiple IP > addresses most of which are dynamically allocated (therefore unknown in > advance). > > We do not want to support permanent email storage at this time. We > just want people to be able to retrieve their mail and provide SMTP > sending capability. We want the IMAP/POP mail server to not require our > linux host to have system logins for any of the mail users. We want a > scalable solution which can be flexible when our requirements change. > We would prefer to allow users to select their own mail client (or at > least provide a list of ones that will work with our setup). > > The SMTP sending capability seems to be the biggest issue. > > Any suggestions would be much appreciated. There are two popular ways of doing this. The first, and preferred by most, is SMTP AUTH - I believe that support of relaying for roaming, authorized users is one of the reasons SMTP AUTH exists. This, however, isn't a Cyrus question, but an MTA question, and is better researched where ever you find information about your MTA of choice. The second is POP-before-SMTP, which is, in general, a bad hack, but might be worth investigating. Essentially, there is a third process (not the POP3d or SMTPd) that looks at the POP logs and tells the SMTPd that it can allow sending from IP addresses that had recent successful POP sessions. This involves a tight integration between your POP server (one of many) and your SMTP server (again, one of many). Because of the large number of possible combinations, this is generally not well supported, but Googling for "pop before send " might turn up something for you. Hope this helps, good luck (and, imo, you should use SMTP AUTH+TLS, with SASL passwords). -- Andrew Caird --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: mass importing/exporting imap folders
Quoting Michael King <[EMAIL PROTECTED]>: > > Hi all. > > I'm not 100% sure this is the right place to ask this, but it's the only > mailing list I saw on the Cyrus IMAP website. Please bear with me. > > I'm trying to figure out if there's a way to export and then import via > IMAP? > > That is, if I have IMAP account 1 on server A, and I have IMAP account 2 > on server B, I want to export all of the folders (and messages, if > possible) off of Server A, and then import them onto Server B - giving > me a "backup" of sorts. > > Is there any way to do this? If not, is there any other interest in > developing a tool such as this? Michael, This is a fine place to ask this question - it gets asked here a lot. If this is just for one person, most modern IMAP clients will let you connect to multiple IMAP servers at once, and you can simply drag folders and messages from one server to another. If you have to do this for many users, there are several options. My favorite is mailutil from the University of Washington pine distribution. You can also search the archives of this list ( http://asg.web.cmu.edu/cyrus/mailing-list.html) for the words "export" and "migrate" and you'll see lots of discussion on this issue. Hope this helps. -- Andrew Caird --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: usage report
Quoting Erik Myllymaki <[EMAIL PROTECTED]>: > I would like to be able to offer a usage report to some managers. > Basically, showing number of inbound and outbound mails per day for some > specific users, maybe on a web interface... > > Can anyone point me in the right direction? Erik, There are lots of options (see Google). I use sma (http://www.klake.org/sma/), and although I don't think it does per-user accounting, it could probably be coerced into doing so. It can produce output in HTML for display on a web page. Good luck. -- Andrew Caird[EMAIL PROTECTED] 215.728.4071 Email Administrator Fox Chase Cancer Center --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Migration from UW-imap
On Mon, 16 Feb 2004, Jordi Pallares wrote: > Hy, > > I've tryed to migrate from an UW-Imap to a Cyrus Imap. > To transport the users mailboxes i use ImapCopy script, but i have a lot > of errors like this: >Getting List of messages in "" Failed (SELECT)- copy aborted for this folder > where xxx are the mailbox name. > > What can i do to sove this. > I can use another program like this to make our migration? Jordi, I strongly suggest you search the recent archives of this list. There have been at least two very informative threads on the topic of migrating IMAP mailboxes to and from Cyrus. http://asg.web.cmu.edu/archive/index.php?mailbox=archive.info-cyrus One keyword that should find at least one message in the thread to get you started is "mailutil". Best of luck. -- Andrew Caird --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: convert from maildir to Cyrus DB
On Fri, 13 Feb 2004, Jukka Salmi wrote: > Hello, > > I'm planing to switch a (Courier) maildir based IMAP system to Cyrus. Jukka, I strongly suggest you search the recent archives of this list. There have been at least two very informative threads on this topic recently. http://asg.web.cmu.edu/archive/index.php?mailbox=archive.info-cyrus One keyword that should find at least one message in the thread to get you started is "mailutil". Best of luck. -- Andrew Caird --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Backup e-mail messages HOW ?
The Cyrus Wiki is an excellent resource. As are the mailing list archives and Google. In particular, you should read: http://acs-wiki.andrew.cmu.edu/twiki/bin/view/Cyrus/Backup as well as the archives of this list: http://asg.web.cmu.edu/archive/index.php?mailbox=archive.info-cyrus -- Andrew Caird[EMAIL PROTECTED] 215.728.4071 Email Administrator Fox Chase Cancer Center On Thu, 12 Feb 2004, Bartosz Jozwiak wrote: > Welcome, > > Could somebody tell me how to backup all e-mail messages from Cyrus-IMAP > 2.1.14 ? > I know that all messages are stored in /var/spool/imap/users/ but I think > thats not all to backup. > Please could somebody tell me how correctly back up all messages from > Cyrus-IMAP ? > > Thank you in advance. > Bartosz Jozwiak > > --- > Home Page: http://asg.web.cmu.edu/cyrus > Wiki/FAQ: http://cyruswiki.andrew.cmu.edu > List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html > --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: saslauthd and ldap and ??? pam
On Wed, 11 Feb 2004, Igor Brezac wrote: > > On Wed, 11 Feb 2004, Shelley Waltz wrote: > > > I am interested in knowing the difference and/or advantages > > of the ways one can use ldap authentication with sasl. > > > > One way is to use saslauthd -a ldap, which uses the auth_ldap > > module for saslauthd. > > > > Another way is to use saslauthd -a pam and then specify ldap > > as the auth mechanism in the various pam.d services such as > > smtp or imap. > > > > saslauthd/ldap combination will give you better performance and in general > it is more stable. Some pam implementations/modules leak memory. And without PAM it's one less layer to debug. And you will be debugging. Cyrus IMAP and SASL are great, but they are not simple. As always, Occam's Razor is a handy tool. If you don't have a clear need for PAM integration with SASL, eliminate it. Good luck. -- Andrew --- Home Page: http://asg.web.cmu.edu/cyrus Wiki/FAQ: http://cyruswiki.andrew.cmu.edu List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Migrating from UW imap to Cyrus
Dwight,
I'm starting a similar project, and have looked at several of the tools
that are out there - if you check the archives for this list for that last
two or three weeks in January 04, you'll see a discussion of this.
Based on my very limited testing, mailutil from the UW IMAP/Pine
distribution is the best tool. It can migrate IMAP-to-IMAP, or from
mboxfile-to-IMAP (and in the other direction). Since, as far as I know,
UW-IMAP doesn't have an authuser option, I've successfully migrated
mailboxes from the UW-IMAP server to the Cyrus IMAP server with a command
like:
mailutil copy /imapfolders/loginid/foldername
"{cyrus/novalidate-cert/authuser=cyrusadmin/user=loginid}user.loginid.foldername"
run as root on the UW-IMAP server. On my servers/network I can transfer
about 5 messages per second. It is interesting to run mailutil with the
-verbose and -debug flags to see what it's doing, but after a while it
just clutters the screen.
Hope this helps.
--
Andrew Caird[EMAIL PROTECTED] 215.728.4071
Email Administrator Fox Chase Cancer Center
On Sun, 1 Feb 2004, Dwight Tovey wrote:
> Hello all
>
> I'm in the process of moving from UW imap to Cyrus Imap. Somewhere in
> my reading up on the compile/install/configure process I could have
> sworn that I saw something about some scripts that can be used to
> transfer my existing mbox mail folders into Cyrus. Unfortuantely, now
> that I'm actually ready for that step, I don't seem to be able to find
> either the scripts or the reference to it. Anybody have any idea what
> I'm talking about?
>
> /dwight
> --
> Dwight N. Tovey
> email: [EMAIL PROTECTED]
> web: http://www.dtovey.net/~dwight
> ---
> Experience is something you don't get until just after you need it.
Re: Migration Strategies?
> Why not just shutdown Cyrus and let your MTA spool incoming messages > while you migrate? > > -- > Kenneth Murchison Oceana Matrix Ltd. Because of the other users who are currently using it. I could shut it down, but if there is a smoother way of migrating individual accounts without impacting everyone else, that would be excellent. -- Andrew
Migration Strategies?
Hello all, I'm starting to think about migrating away from UW-IMAP (for the reasons mentioned a few days ago on this list) to a Cyrus IMAP server that we've been using for some people for a while. It's easy enough to migrate everything except the inbox with mailutil, and in my (so far) limited testing that works pretty well (see below). However, how do people deal with migrating the inbox, given that it can change during the migration? More specifically, the email gateway is currently configured to deliver email to, for example, [EMAIL PROTECTED] I migrate that inbox, and before I can switch delivery to [EMAIL PROTECTED], more email comes to [EMAIL PROTECTED] Then what? Alternatively, if I start delivering email to [EMAIL PROTECTED] and then migrate on top of it, does that cause any problems? It seems like the second choice is the best one, but some confirmation from someone who has done or understands this would be much appreciated. -- Andrew Caird[EMAIL PROTECTED] 215.728.4071 Email Administrator Fox Chase Cancer Center
Re: cyrus logging
On Thu, 8 Jan 2004, Andrei Buryukov wrote: > Can I configure pop3d to log amount and size of messages fetched by user > (like popper)? > > -- > Biryukov Andrei ElVisti Information Center, Kiev, Ukraine. > E-mail[EMAIL PROTECTED] Andrei, This was discussed in a thread on this list within the last 2-3 months. The short answer is "no", sorry. For specific instances, look at the telemetry features of Cyrus IMAP, but for general use, you don't want those on. Perhaps this should be either in the FAQ or considered a feature request? Cheers. -- Andrew
Re: received date and reconstruct
Strangely, I need to do something very similar. So if someone has the answer and was going to hold out because of lack of interest, that's no longer an excuse. :) Thanks a lot. -- Andrew On Thu, 20 Nov 2003, Phil Chambers wrote: > I was planning to have a script go through doing this every night. It > would use the creation dates of the message files, deleting ones that > are too old and then use reconstruct to tidy up. > > I have two questions: > > 1) Can I rely on the creation date? I ask this because I have just seen > an inbox where every message file had the same date/time. The date/time > was 4.05 this morning and the user was not active at that time and the > files had not been recovered from backup. Without any external activity, > I cannot see what would have changed the creation dates. > > 2) Is it safe to use reconstruct on a live system in this way? I am > concerned in case a new message gets delivered into the folder while > reconstruct is doing its thing. > > Phil.
Re: permission denied / var/imap
Frank,
From the strace output, it looks like mkimap is trying to create the
directory /var/imap, and is failing because the user cyrus can't make
directories in /var.
You could add cyrus to whatever group owns /var for the duration of the
mkimap process, then remove it, or dissect the mkimap script and have it
skip that step.
Hope this helps.
--
andrew
On Wed, 19 Nov 2003, frank joseph wrote:
> Hello. I posted yesterday under the subject of "mkimap failure", but did
> not receive any responses. This is my first time posting to any group
> and if I have done something wrong, I apologize. Please let me know what
> it is so that I don't repeat it. Below is a more concise description of
> my problem. I would appreciate it if anyone would be kind enough to
> respond.
>
> packages installed:
>
> db-4.1.25
> openssl-0.9.7c
> cyrus-sasl-2.1.15
> cyrus-imapd-2.1.15
>
> Everything compiles fine. When I su to cyrus and run "tools/mkimap",
> I can see the return is incorrect:
>
> [EMAIL PROTECTED]:/usr/local/src/cyrus-imapd-2.1.15/tools$ ./mkimap
>
> . will configure directory /var/imap
> . saw partition /var/spool/imap
> done
> ...ating /var/imap
> at (eval 1) line 91.ar/imap
>
> This is a fresh install of Slackware-9.1. No permissions other than what
> is required in the manual have been changed. Running Strace on mkimap
> shows the following:
>
> snip---
> reading configure file...open("/etc/imapd.conf", O_RDONLY|O_LARGEFILE) = 4
> ioctl(4, SNDCTL_TMR_TIMEBASE, 0xb600) = -1 ENOTTY (Inappropriate ioctl
> for device)
> _llseek(4, 0, [0], SEEK_CUR)= 0
> fstat64(4, {st_mode=S_IFREG|0644, st_size=155, ...}) = 0
> fcntl64(4, F_SETFD, FD_CLOEXEC) = 0
> brk(0) = 0x814d000
> brk(0x814e000) = 0x814e000
> write(1, "reading configure file...\n", 26reading configure file...
> ) = 26
> brk(0) = 0x814e000
> brk(0x814f000) = 0x814f000
> read(4, "configdirectory: /var/imap\r\npart"..., 4096) = 155
> .rite(1, "i will configure directory /var/"..., 39i will configure directory
> /var/imap
> ) = 39
> .rite(1, "i saw partition /var/spool/imap\r"..., 34i saw partition
> /var/spool/imap
> ) = 34
> read(4, "", 4096) = 0
> write(1, "done\n", 5done
> ) = 5
> close(4)= 0
> ...te(1, "creating /var/imap\r...\n", 23creating /var/imap
> ) = 23
> ", 0755) = -1 EACCES (Permission denied)
> ") = -1 ENOENT (No such file or directory)
> at (eval 1) line 91.hange to /var/imap\r at"..., 51couldn't change to
> /var/imap
> ) = 51
> close(3)= 0
> exit_group(2) = ?
>
> If cyrus owns /var/imap with privileges of 750, and mkimap is being
> run by cyrus, how could write permissions be denied?
>
> Thanks Again
>
> -Frank
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
Re: Cyrus IMAPD with LDAP Authentication
On Wed, 5 Nov 2003, Igor Brezac wrote: > > On Wed, 5 Nov 2003, Robert wrote: > > > Thanks for the correction. The old documentation is to blame. Now, > > the problem is that imap cannot connect to the saslauthd server. I > > tried changing the path in the saslauthd init script. Is there a way > > to speciry the path of the mux socket in the imapd.conf file? What > > directory does imapd look for the mux socket? > > > > sasl_saslauthd_path > which you didn't find because it isn't an imap option, but a sasl option (see doc/sysadmin.html in the cyrus-sasl tree), and prepending sasl_ to it in the imapd.conf file simply passes that option on to sasl, from the imapd.conf man page: sasl_option: Any other SASL option can be set by preceeding it with "sasl_". This file overrides the SASL configuration file. The inter-relation between all of this is non-trivial, but it isn't random, and it all does work in the end. Good luck. -- Andrew
