Re: Long term offline mail retention
On Fri, 21 Jan 2011, Jason L Tibbitts III wrote: > i manage email (and practically everything else) for a medium sized > university department. recently i've been informed that we're adopting a > three-year retention policy for email. we're allowed to keep things for > longer than that, so i'm not looking at forced expiration (which is all i > found when i searched the mailing lists for info). I realise it's an off-topic reply but would google postini be the simplest solution to this? Gavin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Hi, On Tue, 16 Nov 2010, Adam Tauno Williams wrote: > I think the issue you will encounter first is clients will start to fall > down when folders exceed a 'reasonable' number of messages. Common IMAP > clients I've seen start to exhibit severe performance issues beyond a > few hundred thousand messages. As far as I'm aware (the helpdesk guys know better than me so I'm parroting their reply), Outlook 2003's PST file has a limit of 2GB so if it's locally caching folders, you may run into that. If you use Outlook 2007 or later, the limit is more like 20GB. BUT, if you upgrade from 2003 and use the same PST, that PST may continue with the same 2GB limit. Apparently you might need to create a new PST file and move the mail into it¹. Some big users have been moved to Thunderbird to avoid this and to improve performance. Gavin ¹ To be honest, I haven't personally dealt with this issue, but this paraphrases the knowledge of those here who have. I'd think of it as having "[citation required]" beside it. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Does anyone allow unlimited or extremely large quotas?
Hi, On Tue, 16 Nov 2010, Dave McMurtrie wrote: > This may be slightly off-topic, so apologies in advance. Is there > anyone out there who allows unlimited quota for their users or provides > extremely large quotas when asked for? What do you regard as extremely large? 10GB, 100GB, 1TB, ...? Gavin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Tcpwrapper does not work?
Hi, On Fri, 08 Oct 2010, Paul van der Vlis wrote: > Strange, in the manual of tcp-wrappers they say you need to use the > processname... If you use bacula you'll find the same thing. If you run a bacula file daemon, the process name will be bacula-fd but you might call it "doodles" in the bacula-fd config. "doodles" is what you must allow in TCP wrappers. Gavin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Using mbox2cyrus.pl
Hi, On Thu, 07 Oct 2010, Patrick Goetz wrote: > I have a few mbox files that I need to transfer to cyrus (one relatively > large ~3MB). I downloaded the perl script mbox2cyrus.pl, and looked > over the code, and I'm not confident that this will work for a system with > > unixhierarchysep: yes > > Does anyone have any experience with this? Am I going to have to > rewrite this script to get at my mbox files? If you have mutt around the place you can do: mutt -f (should open the mbox) T all (should tag all files) ;C imaps://@/ (copy all tagged mails which should copy all mails to on that imap server. Gavin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: De-duping attachments
Hi, On Wed, 15 Sep 2010, Nik Conwell wrote: > Isn't the easy hack for dedup just looking at the above md5 files and > then doing appropriate hard links? This could be done by a nightly > trawl of the spool space. A bigger win would be to separate the headers > from the messages but that's a lot more work. For what it's worth, I believe the fsdup tool which is part of fslint will do this for you. http://www.pixelbeat.org/fslint/ Gavin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Importing/moving an older cyrus message tree into a new system, without IMAP
Hi, On Mon, 13 Sep 2010, Forrest Aldrich wrote: > I have an older system that crashed - cyrus version is a couple years or > so old. I have 1000's of messages in the spool that I need to preserve. > My question is about whether there's a way to import that huge tree of > messages into a new cyrus installation without imap-to-imap connectivity? We did a migration some months back from an old Kolab v1 (cyrus v2.1) system to a new Kolab v2.2 (cyrus v2.2) system. This was done by writing a script to - dump the ldap database (you might not have this) and load it on the new system - rsync the mailboxes from their location on the old server to the correct location on the new server - recursively reconstruct those mailboxes - copy the .seen and .sub information to the correct new location - copy the quota information to the correct new location - dump the old mailboxes.db and load it on the new system (with cyrus stopped) It's not trivial, but it can be done with some care. We also had to translate usernames from to @ in various places to match the new kolab setup but you probably won't have to worry about that. Gavin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Store documents in IMAP folders
Hi, On Sun, 12 Sep 2010, Julien Vehent wrote: > The goal is to have a PDF library available at any time, with basic file > search on document/message name, so a file share doesn't solve my problem > (and I don't want any document management system, I just want access to > files). I don't imagine IMAP's search would work on MIME attachments, unless you did something like add a plain text version to the body of the email. > If outlook does it, I suppose I could code a basic PHP upload page that > create an email containing the document and stores it into the IMAP > directory (and integrate that to Roundcube, since it's my webmail). Outlook allows you to upload files straight into a mail folder and the mail folder could be shared between users. I'm not sure how well searching them would work. Perhaps if the body of the emails is downloaded by the mail client some clients might search the MIME attachments. Gavin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Store documents in IMAP folders
On Sun, 12 Sep 2010, J. Roeleveld wrote: > I sometimes do this to store a document that I want to keep quickly from a > public PC. > I basically create a draft-email and attach the document to this. > This can be done with any email-client (including webmail) If (heaven forbid) you're using relatively recent versions of Outlook, you can just drag and drop files into mail folders and it will automatically create a mail from you to you on that date in that folder. As you're (presumably) using Cyrus, you can then share the folder with other users as a sort of poor-man's file share. Gavin -- Gavin McCullagh Senior System Administrator IT Services Griffith College South Circular Road Dublin 8 Ireland Tel: +353 1 4163365 http://www.gcd.ie http://www.gcd.ie/brochure.pdf http://www.gcd.ie/opendays This E-mail is from Griffith College. The E-mail and any files transmitted with it are confidential and may be privileged and are intended solely for the use of the individual or entity to whom they are addressed. If you are not the addressee you are prohibited from disclosing its content, copying it or distributing it otherwise than to the addressee. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and delete the e-mail from your computer. Bellerophon Ltd, trades as Griffith College (registered in Ireland No. 60469) with its registered address as Griffith College Campus, South Circular Road, Dublin 8, Ireland. Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: shared \seen flags on shared folders
Hi, On Thu, 09 Sep 2010, Bron Gondwana wrote: > On Wed, Sep 08, 2010 at 06:10:44PM +0100, Gavin McCullagh wrote: > > What happens if the sharedseen attribute is changed during the lifetime of > > a folder, when users may already have their own (or shared) \Seen flags? > > In Cyrus 2.3 and before (i.e. everything that's released now!) they are > completely separate. Under the hood, the \Seen flags are actually owned > by a 'nobody' user. I take it there's no way to sensibly migrate flag data from one to the other? When a user leaves, it's sometimes useful to hang onto their mailboxes and share them with another member of staff. The \Seen flags can be useful in this instance. In general we wouldn't set seenflags by default so you end up wishing to retro-fit it. > In Cyrus 2.4 (when I finish it!) the sharedseen flags are actually the > _owner_'s flags. So if I turned on sharedseen on user.brong.SharedFolder > then everybody would inherit my \Seen flags. If I turned it back on again, > they would each get back their own \Seen flags, and I would keep the shared > flags. That sounds like exactly what I want :-) Thanks for the information, Gavin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: shared \seen flags on shared folders
Hi, On Thu, 02 Sep 2010, Gavin McCullagh wrote: > > "sharedseen" > >Enables the use of a shared \Seen flag on messages rather > > than > >a per-user \Seen flag. The ’s’ right in the mailbox ACL > > still > >controls whether a user can set the shared \Seen flag. > > Brilliant, thanks. I hadn't noticed that. Just a further question on this. If a folder has sharedseen=true set in the metadata from its creation and forever, I would expect shared seen flags. If a folder always has sharedseen=false for its entire life, I expect per-user \Seen flags. What happens if the sharedseen attribute is changed during the lifetime of a folder, when users may already have their own (or shared) \Seen flags? Gavin Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: shared \seen flags on shared folders
Hi, sorry for the very late reply. On Thu, 12 Aug 2010, Wesley Craig wrote: > On 12 Aug 2010, at 09:17, Gavin McCullagh wrote: > >I gather courier shares the \seen flag between users sharing a > >folder. Is it possible to do this on cyrus? > > I believe "sharedseen" does what you're looking for. See "man cyradm": > "sharedseen" >Enables the use of a shared \Seen flag on messages rather than >a per-user \Seen flag. The ’s’ right in the mailbox ACL still >controls whether a user can set the shared \Seen flag. Brilliant, thanks. I hadn't noticed that. > The Kolab client may have direct support for managing sharedseen, so > you might take a look at that, as well. It has a sufficiently new cyradm to support sharedseen, but it doesn't seem to have a web-based way to do this. Gavin -- Gavin McCullagh Senior System Administrator IT Services Griffith College South Circular Road Dublin 8 Ireland Tel: +353 1 4163365 http://www.gcd.ie http://www.gcd.ie/brochure.pdf http://www.gcd.ie/opendays This E-mail is from Griffith College. The E-mail and any files transmitted with it are confidential and may be privileged and are intended solely for the use of the individual or entity to whom they are addressed. If you are not the addressee you are prohibited from disclosing its content, copying it or distributing it otherwise than to the addressee. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and delete the e-mail from your computer. Bellerophon Ltd, trades as Griffith College (registered in Ireland No. 60469) with its registered address as Griffith College Campus, South Circular Road, Dublin 8, Ireland. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: shared \seen flags on shared folders
Hi, thanks guys, that's exactly what I was looking for. Gavin On Thu, 12 Aug 2010, Wesley Craig wrote: > I believe "sharedseen" does what you're looking for. See "man cyradm": On Thu, 12 Aug 2010, Dan White wrote: > The /vendor/cmu/cyrus-imapd/sharedseen annotation will share the seen state > for a given mailbox. > > In cyradm, you enable it with: > > See the cyradm man page. This feature was introduced in version 2.3.9. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
shared \seen flags on shared folders
Hi, we have a kolab server here which as you probably know uses Cyrus for its IMAP/POP/LMTP services. One issue we come across now and then is with a group who share a generic incoming email address as well as each having their own personal address. This email may take general customer queries for example. They want to be able to "share" the \seen flag between them, so that if one user reads the email and deals with it, the others no longer see it as unread in their clients. We can set up their email client to have several accounts and share the same username and password between them. However, this is awkward for various reasons. We'd prefer to share the folder among each user, but this gives each user their own copy of the \seen flag which is awkward. I gather courier shares the \seen flag between users sharing a folder. Is it possible to do this on cyrus? Needless to say I can see advantages in either behaviour but it would be handy to have the option of sharing that flag on cyrus. Gavin -- Gavin McCullagh Senior System Administrator IT Services Griffith College South Circular Road Dublin 8 Ireland Tel: +353 1 4163365 http://www.gcd.ie http://www.gcd.ie/brochure.pdf http://www.gcd.ie/opendays This E-mail is from Griffith College. The E-mail and any files transmitted with it are confidential and may be privileged and are intended solely for the use of the individual or entity to whom they are addressed. If you are not the addressee you are prohibited from disclosing its content, copying it or distributing it otherwise than to the addressee. If you have received this e-mail in error, please immediately notify the sender by replying to this e-mail and delete the e-mail from your computer. Bellerophon Ltd, trades as Griffith College (registered in Ireland No. 60469) with its registered address as Griffith College Campus, South Circular Road, Dublin 8, Ireland. Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Get cyrus-imap folder size without "du" tool
Hi,
On Thu, 01 Apr 2010, David Touzeau wrote:
> This problem occurs when you have more than 50Go mailboxes
> If you launch a "du -h -s /var/spool/cyrus/mail" the server load is on
> top and I/O is at 100% for a long time.
> Is somebody have a tips to get the size status of all maiboxes without
> using du tool even no quota is specified ?
sudo -u cyrus /usr/lib/cyrus-imapd/quota | awk '{ tot+=$3} END { print
"Total="tot }'
(your path to the quota program might vary).
Gavin
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Backup strategy for large mailbox stores
Hi, I'm a relative newbie with cyrus, but I'm interested in this discussion... On Mon, 15 Feb 2010, ram wrote: > We have cyrus servers deployed at many places where clients have varying > mail storage. > We have been taking backups to help in situations of human errors > ( where you get complaints like ..oops, I accidentaly deleted all my > mails!! ) and in case of hardware failures > > Things have been working fine but off late we find that emailusage has > grown and so our backups take too long to complete .. we use dar to take > differential backups and take backups everynight. and transfer the > backup files to a remote server. Have you identified the bottleneck? Is it disk access on the mail server itself, bandwidth to your remote server, something else? > If the backup is still running in the morning people notice a > considerable degradation of the server performance Is this a recent linux server? In principal, you could use ionice to class your dar process "idle" which should mean that users will get a better share of disk access. However, that will also mean your backup takes even longer. Probably not ideal. > Is there a better strategy , probably within the cyrus framework , to > take backups efficiently I've wondered about the best means of backup myself. We've been doing something similar using rsync to sync the mail spools and other associated data to a remote server. This works, but I'm slightly worried that we continue delivering mail throughout the process. So our mail spool is changing as we back it up. I've considered the possibility of stopping all daemons, taking an LVM snapshot, restarting and backing up the snapshot. That way you get a consistent spool where everything was backed up at the same moment. On the other hand, it appears that you can generally reconstruct mailboxes, so perhaps I just don't need to worry about that. I'd prefer the cosy feeling of knowing the data is in a consistent state though. If you simply can't run an incremental or differential backup in the "quiet" time, perhaps it would make more sense to do rolling replication to another server. Then, your backup can stop the replication temporarily, backup the replica and start the replication back up -- leaving the live server alone. I imagine this does add load to the main server, but distributes it over the whole day. http://cyrusimap.web.cmu.edu/imapd/install-replication.html Gavin Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: understanding fields in cyrus.header file
Hi, On Wed, 10 Feb 2010, Dan White wrote: > >http://cyrusimap.web.cmu.edu/twiki/bin/view/Cyrus/CyrusHeaderFormat > > There's also some documentation of the cyrus.header (and other files) > at: > > http://cyrusimap.web.cmu.edu/imapd/internal/mailbox-format.html More very useful info, thanks. I've linked that into that wiki page if that's okay. Gavin Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: understanding fields in cyrus.header file
Hi, On Wed, 10 Feb 2010, Bron Gondwana wrote: > Ok - here's a quick primer on user flags :) (from memory - any mistakes > are my own!) As this information was very useful and I'd had some trouble finding it before, I've added it to the wiki. I hope that's okay: http://cyrusimap.web.cmu.edu/twiki/bin/view/Cyrus/CyrusHeaderFormat Gavin Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: understanding fields in cyrus.header file
Hi On Feb 10, 2010, at 0:29, Bron Gondwana wrote: > > Hope that all made sense! It certainly did. Many thanks for such a thorough explanation. Gavin Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: understanding fields in cyrus.header file
On Wed, 10 Feb 2010, Bron Gondwana wrote: > Check out mailbox_read_header and mailbox_read_header_acl from > imap/mailbox.c in the source code if you want a more detailed > answer (in C!) Many thanks for your help by the way :-) Gavin Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: understanding fields in cyrus.header file
Hi, On Wed, 10 Feb 2010, Bron Gondwana wrote: > > > Junk NonJunk $Forwarded Old > > Check out mailbox_read_header and mailbox_read_header_acl from > imap/mailbox.c in the source code if you want a more detailed > answer (in C!) I guess I'd better do that as while I could guess the rest, I don't understand what the user-flags mean. On the plus side, I'm starting to think I won't need to do any translation of them, which is the main thing for me. Gavin Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
understanding fields in cyrus.header file
Hi, I'm looking at some cyrus.header files while migrating from an old to a new cyrus-based mail server. I think I understand it in general, except for the second last line. A couple of examples follow: -- Cyrus mailbox header "The best thing about this system was that it had lots of goals." --Jim Morris on Andrew user.x^y20756ee342e4c94f Junk NonJunk $Forwarded Old x.y lrswipcda -- -- Cyrus mailbox header "The best thing about this system was that it had lots of goals." --Jim Morris on Andrew user.^bbb 1b3a81794204b40e $Label4 $Label1 $Label2 $Label3 $Label5 Junk NonJunk $Forwarded $MDNSent $NotJunk $Junk JunkRecorded $Label7 urgent Old .bbblrswipcda -- Can someone enlighten me as to the meaning of this second last line? Many thanks in advance, Gavin Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: what happens when cyrus.squat is old?
Hi, On Fri, 22 Jan 2010, Rob Mueller wrote: > Are you using the new incremental mode david carter added? > > -i Incremental updates where squat indexes already exist. I'm not. This is a very old install. However, we're planning a migration to a new server and I'll possibly try it then. Can you just compile squatter separately or do you need to upgrade the rest of cyrus too? > >- use the index for what it can and then look directly at the remaining > > emails? > > Yes, it does this. > > Which is nice, because it means if the index isn't up to date, your > searches still work correctly, just slower. That's what I was hoping for. Good to know. Thanks, Gavin Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
what happens when cyrus.squat is old?
Hi, it seems that running squatter nightly on all mailboxes takes too long for us. I'm thinking of splitting the mailboxes over different nights or doing the job over the weekend. One question though, what happens when a cyrus.squat file is out of date (ie the mailbox has been changed due to mails added/removed from the mailbox). Does it - use the index for what it can and then look directly at the remaining emails? - use the index and ignore the newer mails - ignore the index and look directly at all mails - something else My tests suggest it's probably the first option, but I'd like to be sure. Thanks, Gavin Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
