Re: Problem with folder subscriptions and LIST/LSUB
On 02/02/12 02:04, Anthony L. Awtrey wrote: On 02/01/2012 08:47 PM, Dave McMurtrie wrote: Quick workaround (assuming that you have root access to the server): 1) using your mail client, create a new folder named newfolder. 2) log in to your server and from a root shell, su to your cyrus user. 3). Navigate the filesystem and cp all the mail files from the directory with the funky name that Cyrus won't list to newfolder. 4) reconstruct newfolder. Hth, Dave Thanks Dave, I'll give it a shot. T Just to confirm: is commit 1f0faf282cc918132957d25e8a099105035670c6 (http://git.cyrusimap.org/cyrus-imapd/commit/?h=cyrus-imapd-2.4id=1f0faf282cc918132957d25e8a099105035670c6) the fix for this problem? I think we may be seeing it here. ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Bulk deletion of mailbox ACLs under Cyrus 2.4.4
On 06/09/11 13:29, Jeroen van Meeuwen (Kolab Systems) wrote: Uch, mind where I said just that, I neglected to mention the attached script only removes ACL entries for which the identifier (assuming it's an individual identifier, admittedly) has no corresponding mailbox. My apologies for pressing send too fast! Hi Jeroen, No worries - thanks a lot for taking a look and it's interesting to see another approach to the same problem. Were you able to take a look at my cyradm patch at https://bugzilla.cyrusimap.org/show_bug.cgi?id=3550 at all? Note that I was trying to remove ACLs for accounts which still existed but needed to be removed so they could be replaced with group permissions instead rather than removing dead ACLs entries. Many thanks, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Bulk deletion of mailbox ACLs under Cyrus 2.4.4
On 03/09/11 12:50, Mark Cave-Ayland wrote: Thanks for the heads up. Does that mean I should invoke reconstruct on all the mailboxes whose permissions I've changed in this way in order to bring the backup ACLs back in line with the mailboxes.db changes? Sigh. So as soon as I ran reconstruct on the parts of the tree I had changed using my previous approach, it noticed that the backup ACLs weren't included in mailboxes.db and hence added them all back in again :/ Following on from your previous email, I ended up patching cyradm in order to allow a wildcard ACL deletion which worked really well, although some mailboxes were still confused to the point where I had to remove individual ACLs from the mailbox as a bulk deletion didn't work (I guess again this was confusion caused by a combination of different backup ACLs, reconstruct and mailboxes.db). Since these problem ACLs were removed, everything now works fine so I can recursively drop and rebuild all ACLs on our shared folder tree using a small bash script :) Also is there any reason why cyradm couldn't be modified to accept wildcards for uids in order to remove all of them? It strikes me that this is almost a bug given that I can sam an entire mailbox hierarchy but not do the same with dam. The perl code seemed reasonably easy to follow with a good API design and so the resulting patch is quite neat. I've created a new bug in bugzilla and attached the patch there as it would be very useful to have this included within the main cyrus codebase: https://bugzilla.cyrusimap.org/show_bug.cgi?id=3550. Many thanks, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Bulk deletion of mailbox ACLs under Cyrus 2.4.4
On 03/09/11 06:16, Bron Gondwana wrote: Just for the archives: I managed to find an alternative solution to my problem. I ended up analysing the output of ctl_mboxlist -d and then writing a bit of perl to generate an output file with the same format for just the mailboxes I was interested in changing but with no ACLs. I then fed the resulting file into ctl_mboxlist -u and as if by magic the job was done :) FYI - while that kinda works, it is slightly skanky, and leaves the mailboxes.db and the backup copy of the ACL in the mailbox header out of sync. It's also going to break in future when the content of mailboxes.db changes. It's also somewhat incompatible with replication, murder, etc. The correct way[tm] is to iterate over all the mailboxes and do a setacl for each one you want to change, probably using an external script that talks IMAP. Hi Bron, Thanks for the heads up. Does that mean I should invoke reconstruct on all the mailboxes whose permissions I've changed in this way in order to bring the backup ACLs back in line with the mailboxes.db changes? Also is there any reason why cyradm couldn't be modified to accept wildcards for uids in order to remove all of them? It strikes me that this is almost a bug given that I can sam an entire mailbox hierarchy but not do the same with dam. Many thanks, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Bulk deletion of mailbox ACLs under Cyrus 2.4.4
On 31/08/11 16:20, Mark Cave-Ayland wrote: Hi all, I'm currently trying to recursively remove all ACLs from part of a Cyrus tree so I can replace them with newer ones based upon group membership rather than individual users. However I can't seem to get this to work at the moment using a wildcard under cyradm: localhost cm public.mcatest localhost lam public.mcatest user1 lrs user2 lrs localhost dam public.mcatest * localhost lam public.mcatest user1 lrs user2 lrs localhost dam public.mcatest user1 localhost lam public.mcatest user2 lrs I've also tried using the anyone/all aliases instead of * but that doesn't seem to work either - is anyone able to point me in the right direction as to the correct syntax to completely remove all ACLs for all users from a mailbox? Just for the archives: I managed to find an alternative solution to my problem. I ended up analysing the output of ctl_mboxlist -d and then writing a bit of perl to generate an output file with the same format for just the mailboxes I was interested in changing but with no ACLs. I then fed the resulting file into ctl_mboxlist -u and as if by magic the job was done :) HTH, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Bulk deletion of mailbox ACLs under Cyrus 2.4.4
Hi all, I'm currently trying to recursively remove all ACLs from part of a Cyrus tree so I can replace them with newer ones based upon group membership rather than individual users. However I can't seem to get this to work at the moment using a wildcard under cyradm: localhost cm public.mcatest localhost lam public.mcatest user1 lrs user2 lrs localhost dam public.mcatest * localhost lam public.mcatest user1 lrs user2 lrs localhost dam public.mcatest user1 localhost lam public.mcatest user2 lrs I've also tried using the anyone/all aliases instead of * but that doesn't seem to work either - is anyone able to point me in the right direction as to the correct syntax to completely remove all ACLs for all users from a mailbox? Many thanks, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: crc32_buf
On 18/06/11 20:14, Bron Gondwana wrote: Yeah, I screwed up. We require zlib at the moment. My plan is to embed a basic crc32 algorithm directly into that file for the case where zlib is not available, but I haven't done it yet. Regards, Bron. Hi Bron, What is zlib used for in Cyrus? Is it actually used for mailbox compression or was it just a quick solution for adding a CRC32 implementation? If it is the latter, then it should be fairly easy to add a CRC32 algorithm and remove that dependency. Do you have a preferred generator polynomial? ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
List of Cyrus 2.4 access rights?
Hi all, Having recently upgraded from Cyrus 2.2 to 2.4 it seems that more access rights have been defined. For example: On Cyrus 2.2: localhost sam user.foo foo all localhost lam user.foo foo lrswipcda On Cyrus 2.4: localhost sam user.foo foo all localhost lam user.foo foo lrswipkxtecda I've had a look at the documentation here: http://www.cyrusimap.org/docs/cyrus-imapd/2.4.6/overview.php#acl and I can't see any of these new flags listed. Can anyone tell me what they actually do? ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Incorrect support URL in Cyrus 2.4 ver output
Hi all, Also while I remember from our new Cyrus 2.4 installation: localhost ver name : Cyrus IMAPD version: v2.4.6-Debian-2.4.6-1~6.gbpd84454 35e0e72f 2010-12-21 vendor : Project Cyrus support-url: http://cyrusimap.web.cmu.edu os : Linux os-version : 2.6.32-5-xen-amd64 environment: Built w/Cyrus SASL 2.1.23 Running w/Cyrus SASL 2.1.23 Built w/Berkeley DB 4.8.30: (April 9, 2010) Running w/Berkeley DB 4.8.30: (April 9, 2010) Built w/OpenSSL 0.9.8o 01 Jun 2010 Running w/OpenSSL 0.9.8o 01 Jun 2010 Built w/zlib 1.2.3.4 Running w/zlib 1.2.3.4 CMU Sieve 2.4 TCP Wrappers NET-SNMP mmap = shared lock = fcntl nonblock = fcntl idle = poll Looks like the support-url in imap/version.c needs to be updated to point to the new website. ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: List of Cyrus 2.4 access rights?
On 31/01/11 17:42, Andrew Morgan wrote: I've had a look at the documentation here: http://www.cyrusimap.org/docs/cyrus-imapd/2.4.6/overview.php#acl and I can't see any of these new flags listed. Can anyone tell me what they actually do? The manpage for cyradm has a good description under the command setaclmailbox. These are all described in RFC 4314 as well, which is the standard that has changed since Cyrus 2.2. Andy Hi Andy, That's great - thanks for the references. ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Updated 2.4.6 with autocreate for those who need it
On 20/01/11 18:53, Bron Gondwana wrote: I hope this is useful for those who want to upgrade to 2.4 and can't wait until the auto* feature is implemented upstream - Bron, thanks for looking into it _after_ moving your home and what else :) :) FWIW us at Sirius are very interested in this functionality, and since I'm a dab hand with a C compiler would be happy to help out with patch review, testing etc. Out of interest, what are the objections to the current patch? And would it be applied to the 2.4.x series or wait until 2.5? ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Updated 2.4.6 with autocreate for those who need it
On 21/01/11 14:53, Reinaldo de Carvalho wrote: Out of interest, what are the objections to the current patch? And would it be applied to the 2.4.x series or wait until 2.5? A (commonly) bad MTA configuration that not reject unknown recipients, and try to deliver the message to cyrus will generate thounsands of mailboxes. If this feature will be implemented must have a option to disable it. And, IMHO autocreatemailbox should be disable by default. Oh that's interesting. I must admit I haven't done much with the patches other than noted that they exist within our internal builds, but surely the account auto creation only happens if the recipient exists within a directory somewhere, e.g. LDAP? Can saslauthd be used to determine whether an account exists or not as opposed to just being used for authentication? ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Migrating from Cyrus 2.2 to Cyrus 2.4 - preserving seen db for public folders
Hi folks, I'm currently testing a migration script for migrating across from Cyrus 2.2 to Cyrus 2.4 based on the recipe here: http://cynici.wordpress.com/2010/12/06/how-to-migrate-32-bit-cyrus-imapd-mailboxes-to-64-bit/. This is currently working well, and I have added some extra copy commands to ensure that the seen db information is copied across from /var/imap/user. However, while the seen db information for my personal folders is being transferred across fine, all of the shared folders are currently marked as unread. So I have 2 questions: 1) Where is the seen db information held for the shared (public) folders? It doesn't seem to be under /var/imap/user. 2) Once I find it, how can I migrate it across from the 2.2 server to the 2.4 server? Many thanks, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: New Cyrus project site and bugzilla
Dave McMurtrie wrote: We're very interested in growing the Cyrus project and attracting new volunteers to contribute to the project, and that desire is at the core of why this migration is taking place. The biggest change is that we're trying to separate the environment from Carnegie Mellon University infrastructure as much as possible. Previously, contributions of any kind would end up requiring us to create a CMU computer account for a willing volunteer. We can now simply create local shell accounts as required. Almost the entire website has been created using MediaWiki software, so anyone who is willing to register for an account may update the website content. Wow - this looks really good :) The main criticism I have from a developer point of view is, well, CVS. Enough said. Please please can we have an official git mirror? It makes maintaining out-of-tree patches so much easier in the long run, and therefore much more likely that we can pass the patches back upstream. (On a separate note, if I go to Downloads - Getting Started and click on the AnonymousCVS wiki link then I get redirected back to the front page rather than to a page giving information on how to access CVS) ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: New Cyrus project site and bugzilla
Bron Gondwana wrote: The main criticism I have from a developer point of view is, well, CVS. Enough said. Please please can we have an official git mirror? It makes maintaining out-of-tree patches so much easier in the long run, and therefore much more likely that we can pass the patches back upstream. We're working on it! I'm hoping to chat with Jeroen from Kolab about it again tonight. We've got a partial merge into git - but we just want to make sure all the tags and authors and stuff are imported properly before cutting over. And to give people enough warning to change :) Excellent news! FWIW the PostgreSQL team have been trying to switch to git for the past month, and in the process have involved the cvs2git maintainers and had some fixes committed over the past few weeks to improve the migration process (note that they have also suffered from having to hand-tweak the repository to fix various bugs in CVS). The thread about the entire process is very long, but for those interested the latest summary is here: http://archives.postgresql.org/pgsql-hackers/2010-09/msg00636.php. HTH, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: CVS to GIT
Jeroen van Meeuwen (Kolab Systems) wrote: We have a working sample, with a documented procedure, to move three CVS modules (cmulocal, cyrus and sieve) into one GIT repository: http://www.cyrusimap.org/mediawiki/index.php/Drafts/CVS_to_GIT_Conversion#Stab_.232 There some about branch and tag conversions as well. You can find the result (which is preliminary!!) at http://git.kolabsys.com/cyrus-imapd.git. I'll be working with Dave to get this setup over on cyrusimap.org as soon as possible as well, but meanwhile, feedback is more then welcome! Kind regards, Oooh very nice. It seems to be a common issue that projects have to tweak their CVS repositories by hand to get a reasonable conversion to git. I'll try and take a closer look when I get a spare moment. My other point, of course, was that since the PostgreSQL guys worked to fix a couple of bugs in cvs2git over the past couple of weeks, you may get a better result if you grab the tip version of cvs2git and re-run the conversion. ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: New Cyrus project site and bugzilla
Dave McMurtrie wrote: (On a separate note, if I go to Downloads - Getting Started and click on the AnonymousCVS wiki link then I get redirected back to the front page rather than to a page giving information on how to access CVS) Thanks for pointing this out, Mark. I made that link more useful. Dave Thanks Dave, that looks fixed now. ATB, Mark. -- Mark Cave-Ayland - Senior Technical Architect PostgreSQL - PostGIS Sirius Corporation plc - control through freedom http://www.siriusit.co.uk t: +44 870 608 0063 Sirius Labs: http://www.siriusit.co.uk/labs Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
Re: Cyrus 2.3.13 RC2
Kenneth Marshall wrote: There have been 5 major releases of PostgreSQL since 7.2 was released and 7.2 is EOL in the next few months. I think it is completely reasonable to not support version 7.1/7.2 in a new system considering that 7.1 is EOL and 7.2 will be shortly. Cheers, Ken Oh seriously, don't even waste time worrying about it. 7.2 died a long time ago, 7.3 was EOL the beginning of this year [1], and 7.4 is about to go the same way real soon now. Normally adding 7.3 support is fairly easy if required, whereas going back to 7.2 is often a complete pain, plus you have to live with several unfixable data loss bugs... HTH, Mark. [1] http://www.postgresql.org/about/news.905 -- Mark Cave-Ayland Sirius Corporation - The Open Source Experts http://www.siriusit.co.uk T: +44 870 608 0063 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Cyrus 2.3.8 imapd process periodically sticks at 100% CPU
Hi there,
I'm experiencing a problem with Cyrus 2.3.8 interacting with an Outlook
client and was hoping this would be the right place to get some advice.
What happens is that periodically (maybe around once a month?) we have
one particular user who contacts us complaining that they are unable
access their mailbox. Generally we always find the same thing: there is
an imapd process accessing his seen DB which is running at 100% CPU.
Once this process is killed then things go back to normal and the user
can log in.
The latest report we had of this problem happening again was this
morning, and fortunately I was in a position to attack it with gdb and a
file of debug symbols. This showed that the process in question was
getting stuck in a loop in index_expungeuidlist(). I've uploaded the
transcript of my gdb session to
http://pastebin.siriusit.co.uk/cyrus-imapd-gdb.txt for people who are
familiar with cyrus internals.
The short story appears to be that newseenuids (new) points to an empty
string ('\0') and so the code gets stuck because of the following at
line 532 of imap/index.c in index_checkseen():
oldseen = (*old == ':');
Since *old is an empty string, oldseen will always be 0, and so the
while() loop never exits. Unfortunately this is the first time I've ever
looked at cyrus internals, so am not really sure what the seen list
should look like normally.
The confusing thing is that we have been using these packages for
several clients and this is the *only* particular server and the *only*
user on this server experiencing this problem. The one thing we have
noticed is that this particular user has a larger mailbox compared to
the other users (~1GB) but then it doesn't seem so large as if it would
cause any problems.
Finally, one more thing to add is that we have already gone through the
steps of rebuilding the seen DB skiplist using the skiplist.py script
several times when this has happened in the past, and it has made no
difference.
Many thanks,
Mark.
--
Mark Cave-Ayland
Sirius Corporation - The Open Source Experts
http://www.siriusit.co.uk
T: +44 870 608 0063
Cyrus Home Page: http://cyrusimap.web.cmu.edu/
Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki
List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
Re: Cyrus 2.3.8 imapd process periodically sticks at 100% CPU
Bron Gondwana wrote: No, it won't. You need to fix the mailbox or patch the code to not be put into an infinite loop by a bogus index file. The attached patch might do the trick for you. I just slapped it together on spec. It compiles, that's about all I can offer about it :) Hi Bron, Thanks for taking the time to respond, it is greatly appreciated. My colleague Duncan has taken a look at this (see separate email) and found a corrupted index file which have now been rebuilt. Fingers crossed that the problem is now resolved (I guess we'll find out in a month or so) ;) Many thanks, Mark. -- Mark Cave-Ayland Sirius Corporation - The Open Source Experts http://www.siriusit.co.uk T: +44 870 608 0063 Cyrus Home Page: http://cyrusimap.web.cmu.edu/ Cyrus Wiki/FAQ: http://cyrusimap.web.cmu.edu/twiki List Archives/Info: http://asg.web.cmu.edu/cyrus/mailing-list.html
