[PATCH] Allow setting custom annotations through cyradm
Hello,
please find attached a patch, that allows the configuration of custom
annotations on mailboxes through cyradm.sh.
This patch had been pending submission for a while, as it depended on the
annotation_definitions patch accepted in 2.3.16[1].
I hope you will find this patch applies to CVS HEAD cleanly.
[1] http://lists.andrew.cmu.edu/pipermail/cyrus-announce/2009-
December/69.html
--
Jeroen van Meeuwen
Senior Engineer, Kolab Systems AG
e: vanmeeu...@kolabsys.com
t: +316 42 801 403
w: http://www.kolabsys.com
pgp: 9342 BF08
From bf3ed165f8bfdeb7585e89c059f94b36b4b5aeb9 Mon Sep 17 00:00:00 2001
From: Jeroen van Meeuwen (Ergo Project) jeroen.van.meeu...@ergo-project.org
Date: Mon, 2 Aug 2010 16:36:24 +0200
Subject: [PATCH] Allow configuration of custom annotations through cyradm
---
perl/imap/IMAP/Admin.pm |8
perl/imap/IMAP/Shell.pm |6 +++---
perl/imap/cyradm.sh |4
3 files changed, 11 insertions(+), 7 deletions(-)
diff --git a/perl/imap/IMAP/Admin.pm b/perl/imap/IMAP/Admin.pm
index a33713e..c0a6670 100644
--- a/perl/imap/IMAP/Admin.pm
+++ b/perl/imap/IMAP/Admin.pm
@@ -797,12 +797,12 @@ sub mboxconfig {
return undef;
}
- if(!exists($values{$entry})) {
-$self-{error} = Unknown parameter $entry;
+ if(exists($values{$entry})) {
+$entry = $values{$entry};
+ } else {
+$self-{error} = Unknown parameter $entry unless substr($entry,0,1) eq /;
}
- $entry = $values{$entry};
-
my ($rc, $msg);
$value = undef if($value eq none);
diff --git a/perl/imap/IMAP/Shell.pm b/perl/imap/IMAP/Shell.pm
index 7fed400..c605079 100644
--- a/perl/imap/IMAP/Shell.pm
+++ b/perl/imap/IMAP/Shell.pm
@@ -127,7 +127,7 @@ my %builtins = (exit =
[\_sc_info, '[mailbox]',
'display mailbox/server metadata'],
mboxcfg =
- [\_sc_mboxcfg, 'mailbox [comment|condstore|expire|news2mail|sharedseen|sieve|squat] value',
+ [\_sc_mboxcfg, 'mailbox [comment|condstore|expire|news2mail|sieve|squat|/explicit annotation] value',
'configure mailbox'],
mboxconfig = 'mboxcfg',
reconstruct =
@@ -1437,7 +1437,7 @@ sub _sc_mboxcfg {
while (defined ($opt = shift(@argv))) {
last if $opt eq '--';
if ($opt =~ /^-/) {
- die usage: mboxconfig mailbox [comment|condstore|expire|news2mail|sharedseen|sieve|squat] value\n;
+ die usage: mboxconfig mailbox [comment|condstore|expire|news2mail|sharedseen|sieve|squat|/explicit annotation] value\n;
}
else {
push(@nargv, $opt);
@@ -1446,7 +1446,7 @@ sub _sc_mboxcfg {
}
push(@nargv, @argv);
if (@nargv 2) {
-die usage: mboxconfig mailbox [comment|condstore|expire|news2mail|sharedseen|sieve|squat] value\n;
+die usage: mboxconfig mailbox [comment|condstore|expire|news2mail|sharedseen|sieve|squat|/explicit annotation] value\n;
}
if (!$cyrref || !$$cyrref) {
die mboxconfig: no connection to server\n;
diff --git a/perl/imap/cyradm.sh b/perl/imap/cyradm.sh
index 708aebc..5e2ecf9 100644
--- a/perl/imap/cyradm.sh
+++ b/perl/imap/cyradm.sh
@@ -241,6 +241,10 @@ mailboxes).
Indicates that the mailbox should have a squat index created for it.
+=item C/explicit/annotation
+
+Sets the annotation I/explicit/annotation on Imailbox to Ivalue.
+
=back
=item Crenamemailbox [C--partition Ipartition] Ioldname Inewname
--
1.7.1.1
Re: Closing old idle bugs?
Bron Gondwana wrote: On Mon, Aug 02, 2010 at 05:10:26PM +0200, Jeroen van Meeuwen (Kolab Systems) wrote: Hi there, would it be acceptable if old, idle bugs in bugzilla were closed after due consideration? That would be a great idea! ;-) For example, I encounter bug #2590[1] with status NEW; I would like to close it to clean up the bug database, possibly only after requesting some feedback (like I did with this one). [1] https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2590 Maybe we should do a bug cleaning week? I'm on leave at the moment. I think we should probably go through all the bugs as a group and decide on their status and applicability. I suspect quite a few of them are no longer valid or already solved - just not updated. It seems that I do not have any permissions to change a bug's status; I have a new account. Perhaps if we posted one email per bug to the mailing list and asked for a discussion on its merits. Maybe not all at once so they don't get lost in the haze, but we could group them up. One bug a day through the list would be very inefficient, and I'm sure there's bugs that actually do deserve love attention. There's a number of bugs that have been resolved already, just haven't been closed. Let's remember that if a bug is closed while not satisfactory to the reporter, the bugs can simply be reopened. Most of them look like they just need some follow-up and can be closed in 2/4/6 weeks if no response has been given to any inquiries for more information (like #2590, the time window is a project choice of course). -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Patch to set plaintext refusal string to plaintextloginalert (Bug 3247)
Michael Bacon wrote: This week, we finally dragged UNC into the 21st century and disabled unencrypted plaintext logins. In so doing, we found the default message for rejecting unencrypted LOGIN attempts, Login only available under a layer, to be confusing to our users. This patch just overloads plaintextloginalert to also be the reject message if allowplaintext is set to 0. It is otherwise only used if allowplaintext is set to 1. This is a patch to the 2.3.16 codebase, and is entered in Bugzilla as 3247. I would obfuscate the original message less, by saying something like: Plaintext login not allowed, by policy, without added security layer As opposed to the message that is in there now. Would that make sense? -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Patch to set plaintext refusal string to plaintextloginalert (Bug 3247)
Michael Bacon wrote: In our case, we wanted to put a URL to a help document on our knowledge base system, so I feel like the correct generic solution is to allow it to be configured at runtime in the config file. That's right, when I read back my message I realized I made it sound like something other then what I wanted to say... ;-) First: Excellent! Second: While we're patching this, could we maybe get rid of the obfuscating message as well? Apologies for the confusion. -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
[PATCH] Characters allowed in folder names (#2633)
Hi there,
Current Kolab packages ship a patch to allow more character names in IMAP
mailbox/folder names then is currently permitted with the characters listed in
imap/mboxname.c:724 GOODCHARS definition. Bug #2633[1] also concerns this
topic.
In an attempt to do a little research on the set of characters allowed in a
mailbox, I stumbled upon section 5.1 in RFC 3501[2]. Sub-section 5.1.3
confuses me a little, so I hope some of you can shed some light on that if
necessary.
It seems that mailbox/folder names may contain any of the characters in 7-bit
ASCII, but GOODCHARS does not have all of those characters listed. Attached is
a patch that shows how I would like this a configurable setting. Unless I'm
mistaken, this should preserve the behavior presented today, but allows a
'goodchars' setting to be supplied in imapd.conf.
Please let me know what you think.
[1] https://bugzilla.andrew.cmu.edu/show_bug.cgi?id=2633
[2] http://www.faqs.org/rfcs/rfc3501.html
Kind regards,
--
Jeroen van Meeuwen
Senior Engineer, Kolab Systems AG
e: vanmeeu...@kolabsys.com
t: +316 42 801 403
w: http://www.kolabsys.com
pgp: 9342 BF08
diff --git a/imap/mboxname.c b/imap/mboxname.c
index 1e0a5e2..a0ec2c7 100644
--- a/imap/mboxname.c
+++ b/imap/mboxname.c
@@ -730,8 +730,11 @@ int mboxname_policycheck(char *name)
unsigned c1, c2, c3, c4, c5, c6, c7, c8;
int ucs4;
int unixsep;
+const char *goodchars;
unixsep = config_getswitch(IMAPOPT_UNIXHIERARCHYSEP);
+goodchars = config_getstring(IMAPOPT_GOODCHARS);
+if (!goodchars) goodchars = GOODCHARS;
if (strlen(name) MAX_MAILBOX_NAME) return IMAP_MAILBOX_BADNAME;
for (i = 0; i NUM_BADMBOXPATTERNS; i++) {
@@ -810,7 +813,7 @@ int mboxname_policycheck(char *name)
name++; /* Skip over terminating '-' */
}
else {
- if (!strchr(GOODCHARS, *name)
+ if (!strchr(goodchars, *name)
/* If we're using unixhierarchysep, DOTCHAR is allowed */
!(unixsep *name == DOTCHAR))
return IMAP_MAILBOX_BADNAME;
diff --git a/lib/imapoptions b/lib/imapoptions
index 421b0dd..88715d5 100644
--- a/lib/imapoptions
+++ b/lib/imapoptions
@@ -1270,6 +1270,10 @@ product version in the capabilities */
mailbox hierarchy. The default is to use the netnews separator
character '.'. */
+{ goodchars, NULL, STRING }
+/* Characters to allow in mailbox names. If not configured a default
+ from imap/mboxname.c is being used */
+
{ virtdomains, off, ENUM(off, userid, on) }
/* Enable virtual domain support. If enabled, the user's domain will
be determined by splitting a fully qualified userid at the last '@'
Re: [PATCH] Characters allowed in folder names (#2633)
Bron Gondwana wrote: On Sat, Aug 07, 2010 at 07:22:05PM +0200, Jeroen van Meeuwen (Kolab Systems) wrote: Attached is a patch that shows how I would like this a configurable setting. Unless I'm mistaken, this should preserve the behavior presented today, but allows a 'goodchars' setting to be supplied in imapd.conf. While I like tunables, I don't like this approach. The problem is, the default sucks. I'd prefer to see ALL possible good characters in the source, and the option of setting additional characters you won't allow in the config file, so the default option is best for most people. There can never be enough tunables ;-) This one indeed doesn't necessarily make sense. That said, I recon there is no distinction between normal and expert settings in Cyrus IMAP. Meaning, that if something is made configurable at all, it'll be thrown at the user, in the man page, and throughout the documentation. I usually have the following distinction in types of settings in some applications; - Novice settings in the GUI - just gets the job done, click through, nothing to see here. - Advanced settings on the CLI - gets *your* job done. If you know what you're doing. - Expert settings - you're on your own. This setting, I think, is a typical you're on your own type of setting. While I think it should be configurable, I dislike throwing it in the man page. I would argue this type of thing is way better documented on a cyrusimap.org Wiki article. Cyrus already has too many configuration options for most sites to cope with. Adding more sucks. (also on a technical note: I would put the default in lib/imapoptions rather than testing if the config value existed) I tried, but the way lib/imapoptions is being parsed by some custom scripts in tools made it not work; it would split up the default string at the ',' for some reason I could not determine just like that. -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: [PATCH] Characters allowed in folder names (#2633)
Bron Gondwana wrote: On Sat, Aug 07, 2010 at 07:22:05PM +0200, Jeroen van Meeuwen (Kolab Systems) wrote: While I like tunables, I don't like this approach. The problem is, the default sucks. I'd prefer to see ALL possible good characters in the source, and the option of setting additional characters you won't allow in the config file, so the default option is best for most people. On an additional note; - What would you think would be a sane default? All RFC-compatible chars allowed? Would such be 7bit ASCII in your opinion? - I just verified my suspicions with this version of the patch; the unixhierarchy sep. is required in the list of allowed characters, and is not forcibly included or allowed; opening up the user to a realm of destructive and vague problems. -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: [PATCH] Characters allowed in folder names (#2633)
Henrique de Moraes Holschuh wrote: The OP already has exactly what you want. It was argued we need to be careful throwing all too many configurables at the consumer, for which there is currently only a single mechanism in place. *That's* what I'm trying to work with here. Our demographics are sysadmins. IMO, we should just organize the documentation properly to let them know what options are best left alone, but document them througoutly. Cluster all such options (e.g. under a new advanced options manpage section, and people will just skip over them unless they need them. And so the question becomes; How do you propose we do that, given the manpages are generated from lib/imapoptions by tools/config2man? I suppose one way to get extraordinarily awesome man pages is to not generate the manpages any longer, but instead write them by hand. -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re-organizing the manpages (was: Re: [PATCH] Characters allowed in folder names (#2633))
Wesley Craig wrote: On 10 Aug 2010, at 10:34, Jeroen van Meeuwen (Kolab Systems) wrote: How do you propose we do that, given the manpages are generated from lib/imapoptions by tools/config2man? I haven't experimented with this myself, but it appears to me that lib/imapoptions *is* the man page *and* the code. Take a look at the trailing block of lib/imapoptions: /* .SH SEE ALSO .PP \fBimapd(8)\fR, \fBpop3d(8)\fR, \fBnntpd(8)\fR, \fBlmtpd(8)\fR, \fBtimsieved(8)\fR, \fBidled(8)\fR, \fBnotifyd(8)\fR, \fBdeliver(8)\fR, \fBmaster(8)\fR, \fBciphers(1)\fR */ Looks a lot like a man page, eh? So you could get the desired effect by reorganizing lib/imapoptions, including adding a new section header between the good to change and bad to change options. Personally, I think a goal of sane defaults would be more helpful, but appropriately organizing the information is a great task for a volunteer to engage in! I could at least take a stab at it ;-) My proposition is to distinguish between the following groups of configurables: - Gets the job done (novice) - Gets your particular job done (advanced) - You're on your own (expert) I would also like to split out murder and sync specific settings, and I would probably want them in the Advanced section. Is there anything else we can think of maybe? -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: [PATCH] Allow setting custom annotations through cyradm
Jeroen van Meeuwen (Kolab Systems) wrote: Hello, please find attached a patch, that allows the configuration of custom annotations on mailboxes through cyradm.sh. This patch had been pending submission for a while, as it depended on the annotation_definitions patch accepted in 2.3.16[1]. I hope you will find this patch applies to CVS HEAD cleanly. [1] http://lists.andrew.cmu.edu/pipermail/cyrus-announce/2009- December/69.html Ping, this seems like a pretty straight-forward patch. Can I get some feedback please? -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Draft: Bugzilla Work Flow
Hello there, I'm working on a documented Bugzilla work flow, in an attempt to streamline how we all work with it and what the average consumer may or may not expect. To allow some early feedback, I'm putting the page on the list now as opposed to when I feel like I'm done documenting everything in full ;-) http://www.cyrusimap.org/mediawiki/index.php/User:Jeroen_van_Meeuwen/Drafts/Bugzilla_Work_Flow Please note that these would, in any case, be guidelines, not law. There's no intention to make anyone's life any more difficult ;-) The attempt is to document what mere mortals like myself, and those people that are on the reporting end of bugs might expect, and to allow new contributors like myself to read up on what is an agreeable approach to handling Bugzilla issues. Thank you in advance, Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Draft: Bugzilla Work Flow
Jeroen van Meeuwen (Kolab Systems) wrote: Hello there, Nudging ;-) CC:'ing the info list as well. I'm working on a documented Bugzilla work flow, in an attempt to streamline how we all work with it and what the average consumer may or may not expect. To allow some early feedback, I'm putting the page on the list now as opposed to when I feel like I'm done documenting everything in full ;-) http://www.cyrusimap.org/mediawiki/index.php/User:Jeroen_van_Meeuwen/Drafts/Bugzilla_Work_Flow Please note that these would, in any case, be guidelines, not law. There's no intention to make anyone's life any more difficult ;-) The attempt is to document what mere mortals like myself, and those people that are on the reporting end of bugs might expect, and to allow new contributors like myself to read up on what is an agreeable approach to handling Bugzilla issues. -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Draft: Bugzilla Work Flow
Bron Gondwana wrote: That looks really good actually. I guess the side question is Is Bugzilla the ideal tool for this? I've seen setups where commit messages in the change management tool can be tied directly to tickets. It's probably not essential though - if we have a process and we all know how to follow it. There's git-bugzilla for this purpose. I haven't actually worked with it though. There's also a python-bugzilla CLI client, which we could (possibly) use as a commit hook to update any tickets referred to in the commit message. Speaking of which - the auto bugzilla reminder emails are great :) Occasional annoyances that go away when you do the right thing are very useful. Yeah, so is the automagically put someone in CC: on bug update, having someone as a QA contact, or workflow enhancements (somebody patches some bug, closes the bug and release engineering looks to forward/backporting it). Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Draft: Bugzilla Work Flow
Dave McMurtrie wrote: There was never an intent to really hide the fact that it's a wiki, and you can see on the Contribute page that I prominently announced that the site is almost entirely constructed using Mediawiki. I still don't think we need to do any magic to conceal the mediawiki directory path in the URLs. By saying hiding I actually meant to say strip /mediawiki/index.php/ from the URL. Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Wiki, information on database back ends.
Patrick Goetz wrote: On 09/14/2010 02:24 PM, Patrick Goetz wrote: This raises a number of questions, though: I forgot to add: 4. The Redhat script uses a file called /usr/share/cyrus-imapd/rpm/db.cfg Presumably this is specific to the redhat package, since I can't find a db.cfg file associated with my 2.1.16 cyrus server, or is my version just too old and behind the times? This file is also written out by the same script. The location of the script (the mentioning of 'rpm/' in the path) is completely arbitrary (for all the script cares the file lives in /boot/). Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Draft: Bugzilla Work Flow
Dave McMurtrie wrote: Jeroen van Meeuwen (Kolab Systems) wrote: By saying hiding I actually meant to say strip /mediawiki/index.php/ from the URL. Understood. Someone (Rob Mueller from Fastmail?) actually specifically requested that we do something like this. I would certainly not object to simpler URLs. Would it make sense for me to clone this host so you have a place to test your httpd.conf changes before deploying them in production if we want to pursue this? I guess it's easy enough to change back if something does break. Thoughts? No need to test on a separate host, really, I have said config in production in several places. I'd take a separate vhost on localhost through SSH tunnel to make sure it works as anticipated. Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Wiki, information on database back ends.
Dan White wrote: The current imapd.conf hints at the available data types for each cyrus database. Depending on the option, the following database types are available (in 2.3.16): (...snip...) I don't know why the (rpm) upgrade script chose skiplist as an intermediate format, rather than flat. Perhaps skiplist hasn't changed much between major versions, and offers a speed benefit. Actually the RPM packages patch up lib/imapoptions to default to skiplist for all databases. The script choosing skiplist as the default format to convert databases to is nothing but a choice that makes a default export/import much faster because the script does not actually have to export/import (it merely has to detect/determine/compare/continue). As far as skiplist, that seems to be a proprietary format. skiplist: ? http://git.cyrusimap.org/cyrus-imapd/tree/lib/cyrusdb_skiplist.c is the implementation of http://en.wikipedia.org/wiki/Skip_list inspired by ftp://ftp.cs.umd.edu/pub/skipLists/skiplists.pdf. I fail to see how exactly this could lead to the conclusion it may be proprietary software. Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Cyrus IMAPd packages in Debian
Bron Gondwana wrote: Hi Benjamin, I've been pointed at this: http://lists.alioth.debian.org/pipermail/pkg-cyrus-imapd-debian-devel/2010- August/003427.html One of the things I'm trying to do with Cyrus 2.4 is making the upgrade process easier. I've already made one change that will help enormously. On startup, the command ctl_cyrusdb -r is always run before starting any other commands. I've made it open every configured database file and check the magic number to determine what type the file is, then convert if the configuration says it should be a different type. I'm currently looking into automating BDB file upgrade, so you can link a newer Cyrus against a newer version of the Berkeley libraries and have a clean upgrade path. If there's anything else we can do to make packaging new versions of Cyrus easier for you, let me know! We run Cyrus on Debian ourselves at FastMail, so we care about it being good there :) In addition, I might add, Kolab Systems also has a great stake in the wellfare of the Debian Cyrus packages. We do have some type of alterior motivation as well, in the interest of full disclosure; we will want to keep as much consistency between platforms as possible, for support and documentation (downstream as well as upstream) efforts. You can find some of that work (which I called kolab-cyrus-imapd as to not conflict with work we may be doing on the actual cyrus-imapd packaging), here; http://git.kolabsys.com/apt/kolab-cyrus-imapd/log/?h=work/cross-platform- consistency Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Bugzilla cleanup
Henrique de Moraes Holschuh wrote: On Tue, 12 Oct 2010, Jeroen van Meeuwen (Kolab Systems) wrote: 2) Close the bugs that are not RESOLVED or CLOSED by mass-updating them, providing some text saying that either the bug must be reproducible in a currently supported version, or the reporter somehow proves to us its commitment in getting the issue resolved (insert a link to the wiki page on ways to contribute). Shouldn't the mass-close list be subject to a human cleanup to weed out [from closing] bugs that still obviously apply to current code, or which look important enough that we shouldn't send it to no man's land without further attention? Should there be any bugs that obviously still apply to the current code, then for the 2.1 series, these bugs have apparently been idle for about 5 years, without proper follow-up from either side. Hence, for the closing of bugs, we would send out a notification to the reporter, anyone in CC:, as long as these individuals have set their preferences to be notified about changes to the bug through email. If they have not, then virtually any manual cleanup will turn out to be pointless as well; the questions never reach anyone significant anyways. That said though, is there an example you have in mind, or are we merely speculating on the hypothetical? Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Cyrus 2.4.1
Bron Gondwana wrote: There's a stack of small things accumulating that I think are going to be enough to justify releasing 2.4.1 some time this week. The only major blocker I can see is LSUB across backends. I'm going to have a look at it during this flight to Australia - see what I can get. I would like to (ab)use bugzilla for lists of things we may consider for 2.4- next, in this case 2.4.1; http://bugzilla.cyrusimap.org/bugzilla3/buglist.cgi?cmdtype=runnamednamedcmd=2.4- next What we could do is; 1) create the 2.4.1 milestone, 2) go over this small list of 8 bugs, 3) determine whether we need or want any of the bugs on this list to make it into 2.4.1, 4) set these bugs to the 2.4.1 milestone 5) release 2.4.1 as soon as the list of bugs to be resolved for 2.4.1 (e.g. not CLOSED but against the 2.4.1 milestone) is empty. Would that work for everyone or do we need to find another way to go about (ab)using bugzilla? Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Product versioning and X.Y.Z bumping (was: Re: Cyrus 2.4.1)
Bron Gondwana wrote: On Sun, Oct 17, 2010 at 08:26:49PM -0400, Wesley Craig wrote: On 17 Oct 2010, at 06:53, Jeroen van Meeuwen (Kolab Systems) wrote: I'm in favor to have (in our current X.Y.Z versioning schema) Z be bumped with bug-fixes only. If we keep it to bug-fixes only, this means we can rapidly release the fixes to the community. I'm definitely in favor of that kind of release discipline. We've had cases where untested features were released along side critical security fixes. Pre-defining the branches as above would solve that problem while providing at least enough roadmap for viewers at home to follow along. That sounds good to me too. Ken, Dave and I were talking about making another major release in January - so we'll just do bugfixes and security fixes into the stable releases until then. Let's try and stick to some common terminology - if everyone understands something different for major and minor that doesn't help; I'm just thinking out loud here, based on some past experiences; In a X.Y.Z product versioning schema, X is Major, Y is Minor, Z is Teeny (a.k.a. patch-level, a.k.a. z-stream, a.k.a. Minor.Minor). Let me try and describe some semantics on when a certain X, Y or Z *must* be bumped for any of those particular reasons; Z (Teeny) is typically bumped regularly, and within the same, stable X.Y series only contains bug-fixes. The more a stable branch matures, obviously, the less bug-fixes go in, the less frequent releases are being made. However, the Teeny bumping using this restriction of what can go into a teeny bump then also says; - testing is required for the (supposedly) fixed bugs only (including security fixes which essentially are bugs), - no feature enhancements make it into an existing X.Y stable branch, and no new features may disrupt the use or X.Y series releases in production, - the goal is to let the users/admins/consumers be able to blindly update their way through Teeny releases within a stable Major.Minor series, - rapid release cycle (one bugfix per teeny release is not unsustainable). Y (Minor) is typically bumped for feature enhancements. These new features may require additional configuration, or some type of manual intervention may be required / desirable when upgrading from X.Y to X.(Y+1). However, the former notwithstanding, a minor bump typically allows for a pretty smooth upgrade path. Meaning, no legacy form or operations may be rendered invalid, such as; no database formats supported in 2.3 can be dropped for a 2.4 release, no features can be dropped, and no functional changes of any significant impact to architecture / infrastructure can be required (example: Do not rewrite murder in ways impacting infrastructure topology and so forth). X (Major) is typically where you abandon a form of compatibility operations, and have the opportunity to refactor the lot and break everything down before you build it back up again. This includes (potentially) dropping functionality (take a step backwards in order to then move forward). There's semantics attached to a Major bump; - There may be no upgrade path without manual conversion or any other type of intervention, - The manual conversion may or may not be aided given a set of utilities, which may or may not be available external to the Cyrus IMAP release tarballs, and which may or may not be supported to the same extent as the actual program code (that said, imapsync your way through this upgrade), - Topology changes may be required, - There may or may not be a downgrade path back to the previous major version you know worked. Let me know what you think, we could make this a wiki article of sorts. Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: chk_cyrus bug
Carsey, Robert wrote: Upgraded to 2.4.1. yey! Hi Robert, could you log this in the Cyrus Bugzilla[1]? It'll make sure the report doesn't disappear and the problem will be resolved. If/when too much trouble I can do it for you, please let me know. I want to make it as easy as possible (really I do!). Thanks in advance! Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Cyrus IMAPd 2.4.1 Released
Matt Selsky wrote: On Tue, 19 Oct 2010, Bron Gondwana wrote: Sounds good to me. Of course we need to have someone who has access to each of those platforms. Nice thing about a 'standard library' is how unstandard it is! I would have though strcasestr was OK though. Now I know. Dave (McMurtie), Could we get additional VMs on the project's VM server to install test x86 platforms? Sparc, etc, test platforms will require real hardware, probably. I volunteer to host a Solaris/Sparc test platform once the x86 systems are in place. Since we're on the subject... Please consider allowing me to deploy Hudson Continuous Integration. It integrates with GIT and Bugzilla really well if you tweak it just a tiny little, and it's really easy to add (a (geographically) distributed set of) slaves. Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Bugzilla Cleanup - Your Help Needed and Much Appreciated!
Hello there, The Cyrus Bugzilla is a very important component for all of us, community users and Cyrus developers alike! I suppose most of us have, at least once or twice, logged a new report in Bugzilla, but then what happens with that report? From the other side, the Cyrus team sometimes doesn't sort through the series of legacy tickets to see what (in terms of tickets) it is they are supposed to be working on or which tickets they need to close. As a result, reports quickly back up in the queue and are soon to be rarely paid sufficient attention to. Similarly, I can imagine, it must sometimes be... confusing to see a large number of tickets still be open, or a large list of versions you never know existed, and the version or milestone of a ticket be set to something that doesn't really make sense. Ergo; something needs to be cleaned up. While some things I can just go ahead and do (after short deliberation within the Cyrus team, of course), *WE NEED YOUR HELP!* Let me emphasize that point; Without your help, all I can do is close the tickets and wait for people (probably some of you) to reopen them... Here's some of the things I can do and have done to make it easier on all of us; - I've cleaned up the list of versions by accumulating versions the Cyrus team is unlikely to pay much attention to, into series of versions. Ergo, 2.1.1 through 2.1.13 have all become one single '2.1.x' entry. Note that this includes the 2.2 series, while 2.3 has been released ~5 years ago. Same goes for milestones. - I've installed the BugzillaReports extension for Mediawiki, so we can easily create lists of bugs and share those lists. See, for example, the following Mediawiki page; http://www.cyrusimap.org/mediawiki/index.php/Bugs_Resolved_in_2.4.1 You can help us make Bugzilla just a little bit more accessible! Here's how (the contents of lists are updated instantly); - A list of Open Bugs per legacy version has been created: http://www.cyrusimap.org/mediawiki/index.php/Bugzilla_Cleanup - A description of what you can do with such reports is described here; http://www.cyrusimap.org/mediawiki/index.php/Bugzilla_Cleanup_Guidelines We're talking a list of 174 bugs at the time of this writing. What we would like to see, is have this list be smaller. If you have any questions (like, any, whatsoever, somewhat related to Cyrus), please feel free to drop me a line, or contact my directly in the #cyrus IRC channel on irc.freenode.net. Thank you, in advance or in retrospect or both, for your contribution(s), Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Coding standards / spaces vs tabs
Bron Gondwana wrote: This is the kind of thing that we really need coding standards for. The prevailing spacing standard for the cyrus codebase is: Yup we do, I had looked for the doc. you mentioned Greg had made but I couldn't find it. Sorry to have jumped the gun on this! Now I happen to think this is approximately one of the shittiest coding standards ever - but unless we all agree to change it, rewriting functions to use different indentation styles is counterproductive. Can I motion we change it up? Whether it be 4 spaces to a tab or 4 tabs to a space, I don't care, but I love it to be consistent across all files and definitely on one single line. So please don't. I won't do it again that's for sure. P.S. Here's what I use in vim to make life worth living. It makes tabs visible, and lets me indent blocks quickly by 4 characters while creating the correct tabs. set tabstop=8 set softtabstop=4 set shiftwidth=4 set listchars=tab:- set list! I think whatever coding standard we come up with can be in these little vim footers you see every once in a while, right? FWIW, I would then also like to :argdo normal gg=G over all files to correct the indentation best vim can. FWIW^2, I would then also like to remove all trailing spaces sed -r -i -e 's/\s*$//g' `find * -type f` style. FWIW^3, I would then also like to remove all CVS Id/Exp headers sed -r -i -e '/.*Id.*Exp.*\$\s*$/d' `find * -type f` style Regardless, any of such changes will have to freeze up development for just a moment, since rebasing / merging any pending changes will most likely not succeed after these changes have been pushed. Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Supported Platforms (was: Re: Cyrus IMAPd 2.4.1 Released)
Matt Selsky wrote: Can we decide on a list of supported platforms and document it, ala http://k5wiki.kerberos.org/wiki/Supported_platforms Then we can at least compile-test on these platforms. And maybe even upgrade-test these platforms before each release? I'm working on a documentation effort beyond what would be feasible to include in imapd.conf(5), which would include such list of supported platforms. I'd think a Wiki article would serve similar purpose as well. Similarly, Bugzilla would need to be updated to reflect a final list of platforms supported. Can I get a show of hands who's platform is not listed in the following list to log a ticket in bugzilla (preferred) and/or nudge me (bugzilla will nudge me either way)? - FreeBSD - Linux - Solaris In the interest of full disclosure, my work-in-progress is currently at[1,2]. Noted that if you do add or suggest a platform to the list, we might/may/will request the reporter of such bug to provide our release engineer(s) with access to such box, or inclusion to Hudson CI when we're good and ready. Thank you in advance, Kind regards, [1] http://www.cyrusimap.org/~vanmeeuwen/cyrus-imapd-2.4- docs/Deployment_Guide/html/sect-Deployment_Guide- Supported_Platforms_and_System_Requirements.html [2] http://git.cyrusimap.org/cyrus-imapd-docs/ -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
#cyrus IRC Channel on FreeNode
We found chatting on IRC has just that little more bandwidth available for a conversation as opposed to mailing lists and/or bugzilla, so we would like to invite you to join us on IRC if you're interested; Network: FreeNode (irc.freenode.net) Channel: #cyrus Talk to you later! ;-) Kind regards, -- Jeroen van Meeuwen Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Nov 8th, 21:00 UTC (was Re: IRC Meeting on 2.4.3 Release Mon, Nov 8th, 11:00 UTC)
On Saturday, November 06, 2010 01:46:48 am Bron Gondwana wrote: On Fri, Nov 05, 2010 at 03:14:47PM +0100, Jeroen van Meeuwen (Kolab Systems) wrote: 1) a little after mid-night UTC, closing in on the end-of-business in the U.S., and closing in on the start-of-business in Australia, or; I'm teaching a 6:15am gym class on Tuesday morning Australian time - I should be out and showered by 7:30 and on a train to work in time to get in the office about 8am. We're currently UTC+11 here, so that's actually 21:00 UTC. That sounds like a pretty viable time for a release meeting, since it's still business hours in all of the US, so it's just the Europeans who need to be around outside business hours :) Excellent, that'll work out for me just fine. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Cyrus IMAPd 2.4.4 Released
We are pleased to announce the immediate availability of Cyrus IMAPd version 2.4.4. This is a stable released in the 2.4 series, containing a mere 5 bug-fixes since version 2.4.3, released two days ago. Particular focus of this release has been paid to upgrade paths, for which many of our users have contributed feedback -with a special thanks to Bron Gondwana for making the fixes happen. The URL for this release is: ftp://ftp.cyrusimap.org/cyrus-imapd/cyrus-imapd-2.4.4.tar.gz We recommend that all users of earlier Cyrus IMAP versions in 2.4.x series update to this release. We also recommend users of earlier Cyrus IMAP versions (2.2.x, 2.3.x) test the upgrade path to version 2.4.4 for its seemless upgrade capabilities, and report issues to us through Bugzilla: http://bugzilla.cyrusimap.org/enter_bug.cgi?product=Cyrus%20IMAPversion=2.4.4 A list of bugs logged and fixed in version 2.4.4 can be found on our Wiki: http://cyrusimap.org/mediawiki/index.php/Bugs_Resolved_in_2.4.4 Questions and comments can be directed to our mailing lists: info-cy...@lists.andrew.cmu.edu (public list) or cyrus-devel@lists.andrew.cmu.edu (development) We also like to invite you to join us on IRC: #cyrus on the FreeNode network. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08 signature.asc Description: This is a digitally signed message part.
2.5 snapshot
Just in case anyone's interested, I've uploaded a snapshot release of current master; http://www.cyrusimap.org/releases/snapshots/cyrus- imapd-2.5.git201102032031.tar.gz Note that it is very easy to create one yourself as well: $ git clone git://git.cyrusimap.org/cyrus-imapd $ cd cyrus-imapd $ make snapshot That said, I doubt it's valuable for us to provide the snapshots, unless of course somebody tells me they're awesome and we need to continue providing them. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: RFC patch: Deny removal of folder owner ACLs
Guilherme Maciel Ferreira wrote: Hi, here is the patch against the master. Applied and pushed, with full attribution, thanks! Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Tuning some defaults for 2.5
Hi there, I would like us to take under consideration, changing the default value for the following configuration values for Cyrus IMAP 2.5; The purpose is to improve how Cyrus IMAP works out-of-the-box, and provide defaults that are more likely to not require change. That said, as some of these have a significant impact on runtime environments, which is part of the reason to consider the defaults as well, we'd like to motivate either keeping or changing the default in our documentation. Should we change any of these settings, three things are going to happen; 1) The default is changed for 2.5 (e.g. git's master branch), 2) The new default, rationale and any consequnetial operational change is documented in the Release Notes for 2.5 3) Documentation is provided to; 3.a) Configure a 2.5 server to keep the old behaviour, and 3.b) Deal with the new default (e.g. upgrade the environment to match with the new default), where appropriate -think for example improved_mboxlist_sort ** Your feedback is much appreciated, not to say required ** In no particular order then alphabetical: - allowallsubscribe to 1 I'm considering situations in which not to allow users to subscribe to folders on other servers specific more so then situations in which users are allowed to do so. - allowusermoves to 1 I'm considering preventing RENAMEs and XFERs more of a niche then allowing them by default. - altnamespace to 1 Perhaps only of esthetic value, but let's use the IMAP namespace by default. - hashimapspool to 1 Cyrus IMAP ships with this default on most Linux distributions already, if I recall correctly. - improved_mboxlist_sort to 1 This setting really reads as one that still has the default set to 0 because the upgrade path is not without conversion / manual intervention. I suppose it is more feasible to use the more correct sorting by default, and have those environments that can't or won't dump/convert/undump their mailboxes.db toggle this setting to 0, then vice-versa. - normalizeuid to 1 Somewhat related to username_tolower, in that both lowercase the username, but stripping the blanks by default sounds like a good thing to me. - statuscache to 1 Cache, hell yeah, woohoo! More cache! - unixhierarchysep to 1 This seems to be used so often, because it allows dots to be used in the mailbox and folder names, that I can only conclude (re-)considering the default does it justice. - username_tolower to 1 Not sure I can think of an environment I've been in that used case-sensitive usernames. - virtdomains to userid I'm just throwing this one out there to see if y'all are still awake ;-) It seems to me it has little impact on environments without virtdomains, while allowing what is required in environments with virtdomains set to on. Neither is the same thing, of course, but I think virtdomains: userid perhaps is a better starting point for new environments. If you have other configuration settings for which you would like us to consider changing the default value, please let us know! Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: backport config: make database paths configurable
Øyvind Kolbu wrote: Hi, Hi Oyvind (apologies for bastardizing your name like that), I have a couple of remarks, inline. can 8a23d03fa441f76026e923ac86533d6f85b11064 be backported to 2.4? If so strconcat must be backported as well, that is commit 3f1cc0a7f1e7afb0e52a99d27982a7d94d9bff85. 8a23d03fa441f76026e923ac86533d6f85b11064 seems to change the default database technology for the ptscache, which should not happen in a stable release. This we can overcome, but I would have appreciated you pointing this out alongside your request for the backporting of said commit ;-) That said, for the records to reflect this request, and for things to show up on the radar for the appropriate people (e.g. me in this case), please consider closing #3185 with almost exactly the message you have now sent to the mailing list, and set the 'releng-backport' flag (upper right corner on the bug) to '?'. Procedural detail, I know, but it aids us in various areas. All of the former aside, noted I have no objections, I'm going to let Bron have his say on this. We are running with both patches applied against 2.4.8alpha1, and it works as expected. 8a23d03fa441f76026e923ac86533d6f85b11064 introduced some typos in lib/imapoptions, which is fixed in attached patch. Thanks! Applied to master and pushed out with, of course, full attribution. #3185 can now be closed. Done. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: backport config: make database paths configurable
Øyvind Kolbu wrote: On 2011-04-07 at 16:47, Jeroen van Meeuwen (Kolab Systems) wrote: This we can overcome, but I would have appreciated you pointing this out alongside your request for the backporting of said commit ;-) Sorry, didn't notice. We don't use ptsloader. No problem, it is easy to overlook ;-) That said, for the records to reflect this request, and for things to show up on the radar for the appropriate people (e.g. me in this case), please consider closing #3185 with almost exactly the message you have now sent to the mailing list, and set the 'releng-backport' flag (upper right corner on the bug) to '?'. Procedural detail, I know, but it aids us in various areas. I tried to do that no, but was unable to do so. Perhaps because you closed it? Perhaps that's a permission problem. I'll look into it. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +316 42 801 403 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Automatic database coversion script anyone?
Bron Gondwana wrote: I really, really wanted to have BDB able to handle in-place upgrades as well, but I haven't been able to figure out a way. There is an -upgrade method on databases, but it says it's destructive and can fail If I understood it correctly, I think it actually says that the process may not be interrupted in any way, or the database is damaged in a way that cannot be repaired? That's something else then destructive by definition ;-) I suppose it's easy enough to copy the database to a backup, upgrade the live but if it fails (e.g. the upgraded version is corrupted), try again with the backup copy? Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeu...@kolabsys.com t: +44 144 340 9500 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Cyrus 2.4.9 Released
Rudy Gevaert wrote: I'm sure I'm not the only one that is appreciating the hard work Opera is doing on cyrus. Me too! Me too! Me too! Very glad to have Bron and Greg on board! Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Cyrus Imap and Automake
Ondřej Surý wrote: On Wed, Aug 3, 2011 at 17:59, Jeroen van Meeuwen (Kolab Systems) vanmeeu...@kolabsys.com wrote: Thomas Jarosch wrote: Hi Дилян, here's some feedback about your build system question. Note: I'm not one of the cyrus core developers. if I rewrite the build system of Cyrus imap 2.4(.10) to use Automake to generate the Makefile.in-files, will the patch be accepted in reasonable time in git/master? Have you considered alternatives to GNU Autotools? We have experience with GNU Autotools in our company projects as well as open source projects for several years now. We have found that it has several shortcomings: 1. Autotools version conflicts You can compile a released source package without any Autotools on your system. But as soon as you a) want to develop b) want to install a patch which modifies the build system (like a new path to a library, something that adds a new file,...). This is often happens as part of packaging for .rpm or .deb. you need Autotools on your machine. If the Autotools version on your machine and the one used to build the release are not compatible you can't build. Installing a different Autotools version on a given distribution without breaking something or fixing a huge list of dependency problems is nearly impossible. I have experience with this... I have quite the experience with .rpm and .deb building myself as well, and while I agree autotools *can* be problematic at times, I recon the Linux distributions are not the biggest of problems - the culprit, I think, is with the number of custom / site-specific builds out there, ranging from Sun Solaris to FreeBSD and who knows what versions of autotools are on these systems. With my fancy debian maintainer hat on - I agree, we learnt how to cope with different versions of autotools, that's the minor thing. I personally I would love to have cyrus projects automakized. It's much easy to mangle :). Between the two of us, Debian and Fedora maintainers, are we both saying yes please, no objections? Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Input on patch for ptclient/ldap requested
Hi there, I wanted to ask who is actively using ptclient/ldap, as I have some inhouse patch pending on the canonification using some sort of result_attribute, if you will. We currently have under consideration whether everything, life and the universe should be configurable before the patch is accepted upstream, which is to say (pardon my postfix lingo); - result_attribute_format, - leaf_result_attribute, but also; - group_filter_scope, - group_result_attribute Which is to say, we have a deployment extensively using 'nsroledn' -which functionally behaves like a 'memberOf', and the question then becomes if you want to use the 'cn' attribute for groups -which most often is not enforced to be a unique attribute value for groups, but is automatically unique is the search scope for groups is 'one' and the 'cn' attribute builds the 'rdn'. Long story short, I would like to know of other people who use ptclient/ldap, or have attempted to do so but failed, and the various use-case / deployment scenarios. Thanks in advance, Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Todo lists, 2.4.11 and 2.5 preview
Sébastien Michel wrote: PS: I thought to write regression tests with Cassandane. Is this the future Cyrus framework for functionnal testing ? PS^2; I would also like some input for those of us that enjoy playing, researching, testing and documenting ;-) Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Subject : git best practices
Olivier ROLAND wrote: 2011/8/19 Henrique de Moraes Holschuh h...@debian.org: IMHO, if you're going to come up with rules for that, better do as it is done in the Linux kernel or in git itself. The rules are: * detailed changelogs on every commit * only cleaned up commits are sent up for merging in the mainline. The cleaned-up commits are not large squashes (those are impossible to understand), instead they should look like you did everything perfectly at the first try, but step-by-step. * Bissectability is very important and should be preserved, so the tree should build _and run_ at every commit. * Don't rebase topic branches on top of the latest mainline, unless you actually need to do it for it to run. Base topic branches on stable points of the mainline. Sounds good to me. Bron, you ask us to just point you to a git branch with our patches. The problem is that sometimes we discuss things and the branch become a development branch with (useless for master) history. Maybe it would be more convenient if we can work with pull request like on github : http://help.github.com/send-pull-requests/ If genuinely a topic-branch, try the following: $ git checkout --track -b dev-something origin/master $ git config branch.dev-something.autosetuprebase always ^ I have this in my ~/.gitconfig to apply globally, BTW $ make edit #1 $ git commit $ send patch to mailing list / attach to ticket $ receive feedback, further edits $ git commit (possibly git pull rebasing your working copy changes) $ git diff [options] origin/master [-- files and paths] or $ git diff dev-something...master For what [options] can give you (such as -c and --cc). Is that acceptable? Should we document such (with comments and feedback appreciated!)? Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Cyrus CalDAV design decision
Dave McMurtrie wrote: Good day, Ken, Bron and I have had various disjointed conversations about where CalDAV data should be stored. We're getting to a point where we really need to finalize that design decision, so I'm soliciting feedback here. The current Cyrus CalDAV code stores DAV resources as subfolders of a user's INBOX. For example, user.dave64.#calendars and user.dave64.#addressbooks. Under each resource, of course, may be additional sub-resources like user.dave64.#calendars.Work. This was fairly easy for Ken to implement, so he did so as a proof of concept without much thought beyond that. One issue, however, is that IMAP clients have access to these mailboxes and it would likely be a system administrator's nightmare to roll something like this into production. First: Does the Cyrus CalDAV daemon / interface require to use the same configuration / runtime data (imapd.conf, config and partition directories)? Second: The IMAP clients to a Kolab Groupware solution, heavily dependent on Cyrus IMAP, uses annotations to indicate the folder type, for example; INBOX.Calendar - /vendor/kolab/folder-type value.shared event In these folders we store email messages that have .xml attachments. The .xml attachments are the actual 'object'; an event, a contact, etc. A 'dumb' IMAP client will see a message with a .xml attachment. The message body in our case contains a small note on the object being a Kolab object, and a link to a page listing 'Kolab Smart clients'. We find that we have to make many potential Kolab clients 'smart', however. I suppose, with CalDAV integrating into Cyrus IMAP in the foreseeable future, Kolab Groupware will want to make use if these additional capabilities as well. We have previously found iCalendar to have a number of serious restrictions and/or flaws, however, so I'm not certain what/who/where/how on our side. Anyway, perhaps the way we have successfully used for the past ~7 years inspires someone to come up with a way that works for all of us. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Cyrus CalDAV design decision
Ken Murchison wrote: Bron Gondwana wrote: On Tue, Aug 23, 2011 at 02:44:46PM -0400, Dave McMurtrie wrote: 3) Store DAV resources in a separate hierarchy like the DELETED hierarchy. I think Ken and I initially liked this idea, but the more we talk about it, the more it seems like this is the hardest to implement and we can't remember why we thought it was a good idea. Also, I think Bron suggested that he'd like to move away from having the DELETED hierarchy at some point. I'm pretty sure we were at a bar when we discussed this, which may explain why my memory is so foggy on the details. I actually like this best - put it in a separate namespace at the top level, like: addressbook.brong addressbook.brong.Work calendar.brong calendar.brong.Work This could also be hooked in with altnamespace more sensibly, and even advertised as separate namespaces or suppressed to IMAP clients completely. Where would shared mailboxes reside? I don't know if there is a viable use case for shared mailboxes, (...snip...) There's definitely a use-case for shared calendars (think resources such as conference rooms, beamers and cars). Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Cyrus CalDAV design decision
Georg C. F. Greve wrote: On Wednesday 07 September 2011 09.29:48 Georg C. F. Greve wrote: 10. The /vendor/kolab/folder-type annotation should be updated now that SPECIALUSE has been made an RFC I think that is a very good idea. This should likely be added to KEP 9, which is still in drafting stage, so easy to extend. I've taken a look at the RFC today. It seems to only support the /private/ namespace. That would not work for the folder-type annotation, because the type needs to be visible also to other users with which this folder is shared. Or am I missing something and there is a /public/ annotation as well? It uses SHOULD in where I could find a reference to the private rfc 5464 namespace. It thus allows for the shared namespace to be used as well, in my interpretation of the word SHOULD as described in rfc 2119. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
SPECIAL-USE (was: Re: Cyrus CalDAV design decision)
Bron Gondwana wrote: On 09/07/2011 03:29 PM, Jeroen van Meeuwen (Kolab Systems) wrote: Georg C. F. Greve wrote: On Wednesday 07 September 2011 09.29:48 Georg C. F. Greve wrote: 10. The /vendor/kolab/folder-type annotation should be updated now that SPECIALUSE has been made an RFC I think that is a very good idea. This should likely be added to KEP 9, which is still in drafting stage, so easy to extend. I've taken a look at the RFC today. It seems to only support the /private/ namespace. That would not work for the folder-type annotation, because the type needs to be visible also to other users with which this folder is shared. Or am I missing something and there is a /public/ annotation as well? It uses SHOULD in where I could find a reference to the private rfc 5464 namespace. It thus allows for the shared namespace to be used as well, in my interpretation of the word SHOULD as described in rfc 2119. It also provides a short list of allowed special uses... I think it needs to be extended either way, and the extended version could allow shared :) I think the RFC for SPECIAL-USE lacks the suffix '.default' we currently use in /vendor/kolab/folder-type (example value 'event.default') to indicate which folder is the default folder to save new messages that have a calendar event XML object attached to it. I'm not sure the extension options listed in the RFC allow the addition of '.default' without providing additional requirements, recommendations and rationale. Also, suppose we were to propose \Calendar[.default], I'm not sure that appropriately indicates in what format the folder contents are expected to be in. Perhaps a \calendar[.kolab[.default]] is justified, where the Cyrus IMAP CalDAV folder could have a \calendar.ical -or somesuch. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: SPECIAL-USE (was: Re: Cyrus CalDAV design decision)
Jeroen van Meeuwen (Kolab Systems) wrote: I think the RFC for SPECIAL-USE lacks the suffix '.default' we currently use in /vendor/kolab/folder-type (example value 'event.default') to indicate which folder is the default folder to save new messages that have a calendar event XML object attached to it. I'm not sure the extension options listed in the RFC allow the addition of '.default' without providing additional requirements, recommendations and rationale. Also, suppose we were to propose \Calendar[.default], I'm not sure that appropriately indicates in what format the folder contents are expected to be in. Perhaps a \calendar[.kolab[.default]] is justified, where the Cyrus IMAP CalDAV folder could have a \calendar.ical -or somesuch. Continuing my line of thought, naturally there has not to be a divider such as a '.'; one can set multiple SPECIAL-USE attributes to a folder, such that: - A Cyrus IMAP CalDAV folder containing iCalendar data could have SPECIAL-USE attributes: \Calendar \iCal [\Default] \iCal could be the defined default for a folder marked with SPECIAL-USE attribute \Calendar. - A Kolab Groupware calendar folder could have SPECIAL-USE attributes: \Calendar \KolabXML [\Default] I suppose the requirements and guidelines for combinations of SPECIAL-USE attributes and client implementations are something significant enough to consider, though I'm not sure I can find the section in the RFC that allows requesting new entries the list SPECIAL-USE attributes. Would this new proposition, if we can form some consensus on the topic, be a new RFC to extend 6154? Can somebody more familiar with IETF processes help me out here? Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Cyrus 2.4.11 Released
Bron Gondwana wrote: We are pleased to announce the release of Cyrus IMAPd 2.4.11. This is a stable release in the 2.4.x series. It contains a security fix to issue CVE-2011-3208, a remotely exploitable buffer overflow in the nntpd daemon. This release also contains fixes for quite a few bugs found since the release of 2.4.10, and adds some new features. Packaged up and pushed out to Fedora 15, Fedora 16 and Fedora Rawhide. Also available for Enterprise Linux 5 -by the end of the day- through; http://mirror.kolabsys.com/pub/redhat/kolab-2.4/el5/development/ And for Enterprise Linux 6 -by the end of the day- through; http://mirror.kolabsys.com/pub/redhat/kolab-2.4/el6/development/ Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: SPECIAL-USE (was: Re: Cyrus CalDAV design decision)
Georg C. F. Greve wrote: On Thursday 08 September 2011 10.19:53 Jeroen van Meeuwen wrote: - A Cyrus IMAP CalDAV folder containing iCalendar data could have SPECIAL-USE attributes: \Calendar \iCal [\Default] \iCal could be the defined default for a folder marked with SPECIAL-USE attribute \Calendar. - A Kolab Groupware calendar folder could have SPECIAL-USE attributes: \Calendar \KolabXML [\Default] Indeed, although I still see the bigger issue with the /shared/ and /private/ namespaces. You'd definitely want \Calendar \iCal \KolabXML to live in the /shared/ namespace, but \Default for instance should live in the /private/ namespace, preferably. Right now the RFC only mentions the /private/ namespace It does that because it seems to only concern the list of defined attributes (and doesn't open the floor for new attributes to be added to the list?). Hence I was asking who was more familiar then I am with the processes in IETF to see what, if anything at all, we can do about it -including some of the other things that would then be required, such as valid or certain combinations of SPECIAL-USE attributes requiring or recommending clients treat such folder in a certain way, and conflict resolution / prioritization of private, shared and for cases where multiple folders are marked with some SPECIAL-USE attribute such \Drafts. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Slipstream of #3553: Execute Cassandane functional tests against deployment
Quoting me and Greg from the slipstream of; https://bugzilla.cyrusimap.org/show_bug.cgi?id=3553#c3 Greg Banks wrote: Jeroen van Meeuwen wrote: That said, I'm going to want to be able to have Cassandane run against an existing Murder deployment... Cassandane is designed to set up it's own Cyrus instances rather than run against existing ones. This was entirely deliberate and there's a bunch of reasons for it, but I think that predictable and reproducible setups, and the avoidance of state leakage between tests, are sufficient reasons. Cassandane test cases are by design capable of starting multiple Cyrus instances and controlling their configuration files to connect the instances in any way necessary; this is how we test replication scenarios. It was always my intention to add murder support. The tricky bit is how do we convince Cassandane to go run test X once for single instance and then once again for a murder? I think we could take a different approach. Excuse my French as I'm not a terminology guru, and apologies if the entire concept is crazy / way out of scope; I'm just thinking out loud. Because the alternative is to have two copies of every test that needs to be run against a murder, which would be just nuts. I'm not entirely sure what the best approach is for this, but I'm betting that something like parameterised tests is the Right Way...except that Test::Unit doesn't seem to support those. The tests that can be executed against a deployed environment are a type of tests usually qualified as functional tests, or at least a sub-set thereof, right? If running those tests could be provided the configuration of a deployed instance of Cyrus IMAP(single-server?), - admin credentials, - sample user credentials, possibly a set of login names for the same user (uid, mail, alias, ...) - Cyrus IMAP server IP / DNS name (or 'localhost') a variety of (functional) tests could be executed for a deployed instance rather then a snapshot of the codebase. I would just execute this against a deployed frontend. The unit tests can be performed using a local checkout of the tag used to built the deployed package. Later on perhaps such set of functional tests could be extended to perform murder specific stuff (MUPDATE capability would display part of the murder topology?) such as transfer, rename, setquota, reconstruct against frontend, and many, many other things. -- Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: RSS support in Cyrus
On 19.09.2011 18:07, Bron Gondwana wrote: On Mon, Sep 19, 2011 at 12:37:52PM -0400, Ken Murchison wrote: +1 for RSS (obviously) I think if it's going into production it should go onto the roadmap for 2.5 as well, unless we get 2.5 out before then. Otherwise the development effort of keeping it separate is too much work. And +1 for RSS - we want it too. Hey Bron, given you want it too, what are your thoughts on branching off 2.5 in a timely fashion? That would be not all too soon from now, perhaps even as soon as we can declare database formats final -which would get it to a state that can be released in the form of development snapshots pretty early in the stabilization cycle before the .0 release, no? From what Dave was saying, your first implementation is going to be on top of 2.4 to get a stable base for running by the end of this year. Good choice :) I'm really hoping to have things nailed down for 2.5 soon. I'll send a separate email to cyrus-devel to flesh out some things we discussed in the IRC channel once I've researched a bit more. Part of the answer to my previous question, I suppose, is going to the list :P Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: RSS support in Cyrus
On 19.09.2011 23:25, Alexey Melnikov wrote: Sounds like a good idea. But my question is: which version(s) of RSS? I suppose different URI handlers could spit out different version(s) of RSS. Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: RSS support in Cyrus
On 21.09.2011 15:01, Ken Murchison wrote: Alexey Melnikov wrote: Sounds like a good idea. But my question is: which version(s) of RSS? RSS 2.0: http://www.rssboard.org/rss-specification Hi Ken, I'm not aware of what Outlook allows in terms of configuration, but would -in the future- a different URI for different RSS formats be possible? Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: SQL database backend
On 11.10.2011 00:04, Bron Gondwana wrote: Hi, Is there anyone out there using the SQL backend in production? Would you be really sad if I redesigned it? Were/Are you planning to include a database schema beyond key-value blobs? Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Bugzilla (was: Re: [PATCH] Fix incorrect readlink() buffer handling)
On 2011-10-13 11:38, Thomas Jarosch wrote: On Thursday, 13. October 2011 12:15:17 Dave McMurtrie wrote: @Dave: Any issue known with bugzilla? Not that I'm aware of. Why do you ask? If there's something broken, let me know and I'll take care of it. I tried to create a new bug (twice) and it started processing after hitting submit. Shortly after that I only got a blank page and no bug was created. If other people still can enter new submissions, then something is messed up here locally. So just sit back and watch if new submission come in :) I was able to successfully create a new ticket: https://bugzilla.cyrusimap.org/show_bug.cgi?id=3580 Kind regards, Jeroen van Meeuwen -- Senior Engineer, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: I/O stats for lmtp, imap and pop
On 2011-07-13 8:43, Olivier ROLAND wrote: Thanks, you are perfectly right :-) 2011/7/13 Wesley Craig wescr...@columbia.edu: I'd think you'd want to add this to telemetry_rusage(). Seems like you can get this data from getrusage() since kernel 2.6.22? Hi there, apologies for bumping the old topic, but in preparation for what I think I want to have included in 2.5 (now the master branch still), I successfully merged the two branches. I can, and would like to, push this out to the git.cyrusimap.org/cyrus-imapd master branch for inclusion as is in 2.5, if there's no objections. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: I/O stats for lmtp, imap and pop
On 2012-01-06 17:47, Bron Gondwana wrote: On Fri, Jan 06, 2012 at 05:15:48PM +0100, Jeroen van Meeuwen (Kolab Systems) wrote: I can, and would like to, push this out to the git.cyrusimap.org/cyrus-imapd master branch for inclusion as is in 2.5, if there's no objections. Sounds good. Done. Apologies for not following up on that thread. Don't worry about it, it gives me something to do ;-) Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Cassandane Troubles
Hello there,
I'm trying to run Cassandane on my laptop, because I'm interested in
writing tests and automating the execution thereof.
Having followed the instructions in doc/setting_up.txt -liberally, I
must admit-, I notice that;
- While Cyrus IMAP is installed with a proverbial './configure; make;
make install DESTDIR=/var/tmp/cyrus-imapd-2.4/', and the binaries
therefore end up in /var/tmp/cyrus-imapd-2.4/usr/cyrus/bin/, the
configuration that Cassandane writes out refers to binaries in
/var/tmp/cyrus-imapd-2.4/bin/, a directory that does not exist.
Please find that:
$ cat /var/tmp/cass/cassandane/conf/cyrus.conf
START {
# integrity check and setup of databases
recover cmd=ctl_cyrusdb -C /var/tmp/cass/cassandane/conf/imapd.conf
-r
}
SERVICES {
imap listen=127.0.0.1:9100 cmd=/var/tmp/cyrus-imapd-2.4//bin/imapd
-C /var/tmp/cass/cassandane/conf/imapd.conf
}
and;
$ cat cassandane.ini | grep -v ^#
[cassandane]
rootdir = /var/tmp/cass
[cyrus default]
prefix = /var/tmp/cyrus-imapd-2.4/usr/cyrus
[gdb]
[config]
cyrus_prefix = /usr/cyrus
prefix = /var/tmp/cyrus-imapd-2.4
- When /var/tmp/cyrus-imapd-2.4/bin/ is created a symbolic link for, to
/var/tmp/cyrus-imapd-2.4/usr/cyrus/bin/, Cyrus IMAP / Cassandane
ultimately fails authenticating. I recon this is a Cyrus SASL thing, but
I was wondering whether Cassandane requires the system to have a valid
SASL configuration with an 'admin' user, whether any additional users
would be required, and whether it could be made so that no such
system-wide configuration is required (by starting an SASL auth daemon
with a different user database then the system database?).
Kind regards,
Jeroen van Meeuwen
--
Systems Architect, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
Re: Cassandane Troubles
On 2012-01-09 4:29, Greg Banks wrote:
Having followed the instructions in doc/setting_up.txt -liberally, I
must admit-, I notice that;
- While Cyrus IMAP is installed with a proverbial './configure;
make;
make install DESTDIR=/var/tmp/cyrus-imapd-2.4/', and the binaries
therefore end up in /var/tmp/cyrus-imapd-2.4/usr/cyrus/bin/, the
configuration that Cassandane writes out refers to binaries in
/var/tmp/cyrus-imapd-2.4/bin/, a directory that does not exist.
It seems that this feature is currently broken, sorry :(
However if you follow section 4 in that document precisely, it should
work.
I'd love to, but I don't want to mess with the system installation on
'/', if you will. I also want to be able to do the same thing over and
over again with multiple versions of Cyrus IMAP. I recon just creating
the symbolic link /var/tmp/cyrus-imapd-2.4/bin/ -
/var/tmp/cyrus-imapd-2.4/usr/cyrus/bin/ is the workaround for now then.
$ cat /var/tmp/cass/cassandane/conf/cyrus.conf
START {
# integrity check and setup of databases
recover cmd=ctl_cyrusdb -C
/var/tmp/cass/cassandane/conf/imapd.conf
-r
This is a bug, it should have been a full path to ctl_cyrusdb. Today
I
pushed some changes from a development branch which should fix that.
- When /var/tmp/cyrus-imapd-2.4/bin/ is created a symbolic link for,
to
/var/tmp/cyrus-imapd-2.4/usr/cyrus/bin/, Cyrus IMAP / Cassandane
ultimately fails authenticating. I recon this is a Cyrus SASL thing,
but
I was wondering whether Cassandane requires the system to have a
valid
SASL configuration with an 'admin' user, whether any additional
users
would be required, and whether it could be made so that no such
system-wide configuration is required (by starting an SASL auth
daemon
with a different user database then the system database?).
Cassandane relies on the no-password hack in libsasl which is enabled
using
sasl_pwcheck_method: alwaystrue
which might not be available on your build of libsasl?
Cool, that's good to know. I'll find it out and make sure Fedora + RHEL
will have this in the future.
Kind regards,
Jeroen van Meeuwen
--
Systems Architect, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
t: +44 144 340 9500
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
Fedora / Red Hat / CentOS dependencies (was: Re: Cassandane Troubles)
On 2012-01-08 13:00, Jeroen van Meeuwen (Kolab Systems) wrote: Hello there, I've packaged up the dependencies for Cassandane, so hopefully they'll be available as just packages shortly. For those of you that are packagers, I'd appreciate if you reviewed; - perl-BSD-Resource at https://bugzilla.redhat.com/show_bug.cgi?id=772617 - perl-News-NNTPClient at https://bugzilla.redhat.com/show_bug.cgi?id=772618 - perl-Encode-IMAPUTF7 at https://bugzilla.redhat.com/show_bug.cgi?id=772620 I've also enabled the alwaystrue sasl_pwcheck_method in Fedora Rawhide's cyrus-sasl package, but I won't be able to make this happen for existing Red Hat / CentOS releases I'm afraid, or not through the proper channels. I'll have those packages available through our Kolab repositories though, as we also require the httpform module to be available. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: FOSDEM Report - Saturday
On 2012-02-08 5:08, Greg Banks wrote: Sounds like you're having fun at FOSDEM :) Yes. On Tue, Feb 7, 2012, at 02:27 PM, Bron Gondwana wrote: http://fosdem.org/2012/schedule/event/keynotes_welcome [...] There are 420 talks, 273 hours of scheduled content. You can't see it all! As much as possible will be videoed. Have they announced the URL for the videos? I think the links are on every talk page on the FOSDEM schedule on fosdem.org. Saturday 11:00 - 2:20 - Mail Track = [...] Cyrus process: - Gerrit? Some sort of code review process to make it easier to keep track of the work from drive-by contributers. I see two benefits to Gerrit or something like it a) casual contributions don't get lost in the noise b) regular contributors get regular code reviews There's a couple more; - Jenkins can be put in charge of giving the first thumbs-up for the commit(-series) but letting a build using the commit(s) succeed, possibly along with Cassandane executing successful tests. - New contributors can be trusted to commit to what goes through Gerrit, lowering the entry barrier. Suppose Kolab Systems, for example, were to find a C programmer with some C experience, that person will obviously need quite some time to catch up with everything that happens within Cyrus IMAP. One can imagine how automating (a large part of) as well as requiring such a person's contributions be reviewed serves both parties quite well. - Bugzilla - use it for everything. If it doesn't have a bug number where discussion took place, it doesn't get accepted. It's a transparency thing, as well as a planning thing. Then afterwards, its a reporting thing. This is a major workflow change, and is probably more of a challenge for me than anyone else. Sounds good. Another thing I'd like to see is a push hook on git.cyrusimap.org's master and 2.4 branches, so that if a git commit has the text Bug in the subject then the commit message is copied to Bugzilla and the bug marked RESOLVED - FIXED. What this does is create properly hard dependencies between GIT branches and Bugzilla versions/milestones. This isn't a bad thing, but we are currently winging it between the two. It means every time a thing happens on the one, the thing needs to happen on the other as well, and properly so that the match between the two things can be made by such program. - Websites in git. Yes! - Release process - simplify to save the repeated typing involved. I wind up writing the changelog, the website release note and the email release note, plus manually changing a bunch of things in the website PHP every time I do a release. I wonder if DIstZilla or something like it could be used to automate the release process? I'm gonna go out on a limb and say Not Your Problem(TM) - it's why non-coders like me exist and can have some value to the project ;-) I'm strongly in favor of you just throwing things over the fence while we have not yet found a working solution in order for me to delegate creating releases in a way that makes everything happen properly in a variety of places. It allows you to focus on development (on master, and other development branches), and of course I'll be able to follow up with any of the developers if I get stuck backporting things or zapping bugs. Special-Use: (...snip...) Yes, our RFC compliance is basic here. Technically, it's been outlined what Kolab Systems is seeking to do here, and as it is not so much on the roadmap for other parties involved, we're therefore seeking ways to allow us to also actually do the work (instead of asking other parties to do the work for us). I'm looking forward to it, as currently I may have seemed to ill- / ambiguously define what it is we're trying to do exactly. E-Discovery, deletion controls: - Kolab are planning to use the msg bus stuff from Worldline to have a listener that collects data for e-discovery. Kind of a cyrus watcher. I'd love to have a more generic infrastructure for listening to changes in, and interacting with, the cyrus data model, on top of which we could implement the Annotator daemon, mupdate, and NOTIFY. We have a quite large but also long-standing agenda item that is in the realm of e-Discovery (legal shit about what the f happened), Archiving, Accounting and the more assisting sysadmins side of discovery (log aggregation and such to track mail flow), to name a few. ActiveSync: - MetaWays have a very good open-source ActiveSync stack: http://www.h-online.com/open/news/item/Tine-2-0-supports-ActiveSync-740315.html That seems to be another PHP implementation like Z-Push. Yes, but not entirely the same - Z-Push is basically tied down to one groupware product, even though it may seem not to be the case, whereas Metaways is launching their ActiveSync implementation as a separate, not-ruled-by-any-company project. * Undo.
Re: FOSDEM Report - Saturday
On 2012-02-10 4:26, Greg Banks wrote: On Thu, Feb 9, 2012, at 09:58 AM, Jeroen van Meeuwen wrote: Special-Use: Technically, it's been outlined what Kolab Systems is seeking to do here, and as it is not so much on the roadmap for other parties involved, we're therefore seeking ways to allow us to also actually do the work (instead of asking other parties to do the work for us). I'm looking forward to it, as currently I may have seemed to ill- / ambiguously define what it is we're trying to do exactly. What we're specifically looking for, paraphrasing a 30.000 ft high-level birds view, is xCal and xCard stored in IMAP folders that are also made available through CalDAV / CardDAV, Ok. where such folders are annotated through SPECIAL-USE, Sounds good. so that any IMAP client is automatically compatible. I'm not entirely sure what you mean by compatible. If a client understood Content-Type: application/calendar+xml and did something useful with it, like display the calendar, or start up a separate calendar tool, then it's compatible. If a client didn't understand that, a user opening that folder would see a bunch of XML gibberish, which is very nearly the moral equivalent of not showing anything at all, except more confusing. SPECIAL-USE already requires ENABLE SPECIAL-USE by the client, No it doesn't. RFC6154 says no such thing. Of course, you are right... I must have been seriously confused with something else. Well, some of the reasoning behind what I said would supposedly be possible to do were based on this assumption, actually. I suppose we'll need to take this into account on whatever update to the RFC we come up with. (...snip...) I've pointed Georg to your comments, they are much appreciated. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Build failed in Jenkins: cyrus-imapd-master #402
On 2012-02-20 11:44, Greg Banks wrote: Sent from my iPhone On 20/02/2012, at 19:13, Bron Gondwana br...@fastmail.fm wrote: On Mon, Feb 20, 2012 at 07:00:52PM +1100, Greg Banks wrote: 4) Cassandane sometimes leaves master and lemming processes lying around. I haven't been able to reproduce that problem, although I have solved it several times before. Those leaked processes are never cleaned up and hog the TCP ports that Cassandane expects to be able to use, causing subsequent Cassandane runs to fail spuriously. I'm not entirely sure of the best way to address this, but I'm thinking of something like a sledgehammer which kills all processes running as the cyrus userid. My 'cyrus-devtools' scripts use a sledgehammer that looks for the -C option that I set on all my processes. Yeah, I could identify master processes started by Cassandane, and service processes started by those masters, by looking for -C options. I might make --cleanup do that. There's also the opportunity to have Jenkins execute a 'sudo pkill -9 -u cyrus' at the start of a run, perhaps? Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com t: +44 144 340 9500 m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Cyrus IMAP 2.4.14 released
We are pleased to announce the release of Cyrus IMAPd 2.4.14. This is a stable release in the 2.4.x series. The release mainly contains bug fixes, mostly small but significant. Please find an overview of all bugs resolved in this release at: http://www.cyrusimap.org/mediawiki/index.php/Bugs_Resolved_in_2.4.14 You can download via HTTP or FTP: http://cyrusimap.org/releases/cyrus-imapd-2.4.14.tar.gz ftp://ftp.cyrusimap.org/cyrus-imapd/cyrus-imapd-2.4.14.tar.gz Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08 signature.asc Description: This is a digitally signed message part.
Re: Cyrus IMAP 2.4.14 released
On 2012-03-12 12:56, OBATA Akio wrote: doc/changes.html and doc/text/changes in released tarball are not updated to 2.4.14? (I don't know about other document files) Hi, You are right, these files have not been updated with 2.4.14 release information - we're missing a piece in our release process. These docs have not been updated for 2.4.13 either, now that I examine them a little closer. The information normally contained within those pages is available though; - http://www.cyrusimap.org/mediawiki/index.php/Bugs_Resolved_in_2.4.14 - http://www.cyrusimap.org/mediawiki/index.php/Bugs_Resolved_in_2.4.13 Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
What do we want from the Changelog?
Hi there, after my little screw-up not updating doc/changes.html, which has to happen manually, we're now in the realm of fixing this once and for all. We have the following sources for what changed: - git commit logs, - bugzilla entries, Sometimes, fixes are applied after one of our developers finds it (i.e. only occurs in git commit log) and sometimes a post to a mailing list triggers someone to fix one or the other thing. Obviously, the most complete changelog is the git commit log. Distilling this log in some sort of a readable format is perfectly doable; [user@system cyrus-imapd.git/] $ (GIT_DIR=.git git log $(git tag -l | grep -E ^cyrus-imapd-[0-9]+\.[0-9]+\.[0-9]+$ | grep -B 1 $(PACKAGE)-$(VERSION) | head -n 1)..$(PACKAGE)-$(VERSION) .changelog.tmp mv .changelog.tmp ChangeLog; rm -f .changelog.tmp) || (touch ChangeLog; echo 'git directory not found: installing possibly empty changelog.' 2) The list of bugs closed for a certain milestone is easily created as well: http://www.cyrusimap.org/mediawiki/index.php/Bugs_Resolved_in_2.4.14 I, personally, am in favour of using these lists of things as they are (maybe we can enhance the formatting a little but), but adding Release Notes to a release should anything need a little bit of text to accompany the change. Today, however, there's a doc/changes.html which needs to be updated as well - manually. The latter is where today's confusion comes from, really. I had failed to update that document (it wasn't listed in the things to do for a release[1]). Any thoughts/comments/gripes? Kind regards, Jeroen van Meeuwen [1] http://www.cyrusimap.org/mediawiki/index.php/Release_Processes_and_Tasks -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Cyrus SEEN and uidl patch
On 2012-04-01 10:25, ego...@ramattack.net wrote: Good morning, Good morning, Not speaking to the code changes itself, I have a couple of suggestions / questions. If you are familiar with GIT, could you please clone the git repository[1] and create a patch(-set) with git-format-patch? In case you're not all too familiar with GIT, you could, for example: $ git clone git://git.cyrusimap.org/git/cyrus-imapd $ cd cyrus-imapd $ git checkout cyrus-imapd-2.3 $ git checkout -b dev/pop3_uidl-patch (...make some changes...) $ git commit /path/to/changed/files (...make some more changes...) $ git commit /path/to/changed/files $ git format-patch origin/cyrus-imapd-2.3 And send the resulting -*.patch files as an attachment to an email, but better yet, a Bugzilla[2] ticket. If you would be able to do that, that'd be great. I'm running an ISP environment with Cyrus 2.3.18. Given that Cyrus IMAP 2.3 is... well... rather old, it is maintained by only a small group of people having volunteered to do so. The main focus for the project is Cyrus IMAP 2.4, and future releases currently in development, so a question that pops up in my mind is - can this issue be reproduced in Cyrus IMAP 2.4.14, or even GIT master (future 2.5)? Kind regards, Jeroen van Meeuwen [1] http://git.cyrusimap.org/cyrus-imapd [2] https://bugzilla.cyrusimap.org -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Automake Support for cyrus-imapd 2.5
On 2012-04-15 11:27, Bron Gondwana wrote: On Sun, Apr 15, 2012, at 01:31 AM, Дилян Палаузов wrote: Hello, at git://demo.aegee.org:143/cyrus-imapd.git , branch dpa/automake I have patched cyrus-imapd/master to support Automake . With the exception of perl and CUnit, I it works fine and permits building in a directory, different from the source directory. All dependencies are moved to a single Makefile.am (except CUnit and Perl). The point with CUnit is, that I did not manage to get it running on my system, so I left it as is. The Makefile's generated from Makefile.PL do not permit compiling perl files in a directory, different from the source directory. I wonder if we can change from Makefile.PL to one of the more modern Perl build systems... I reckon the Perl side of things is due a refactor anyways, no? I do not have the purify-tool, so I was not able to compile the imap/imap.pure, imap/imap.quant, imap/lmtpd.pure, imap/muptead.pure, imtest/imtest.pure, notifyd/notifyd.pure, ptclient/ptdump.pure, ptclient/ptexpire.pure, ptclient/ptloader.pure and timsieved/timsieved.pure targets. Those are not transferred to Makefile.am . I guess, the one who use those pure-things, can easily extend Makefile.am to support them. We don't use them. I wonder if anyone actually does :) Let's put a stake in the ground and remove them. make dist shall build .tar.bz2 including all files. However I have copied the dist-target from the old Makefile.in, removed all Makefile.dist files. The one who make tarballs/snapshots shall consider if it is wiser to use the Automake system to make tarballs, or the system used so far (with git). So or so, the xversion.h file needs to be generated by git. So long as it can be done easily, I dno't much care. Can configure.ac - ./configure please be made in charge of xversion.h? I'll log a ticket with this request. I hope you will like the result. Let me know, if you experience any (compilation) problems, after merging the changes. I will respond promptly. By the way, when approximately will be v2.5 released? We don't know for sure. I had a good chat with Greg in Melbourne last week about what still needs to be done. My plan is to build a set of bugs that need to be resolved, and make sure everything we're thinking about is on that list! Then we'll have a clearer idea. Perhaps it's time we call a meeting to discuss 2.5. Shall I announce tickets need to be created as per the #3669[1] model? Then if it isn't a ticket in Bugzilla it's not going to be planned. What do we think is a suitable timeframe for people to start creating tickets? [1] https://bugzilla.cyrusimap.org/show_bug.cgi?id=3669 Regardless, I think we should merge your automake code now, so we have a chance to test it for a while! Please let me work on this for a few, so we have some more eyeballs rolling over this. I propose we have Dilyan send in his SSH key for access to git.cyrusimap.org, and we set the distribution component to be assigned to him by default. Perhaps we rename the distribution component to build / autoconf / automake / autofoo? Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Let's map tickets to milestones
A conversation has been started on when Cyrus IMAP 2.5 would be released. So far, there's little certainty about it, but we can try and make it more visible by creating tracker tickets and logging every bug, enhancement and/or task into Bugzilla. If we were to give people a week or so to do all of this, then I reckon next week we could have a meeting to review everything we want to do for 2.5. So, I've created a 2.5.0 release blocker ticket; https://bugzilla.cyrusimap.org/show_bug.cgi?id=3674 Its dependency tree[1] indicates what needs to actually happen. Its direct dependencies are considered high-level bugs, enhancements or tasks we aim to complete before we release 2.5.0. For example, ticket #3669[2] is a high-level enhancement to convert to using autofoo. While we work on this, of course we run into issues, which we log as blocking #3669. Now, all tickets the depencency tree if #3669 auto-magically also become visible in the dependency tree for the release blocker ticket. All of the enhancement specific bugs need to be resolved in order for the feature to be complete. Another example enhancement, #3343[3] (conversations support), currently has no dependencies... but I'm sure these will pop up as the work on the feature moves back to origin and closer and closer to master. If you'll allow me to wing it as we go along, please don't hesitate and make your mark, as follows: - Log your new enhancement request and set Blocks: 3674. Please use version 2.5-next and milestone 2.5-next. - Take an existing enhancement request and set Blocks: 3674 - you can leave the version untouched but you are at liberty to set the milestone to 2.5-next, even if it is currently set to 2.4-next. - We would like existing bugs to mostly remain as they are, but feel free to set Blocks: 3674 to existing bug reports. - Add yourself to CC: on tracker ticket 3674 to receive a moderate amount of traffic as new dependencies are added and existing dependencies are removed/implemented. I plan on releasing development snapshots as work on enhancements is progressing, so that people can take an interest in a specific thing (before it's merged back into master?) and find whether or not it works for them. Ideas? Questions? Comments? Gripes? Kind regards, Jeroen van Meeuwen [1] https://bugzilla.cyrusimap.org/showdependencytree.cgi?id=3674 [2] https://bugzilla.cyrusimap.org/show_bug.cgi?id=3669 [3] https://bugzilla.cyrusimap.org/show_bug.cgi?id=3343 -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08 signature.asc Description: This is a digitally signed message part.
Re: Build failed in Jenkins: cyrus-imapd-master #542
On 2012-04-27 8:32, Greg Banks wrote: So it turns out there were several things wrong with the code in configure.ac which checked for LDAP. Most of these have been broken for a very long time. - we check for ldap_initialize() twice, pointlessly - if given --with-ldap i.e. with no argument, configure checks for a directory called yes and then generates -I/usr/local/include for LDAP_CPPFLAGS instead of an empty value - if the LDAP libraries are not detected, we get spurious non-empty values of LDAP_CPPFLAGS etc - if LDAP libraries are detected, the final Makefile gets a spurious extra copy of LDFLAGS - the recently added automake conditional never worked, as it tests for $HAVE_LDAP which is never set, because the AC_CHECK_LIB has a non-default action-if-set argument, so we compile against the LDAP headers but don't link against the libraries. So I fixed all those. Maybe *this* time it'll build... The AFSKRB checks are probably similarly broken; certainly the automake conditional is checking for a variable which is never set. But I'll let someone who cares fix that. Thanks Greg, for this fix - I had run into these a couple of times as well but had failed in my attempts to fix it. Seems you did it! Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
In preparation of Cyrus IMAP 2.5: autoconf and automake
Hello there, With many thanks to Дилян Палаузов dilyan.palau...@aegee.org, we would like to let you know about one particular feature now definitely included for a pending Cyrus IMAP 2.5 release. As a feature for the upcoming 2.5 release of Cyrus IMAP, though the exact schedule is yet unknown, we have merged into master the grand overhaul to using autoconf / automake. This marks a first significant milestone closing in on actually producing a 2.5 series release, but, and this is very important: NOT without your help. We would like those of you that have a need to or experience with building Cyrus IMAP from source to let us know whether the autoconf and automake (or, as I like to call it, autofu) Works For You(TM). To this end, we encourage you to clone the GIT repository master branch and attempt a build, or, alternatively, download the following snapshot release: http://git.cyrusimap.org/cyrus-imapd/snapshot/cyrus-imapd-2.5-snapshot- autoconf-and-automake.tar.gz The canonical build process we think applies, generally speaking, is: $ autoreconf -v $ ./configure [your-options] $ make This process currently requires autoconf = 2.67. We would appreciate you let us know whether or not such process works for you, preferrably though Bugzilla (please use product 'Cyrus IMAP' and component 'Distribution'). Thank you, in advance, On behalf of the Cyrus team, Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08 signature.asc Description: This is a digitally signed message part.
Re: Libtool and Support for Shared Libraries
On 2012-05-10 17:32, Bron Gondwana wrote: On Thu, May 10, 2012 at 04:03:26PM +0200, Дилян Палаузов wrote: Hello, While we're at it, I'd love to split things like 'mailbox.c' which are really libraries into a separate directory from 'imapd.c' which is a system daemon only run by master from cyrus.conf and separate again from cyr_dbtool.c which is a tool designed to be mainly run by humans. I would suggest, that the sources of libcyrus are moved to src/lib/cyrus, the sources of libcyrus_min are moved to src/lib/cyrus_min, the sources of libimap are moved to src/lib/imap, the sources of libsieve are moved to src/lib/sieve, the sources of the services from imap/ are moved to src/service, probably the installable header files are moved to src/include, and then check what to do with the remaining files. With services I mean fud, imapd, lmtpd, pop3d, smmapd and sync_server. May I suggest the services end up in src/libexec/ perhaps? There's still things like master as well. I'd really like to move that elsewhere so the name 'master' doesn't clash with the git branch name 'master'. It makes a few commands more annoying to run. src/master or something would be better. It could still match remote 'src' branch 'master' - if we consider renaming the installed utility to cyrus-master, why not name the source file cyrus-master as well? A location could be (this is the only service process to be executed, really) src/sbin/. The utilities could live in src/bin/ (cyrus-quota, cyrus-mbpath). But - it will invalidate basically every patch out there, Yes indeed it will, and we have another (huge) pending task; fixing the indentation. so I don't want to embark on something like this until we've merged everything we're planning to merge, and everyone has advance notice of the plan. Even the automake changes so far have been quite invasive. I agree. This tidying up is going to majorly invasive - don't forget we want to merge in caldav before this happens as well. I strongly recommend any major really tidying things up is postponed to 2.6. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: libzephyr and notifications
On 2012-05-22 9:40, Bron Gondwana wrote: We're having issues building zephyr with the new automake stuff, and before we spend too much time fixing it - there's a question worth asking... Does anyone actually use zephyr? I (we) don't. If not, I'd prefer to remove it and integrate worldline's notification bus work rather than having multiple competing notification systems. No objections from me (us). Andreas Osowski has successfully merged in / rebased the notification bus work from worldline's github, BTW, so if it can be given a glance-over before it's finally merged into origin/master, that'd be great. I've given Andreas commit / push access to origin so he can do the final merge and push himself on whatever is origin/master HEAD by then. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: libzephyr and notifications
On 2012-05-29 1:38, Greg Banks wrote: Ok, I'll review that, starting today or tomorrow (flu permitting). I saw Andreas' message go past but I can't seem to find it now that I need it :( so what's the git URL? He's git://github.com/aosowski/cyrus-imapd.git The branch is ticket/3605 Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Libtool and Support for Shared Libraries (2)
On 2012-05-29 1:25, Greg Banks wrote:
On Mon, May 28, 2012, at 09:33 PM, Dilyan Palauzov wrote:
[...]probably using make install DESTDIR= with
libtool is either wrong or implemented/handled wrong in
Automake/libtool.
Well, that sucks.
Why do not you use ./configure --prefix=$(DESTDIR), so that make
install DESTDIR=somewhere is not necessary? To my understanding
installing in DESTDIR is used to create packages,
So we now generate dozens of warnings when doing a straightforward,
entirely normal, and unavoidable step in the packaging process? I
don't
see how that's acceptable.
While of course in my realm of packaging, I could use ./configure, what
I actually use is %configure. It expands to a predefined set of standard
configure options such as --prefix=/usr, --libdir=/usr/lib64, etc, along
with first exporting a bunch of variables.
When the make install DESTDIR=/some/where is issued, we point it to the
root directory of a buildroot (%{buildroot}) and expect the other
./configure options to kick in so that everything is still finally
placed in the correct directories (i.e.%{buildroot}etc/ for
--sysconfdir=%{_sysconfdir}, %{buildroot}usr/lib64 for
--libdir=%{_libdir}, etc. exec dir, libexec dir, ...) for which, if the
prefix were to be set to the buildroot root directory, we would need to
add the options for all the other --*dir= configure options as well.
Kind regards,
Jeroen van Meeuwen
--
Systems Architect, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
Re: official dev team position regarding multiple times requested feature (global sieve)
On 2012-07-23 22:26, mailing list subscriber wrote: With all due respect, what is the development's team position regarding this feature and how do the development team see a solution that meets both requirements? Users will most likely continue to require write access to a script that allows them to set what level of spam is to be filtered to a different folder then one's INBOX, which addresses are on a whitelist, and whether or not the vacation or out of office responses is active/de-active. If an organization wishes to enforce a particular script (with or without particular settings, and with or without allowing the user some level of editing), then it is (now) mostly some or the other management solution on top of the Cyrus IMAP deployment that takes care of this level of management. I'm curious to learn what exactly would be the set of requirements that would enable a mandatory Sieve script feature to be integrated into Cyrus IMAP; - Would setting a mandatory script implicitly disallow users to write additional(?), new scripts? Would this be Yet Another Setting? Would Cyrus IMAP magically adjust any user-uploaded scripts to conform with the mandatory script policy? - Would a script (if it were read-only for the user) read settings from a location not Cyrus IMAP, such as the proverbial boolean Yes I am out-of-office and/or my vacation lasts until $x? - How many Sieve editing clients would remain compatible / could become compatible with such Sieve semantics? Please allow me to shamelessly plug some thoughts from Kolab[1] here, that relate to but are not entirely in the same realm. Kind regards, Jeroen van Meeuwen [1] https://wiki.kolab.org/KEP:14 -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Build failed in Jenkins: cyrus-imapd-master #1059
On 2013-01-02 15:07, Sébastien Michel wrote: I used initially json-c that is the most common. However, only the trunk offered support to 64bit integers. This is why I switched to jansson that is also valuable. Unfortunately, it is less common than json-c, and for which the version 0.10 is now available in EPEL. Should I change back the JSON library? there is still time before releasing Cyrus 2.5 We have discussed this before, though perhaps not on the list; I have no trouble *not* having event notifications available with Cyrus IMAP 2.5 on Enterprise Linux 5, but; - We should consider upgrading our CI system to Enterprise Linux 6, - It is not very unlikely nobody will even attempt to build Cyrus IMAP 2.5 let alone ship off to production, based on Enterprise Linux 5... So, it is really up to the rest of the list (especially those with the Solaris, -UX and other UNIX systems), for which, if I recall correctly, json-c and jansson are both equally unavailable and need to be built on/for these platforms regardless. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: CI Builld #1169 has not completed for 50 days
On 2013-04-20 01:11, Дилян Палаузов wrote: Hello, ci.cyrusimap.org/job/cyrus-imapd-master/1169/ ) has not completed for over a month. Ideas? I aborted the build, so we're rolling forward with build #1170 now. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
git master notifyd/notify_external.c and *user
Hi there,
I was trying to get git master HEAD to execute an external program
(notify_external: /usr/bin/myprogram), but it complained no recipient
sepcified - short of the typo that I corrected, it seemed that the
check:
if (!*user) {
syslog(LOG_ERR, ERROR: recipient not specified);
return strdup(NO external recipient not specified);
}
at [1] was causing this.
For the sake of argument, I removed the check, and voila - my program
is actually being executed.
Does anyone have any idea what the -u USER parameter is supposed to
be doing / does anyone actually successfully use notify_external?
Thanks.
Kind regards,
Jeroen van Meeuwen
[1]
http://git.cyrusimap.org/cyrus-imapd/tree/notifyd/notify_external.c#n78
--
Systems Architect, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08
Re: [PATCH/FEATURE] Per-user XLIST support for cyrus-imapd
On 2013-04-30 09:20, Thomas Jarosch wrote: Hi Bron, On Tuesday, 30. April 2013 08:53:24 Bron Gondwana wrote: Hi - have you looked at all at the special-use support in mainline Cyrus? The xlist-* behaviour is planned to be removed, in favour of using the RFC 6154 mandated annotation. The implementation in git at the moment doesn't quite match the standard, because I wrote it before the RFC was released, but I have a branch which makes it fully compatible. I wrote to this list just the other day asking about it! Eduardo is innocent, we (=Intra2net) needed support for this :) We didn't upgrade to 2.4.x yet, so we needed to develop this for 2.3.16 anyway. As far as I understood it, Outlook 2013 does not implement the RFC properly. I suppose you would then use the Cyrus IMAP internal SPECIAL-USE folder flags storage(?) to represent a response to X-LIST(?) rather than supplying additional configuration that can quickly get out-of-sync with other clients (that do support SPECIAL-USE?) as well as across folders. I haven't actually looked at anything to back up this rambling with, tell me if it makes sense ;-) Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Special-Use and Cyrus 2.5 git
On 2013-04-26 16:17, Bron Gondwana wrote: Is anybody actually using cyrus 2.5 git in production other than FastMail? Yes, though admittedly I should say production. I'm planning to do the mailboxes.db format changes before releasing 2.5, so that we don't have to support a stupid intermediate format. It's been on my todo list for AGES as one of the final blockers to 2.5 being released, and I've finally got some spare cycles away from our internal search code. But it would simplify things considerably if I don't have to write an importer that goes upstream. I'd just manually script a conversion of the FastMail servers, and then toss away the intermediate code. None of our systems with 2.5 in production use special-use, nor are they expected to be upgraded smoothly without manual / scripted sysadmin intervention. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: [PATCH] 2.4 branch does not build anymore
On 2013-05-18 16:34, Thomas Cataldo wrote: As conflict marks were committed. Thanks! Committed and pushed. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Changing LIST (again)
On 2013-05-02 07:29, Bron Gondwana wrote: One of my release goals for Cyrus 2.5 is to be correct in our implementation of every standard that we claim to support. This is why I emailed the list last week asking if anyone is using the intermediate SPECIALUSE representation in git. Since there was no reply, I'm assuming it will be OK :) I've just started testing it into production at FastMail, and will roll it out to all our users later this week. Assuming performance is acceptable, I'll push that upstream. Meanwhile, there's one glaring problem remaining, as evidenced by Timo's imaptest tool. Our LIST-EXTENDED implementation is broken in some places. I've already had reports of users of recent versions of some clients having problems with it. That sucks. Rob M (CC'd) and I had a good talk about it yesterday. We're looking at switching the default at FastMail to use altnamespace - it works better with Outlook 2013 and also with some phones. One problem is that you can't create subfolders of Inbox. We've looked at the standard, and we don't see any reason why we can't allow them, The clients that do create (allow) a sub-folder of the INBOX (to be created) apparently do not adhere to the \NoInferiors flag... Should you make the change to allow sub-folders (of the INBOX) to in fact be created (while using altnamespace), would that eliminate the use of \NoInferiors? Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
SETMETADATA: missing annotation value
Hi there, I'm wondering whether it is me (us) doing something wrong, or whether Cyrus IMAP git origin master HEAD is at fault... C: A0019 SETMETADATA INBOX (/shared/vendor/kolab/folder-type mail /private/vendor/kolab/folder-type mail.inbox) S: A0019 BAD Missing metadata value C: A0020 SETMETADATA INBOX (/private/vendor/kolab/folder-type mail.inbox) S: A0020 BAD Missing metadata value or: C: . GETMETADATA INBOX /shared/comment S: * METADATA INBOX (/shared/comment NIL) S: . OK Completed C: . SETMETADATA INBOX (/shared/comment hey) S: . BAD Missing metadata value C: . SETMETADATA INBOX (/shared/comment 'hey') S: . BAD Missing metadata value C: . SETMETADATA INBOX /shared/comment 'hey' S: . BAD Missing metadata entry list Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Changing LIST (again)
On 2013-05-19 21:45, Bron Gondwana wrote: On Sun, May 19, 2013, at 11:21 PM, Jeroen van Meeuwen (Kolab Systems) wrote: On 2013-05-19 15:07, Bron Gondwana wrote: On Sun, May 19, 2013, at 08:36 PM, Jeroen van Meeuwen (Kolab Systems) wrote: Should you make the change to allow sub-folders (of the INBOX) to in fact be created (while using altnamespace), would that eliminate the use of \NoInferiors? Indeed it would :) I really, really like calling out stupid clients that do not honour \NoInferiors though... I guess everyone needs a hobby... *g* Can we please keep a sane default of (enforcing) RFC(s)-compliant behaviour? Default, sure. Thanks! Do you suggest to remove the \NoInferiors flag for deployments with sub-folder of INBOX w/ altnamespace allowed type of configurations, or would you keep it around (and simply behave differently by honouring the request instead of returning an error)? You kind of have to - since you want RFC-compliant clients to be able to create subfolders of Inbox. That's the whole point. Fine with me - I just wanted to understood the changes you were about to make correctly ;-) Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +44 74 2516 3817 w: http://www.kolabsys.com pgp: 9342 BF08
Cyrus IMAP 2.5 Discrete Murder
Hi there,
I've attempted to deploy a discrete murder topology setup using git master @
534066.
I run in to a particular issue attempting to create a mailbox, for which I
have attempted two scenarios:
a) have the frontends configured with no partition-default setting,
b) have the frontends configured with a partition-default of
/var/spool/imap/
The configuration is working fine for cyrus-imapd-2.4.17.
In case a), a . CREATE user/someu...@domain.tld results in a
Unknown/invalid partition error,
In case b), the mailbox does get created, but on the frontend, which of course
is not the intention, and while I would be able to reproduce the error message
I subsequently get, it's going to take me a while to move forward and then
backwards again.
Attached are the configuration files, for reference.
Kind regards,
Jeroen van Meeuwen
--
Systems Architect, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08# standard standalone server implementation
START {
# do not delete this entry!
recover cmd=ctl_cyrusdb -r
# this is only necessary if using idled for IMAP IDLE
idled cmd=idled
# Push out my current list of mailboxes to the mupdate server
mupdate cmd=ctl_mboxlist -m
}
# UNIX sockets start with a slash and are put into /var/lib/imap/sockets
SERVICES {
# add or remove based on preferences
imapcmd=imapd listen=imap prefork=5
imaps cmd=imapd -s listen=imaps prefork=1
pop3cmd=pop3d listen=pop3 prefork=3
pop3s cmd=pop3d -s listen=pop3s prefork=1
sieve cmd=timsieved listen=sieve prefork=0
lmtpunixcmd=lmtpd listen=/srv/imap/config/socket/lmtp prefork=1
# this is only necessary if using notifications
notify cmd=notifyd listen=/srv/imap/config/socket/notify
proto=udp prefork=1
}
EVENTS {
# this is required
checkpoint cmd=ctl_cyrusdb -c period=30
# this is only necessary if using duplicate delivery suppression,
# Sieve or NNTP
duplicateprune cmd=cyr_expire -E 3 at=0400
# Expire data older then 69 days. Two full months of 31 days
# each includes two full backup cycles, plus 1 week margin
# because we run our full backups on the first sat/sun night
# of each month.
deleteprune cmd=cyr_expire -E 4 -D 69 at=0430
expungeprune cmd=cyr_expire -E 4 -X 69 at=0445
# this is only necessary if caching TLS sessions
tlsprunecmd=tls_prune at=0400
## Create search indexes regularly
#squattercmd=squatter -s -i at=0530
}
# standard standalone server implementation
START {
# do not delete this entry!
recover cmd=ctl_cyrusdb -r
# this is only necessary if using idled for IMAP IDLE
idled cmd=idled
}
# UNIX sockets start with a slash and are put into /var/lib/imap/sockets
SERVICES {
# add or remove based on preferences
imapcmd=proxyd listen=imap prefork=5
imaps cmd=proxyd -s listen=imaps prefork=1
pop3cmd=proxyd listen=pop3 prefork=3
pop3s cmd=proxyd -s listen=pop3s prefork=1
sieve cmd=timsieved listen=sieve prefork=0
mupdate cmd=mupdate listen=3905 prefork=1
}
EVENTS {
# this is required
checkpoint cmd=ctl_cyrusdb -c period=30
# this is only necessary if caching TLS sessions
tlsprunecmd=tls_prune at=0400
}
# standard standalone server implementation
START {
# do not delete this entry!
recover cmd=ctl_cyrusdb -r
}
# UNIX sockets start with a slash and are put into /var/lib/imap/sockets
SERVICES {
mupdate cmd=mupdate -mlisten=3905 prefork=1
}
EVENTS {
# this is required
checkpoint cmd=ctl_cyrusdb -c period=30
# this is only necessary if caching TLS sessions
tlsprunecmd=tls_prune at=0400
}
/vendor/horde/share-params,mailbox,string,backend,value.shared value.priv,a
/vendor/kolab/color,mailbox,string,backend,value.shared value.priv,a
/vendor/kolab/folder-test,mailbox,string,backend,value.shared value.priv,a
/vendor/kolab/folder-type,mailbox,string,backend,value.shared value.priv,a
/vendor/kolab/incidences-for,mailbox,string,backend,value.shared value.priv,a
/vendor/kolab/pxfb-readable-for,mailbox,string,backend,value.shared value.priv,a
/vendor/kolab/h-share-attr-desc,mailbox,string,backend,value.shared value.priv,a
/vendor/kolab/activesync,mailbox,string,backend,value.priv,r
/vendor/x-toltec/test,mailbox,string,backend,value.shared value.priv,aconfigdirectory: /var/lib/imap
admins: cyrus-admin
sievedir: /var/lib/imap/sieve
sendmail: /usr/sbin/sendmail
sasl_pwcheck_method: saslauthd
sasl_mech_list: PLAIN LOGIN
allowplaintext: no
tls_cert_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_key_file: /etc/pki/cyrus-imapd/cyrus-imapd.pem
tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt
# uncomment this if you're operating in a
ACL Change notifications
Hi there,
as part of an exercise to make use of event notifications for the
purposes of auditing (non-syslog), I wanted to add an event notification
for ACL changes.
Please find attached a patch for your review, an aggregate of the work
in dev/acl-change-notification[1].
I have a couple of things I myself am pondering as well;
- ACL change notifications are not a part of any RFC as such (but for
[2]), and therefore the fields aclSubject / aclRights may need a
'vnd.cmu' prefix? Does the event name AclChange need a similar prefix?
- In relation to the previous consideration, this change (in part) could
relate to Access Control List Changes in IMAP NOTIFY[2].
- The event (type) could perhaps use a separate event_groups in
imapd.conf(5), but for now I stuffed them under access - the fields
themselves could also be subject to inclusion in event_extra_params
instead, perhaps.
Thanks, in advance,
Kind regards,
Jeroen van Meeuwen
[1]
http://git.cyrusimap.org/cyrus-imapd/log/?h=dev/acl-change-notification
[2] http://tools.ietf.org/html/rfc5465#section-5.9
--
Systems Architect, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08diff --git a/imap/mboxevent.c b/imap/mboxevent.c
index db45821..2ad0fc7 100644
--- a/imap/mboxevent.c
+++ b/imap/mboxevent.c
@@ -72,7 +72,7 @@
#define MAILBOX_EVENTS (EVENT_MAILBOX_CREATE|EVENT_MAILBOX_DELETE|\
EVENT_MAILBOX_RENAME|EVENT_MAILBOX_SUBSCRIBE|\
- EVENT_MAILBOX_UNSUBSCRIBE)
+ EVENT_MAILBOX_UNSUBSCRIBE|EVENT_ACL_CHANGE)
#define QUOTA_EVENTS (EVENT_QUOTA_EXCEED|EVENT_QUOTA_WITHIN|EVENT_QUOTA_CHANGE)
@@ -120,6 +120,8 @@ static struct mboxevent event_template =
{ EVENT_FLAG_NAMES, flagNames, EVENT_PARAM_STRING, 0, 0 },
{ EVENT_USER, user, EVENT_PARAM_STRING, 0, 0 },
{ EVENT_MESSAGE_SIZE, messageSize, EVENT_PARAM_INT, 0, 0 },
+{ EVENT_ACL_SUBJECT, aclSubject, EVENT_PARAM_STRING, 0, 0 },
+{ EVENT_ACL_RIGHTS, aclRights, EVENT_PARAM_STRING, 0, 0 },
/* always at end to let the parser to easily truncate this part */
{ EVENT_BODYSTRUCTURE, bodyStructure, EVENT_PARAM_STRING, 0, 0 },
{ EVENT_MESSAGE_CONTENT, messageContent, EVENT_PARAM_STRING, 0, 0 }
@@ -169,7 +171,7 @@ EXPORTED void mboxevent_init(void)
enabled_events |= FLAGS_EVENTS;
if (groups IMAP_ENUM_EVENT_GROUPS_ACCESS)
- enabled_events |= (EVENT_LOGIN|EVENT_LOGOUT);
+ enabled_events |= (EVENT_LOGIN|EVENT_LOGOUT|EVENT_ACL_CHANGE);
if (groups IMAP_ENUM_EVENT_GROUPS_MAILBOX)
enabled_events |= MAILBOX_EVENTS;
@@ -360,6 +362,10 @@ static int mboxevent_expected_param(enum event_type type, enum event_param param
return extra_params IMAP_ENUM_EVENT_EXTRA_PARAMS_SERVICE;
case EVENT_TIMESTAMP:
return extra_params IMAP_ENUM_EVENT_EXTRA_PARAMS_TIMESTAMP;
+case EVENT_ACL_SUBJECT:
+ return type EVENT_ACL_CHANGE;
+case EVENT_ACL_RIGHTS:
+ return type EVENT_ACL_CHANGE;
case EVENT_UIDNEXT:
if (!(extra_params IMAP_ENUM_EVENT_EXTRA_PARAMS_UIDNEXT))
return 0;
@@ -621,6 +627,16 @@ EXPORTED void mboxevent_set_access(struct mboxevent *event,
}
}
+EXPORTED void mboxevent_set_acl(struct mboxevent *event, const char *identifier,
+const char *rights)
+{
+if (!event)
+ return;
+
+FILL_STRING_PARAM(event, EVENT_ACL_SUBJECT, xstrdup(identifier));
+FILL_STRING_PARAM(event, EVENT_ACL_RIGHTS, xstrdup(rights));
+}
+
EXPORTED void mboxevent_extract_record(struct mboxevent *event, struct mailbox *mailbox,
struct index_record *record)
{
@@ -979,6 +995,8 @@ static const char *event_to_name(enum event_type type)
return MailboxSubscribe;
case EVENT_MAILBOX_UNSUBSCRIBE:
return MailboxUnSubscribe;
+case EVENT_ACL_CHANGE:
+ return AclChange;
default:
fatal(Unknown message event, EC_SOFTWARE);
}
@@ -1178,6 +1196,12 @@ EXPORTED void mboxevent_set_access(struct mboxevent *event __attribute__((unused
{
}
+EXPORTED void mboxevent_set_acl(struct mboxevent *event __attribute__((unused)),
+const char *identifier __attribute__((unused)),
+const char *rights __attribute__((unused)))
+{
+}
+
EXPORTED void mboxevent_extract_record(struct mboxevent *event __attribute__((unused)),
struct mailbox *mailbox __attribute__((unused)),
struct index_record *record __attribute__((unused)))
diff --git a/imap/mboxevent.h b/imap/mboxevent.h
index 1171bd2..9a85927 100644
--- a/imap/mboxevent.h
+++ b/imap/mboxevent.h
@@ -79,10 +79,11 @@ enum event_type {
EVENT_MAILBOX_DELETE = (116),
EVENT_MAILBOX_RENAME = (117),
EVENT_MAILBOX_SUBSCRIBE = (118),
-EVENT_MAILBOX_UNSUBSCRIBE = (119)
+EVENT_MAILBOX_UNSUBSCRIBE = (119),
+EVENT_ACL_CHANGE = (120),
};
-#define MAX_PARAM 21 /* messageContent number that is always the last */
+#define MAX_PARAM 23 /* messageContent number that is always the last */
/*
* event parameters defined in RFC
multi-domain/multi-rootdn for ptclient/ldap.c
Hi there,
I've previously (a long time ago, actually, too long if you ask me) made
inquiries as to who might be using ptclient/ldap.c[1,2], and in which
fashion; I got three points from the responses;
- Everything should be configurable as LDAP deployments typically vary
widely and often pre-date a Cyrus IMAP deployment[3],
- It should handle groups better[4], namely nested/recursive groups,
the 'memberOf' attribute,
- It should handle memberUrls[5].
While these are all valid points and worth working on for me as well,
today I have another aspect; the handling of multi-domain deployments,
with isolated root dns for each parent domain. A very ugly and
presumptuous patch is attached, that needs extra careful checking and a
nice cleanup.
This scenario (of multiple domains separated in to multiple, different
root DNs) is widely in use with Kolab Groupware, while canonification
nor group ACLs would work.
The scenario for such a deployment could be described as follows:
- A list of objectClass=domainRelatedObject LDAP objects exists in
cn=kolab,cn=config.
- A domain example.org may have a root dn of dc=example,dc=org,
and would be an LDAP entry as follows:
dn: associatedDomain=example.org,cn=kolab,cn=config
objectClass: top
objectClass: domainRelatedObject
associatedDomain: example.org
- To translate example.org to dc=example,dc=org in this particular
case, the C equivalent of:
$root_dn = 'dc=' . implode(',dc=', explode('.', example.org));
can be used.
- Another domain example.com mayhave a root dn of o=example,c=de,
and could be an LDAP entry as follows:
dn: associatedDomain=example.com,cn=kolab,cn=config
objectClass: top
objectClass: domainRelatedObject
objectClass: inetDomain
associatedDomain: example.com
inetDomainBaseDN: o=example,dc=de
- Here, the inetDomainBaseDN attribute should be used to translate a
user login of lucy.me...@example.com to a search against
o=example,dc=de.
This leads me to believe the following items should be configurable:
- domain_base_dn
The base dn to use when searching for domains or domain name
spaces.
For Kolab Groupware deployments, this is a default of
cn=kolab,cn=config, but could of course be
ou=Domains,dc=domain,dc=tld as well.
- domain_filter
The filter to use.
Perhaps something like
((objectClass=domainRelatedObject)(inetDomainStatus=on)(associatedDomain=%s))
- domain_scope
The search scope. sub, one or base, with a default of sub.
- domain_name_attribute
The attribute name for actual domain name spaces, such as
associatedDomain.
For LDAP deployments without the Netscape schemas I suppose this
attribute name might be cn.
For situations in which a domainRelatedObject does not contain one,
but multiple values for the domain_name_attribute, the first value
returned by LDAP is used (this typically corresponds with the relative
DN attribute value, and should be consistent).
- domain_result_attribute
Name of the attribute to look for, for example inetDomainBaseDN.
If no such attribute exists (for the object found), fall back to the
standard root dn described above (the implode over explode thing).
I would appreciate your thoughts and feedback.
Thanks, in advance,
Kind regards,
Jeroen van Meeuwen
[1]
http://lists.andrew.cmu.edu/pipermail/cyrus-devel/2011-August/001923.html
[2]
http://lists.andrew.cmu.edu/pipermail/info-cyrus/2011-August/035257.html
[3]
http://lists.andrew.cmu.edu/pipermail/info-cyrus/2011-August/035259.html
[4]
http://lists.andrew.cmu.edu/pipermail/info-cyrus/2011-August/035262.html
[5]
http://lists.andrew.cmu.edu/pipermail/info-cyrus/2011-August/035294.html
--
Systems Architect, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
m: +44 74 2516 3817
w: http://www.kolabsys.com
pgp: 9342 BF08diff --git a/lib/imapoptions b/lib/imapoptions
index ecb54ef..0725fd9 100644
--- a/lib/imapoptions
+++ b/lib/imapoptions
@@ -597,6 +597,21 @@ Blank lines and lines beginning with ``#'' are ignored.
{ ldap_deref, never, STRINGLIST(search, find, always, never) }
/* Specify how aliases dereferencing is handled during search. */
+{ ldap_domain_base_dn, , STRING }
+/* Base DN to search for domain name spaces. */
+
+{ ldap_domain_filter, ((objectclass=domainrelatedobject)(associateddomain=%s)), STRING }
+/* Filter to use searching for domains */
+
+{ ldap_domain_name_attribute, associateddomain, STRING }
+/* The attribute name for domains. */
+
+{ ldap_domain_scope, sub, STRINGLIST(sub, one, base) }
+/* Search scope */
+
+{ ldap_domain_result_attribute, inetdomainbasedn, STRING }
+/* Result attribute */
+
{ ldap_filter, (uid=%u), STRING }
/* Specify a filter that searches user identifiers. The following tokens can be
used in the filter string:
diff --git a/ptclient/ldap.c b/ptclient/ldap.c
index beb31d9..e16e74a 100644
--- a/ptclient/ldap.c
+++ b/ptclient/ldap.c
@@ -131,42
Re: Patch for adding tls_honor_cipher_order
On 2014-10-16 19:32, Kristian Kræmmer Nielsen wrote: Hi, Patch attached. Something similar is already in cyrus-imapd-2.4: http://git.cyrusimap.org/cyrus-imapd/commit/?h=cyrus-imapd-2.4id=4b26d2d7244eeaa481871c337e57cd393fd76dfe For master / 2.5, I have a push pending of a similar nature, while it also addresses some client vs. server certificate chain configuration options (i.e. Internet-facing public CA, verify client certificates against private CA, offer client certificates between Cyrus IMAP servers, and allow requiring certs to be set to optional or required). Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +41 79 951 9003 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Pull Request: Sieve imap4flags extension
On 2014-10-20 13:11, James Cassell wrote:
Hello,
After over a year since I began, I have finished implementing the
imap4flags extension. It is ready for 2.5.
Hi James,
I've pushed your enhancement, all individual commits nicely rebased
against the then current master HEAD, nicely done!
Just in time for 2.5.0 -- slated to be alpha'd and beta'd one of these
next few weeks, Bron/Ken/Dave are having face-to-face sessions about it.
Could you please make sure a full build of 531b18a works exactly the way
you intended? I've verified (quickly, dirty) things build and run, but
not the functionality.
If you're interested, I have my autogen.sh script attached representing
the way I also build RPM packages -- makes it nice and reproducible ;-)
Kind regards,
Jeroen van Meeuwen
--
Systems Architect, Kolab Systems AG
e: vanmeeuwen at kolabsys.com
m: +41 79 951 9003
w: http://www.kolabsys.com
pgp: 9342 BF08#!/bin/bash -x
git_branch=$(git symbolic-ref HEAD | sed -e 's|refs/heads/||g')
git_branch_friendly=$(echo ${git_branch} | sed -e 's|/|_|g')
git_remote=$(git config branch.${git_branch}.remote)
(
automake --add-missing
libtoolize
autoconf -v
autoreconf -v --install
export CPPFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -I/usr/include/et -I/usr/include/kerberosIV'
export CFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC'
export CXXFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC'
export CCDLFLAGS=-rdynamic
export LDFLAGS=' -pie'
export CFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC'
export CXXFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC'
export FFLAGS='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -I/usr/lib64/gfortran/modules'
./configure \
--build=x86_64-redhat-linux-gnu \
--host=x86_64-redhat-linux-gnu \
--target=x86_64-redhat-linux-gnu \
--program-prefix= \
--prefix=/usr \
--exec-prefix=/usr \
--bindir=/usr/bin \
--sbindir=/usr/sbin \
--sysconfdir=/etc \
--datadir=/usr/share \
--includedir=/usr/include \
--libdir=/usr/lib64 \
--libexecdir=/usr/libexec \
--localstatedir=/var \
--sharedstatedir=/var/lib \
--mandir=/usr/share/man \
--infodir=/usr/share/info \
--enable-autocreate \
--enable-event-notification \
--enable-gssapi \
--enable-idled \
--enable-maintainer-mode \
--enable-murder \
--enable-netscapehack \
--enable-nntp \
--enable-replication \
--enable-unit-tests \
--enable-xapian \
--without-bdb \
--with-cyrus-prefix=/usr/lib/cyrus-imapd \
--with-extraident=Kolab-2.5-0.1.dev20140802.el6 \
--without-wrap \
--with-krbimpl=mit \
--with-ldap=/usr \
--with-perl=/usr/bin/perl \
--with-service-path=/usr/lib/cyrus-imapd \
--with-snmp \
--with-syslogfacility=MAIL
make clean
make -B -j4
make check
) 21 | tee ${TMPDIR:-/tmp}/cyrus-imapd-${git_remote}.${git_branch_friendly}-$(date +'%s').log
Re: POLL: per-domain shared folder/sieve/etc
On 2014-10-22 23:02, Bron Gondwana wrote: Yes, that means a massive change, instead of internally: example.com!user.foo.bar = user/foo/b...@example.com (which is a million ways of bogus) we would have: user.foo@example^com.bar = user/f...@example.com/bar Or in alt namspace: Other Users/f...@example.com/bar This means we will finally be able to share things across domains. It creates a single consistent way to access everything. The domain used to be used as an authorization realm, where a user j...@example.com would only see Other Users/foo/bar -- without the domain. How would this translate to the new way? If the external name (the new default) uses unix hierarchy separators, would it be reasonable to update the internal format to that as well, and translate user/foo/b...@example.org or user/f...@example.org/bar back to using the netnews hierarchy separator if so configured? The problem is, it means you can't set quotas per domain, you can't have sieve scripts per domain, and most of all - you can't have shared folders in a domain. example.com!shared.stuff worked fine, but shared.example^com.stuff would be weird. It's just a folder, and wouldn't be treated specially in any way. The domain would have no special meaning. This is now shared/st...@example.org, I suppose a hierarchy of such folders would lead to shared/st...@example.org/something? This is all, obviously, Cyrus 3.0 stuff. In the multi-domain environments we typically run, while sharing between domains is indeed an often requested feature, we love the inability to share cross-realm -- preventing accidentally sharing your @company.com content with @competitor.com people. If the new way of doing things is to allow cross-realm sharing, I would like to ensure some sort mandatory access policy is in place, where one has to specify @something can in fact share with @else. On 2014-10-24 02:59, Bron Gondwana wrote: No, the per-user namespace is still fine - users can still share with other users in their own domain - just currently it is technically impossible to share with users in other domains right now - because the mailbox naming is not RFC compliant, so it's not compatible with real IMAP client, only with Cyrus management tools. You mentioned in another post (quote above) that the current naming of mailboxes is not necessarily RFC-compliant, and that only the Cyrus tooling is compatible. I may be misunderstanding what this means, because only an administrator without a realm (as part of its login username) is currently able to view lists of mailboxes across realms -- bear with me while I transcribe from the top of my head: Settings: virtdomains: userid admins: cyrus-admin ad...@example.org cyrus-admin: C: . LIST * S: * user/j...@company.com S: * user/j...@example.org S: * user/m...@example.org ad...@example.org: C: . LIST * S: * user/jane S: * user/max j...@example.org: C: . LIST * S: * INBOX S: * Other Users/max Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +41 79 951 9003 w: http://www.kolabsys.com pgp: 9342 BF08
Re: POLL: per-domain shared folder/sieve/etc
On 2014-11-01 21:29, Bron Gondwana wrote: We already have one at FastMail to stop users setting an 'anyone' ACL. I think this may already be in upstream, unless you're talking about a different implementation/solution? http://git.cyrusimap.org/cyrus-imapd/tree/lib/imapoptions#n179 This is all, obviously, Cyrus 3.0 stuff. In the multi-domain environments we typically run, while sharing between domains is indeed an often requested feature, we love the inability to share cross-realm -- preventing accidentally sharing your @company.com content with @competitor.com people. Yes, that's pretty dangerous, isn't it. If the new way of doing things is to allow cross-realm sharing, I would like to ensure some sort mandatory access policy is in place, where one has to specify @something can in fact share with @else. This is tricky, particularly for FastMail where multiple companies can have addresses in the shared domains (e.g. fastmail.com) as well. It sounds like the right general approach is to allow an external daemon to be queried for policy responses. I suppose the level of complexity depends on the level of flexibility / features required. Suppose that the default is to not have any realm be allowed to use any other realm (no other realm's mailboxes are visible, no ACL subject identifiers validate). This, I would say, represents the current situation most accurately. Suppose a list of source realms (the authenticated user is in this realm) is used as a lookup key, and for any other realm that realm must have presence in the lookup value) -- admittedly a very simplistic view: company.com: subsidiary.com partner.com subsidiary.com: company.com partner.com: company.com competitor.com competitor.com: partner.com Suppose this means that @company.com people (source, lookup key) can share @company.com mailboxes (source, lookup key, must match logged in account?) with @subsidiary.com and with @partner.com ACL subject identifiers. Suppose this means @partner.com (target lookup value) can therefore see @company.com mailboxes -- but cannot share with @company.com because of the first rule, but because of the third rule. I do not suppose there is any use-case to nesting such rules to the tune to, in the above list, allow subsidiary to partner descending to company authorizations (or any other way). I suppose in the case of FastMail, you would list fastmail.com as a lookup key and perhaps use a regular expression .* to ensure @fastmail.com mailboxes are visible to all other realms? Of course, to a certain degree this is trying to make a technical solution to a human problem. If it's that vital that they can't see each other, they should be on separate Cyrus instances at the very least, if not entirely separate infrastructure. There's nothing to stop mall...@example.org just adding a sieve rule to forward a copy of everything to j...@company.com, or handing over credentials, or any number of things. I agree with the general sentiment, but disagree with such separation on the infrastructure level being that kind of a must (for that level of vital). There are other considerations that could require an organization to have infrastructure isolated from a multi-customer hosted environment, but such are in the gut-feeling-more-so-than-technical-literacy, compliance and telco regulatory domains. While to share or not is certainly a social problem, and to enable sharing or not probably is so too, to allow sharing or not is definitely a more technical one if the implementation thereof leaves behind a Free-for-All. For Sieve rules forwarding content, cross-realm ACLs are rather irrelevant. One could (define to) forward content using Sieve regardless, unless Cyrus IMAP's Sieve implementation is extended to allow a similar level of access control. The current methodology to prevent this from happening lays in limiting the user interfaces, not including Sieve extensions, MTA configuration and Data-Loss Prevention systems -- neither of which are helped or negated by introducing cross-realm ACLs, and neither of which requires a given customer to run off of completely separate infrastructure. Should sharing across domains be allowed, but without mandatory access control, however, then you move Cyrus IMAP from the perfect for hosted environments with multiple customers universe to the it's such a Free-for-All you require separate infrastructure for each customer. On 2014-10-24 02:59, Bron Gondwana wrote: No, the per-user namespace is still fine - users can still share with other users in their own domain - just currently it is technically impossible to share with users in other domains right now - because the mailbox naming is not RFC compliant, so it's not compatible with real IMAP client, only with Cyrus management tools. You mentioned in another post (quote above) that the current naming of mailboxes is not necessarily RFC-compliant, and
Re: Mailbox URI format in Event Notifications
On 2014-11-05 11:35, Bron Gondwana wrote: On Wed, Nov 5, 2014, at 09:28 PM, Jeroen van Meeuwen (Kolab Systems) wrote: For those of you using event notifications, I'm wondering how you make other software interpret these things -- our other software looks at everything from the administrative perspective, and so we'd opt for a format of imap://imap.example.org/user/j...@example.org -- but I'm suspecting this may have implications I'm unaware of. Totally agree, that's how it should be. I guess we aren't using altnamespace, so we didn't notice - but I'd agree with both the location of the domain (notice how I'm wanting that elsewhere too) and of course the use of non-alt namespace for administrative things like events. Could you tell me more about how you are using event notifications? We use event notifications for the sake of audit trails, in that client applications do not currently consume the event notifications nor their payload, so what URI is being used in the notification payload is preferably (for us) a consistent URI (i.e. the one from a global, not domain-specific administrator). Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +41 79 951 9003 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Mailbox URI format in Event Notifications
On 2014-11-05 12:27, Bron Gondwana wrote: So we do a ton of stuff with them now :) Most importantly, they feed into the EventSource pipeline for web browser clients to get immediate updates, and likewise the Apple and Google push notification channels if you have logged in with our app on those platforms. We also have alarms from the calendar as events. It's kind of our go-to hammer ;) Does the payload for and/or format of the uri parameter matter anywhere? Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +41 79 951 9003 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Mailbox URI format in Event Notifications
On 2014-11-05 15:16, Sébastien Michel wrote: For those of you using event notifications, I'm wondering how you make other software interpret these things -- our othersoftware looks at everything from the administrative perspective, and so we'd opt for a format of imap://imap.example.org/user/j...@example.org -- but I'm suspecting this may have implications I'm unaware of. Not sure that 'imap://imap.example.org/user/j...@example.org' is right because the format defined in the RFC 5092 - IMAP URL : imap://iserver/enc-mailbox[uidvalidity]iuid[isection][ipartial][iurlauth] The iserver component common to all types of absolute IMAP URLs has the following syntax expressed in ABNF [ABNF]: [iuserinfo @] host [ : port ] If I understand this correctly, the uri here then is defined to have to be a uri that a client could interpret and follow and end up OK with? This makes me wonder why it would be the mailbox owner being in the spot of iuserinfo, because imap://j...@imap.example.org/Other Users/jane will not likely be understood correctly by clients -- as jane, Other Users/jane will not exist. As not jane, the notification is not necessarily invalid nor irrelevant. Neither would, I suppose, imap://j...@imap.example.org/INBOX make all that much sense -- not unless a client is taught to understand jane@'s INBOX is Other Users/jane. The jane@ part should go away or be substituted for the actual user that had the perspective of Jane's INBOX being at Other Users/jane when triggering the event notification. If user john is doing something to a user jane mailbox, the notification should contain the user john in the field user and the URI imap://j...@imap.example.org/INBOX in the fields uri and mailboxID. This is how I understand the RFC 5423. Many of the event notifications have turned out to not contain the user triggering the event at all, and I suppose I'll need to double-check if it is in fact the authenticated user id being used, or the mailbox owner. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +41 79 951 9003 w: http://www.kolabsys.com pgp: 9342 BF08
Re: Possible TLS dir option name discrepancy?
On 2015-01-12 13:23, Patrick Goetz wrote: The 2.5 documentation here (http://www.cyrusimap.org/~vanmeeuwen/imap/release-notes/2.5.0.html) states that some of the TLS options will change in 2.5, namely tls_client_ca_dir (was: tls_ca_dir) However, there is no tls_ca_dir option given here (https://cyrusimap.org/docs/cyrus-imapd/2.4.17/man/imapd.conf.5.php). I've been using tls_ca_path as per the 2.4.17 man page. Am I missing something, or is this just a typo in the 2.5 documentation? Typo. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +41 79 951 9003 w: https://kolabsystems.com pgp: 9342 BF08
Re: What would it take for FastMail to run murder
On 2015-03-18 01:51, Bron Gondwana wrote: On Wed, Mar 18, 2015, at 09:00 AM, Jeroen van Meeuwen (Kolab Systems) wrote: We promote a standby frontend not otherwise used, to become the new mupdate server. The interruption is a matter of seconds this way, unless of course you're in the typical stalemate. Hmm so maybe it's affordable. It scales up with number-of-servers as well though. Making sure it's up to date costs at least O(number of backends). I suppose in your specific case, which I'm not at all too familiar with, perhaps enhancing murder/mupdate to allow cascading and/or (geo-based) replication and/or multi-master would serve your deployment yet even better? I'm suggesting so because I would be concerned with the round-trip times between datacenters if there were only one mupdate master across all -- and perhaps the replicas are faster in issuing the cmd_set() than the mupdate master is(?). Interesting. Does it also handle the case where the same mailbox gets accidentally created on two servers which aren't replica pairs though? Or do you get a mailbox fork? The race condition is not addressed with it, like it is not addressed currently. I'm not 100% happy living with unaddressed race conditions. Addressing this would be an important part of making FastMail happy to run it. Ghe, neither am I, but c'est la vie. That said, in ~5 years and dozens of murder deployments, I have yet to encounter a situation or even a support case in which one mailbox is -- accidentally or otherwise -- created in two locations without the second failing / being rejected. It solely makes the MUPDATE server not reject the reservation request from a server that uses the same servername if it already has an entry for the same servername!partition, so that the replica successfully creates the local copy -- after which replication is happy. Yeah, that makes sense. Of course, the backend should probably not be reserving so much. There are two things conflated here: 1) I'm running cmd_create in an IMAPd and I want to see if this folder already exists. 2) I'm a replica backend getting a copy of an existing folder (or indeed, a backend which already has a folder) and I'm informing mupdate of the fact. Those two should be treated differently. The first is does this already exist, which is a legitimate question to ask. The second should always succeed. MUPDATE is a representation of facts, and the backends are the masters of those facts. With two-way replication safety however (and in your case, channelled as well, right?), which end of the replication (just in case things end up load-balanced across replicas?) gets to submit the original cmd_set() is up in the air, no? So this would build a scenario in which: pair-1-replica-1.example.org and pair-1-replica-2.example.org present themselves as pair-1.example.org A DNS IN A RR is created for the fail-over address(es) for pair- 1.example.org and attached to whichever replica in the pair is considered the active node. Both replicas would be configured to replicate to one another, which works in a PoC scenario but may seem to require lmtpd/AF_INET delivery. So they both have the same server name in mupdate. Yes, and frontends proxy the connections for mailboxes on the backend to the same fake server address. My plan is that they have different server names in mupdate. There's a separate channel somehow to say which is the primary out of those servers, which can be switched however (failover tooling) based on which servers are up, but the murder has the facts about where the mailbox really exists. It may even have statuscache. Man, how awesome would distributed statuscache be. So there are multiple records for the same mailbox, with different server names, in the murder DB. Would this not open back up a route to entertaining a variety of race conditions (that would need to be addressed somehow) though? Should then one of the duplicate mailboxes be marked as the primary? A scenario that comes up often is the geographically close-yet-distant secondary site for disaster recovery, where a set of backends on the primary site replicate to a set of backends on the secondary site. While initially this succeeds perfectly fine, and the backends on the secondary site can participate in a local-to-site murder, transferring mailboxes from one backend to another on the primary site will fail to replicate to the secondary site's backends (because of their participation in the murder). This is in part because it is not the XFER being replicated as such, but the target backend's CREATE/cmd_set(), which will fail because the mailbox already resides on another backend. I suppose a scenario in which the mupdate master is in fact able to hold multiple records for the same mailbox might also allow us to overcome this conundrum? Would using shared memory address the in-memory problem
Re: What would it take for FastMail to run murder
On 2015-03-13 23:50, Bron Gondwana wrote: So I've been doing a lot of thinking about Cyrus clustering, with the underlying question being what would it take to make FastMail run a murder. We've written a fair bit about our infrastructure - we use nginx as a frontend proxy to direct traffic to backend servers, and have no interdependencies between the backends, so that we can scale indefinitely. With murder as it exists now, we would be pushing the limits of the system already - particularly with the globally distributed datacentres. Why would FastMail consider running murder, given our existing nice system? a) we support folder sharing within businesses, so at the moment we are limited by the size of a single slot. Some businesses already push that limit. How, though, do you ensure that a mailbox for a new user in such business is created on the same backend as all the other users of said business? Here are our deal-breaker requirements: 1) unified murder - we don't want to run both a frontend AND a backend imapd process for every single connection. We already have nginx, which is non-blocking, for the initial connection and auth handling. There's one particular problem with using NGINX as the IMAP proxy -- it requires that external service that responds with the address to proxy to. I say problem in quotes to emphasize I use the term problem very loosely -- whether it be a functioning backend+mupdate+frontend or a functioning backend+mupdate+frontend+nginx+service is a rather futile distinction, relatively speaking. 2) no table scans - anything that requires a parse and ACL lookup for every single row of mailboxes.db is going to be a non- starter when you multiply the existing mailboxes.db size by hundreds. I don't understand how this is an established problem already -- or not as much as I probably should. If 72k users can be happy on a murder topology, surely 4 times as many could also be happen -- inefficiencies notwithstanding, they're only a vertical scaling limitation. That said of course I understand it has it's upper limit, but getting updated lookup tables in-memory pushed there when an update happens would seem to resolve the problem, no? 3) no single-point-of-failure - having one mupdate master which can stop the entire cluster working if it's offline, no thanks. This is not necessarily what a failed mupdate server does though -- new folders and folder renames (includes deletions!) and folder transfers won't work, but the cluster remains functional under both the SMTP-to-backend and LMTP-proxy-via-frontend topology -- autocreate for Sieve fileinto notwithstanding, and mailbox hierarchies distributed over multiple backends when also using the SMTP-to-backend topoplogy notwithstanding. Thankfully, the state of the art in distributed databases has moved a long way since mupdate was written. I have also written a one-or-two line patch that enables backends that replicate, to both be a part of the same murder topology, to prevent the replica slave from bailing out on the initial creation of a mailbox -- consulting mupdate and finding that it would already exist. Along with this, we need a reverse lookup for ACLs, so that any one user doesn't ever need to scan the entire mailboxes.db. This might be hooked into the distributed DB as well, or calculated locally on each node. I reckon this may be the rebuild more efficient lookup trees in-memory or otherwise I may have referred to just now just not in so many words. And that's pretty much it. There are some interesting factors around replication, and I suspect the answer here is to have either multi- value support or embed the backend name into the mailboxes.db key (postfix) such that you wind up listing the same mailbox multiple times. In a scenario where only one backend is considered active for the given (set of) mailbox(es), and the other is passive, this has been more of a one-line patch in mupdate plus the proper infrastructure in DNS/keepalived type of failover service IP addresses than it has been about allowing duplicates and suppressing them. Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +41 79 951 9003 w: https://kolabsystems.com pgp: 9342 BF08
RE: What would it take for FastMail to run murder
On 2015-03-13 23:54, Dave McMurtrie wrote: From my phone, so excuse brevity and top-posting, but Fastmail running murder would be a huge bonus. I not-so-fondly recall the intimate relationship I developed with gdb debugging murder issues when we upgraded from 2.3 to 2.4 :) You won't have to for 2.5 (as much) because we're running it at supported customer sites, and I'm to blame for the alleged fixes ;-) Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +41 79 951 9003 w: https://kolabsystems.com pgp: 9342 BF08
Re: What would it take for FastMail to run murder
On 2015-03-14 22:48, Bron Gondwana wrote: On Sun, Mar 15, 2015, at 07:18 AM, Jeroen van Meeuwen (Kolab Systems) wrote: How, though, do you ensure that a mailbox for a new user in such business is created on the same backend as all the other users of said business? If the business already exists, the create user code will fetch the server name from the business database table and make that the creation server. There's a cron job which runs every hour and looks for users who aren't on the right server, so if we import a user to the business, they get moved. Right -- so you seem to agree that one business is limited to one backend server, which is precisely what the larger businesses that are our customers need to work around, when the number of mailboxes is typically tens of thousands, and the mechanism you describe stops working. There's one particular problem with using NGINX as the IMAP proxy -- it requires that external service that responds with the address to proxy to. T108 I say problem in quotes to emphasize I use the term problem very loosely -- whether it be a functioning backend+mupdate+frontend or a functioning backend+mupdate+frontend+nginx+service is a rather futile distinction, relatively speaking. Sure, but backend+distributed mailbox service+nginx would be a much simpler setup. Yes, T108 here ;-) I don't understand how this is an established problem already -- or not as much as I probably should. If 72k users can be happy on a murder topology, surely 4 times as many could also be happen -- inefficiencies notwithstanding, they're only a vertical scaling limitation. happy is a relative term. You can get most of the benefit from using foolstupidclients, but otherwise you're paying O(N) for the number of users - and taking 4 times as long to do every list command is not ideal. Sure -- the majority of the processing delays seem to lay on the client side taking off the wire what is being dumped on it, however. You're far better entitled to speak to what is in a mailboxes.db and/or its in-memory representation by the time you get to scanning the complete list for items to which a user might have access, I just have to say we've not found this particular part to be as problematic for tens of thousands of users (yet). That said of course I understand it has it's upper limit, but getting updated lookup tables in-memory pushed there when an update happens would seem to resolve the problem, no? Solving the problem is having some kind of index/lookup table indeed. Whether this is done all in-memory by some sort of LIST service which scans the mailboxes.db at startup time and then gets updates from mupdate. For frontends specifically (discrete murder), we're able to use tmpfs for mailboxes.db (and some other stuff of course) solving a bit of the I/O constraints, but it's still a list of folders with parameters containing whether the user has access, and what I meant was perhaps the list can (in addition) be inverted to be a list of users with folders (and rights?). This is not necessarily what a failed mupdate server does though -- new folders and folder renames (includes deletions!) and folder transfers won't work, but the cluster remains functional under both the SMTP-to-backend and LMTP-proxy-via-frontend topology -- autocreate for Sieve fileinto notwithstanding, and mailbox hierarchies distributed over multiple backends when also using the SMTP-to-backend topoplogy notwithstanding. Yeah, until you start up the mupdate server again or configure a new one. Again, you get user visible failures (folder create, etc) while the server is down. The reason I want to shave off all these edge cases is that in a big enough system over a long enough time, you will hit every one of them. We promote a standby frontend not otherwise used, to become the new mupdate server. The interruption is a matter of seconds this way, unless of course you're in the typical stalemate. Thankfully, the state of the art in distributed databases has moved a long way since mupdate was written. I have also written a one-or-two line patch that enables backends that replicate, to both be a part of the same murder topology, to prevent the replica slave from bailing out on the initial creation of a mailbox -- consulting mupdate and finding that it would already exist. Interesting. Does it also handle the case where the same mailbox gets accidentally created on two servers which aren't replica pairs though? Or do you get a mailbox fork? The race condition is not addressed with it, like it is not addressed currently. It solely makes the MUPDATE server not reject the reservation request from a server that uses the same servername if it already has an entry for the same servername!partition, so that the replica successfully creates the local copy -- after which replication is happy. So this would build a scenario in which: pair-1-replica
Re: Docker/Cyrus user
On 2015-03-19 05:29, Chris Davies wrote: I think it's because the cyrus user isn’t being created. Perl exception: No user named 'cyrus' Has anyone else got the docker images working? Am I missing a step or command somewhere? Are you sure you have the latest version of the GIT repo? https://git.cyrus.foundation/diffusion/ID/browse/master/santiago;6c242b5541d257fb6464f86b99c9ff35b7bcc959$75 I'm new to docker, any attempt to get command line access to my container is failing the the run script getting rerun, which takes another 5+ minutes Any suggestions? To get an interactive shell, specify the entrypoint (and a -s at the end): $ docker run -it --entrypoint=/bin/bash cyrusimapd/santiago -s Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +41 79 951 9003 w: https://kolabsystems.com pgp: 9342 BF08
Re: Docker/Cyrus user
On 2015-03-20 07:39, Chris Davies wrote: Not sure if you saw my previous IRC messages. The IRC server rejected some of them because I used copy paste which broke each line down to individual messages. I'm currently rerunning the run command '*docker run -t -i cyrus- imapd-3.0.santiago*'. It failed multiple times but continued running tests. Unfortunately my terminal only shows the last 512 lines, so I’ve updated it to show unlimited lines so I can see what actually failed first. Should the tests stop once they hit a failure? or capture that and output it at the end? It should do its best to run as many tests if possible. So, if a relaxed make succeeds but a strict make fails, it can continue doing unit tests and cassandane based on a relaxed build. All output should be available on your stdout/stderr, so you might as well: $ docker run -it cyrusimapd/santiago 21 | tee santiago.log Kind regards, Jeroen van Meeuwen -- Systems Architect, Kolab Systems AG e: vanmeeuwen at kolabsys.com m: +41 79 951 9003 w: https://kolabsystems.com pgp: 9342 BF08
