Thanks Michael - I don't have much time to give a longer response right now,
but this is really valuable feedback into what we focus on over the next few
months and into the 3.2 release plans. I really appreciate you taking the time
to spell this out in detail.
Cheers,
Bron.
On Tue, Mar 5, 2019, at 02:17, Michael Menge wrote:
> Hi Ellie and Bron,
>
> first thank you for your ideas for the workaround and for opening the issue.
>
> I have to apologize about the original subject, but I was a bit
> frustrated that
> I have encountered again a problem with the conversation db.
>
> As I have discovered jet another, not jet reported problem with
> conversation db
> (I am unsure if it's a problem with the conversation db or if only
> shows an other
> problem some where else), I have decided to deactivate the conversation db for
> the moment.
>
> I think I should elaborate the general problem I have with this
> feature and the
> cyrus development as i have observed it.
>
> I am an experienced linux and cyrus administrator. I am not an
> software developer,
> or programmer, but I understand enough about programming to fix small problems
> and narrow down problems.
>
> Cyrus code quality has grown in the last 14 years since i first set up
> our first
> cyrus imap (2.3) server. Especially since Fastmail dedicated personal for the
> cyrus project. But also automatic testing and other design decisions helped to
> bring the project a big step forward.
>
> I know we have a complex setup (murder, replication, meta- and archive
> partition,
> delayed delete, delayed expunge) so that we use combinations of
> features that are not
> that common. So fare we are happy with our cyrus setup.
>
> We did not encounter any data loss, and where able to fix most
> problems in short time.
> The system is very stable and expect for the slow search can't
> complain about the performance.
> So thanks again to the devs for the work and also for the community
> the help I received
> in the last 14 years.
>
> I like new features, but I have always to balance the advantages of
> these new features
> with the impact on stability, performance and administration overhead.
> In that regard I
> tend to exercise caution.
>
> While testing cyrus 3.0, on one hand neither conversation db nor
> sphinx or xapian search
> engines looked particular interesting, as I had no need to improve
> search speed in
> cyrus 2.4 with squatter ("If it isn't broken don't fix it"). On the
> other hand i didn't
> have time to rigorous test these features, so I decided not to use
> them as new feature
> have a tendency to contain more new bugs.
>
> After upgrading the production system to cyrus 3.0 I discovered that
> search was slow with
> squatter, conversation db did more than the documentation suggested
> that it does.
> I don't know if it was intended that conversation db was required for
> squatter to work
> or if it broke by one change to support multiple search engines but
> the requirement
> was surprising. I did try to find the commit that did break the
> squatter search but
> failed, as was unable to compile most commits git bisect suggested.
>
> The problems I encountered with conversation db, confirmed my initial
> caution not to enable
> conversation db without testing. But it would be unfair to blame only
> the implementation of
> conversation db.
>
> One problem was caused by "reconstruct -V max" not upgrading all
> mailboxes, which i did
> miss in the logs. It would be nice if ctl_conversationsdb would check
> mailbox version
> before creating a huge conversation db file in an endless loop.
>
> The problem with "IOERROR: conversations_audit on load:" and "IOERROR:
> conversations_audit on store:"
> is still a mystery to so it is unclear if it is really a bug in the
> conversation db of if it shows
> an other problem with my installation.
>
> The same is with the new problem i had no time to report jet. Deleting
> users i see the following error
> popping up for some accounts.
> "Fatal error: Internal error: assertion failed: imap/conversations.c:
> 2205: !status.exists"
>
> But breaking an common use case of an other feature like "delete_mode:
> delayed" is
> an other case. This should have been fixed before it was released in
> the stable
> cyrus version. At least a WARNING in the documentation is required.
>
> I would like to help to improve the documentation, but there are some
> questions that need to be answered:
>
> 1. Which search engines and combinations are currently supported?
> Is a stand alone squatter still supported?
>
> 2. What are to pros and cons for the supported search engines?
>
> 3. Should, or to which extend should, the search engines work without
> conversations db?
> Or is enabling the conversations db a new requirement for some/all
> search engines?
>
> 4. Is the conversations db murder aware? And how do shared folders
> (one user shared one
> of his folders with
