Thanks Michael - I don't have much time to give a longer response right now, but this is really valuable feedback into what we focus on over the next few months and into the 3.2 release plans. I really appreciate you taking the time to spell this out in detail.
Cheers, Bron. On Tue, Mar 5, 2019, at 02:17, Michael Menge wrote: > Hi Ellie and Bron, > > first thank you for your ideas for the workaround and for opening the issue. > > I have to apologize about the original subject, but I was a bit > frustrated that > I have encountered again a problem with the conversation db. > > As I have discovered jet another, not jet reported problem with > conversation db > (I am unsure if it's a problem with the conversation db or if only > shows an other > problem some where else), I have decided to deactivate the conversation db for > the moment. > > I think I should elaborate the general problem I have with this > feature and the > cyrus development as i have observed it. > > I am an experienced linux and cyrus administrator. I am not an > software developer, > or programmer, but I understand enough about programming to fix small problems > and narrow down problems. > > Cyrus code quality has grown in the last 14 years since i first set up > our first > cyrus imap (2.3) server. Especially since Fastmail dedicated personal for the > cyrus project. But also automatic testing and other design decisions helped to > bring the project a big step forward. > > I know we have a complex setup (murder, replication, meta- and archive > partition, > delayed delete, delayed expunge) so that we use combinations of > features that are not > that common. So fare we are happy with our cyrus setup. > > We did not encounter any data loss, and where able to fix most > problems in short time. > The system is very stable and expect for the slow search can't > complain about the performance. > So thanks again to the devs for the work and also for the community > the help I received > in the last 14 years. > > I like new features, but I have always to balance the advantages of > these new features > with the impact on stability, performance and administration overhead. > In that regard I > tend to exercise caution. > > While testing cyrus 3.0, on one hand neither conversation db nor > sphinx or xapian search > engines looked particular interesting, as I had no need to improve > search speed in > cyrus 2.4 with squatter ("If it isn't broken don't fix it"). On the > other hand i didn't > have time to rigorous test these features, so I decided not to use > them as new feature > have a tendency to contain more new bugs. > > After upgrading the production system to cyrus 3.0 I discovered that > search was slow with > squatter, conversation db did more than the documentation suggested > that it does. > I don't know if it was intended that conversation db was required for > squatter to work > or if it broke by one change to support multiple search engines but > the requirement > was surprising. I did try to find the commit that did break the > squatter search but > failed, as was unable to compile most commits git bisect suggested. > > The problems I encountered with conversation db, confirmed my initial > caution not to enable > conversation db without testing. But it would be unfair to blame only > the implementation of > conversation db. > > One problem was caused by "reconstruct -V max" not upgrading all > mailboxes, which i did > miss in the logs. It would be nice if ctl_conversationsdb would check > mailbox version > before creating a huge conversation db file in an endless loop. > > The problem with "IOERROR: conversations_audit on load:" and "IOERROR: > conversations_audit on store:" > is still a mystery to so it is unclear if it is really a bug in the > conversation db of if it shows > an other problem with my installation. > > The same is with the new problem i had no time to report jet. Deleting > users i see the following error > popping up for some accounts. > "Fatal error: Internal error: assertion failed: imap/conversations.c: > 2205: !status.exists" > > But breaking an common use case of an other feature like "delete_mode: > delayed" is > an other case. This should have been fixed before it was released in > the stable > cyrus version. At least a WARNING in the documentation is required. > > I would like to help to improve the documentation, but there are some > questions that need to be answered: > > 1. Which search engines and combinations are currently supported? > Is a stand alone squatter still supported? > > 2. What are to pros and cons for the supported search engines? > > 3. Should, or to which extend should, the search engines work without > conversations db? > Or is enabling the conversations db a new requirement for some/all > search engines? > > 4. Is the conversations db murder aware? And how do shared folders > (one user shared one > of his folders with other users) on the same server/cross server > affect search results and performances > > 5. What is stored in the conversations db? > > https://www.cyrusimap.org/dev/imap/concepts/deployment/databases.html#conversations-userid-conversations > is incomplete as conversations db also contains hashes of mime parts. > > 6. Which Information is affected by conversations_expire_days > > TLDR; I like cyrus, but there is some work to do regarding to > conversation db and search engines, > in the field of documentation, code testing and feature interaction > > > -------------------------------------------------------------------------------- > M.Menge Tel.: (49) 7071/29-70316 > Universität Tübingen Fax.: (49) 7071/29-5912 > Zentrum für Datenverarbeitung mail: > michael.me...@zdv.uni-tuebingen.de > Wächterstraße 76 > 72074 Tübingen > > ---- > Cyrus Home Page: http://www.cyrusimap.org/ > List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ > To Unsubscribe: > https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus -- Bron Gondwana, CEO, FastMail Pty Ltd br...@fastmailteam.com
---- Cyrus Home Page: http://www.cyrusimap.org/ List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/ To Unsubscribe: https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus