On 07/19/2016 11:49 AM, Gabriele Bulfon wrote:
I found that expecially the function "_index_thread_ref" is quite different (arguments and implementation) in 2.4.12, while it's very similar in 2.5.8 : is it safe to upgrade binaries from 2.4.12 to 2.5.8 on a running system, or do I need any kind of conversion?

doc/install-upgrade.html



Also, the diff around "index_msgdata_free" where "if (!md) continue;" is added, is quite different from 2.5.8 too, but it's probably not necessary, as far as I can see...can you check this?

It is necessary. It would be necessary in 2.5.8 if this patch were to be backported.



----------------------------------------------------------------------------------------
*Sonicle S.r.l. *: http://www.sonicle.com <http://www.sonicle.com/>
*Music: *http://www.gabrielebulfon.com <http://www.gabrielebulfon.com/>
*Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon

------------------------------------------------------------------------


*Da:* Ken Murchison <mu...@andrew.cmu.edu>
*A:* gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu
*Data:* 19 luglio 2016 15.02.36 CEST
*Oggetto:* Re: thread=refs


    I don't think much has changed in the threading code for a while.
    I would expect that the patch would apply pretty cleanly to 2.4.x.


    On 07/19/2016 02:58 AM, Gabriele Bulfon wrote:
    Wow! This is really interesting!
    What minimum version of cyrus sources can I use for these changes?
    At the moment I'm running servers with 2.4.12, on our illumos
    based distro XStreamOS.

    
----------------------------------------------------------------------------------------
    *Sonicle S.r.l. *: http://www.sonicle.com
    *Music: *http://www.gabrielebulfon.com
    *Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon

    ------------------------------------------------------------------------


    *Da:* Ken Murchison <mu...@andrew.cmu.edu>
    *A:* gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu
    *Data:* 14 luglio 2016 17.14.18 CEST
    *Oggetto:* Re: thread=refs


        I went ahead and committed an implementation of THREAD-REFS:
        
https://github.com/cyrusimap/cyrus-imapd/commit/16747e608f32f9dd9348988d3f83cb8f1b037ff6

        Per the draft, grouping by subject is skipped and threads
        (toplevel messages) are sorted by INTERNALDATE, while the
        messages within the threads are still sorted by SENTDATE.

        I confirmed that THREAD=REFERENCES is still correct, but I
        have nothing to compare THREAD=REFS results to. The current
        threading in Thunderbird is close, but it might be using
        INTERNALDATE throughout.


        On 07/12/2016 04:44 PM, Ken Murchison via Info-cyrus wrote:
        Gabriele,

        The attached patch is what I was thinking in terms of
        implementation. It skips the grouping by subject for REFS,
        but I didn't do the REFS-specific date handling. Contrary to
        what the THREAD=REFS draft says, I'm not sure if the special
        date handling should be done in step 4 or 6. I would have to
        did deeper into the code to see where is belongs.


        On 07/08/2016 11:36 AM, Gabriele Bulfon via Info-cyrus wrote:
        Mainly web clients, installed clients usually implements
        threading internally to overcome problems with the original
        references algorithm that is often confusing.

        The problem with references is that it includes subject
        grouping, that is an old netscape model of the 90s: today,
        we just need references within the headers ids, or we may
        take a wrong message in the thread just because it has a
        similar subject (for example automatic mails with same
        subjects would be treated as a thread, which is wrong).

        Now, we're staring to implement threading view on our web
        collaboration software, running on cyrus.
        So we are investigating how RoundCube is doing threading on
        a dovecot installation, and we found it to be the same as
        the client algrithm of Thunderbird, which is fine. Looking
        at the protocol, it uses REFS first, probably because it
        has no subject grouping by definition, and it should have a
        better date sorting. Should, because I found that Dovecot
        does not sort it reversed...

        Maybe I will ask Dovecot guys why they choose to keep sort
        same as references: I suspect that claim to support refs,
        but actually the do the same references functions, but
        never do subject grouping.

        
----------------------------------------------------------------------------------------
        *Sonicle S.r.l. *: http://www.sonicle.com
        *Music: *http://www.gabrielebulfon.com
        *Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon

        ------------------------------------------------------------------------


        *Da:* Ken Murchison <mu...@andrew.cmu.edu>
        *A:* gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu
        *Data:* 8 luglio 2016 17.17.32 CEST
        *Oggetto:* Re: thread=refs




            On 07/08/2016 11:08 AM, Gabriele Bulfon wrote:
            Ok, sure, but still two issues remain other than the
            draft:
            - we need to get rid of subject grouping in REFS, it
            only brings disorder, merging stuff that is not related

            I believe that the parameterization of the core
            functions should be able to handle this.


            - I would try to guess why dovecot does not change
            sorting in REFS, keeping it same as REFERENCES

            I would contact that Dovecot authors and find out which
            version of the THREAD=REFS draft they based their work on.

            BTW, which clients use THREAD=REFS?



            
----------------------------------------------------------------------------------------
            *Sonicle S.r.l. *: http://www.sonicle.com
            *Music: *http://www.gabrielebulfon.com
            *Quantum Mechanics :
            *http://www.cdbaby.com/cd/gabrielebulfon

            
------------------------------------------------------------------------


            *Da:* Ken Murchison <mu...@andrew.cmu.edu>
            *A:* gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu
            *Data:* 8 luglio 2016 16.39.17 CEST
            *Oggetto:* Re: thread=refs


                Is there an actual RFC? All I find is
                draft-ietf-morg-inthread-01. Looking at that
                draft, the only difference between REFS ad
                REFERENCES is this:

                     THREAD=REFS sorts threads by the most recent INTERNALDATE 
in each
                     thread, replacing THREAD=REFERENCES step (4). This means 
that when a
                     new message arrives, its thread becomes the latest thread. 
(Note
                     that while threads are sorted by arrival date, messages 
within a
                     thread are sorted by sent date, just as for 
THREAD=REFERENCES.)


                This being the case, I don't think we need two
                copies of the threading functions. I'd modify the
                exiting functions to take an additional parameter
                to specify whether we're doing REFS or REFERENCES
                and then have 2 wrapper functions which call the
                main function with the parameter set appropriately
                for the given algorithm.


                On 07/08/2016 10:03 AM, Gabriele Bulfon wrote:
                Ok, it works :) but checking against dovecot
                implementation, it looks like they have refs
                order same as references, but without subject
                grouping. AFAIK the RFC on refs says ordering of
                dates within the group should be reversed. Am I
                wrong?

                
----------------------------------------------------------------------------------------
                *Sonicle S.r.l. *: http://www.sonicle.com
                *Music: *http://www.gabrielebulfon.com
                *Quantum Mechanics :
                *http://www.cdbaby.com/cd/gabrielebulfon

                
------------------------------------------------------------------------


                *Da:* Gabriele Bulfon via Info-cyrus
                <info-cyrus@lists.andrew.cmu.edu>
                *A:* Ken Murchison <mu...@andrew.cmu.edu>
                info-cyrus@lists.andrew.cmu.edu
                *Data:* 8 luglio 2016 15.22.56 CEST
                *Oggetto:* Re: thread=refs


                    Testing ;) and checking against a dovecot
                    machine with refs and same messages.
                    Will let you know

                    
----------------------------------------------------------------------------------------
                    *Sonicle S.r.l. *: http://www.sonicle.com
                    *Music: *http://www.gabrielebulfon.com
                    *Quantum Mechanics :
                    *http://www.cdbaby.com/cd/gabrielebulfon

                    
------------------------------------------------------------------------


                    *Da:* Ken Murchison <mu...@andrew.cmu.edu>
                    *A:* gbul...@sonicle.com
                    info-cyrus@lists.andrew.cmu.edu
                    *Data:* 8 luglio 2016 15.02.38 CEST
                    *Oggetto:* Re: thread=refs




                        On 07/07/2016 02:03 PM, Gabriele Bulfon
                        via Info-cyrus wrote:
                        I can finally get back to this after so
                        many months!
                        I checked the sources, and I actually
                        see it doesn't look very hard.

                        Looks like:
                        - renaming all functions like
                        "index_thread_ref" into
                        "index_thread_references"
                        - duplicate them as "index_thread_refs"
                        - let "references" alg call the
                        "references" funcs
                        - add support for "refs" in thread_algs
                        and let them call the "refs" funcs

                        Makes sense.



                        then:
                        - completely remove the call to
                        "ref_group_subjects", we don't want it
                        at all in refs
                        - change the sortcrit to use the
                        SORT_REVERSE modifier

                        As long as you mean making these changes
                        for just the "refs" variant and not both.




                        what do you think? may be fine?

                        
----------------------------------------------------------------------------------------
                        *Sonicle S.r.l. *: http://www.sonicle.com
                        *Music: *http://www.gabrielebulfon.com
                        *Quantum Mechanics :
                        *http://www.cdbaby.com/cd/gabrielebulfon

                        
------------------------------------------------------------------------


                        *Da:* Ken Murchison <mu...@andrew.cmu.edu>
                        *A:* info-cyrus@lists.andrew.cmu.edu
                        *Data:* 5 ottobre 2015 14.04.02 CEST
                        *Oggetto:* Re: thread=refs


                            As far as I can tell, the last
                            specification for thread=refs was
                            
here:https://tools.ietf.org/html/draft-ietf-morg-inthread-01

                            To implement this you want to look
                            at index.c in the Cyrus source and
                            add another entry to the
                            thread_algs[] array. I'm guessing
                            that you can reuse a lot of the
                            existing index_thread_ref() code
                            (which is probably needs to be
                            renamed to index_thread_references()).



                            On 10/05/2015 06:07 AM, Gabriele
                            Bulfon wrote:
                            Great, Ken. Can you give me some
                            advice / pointer to the sources I
                            should look at?

                            Gabriele

                            
------------------------------------------------------------------------


                            *Da:* Ken Murchison
                            <mu...@andrew.cmu.edu>
                            *A:* info-cyrus@lists.andrew.cmu.edu
                            *Data:* 2 ottobre 2015 19.08.04 CEST
                            *Oggetto:* Re: thread=refs


                                On 10/02/2015 10:53 AM,
                                Gabriele Bulfon wrote:
                                Nice, it's not a big deal for
                                us to upgrade to new versions,
                                surely easier than porting to
                                Dovecot! ;)

                                So, maybe we can help with the
                                implementation.
                                In my mind, it's almost about
                                changing the
                                "thread=reference" and let it
                                omit the subject matching,
                                change sorting
                                and...maybe just this? How
                                much hard do you think it is?


                                That sounds about right from
                                what I remember of
                                THREAD=REFERENCES (which I
                                co-authored and implemented)
                                and THREAD=REFS (which I think
                                was last documented in 2010).



                                
------------------------------------------------------------------------


                                *Da:* Bron Gondwana
                                <br...@fastmail.fm>
                                *A:*
                                info-cyrus@lists.andrew.cmu.edu
                                *Data:* 2 ottobre 2015
                                12.59.08 CEST
                                *Oggetto:* Re: thread=refs


                                    No, there isn't. The
                                    conversations work in 3.0
                                    beta contains a lot of
                                    what would be required to
                                    efficiently implement
                                    THREAD=REFS, but nobody
                                    has done the work to
                                    implement it.
                                    It certainly will never be
                                    backported to the 2.4
                                    series, which is only
                                    getting security updates
                                    and fixes for major bugs now.
                                    Regards,
                                    Bron.
                                    On Fri, Oct 2, 2015, at
                                    18:40, Gabriele Bulfon wrote:
                                    Hi,
                                    we have systems running
                                    cyrus 2.4.12, where
                                    thread algorithms are
                                    only references and
                                    orderedsubject.
                                    Is there support for the
                                    thread=refs algorithm?
                                    Thanks
                                    Gabriele
                                    ----
                                    Cyrus Home Page:
                                    http://www.cyrusimap.org/
                                    List Archives/Info:
                                    
http://lists.andrew.cmu.edu/pipermail/info-cyrus/
                                    To Unsubscribe:
                                    
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
                                    --
                                    Bron Gondwana
                                    br...@fastmail.fm



                                ----
                                Cyrus Home Page:http://www.cyrusimap.org/
                                List 
Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
                                To Unsubscribe:
                                
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


-- Kenneth Murchison
                                Principal Systems Software Engineer
                                Carnegie Mellon University



                            ----
                            Cyrus Home Page:http://www.cyrusimap.org/
                            List 
Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
                            To Unsubscribe:
                            
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-- Kenneth Murchison
                            Principal Systems Software Engineer
                            Carnegie Mellon University



                        ----
                        Cyrus Home Page:http://www.cyrusimap.org/
                        List 
Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
                        To Unsubscribe:
                        https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-- Kenneth Murchison
                        Principal Systems Software Engineer
                        Carnegie Mellon University

                    ----
                    Cyrus Home Page:http://www.cyrusimap.org/
                    List 
Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
                    To Unsubscribe:
                    https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus


-- Kenneth Murchison
                Principal Systems Software Engineer
                Carnegie Mellon University


-- Kenneth Murchison
            Principal Systems Software Engineer
            Carnegie Mellon University



        ----
        Cyrus Home Page:http://www.cyrusimap.org/
        List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
        To Unsubscribe:
        https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-- Kenneth Murchison
        Principal Systems Software Engineer
        Carnegie Mellon University


        ----
        Cyrus Home Page:http://www.cyrusimap.org/
        List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
        To Unsubscribe:
        https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

-- Kenneth Murchison
        Principal Systems Software Engineer
        Carnegie Mellon University


-- Kenneth Murchison
    Principal Systems Software Engineer
    Carnegie Mellon University


--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University

----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus

Reply via email to