Gabriele,
The attached patch is what I was thinking in terms of implementation.
It skips the grouping by subject for REFS, but I didn't do the
REFS-specific date handling. Contrary to what the THREAD=REFS draft
says, I'm not sure if the special date handling should be done in step 4
or 6. I would have to did deeper into the code to see where is belongs.
On 07/08/2016 11:36 AM, Gabriele Bulfon via Info-cyrus wrote:
Mainly web clients, installed clients usually implements threading
internally to overcome problems with the original references algorithm
that is often confusing.
The problem with references is that it includes subject grouping, that
is an old netscape model of the 90s: today, we just need references
within the headers ids, or we may take a wrong message in the thread
just because it has a similar subject (for example automatic mails
with same subjects would be treated as a thread, which is wrong).
Now, we're staring to implement threading view on our web
collaboration software, running on cyrus.
So we are investigating how RoundCube is doing threading on a dovecot
installation, and we found it to be the same as the client algrithm of
Thunderbird, which is fine. Looking at the protocol, it uses REFS
first, probably because it has no subject grouping by definition, and
it should have a better date sorting. Should, because I found that
Dovecot does not sort it reversed...
Maybe I will ask Dovecot guys why they choose to keep sort same as
references: I suspect that claim to support refs, but actually the do
the same references functions, but never do subject grouping.
----------------------------------------------------------------------------------------
*Sonicle S.r.l. *: http://www.sonicle.com <http://www.sonicle.com/>
*Music: *http://www.gabrielebulfon.com <http://www.gabrielebulfon.com/>
*Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon
------------------------------------------------------------------------
*Da:* Ken Murchison <mu...@andrew.cmu.edu>
*A:* gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu
*Data:* 8 luglio 2016 17.17.32 CEST
*Oggetto:* Re: thread=refs
On 07/08/2016 11:08 AM, Gabriele Bulfon wrote:
Ok, sure, but still two issues remain other than the draft:
- we need to get rid of subject grouping in REFS, it only brings
disorder, merging stuff that is not related
I believe that the parameterization of the core functions should
be able to handle this.
- I would try to guess why dovecot does not change sorting in
REFS, keeping it same as REFERENCES
I would contact that Dovecot authors and find out which version of
the THREAD=REFS draft they based their work on.
BTW, which clients use THREAD=REFS?
----------------------------------------------------------------------------------------
*Sonicle S.r.l. *: http://www.sonicle.com
*Music: *http://www.gabrielebulfon.com
*Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon
------------------------------------------------------------------------
*Da:* Ken Murchison <mu...@andrew.cmu.edu>
*A:* gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu
*Data:* 8 luglio 2016 16.39.17 CEST
*Oggetto:* Re: thread=refs
Is there an actual RFC? All I find is
draft-ietf-morg-inthread-01. Looking at that draft, the only
difference between REFS ad REFERENCES is this:
THREAD=REFS sorts threads by the most recent INTERNALDATE in each
thread, replacing THREAD=REFERENCES step (4). This means that when
a
new message arrives, its thread becomes the latest thread. (Note
that while threads are sorted by arrival date, messages within a
thread are sorted by sent date, just as for THREAD=REFERENCES.)
This being the case, I don't think we need two copies of the
threading functions. I'd modify the exiting functions to take
an additional parameter to specify whether we're doing REFS
or REFERENCES and then have 2 wrapper functions which call
the main function with the parameter set appropriately for
the given algorithm.
On 07/08/2016 10:03 AM, Gabriele Bulfon wrote:
Ok, it works :) but checking against dovecot implementation,
it looks like they have refs order same as references, but
without subject grouping. AFAIK the RFC on refs says
ordering of dates within the group should be reversed. Am I
wrong?
----------------------------------------------------------------------------------------
*Sonicle S.r.l. *: http://www.sonicle.com
*Music: *http://www.gabrielebulfon.com
*Quantum Mechanics : *http://www.cdbaby.com/cd/gabrielebulfon
------------------------------------------------------------------------
*Da:* Gabriele Bulfon via Info-cyrus
<info-cyrus@lists.andrew.cmu.edu>
*A:* Ken Murchison <mu...@andrew.cmu.edu>
info-cyrus@lists.andrew.cmu.edu
*Data:* 8 luglio 2016 15.22.56 CEST
*Oggetto:* Re: thread=refs
Testing ;) and checking against a dovecot machine with
refs and same messages.
Will let you know
----------------------------------------------------------------------------------------
*Sonicle S.r.l. *: http://www.sonicle.com
<http://www.sonicle.com/>
*Music: *http://www.gabrielebulfon.com
*Quantum Mechanics :
*http://www.cdbaby.com/cd/gabrielebulfon
------------------------------------------------------------------------
*Da:* Ken Murchison <mu...@andrew.cmu.edu>
*A:* gbul...@sonicle.com info-cyrus@lists.andrew.cmu.edu
*Data:* 8 luglio 2016 15.02.38 CEST
*Oggetto:* Re: thread=refs
On 07/07/2016 02:03 PM, Gabriele Bulfon via
Info-cyrus wrote:
I can finally get back to this after so many months!
I checked the sources, and I actually see it
doesn't look very hard.
Looks like:
- renaming all functions like "index_thread_ref"
into "index_thread_references"
- duplicate them as "index_thread_refs"
- let "references" alg call the "references" funcs
- add support for "refs" in thread_algs and let
them call the "refs" funcs
Makes sense.
then:
- completely remove the call to
"ref_group_subjects", we don't want it at all in refs
- change the sortcrit to use the SORT_REVERSE modifier
As long as you mean making these changes for just
the "refs" variant and not both.
what do you think? may be fine?
----------------------------------------------------------------------------------------
*Sonicle S.r.l. *: http://www.sonicle.com
*Music: *http://www.gabrielebulfon.com
*Quantum Mechanics :
*http://www.cdbaby.com/cd/gabrielebulfon
------------------------------------------------------------------------
*Da:* Ken Murchison <mu...@andrew.cmu.edu>
*A:* info-cyrus@lists.andrew.cmu.edu
*Data:* 5 ottobre 2015 14.04.02 CEST
*Oggetto:* Re: thread=refs
As far as I can tell, the last specification
for thread=refs was
here:https://tools.ietf.org/html/draft-ietf-morg-inthread-01
To implement this you want to look at index.c
in the Cyrus source and add another entry to
the thread_algs[] array. I'm guessing that you
can reuse a lot of the existing
index_thread_ref() code (which is probably
needs to be renamed to index_thread_references()).
On 10/05/2015 06:07 AM, Gabriele Bulfon wrote:
Great, Ken. Can you give me some advice /
pointer to the sources I should look at?
Gabriele
------------------------------------------------------------------------
*Da:* Ken Murchison <mu...@andrew.cmu.edu>
*A:* info-cyrus@lists.andrew.cmu.edu
*Data:* 2 ottobre 2015 19.08.04 CEST
*Oggetto:* Re: thread=refs
On 10/02/2015 10:53 AM, Gabriele Bulfon wrote:
Nice, it's not a big deal for us to
upgrade to new versions, surely easier
than porting to Dovecot! ;)
So, maybe we can help with the
implementation.
In my mind, it's almost about changing
the "thread=reference" and let it omit
the subject matching, change sorting
and...maybe just this? How much hard do
you think it is?
That sounds about right from what I
remember of THREAD=REFERENCES (which I
co-authored and implemented) and
THREAD=REFS (which I think was last
documented in 2010).
------------------------------------------------------------------------
*Da:* Bron Gondwana <br...@fastmail.fm>
*A:* info-cyrus@lists.andrew.cmu.edu
*Data:* 2 ottobre 2015 12.59.08 CEST
*Oggetto:* Re: thread=refs
No, there isn't. The conversations
work in 3.0 beta contains a lot of
what would be required to efficiently
implement THREAD=REFS, but nobody has
done the work to implement it.
It certainly will never be backported
to the 2.4 series, which is only
getting security updates and fixes
for major bugs now.
Regards,
Bron.
On Fri, Oct 2, 2015, at 18:40,
Gabriele Bulfon wrote:
Hi,
we have systems running cyrus
2.4.12, where thread algorithms are
only references and orderedsubject.
Is there support for the thread=refs
algorithm?
Thanks
Gabriele
----
Cyrus Home Page:
http://www.cyrusimap.org/
List Archives/Info:
http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
--
Bron Gondwana
br...@fastmail.fm
----
Cyrus Home Page:http://www.cyrusimap.org/
List
Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University
----
Cyrus Home Page:http://www.cyrusimap.org/
List
Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University
----
Cyrus Home Page:http://www.cyrusimap.org/
List
Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University
----
Cyrus Home Page:http://www.cyrusimap.org/
List Archives/Info:http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University
--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
--
Kenneth Murchison
Principal Systems Software Engineer
Carnegie Mellon University
diff --git a/imap/imapd.c b/imap/imapd.c
index f112fd3..2f0fbce 100644
--- a/imap/imapd.c
+++ b/imap/imapd.c
@@ -323,6 +323,7 @@ static struct capa_struct base_capabilities[] = {
{ "SORT=UID", 2 }, /* not standard */
{ "THREAD=ORDEREDSUBJECT", 2 },
{ "THREAD=REFERENCES", 2 },
+ { "THREAD=REFS", 2 }, /* not standard */
{ "ANNOTATEMORE", 2 },
{ "ANNOTATE-EXPERIMENT-1", 2 },
{ "METADATA", 2 },
diff --git a/imap/index.c b/imap/index.c
index a7d2ce5..e80199e 100644
--- a/imap/index.c
+++ b/imap/index.c
@@ -167,9 +167,12 @@ static void index_thread_orderedsubj(struct index_state *state,
static void index_thread_sort(Thread *root, const struct sortcrit *sortcrit);
static void index_thread_print(struct index_state *state,
Thread *threads, int usinguid);
-static void index_thread_ref(struct index_state *state,
- unsigned *msgno_list, unsigned int nmsg,
- int usinguid);
+static void index_thread_references(struct index_state *state,
+ unsigned *msgno_list, unsigned int nmsg,
+ int usinguid);
+static void index_thread_refs(struct index_state *state,
+ unsigned *msgno_list, unsigned int nmsg,
+ int usinguid);
static struct seqset *_parse_sequence(struct index_state *state,
const char *sequence, int usinguid);
@@ -178,7 +181,8 @@ static void massage_header(char *hdr);
/* NOTE: Make sure these are listed in CAPABILITY_STRING */
static const struct thread_algorithm thread_algs[] = {
{ "ORDEREDSUBJECT", index_thread_orderedsubj },
- { "REFERENCES", index_thread_ref },
+ { "REFERENCES", index_thread_references },
+ { "REFS", index_thread_refs },
{ NULL, NULL }
};
@@ -6069,11 +6073,11 @@ static void index_thread_search(struct index_state *state,
}
/*
- * Guts of the REFERENCES algorithms. Behavior is tweaked with loadcrit[],
+ * Guts of the REFERENCES/REFS algorithms. Behavior is tweaked with loadcrit[],
* searchproc() and sortcrit[].
*/
static void _index_thread_ref(struct index_state *state, unsigned *msgno_list,
- unsigned int nmsg,
+ unsigned int nmsg, int groupbysubj,
const struct sortcrit loadcrit[],
int (*searchproc) (MsgData *),
const struct sortcrit sortcrit[], int usinguid)
@@ -6137,8 +6141,10 @@ static void _index_thread_ref(struct index_state *state, unsigned *msgno_list,
/* Step 4: sort the root set */
ref_sort_root(rootset.root);
- /* Step 5: group root set by subject */
- ref_group_subjects(rootset.root, rootset.nroot, &newnode);
+ if (groupbysubj) {
+ /* Step 5: group root set by subject */
+ ref_group_subjects(rootset.root, rootset.nroot, &newnode);
+ }
/* Optionally search threads (to be used by REFERENCES derivatives) */
if (searchproc) index_thread_search(state, rootset.root, searchproc);
@@ -6159,9 +6165,9 @@ static void _index_thread_ref(struct index_state *state, unsigned *msgno_list,
/*
* Thread a list of messages using the REFERENCES algorithm.
*/
-static void index_thread_ref(struct index_state *state,
- unsigned *msgno_list, unsigned int nmsg,
- int usinguid)
+static void index_thread_references(struct index_state *state,
+ unsigned *msgno_list, unsigned int nmsg,
+ int usinguid)
{
static const struct sortcrit loadcrit[] =
{{ LOAD_IDS, 0, {{NULL,NULL}} },
@@ -6172,7 +6178,27 @@ static void index_thread_ref(struct index_state *state,
{{ SORT_DATE, 0, {{NULL,NULL}} },
{ SORT_SEQUENCE, 0, {{NULL,NULL}} }};
- _index_thread_ref(state, msgno_list, nmsg, loadcrit, NULL, sortcrit, usinguid);
+ _index_thread_ref(state, msgno_list, nmsg, 1 /* groupbysubj */,
+ loadcrit, NULL, sortcrit, usinguid);
+}
+
+/*
+ * Thread a list of messages using the REFS algorithm.
+ */
+static void index_thread_refs(struct index_state *state,
+ unsigned *msgno_list, unsigned int nmsg,
+ int usinguid)
+{
+ static const struct sortcrit loadcrit[] =
+ {{ LOAD_IDS, 0, {{NULL,NULL}} },
+ { SORT_DATE, 0, {{NULL,NULL}} },
+ { SORT_SEQUENCE, 0, {{NULL,NULL}} }};
+ static const struct sortcrit sortcrit[] =
+ {{ SORT_DATE, 0, {{NULL,NULL}} },
+ { SORT_SEQUENCE, 0, {{NULL,NULL}} }};
+
+ _index_thread_ref(state, msgno_list, nmsg, 0 /* groupbysubj */,
+ loadcrit, NULL, sortcrit, usinguid);
}
/*
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus
