st UID. (Supporting out-of-order
indexing would be rather difficult to keep track of.)
Q2 : WHen Indexing an email, the data is not passed by "build_key". Why so ? What is the
link with "build_more" ?
The idea is that it calls something like:
- build_key(type=hdr, hdr_name=From)
-
On 13 Jan 2019, at 10.45, Joan Moreau via dovecot wrote:
>
> Now, I can see in the logs that several times, the dovecot calls the
> fts_backend_xapian_update_set_mailbox with box == NULL. WHy so ?
>
fts-api.h says:
/* Switch to updating the specified mailbox. box may also be set to NULL to
because fts_squat is set to be deleted
Xapian and similar libraries offers a very easy interface for FTS
(and basically, I have done it already)
On 2019-01-07 18:31, Michael Slusarz wrote:
Maybe a dumb question (I admit I haven't followed this thread very closely)...
But why are you writi
I found the solution o this using
SEQ_RANGE_ARRAY_ADD(&RESULT->DEFINITE_UIDS, UID);
Now, I can see in the logs that several times, the dovecot calls the
fts_backend_xapian_update_set_mailbox with box == NULL. WHy so ?
THank you
On 2019-01-12 21:40, Joan Moreau via dovecot wrote:
I somehow
I somehow fixed the folder issue. (seems some unix rights after too many
tests)
Getting back on the "fts_results" structure:
I am trying:
I_ARRAY_INIT(&(RESULT->DEFINITE_UIDS),R->SIZE);
I_ARRAY_INIT(&(RESULT->MAYBE_UIDS),0);
uint32_t uid;
for(i=0;isize;i++)
{
try
{
uid=atol(back
additionally, my logic is that the backend stores one databalse per
mailox in /xapian-indexes (in the "root" dir of the user), the name od
the database is the GUID of the mailbox
For INBOX, that works perfectly, and database is properly createdm and
backed starts indexing all emails
For othe
THank you
Now, for the results
I see the member of fts_result is :
ARRAY_TYPE(seq_range) definite_uids;
I have the UID as a aray of uint32_t *
How to put my UIDs into this "definite_uids" ? Obviously this is not a
simple array/pointer. How to say someting similar to
result->definite_uids
On 11 Jan 2019, at 21.23, Joan Moreau via dovecot wrote:
>
> The below patch resolves the compilation error
>
> $ diff -p compat.h compat.h.joan
> *** compat.h 2019-01-11 20:21:00.726625427 +0100
> --- compat.h.joan 2019-01-11 20:14:41.729109919 +0100
> *** struct iovec;
> *** 202,2
El 04/01/19 a las 03:20, Joan Moreau via dovecot escribió:
> What about consedering linking Dovecot with Xapian librairies instead of
> going to nightmare Solr ?
> https://xapian.org/features
given that notmuch already does a good job at indexing email (although
only supports maildirs afaik), woul
first indexes all the
missing mails. So the latest UID is supposed to be the greatest UID.
(Supporting out-of-order indexing would be rather difficult to keep
track of.)
Q2 : WHen Indexing an email, the data is not passed by "build_key". Why
so ? What is the link with "build_
The idea is that it calls something like:
- build_key(type=hdr, hdr_name=From)
- build_more(" t...@iki.fi")
- build_key(type=hdr, hdr_name=Subject)
- build_more("Re: Solr -> Xapian ?")
- build_key(type=body_part)
- build_more("message body piece")
- build_more(&q
so ? What is the link with "build_more" ?
The idea is that it calls something like:
- build_key(type=hdr, hdr_name=From)
- build_more("
t...@iki.fi")
- build_key(type=hdr, hdr_name=Subject)
e rather difficult to keep
track of.)
Q2 : WHen Indexing an email, the data is not passed by "build_key". Why
so ? What is the link with "build_more" ?
The idea is that it calls something like:
- build_key(type=hdr, hdr_name=From)
- build_more("t...@iki.fi")
-
idea is that it calls something like:
- build_key(type=hdr, hdr_name=From)
- build_more("t...@iki.fi")
- build_key(type=hdr, hdr_name=Subject)
- build_more("Re: Solr -> Xapian ?")
- build_key(type=body_part)
- build_more("message body piece")
- build_more("messa
2 : WHen Indexing an email, the data is not passed by "build_key". Why so ? What is the
link with "build_more" ?
The idea is that it calls something like:
- build_key(type=hdr, hdr_name=From)
- build_more("t...@iki.fi")
- build_key(type=hdr, hdr_name=Subjec
name=From)
- build_more("t...@iki.fi")
- build_key(type=hdr, hdr_name=Subject)
- build_more("Re: Solr -> Xapian ?")
- build_key(type=body_part)
- build_more("message body piece")
- build_more("message body piece2")
...
> Q3 : Searching/Lookup : TH
Maybe a dumb question (I admit I haven't followed this thread very closely)...
But why are you writing a new FTS driver? If squat allegedly does everything
you need it to do, why don't you just take that plugin and fix it up to do what
you need? That seems way easier than trying to create a FT
Hi
ANyone to answer specifically ?
Q1 : get_last_uid -> Is this the last UID indexed (which may be not the
greatest value), or the gratest value (which may not be the latest) (the
code of existing plugins is unclear about this, Solr looks for the
greatest for insance)
Q2 : WHen Indexing an
and finally , for fts_backend__lookup_multi, why is that backend
dependent ?
Would- nt the below function below be the same for any backend ?
Waiting fro your feedback on all those questions
Thank you
JM
-
static int fts_backend_xapian_lookup_multi(struct fts_backe
for fts_backend_xxx_lookup, where is specidifed in which field (to, cc,
subject, body, from, all) to lookup ?
On 2019-01-06 16:03, Joan Moreau wrote:
For "rescan " and "optimize", wouldn't it be the dovecot core who indicate which are to be dismissed (expunged), or re-ask for indexing a particul
For "rescan " and "optimize", wouldn't it be the dovecot core who
indicate which are to be dismissed (expunged), or re-ask for indexing a
particular (or all) uid ? WHy would the backend be aware of the
transactions on the mailbox ???
There is alredy "fts_backend_xxx_update_expunge", so I beleiv
also, for fts_backend_solr_update_set_build_key -> where is the data (of
the hdr_name or the body) ?
On 2019-01-06 14:10, Joan Moreau wrote:
for the "last uid"-> this is not the last added, but the maximum of the UID in the indexed emails, right ?
On 2019-01-06 11:53, Joan Moreau via dovecot
for the "last uid"-> this is not the last added, but the maximum of the
UID in the indexed emails, right ?
On 2019-01-06 11:53, Joan Moreau via dovecot wrote:
Thank you
I still don't get the "build_key" function. The email (body, hearders, .. and the uid) is the one (and only) to index . Wha
Thank you
I still don't get the "build_key" function. The email (body, hearders,
.. and the uid) is the one (and only) to index . What "key" is that
function referring to ? Or is the "key" here the actual email ?
On 2019-01-06 08:43, Stephan Bosch wrote:
Op 06/01/2019 om 01:00 schreef Joan M
Op 06/01/2019 om 01:00 schreef Joan Moreau:
Anyone willing to explain those functions ?
Most notably " get_last_uid"
From src/plugins/fts/fts-api.h:
/* Get the last_uid for the mailbox. */
int fts_backend_get_last_uid(struct fts_backend *backend, struct mailbox
*box,
uint
Anyone willing to explain those functions ?
Most notably " get_last_uid" "set_build_key" "build_more" , what is refresh
versus rescan ?
On January 5, 2019 14:23:10 Joan Moreau via dovecot
wrote:
Thank Stephan
I basically need to know the role/description of each of the functions of
th
Anyone willing to explain those functions ?
On January 5, 2019 14:23:10 Joan Moreau via dovecot
wrote:
Thank Stephan
I basically need to know the role/description of each of the functions of
the fts_backend:
struct fts_backend fts_backend_xapian = {
.name = "xapian",
.flags = FTS_BACKEND_F
Thank Stephan
I basically need to know the role/description of each of the functions
of the fts_backend:
struct fts_backend fts_backend_xapian = {
.name = "xapian",
.flags = FTS_BACKEND_FLAG_NORMALIZE_INPUT, -> WHAT OTHER FLAGS ?
{
fts_backend_xapian_alloc,
fts_backend_xapian_init,
fts_bac
Op 04/01/2019 om 11:17 schreef Joan Moreau via dovecot:
Why not, but please guide me about the core structure (mandatory
funcitons, etc..) of a typical Dovecot FTS plugin
The Dovecot API documentation is not exhaustive everywhere, but the
basics are documented. The remaining questions can
Also, a description of the "to be" functions of the backend:
struct fts_backend fts_backend_xapian = {
.name = "xapian",
.flags = FTS_BACKEND_FLAG_NORMALIZE_INPUT, -> WHAT OTHER FLAGS ?
{
fts_backend_xapian_alloc,
fts_backend_xapian_init,
fts_backend_xapian_deinit,
fts_backend_xapian_get_last
Yes but:
1 - is there a documentation of the main object ? (fts_backend,
mail_user, mailbox, etc..)
2 - What are the mandatory functions ?
3 - Search : Supposedly, the FTS shall have several parameters : the
keyword(s), the user & mailbox, and the fields (to, from, body, etc..)
to be includ
A starting point would be to have a look at the current FTS plugins:
https://github.com/dovecot/core/tree/master/src/plugins/fts-solrandhttps://github.com/dovecot/core/tree/master/src/plugins/fts-squat
-M
Am Freitag, den 04.01.2019, 18:17 +0800 schrieb Joan Moreau via
dovecot:
> Why not, but plea
Why not, but please guide me about the core structure (mandatory
funcitons, etc..) of a typical Dovecot FTS plugin
On 2019-01-04 17:20, Aki Tuomi wrote:
I hope you are aware that "linking with Xapian" requires somewhat more work than just -lxapian in linker? If you or someone feels like writin
I hope you are aware that "linking with Xapian" requires somewhat more work
than just -lxapian in linker? If you or someone feels like writing fts_xapian,
go for it.
Aki
> On 04 January 2019 at 08:20 Joan Moreau via dovecot
> wrote:
>
>
> What about consedering linking Dovecot with Xapian
What about consedering linking Dovecot with Xapian librairies instead of
going to nightmare Solr ?
https://xapian.org/features
On 2019-01-02 17:10, John Tulp wrote:
On Wed, 2019-01-02 at 00:59 -0800, M. Balridge wrote: The main problem is :
After some time of indexing from Dovecot, Dovecot
35 matches
Mail list logo
