Re: Sieve not getting recompiled

2024-04-20 Thread Joan Moreau via dovecot
I changed it to the following to stick to the doc sieve = file:/mails/%d/%n/sieve/ sieve_after = file:/mails/sieve/after.sieve sieve_default = file:/mails/sieve/before.sieve sieve_before = file:/mails/sieve/before.sieve Still no scripts are compiled/executed (and

Sieve not getting recompiled

2024-04-20 Thread Joan Moreau via dovecot
Hi I have sieve = /mails/%d/%n/sieve/roundcube.sieve sieve_after = /mails/sieve/after.sieve sieve_before = /mails/sieve/before.sieve sieve_dir = /mails/%d/%n/sieve/ sieve_global_dir = /mails/sieve/ But sieve scripts are not compiled and not executed It

Re: exfat not supported ?

2024-04-20 Thread Joan Moreau via dovecot
That resolve the fisrt bug but I get now : Error: link(/xxx/dovecot.list.index.log, /xxx/dovecot.list.index.log.2) failed: Operation not permitted On 2024-04-21 02:02, Aki Tuomi via dovecot wrote: Try setting lock_method = dotlock Aki On 20/04/2024 15:32 EEST Joan Moreau via dovecot

Re: exfat not supported ?

2024-04-20 Thread Joan Moreau via dovecot
: On 20/04/2024 12:27 EEST Joan Moreau via dovecot wrote: Hi Would placing my storage on a exfat partition work ? If no, why ? Thank you ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org I can't see

thread->detach() creates confusion of dovecot

2024-04-20 Thread Joan Moreau via dovecot
Hi When I try to "detach" (https://en.cppreference.com/w/cpp/thread/thread/detach) a thread running inside a plugin, it seems the core dovecot has some influence on that , tries to close this for some unknown reason and usually ends up crashing What is the cause of this ? Thank you

exfat not supported ?

2024-04-20 Thread Joan Moreau via dovecot
Hi Would placing my storage on a exfat partition work ? If no, why ? Thank you ___ dovecot mailing list -- dovecot@dovecot.org To unsubscribe send an email to dovecot-le...@dovecot.org

Re: [EXT] Re: How to get a memory pointer in the core process

2024-03-14 Thread Joan Moreau via dovecot
Thanks Eduardo I am trying to avoid closing/ reopening a file pointer to the exact same file between each call to the plugin On 14 March 2024 20:08:37 Eduardo M KALINOWSKI via dovecot wrote: On 14/03/2024 02:49, Joan Moreau via dovecot wrote: No, you don´t understand

Re: [EXT] Re: How to get a memory pointer in the core process

2024-03-13 Thread Joan Moreau via dovecot
th a) services (internal or external), such as redis, sql, or something else b) storing things to disk Aki On 13/03/2024 18:45 EET Joan Moreau via dovecot wrote:   No, I am not referring to that I want to create an

Re: [EXT] Re: How to get a memory pointer in the core process

2024-03-13 Thread Joan Moreau via dovecot
No, I am not referring to that I want to create an object at first call in memory that object would be retrievable at second and furthers calls of the plugin, as long as dovecot is running On 2024-03-13 16:29, Aki Tuomi via dovecot wrote: Not really no. You should use e.g. dict inteface

Re: How to get a memory pointer in the core process

2024-03-13 Thread Joan Moreau via dovecot
Keep a pointer in memory retrievable each time a plugin is called So the plugin keep the memory, not has to restart everything at each call On 12 March 2024 08:53:38 Aki Tuomi via dovecot wrote: On 11/03/2024 10:42 EET Joan Moreau wrote: Hi Is it possible,

How to get a memory pointer in the core process

2024-03-11 Thread Joan Moreau via dovecot
Hi Is it possible, from a plugin perspective, to create and recover a pointer in the core process (i.e. memory not lost between 2 calls to the plugin, even after the "deinit" of the plugin" ) ? Thanks ___ dovecot mailing list -- dovecot@dovecot.org To

Re: FTS Xapian -> FTS core issue

2019-06-09 Thread Joan Moreau via dovecot
The issue is not in plug-in then. Maybe Aki or Timo knows where does this bug come from ? On 2019-06-07 14:09, Daniel Miller wrote: Yes, latest git version. The logs show (as I read them) returned results - yet nothing shows in the client. The logs look the same (with different numbers)

Re: FTS Xapian

2019-06-09 Thread Joan Moreau via dovecot
Hi Are you using the latest git version ? WHich part exactly of your logs relates to "virtual folders do not work" ? On 2019-06-05 13:08, Daniel Miller via dovecot wrote: Logs: Jun 5 06:02:25 bubba dovecot: imap(dmil...@amfes.com)<25877>: Opening DB (RO)

Re: FTS Xapian

2019-06-04 Thread Joan Moreau via dovecot
Hi Can you post your dovecot conf file and the subset of the log files related to the issue ? thanks On June 5, 2019 9:29:13 AM Daniel Miller via dovecot wrote: For my primary namespace this is working fine - thanks to the developers! It also appears to work great for shared folders

Further issues on FTS engine

2019-05-20 Thread Joan Moreau via dovecot
Hi, Additionally to the long list of problem on the FTS previously discussed, here a new: WHen I reset the indexes, the indexer-worker seems paralelleizing the indexing (which is good), however, the number available in "ps aux | grep dove" shows that it does not move: dovecot 28549 0.0

Re: FTS delays

2019-04-21 Thread Joan Moreau via dovecot
jserver dovecot[14251]: imap(j...@grosjo.net)<15709>: Testing if wildcard Apr 21 11:08:39 gjserver dovecot[14251]: imap(j...@grosjo.net)<15709>: Query : ( bcc:milao ) OR ( cc:milao ) OR ( from:milao ) OR ( subject:milao ) OR ( to:milao ) Apr 21 11:08:39 gjserver dovecot[14251]: ima

Re: FTS delays

2019-04-21 Thread Joan Moreau via dovecot
0095331209 3121 -> LOOPING FOREVER On 2019-04-21 09:57, Timo Sirainen via dovecot wrote: On 3 Apr 2019, at 20.30, Joan Moreau via dovecot wrote: doveadm search -u j...@grosjo.net mailbox inbox text milan output doveadm(j...@grosjo.net): Info: Query : ( bcc:inbox OR body:inbox OR cc:inbo

Re: FTS delays

2019-04-21 Thread Joan Moreau via dovecot
Sirainen via dovecot wrote: On 3 Apr 2019, at 20.30, Joan Moreau via dovecot wrote: doveadm search -u j...@grosjo.net mailbox inbox text milan output doveadm(j...@grosjo.net): Info: Query : ( bcc:inbox OR body:inbox OR cc:inbox OR from:inbox OR message-id:inbox OR subject:inbox OR to:inbox

Re: FTS delays

2019-04-21 Thread Joan Moreau via dovecot
h parameters On 2019-04-21 10:31, Joan Moreau via dovecot wrote: For this first point, the problem is that dovecot core sends TWICE the request and "Inbox" appears in the list of arguments ! (inbox shall serve to select teh right mailbox, never sent to the backend) And even if th

Re: FTS delays

2019-04-21 Thread Joan Moreau via dovecot
ER On 2019-04-21 09:57, Timo Sirainen via dovecot wrote: On 3 Apr 2019, at 20.30, Joan Moreau via dovecot wrote: doveadm search -u j...@grosjo.net mailbox inbox text milan output doveadm(j...@grosjo.net): Info: Query : ( bcc:inbox OR body:inbox OR cc:inbox OR from:inbox OR mes

Re: FTS delays

2019-04-20 Thread Joan Moreau via dovecot
On 2019-04-06 09:56, Joan Moreau via dovecot wrote: For the point 1, this is not "suboptimal", it is plain wrong (results are damn wrong ! and this is not related to the backend, but the FTS logic in Dovecot core) For the point 2 , this has been discussed already numerous time

Re: FTS delays

2019-04-14 Thread Joan Moreau via dovecot
g and must be totally reviewed. I do not have the time but I can participate in testing if someone is ready to roll up its sleeves on teh mater THe "loop" part seems the most urgent : It breaks everything (search timeout 100% of the time) On 2019-04-06 09:56, Joan Moreau via dovecot wrot

Re: FTS delays

2019-04-05 Thread Joan Moreau via dovecot
returns X, then dovecot core choose to select only Y emails. THis is a clear bug. On 2019-04-05 20:08, Josef 'Jeff' Sipek via dovecot wrote: On Fri, Apr 05, 2019 at 19:33:57 +0800, Joan Moreau via dovecot wrote: Hi If you plan to fix the FTS part of Dovecot, I will be very gratefull. I'm trying to

Re: FTS delays

2019-04-05 Thread Joan Moreau via dovecot
dovecot wrote: On Wed, Apr 03, 2019 at 19:02:52 +0800, Joan Moreau via dovecot wrote: issue seems in the Git version : Which git revision? Before you updated to the broken revision, which revision/version were you running? Can you try it with 5f6e39c50ec79ba8847b2fdb571a9152c71cd1b6 (the commi

Re: FTS delays

2019-04-05 Thread Joan Moreau via dovecot
On 2019-04-05 16:36, Josef 'Jeff' Sipek via dovecot wrote: On Wed, Apr 03, 2019 at 19:02:52 +0800, Joan Moreau via dovecot wrote: issue seems in the Git version : Which git revision? Before you updated to the broken revision, which revision/version were you running? C

Re: FTS delays

2019-04-03 Thread Joan Moreau via dovecot
issue seems in the Git version : FTS search in teh body ends up with looping Other search call twice the FTS plugin (for no reason) On 2019-04-03 18:58, @lbutlr via dovecot wrote: On 3 Apr 2019, at 04:30, Joan Moreau via dovecot wrote: doveadm search -u j...@grosjo.net mailbox inbox

Re: FTS delays

2019-04-03 Thread Joan Moreau via dovecot
d82b4b0f550d3859364495331209 3038 d82b4b0f550d3859364495331209 3121 d82b4b0f550d3859364495331209 3170 1 - The query is wrong 2 - teh last line "d8...209 3170" gets repeated for ages On 2019-04-02 16:30, Timo Sirainen wrote: On 2 Apr 2019, at 6.38, Joan Moreau via dovecot wrote: F

Re: FTS delays

2019-04-01 Thread Joan Moreau via dovecot
Moreau via dovecot wrote: Hi When I do a FTS search (using Xapian plugin) in the BODY part, the plugins returns the matching IDs within few milliseconds (as seen in the log). However, roundcube (connected on dovecot) takes ages to show (headers only vie IMAP) the few results (I tested

Re: FTS delays

2019-03-30 Thread Joan Moreau via dovecot
it is already on On March 31, 2019 03:47:52 Aki Tuomi via dovecot wrote: On 30 March 2019 21:37 Joan Moreau via dovecot wrote: Hi When I do a FTS search (using Xapian plugin) in the BODY part, the plugins returns the matching IDs within few milliseconds (as seen in the log

FTS delays

2019-03-30 Thread Joan Moreau via dovecot
Hi When I do a FTS search (using Xapian plugin) in the BODY part, the plugins returns the matching IDs within few milliseconds (as seen in the log). However, roundcube (connected on dovecot) takes ages to show (headers only vie IMAP) the few results (I tested with a matching requests of 9

Re: [grosjo/fts-xapian] `doveadm fts rescan` removes all indices (#15)

2019-02-18 Thread Joan Moreau via dovecot
d not get done. Aki On 17 February 2019 at 10:56 Joan Moreau via dovecot wrote: In such case, as long as the API is not upgraded, should doveadm index -A -q \* be considered a replacement of doveadm fts rescan On 2019-02-14 16:24, Timo Sirainen via dovecot wrote: Hi, The rescan() fun

Re: [grosjo/fts-xapian] `doveadm fts rescan` removes all indices (#15)

2019-02-17 Thread Joan Moreau via dovecot
Anyway, we don't really have time to implement this new API soon. I'm not sure if this is a big problem though. I don't think most people running FTS have ever run rescan. On 8 Feb 2019, at 9.54, Joan Moreau via dovecot wrote: Hi, THis is a core problem in Dovecot in my understanding.

Re: Fwd: [grosjo/fts-xapian] `doveadm fts rescan` removes all indices (#15)

2019-02-13 Thread Joan Moreau via dovecot
Hi Anyone ? On 2019-02-08 08:54, Joan Moreau via dovecot wrote: Hi, THis is a core problem in Dovecot in my understanding. In my opinion, the rescan in dovecot should send to the FTS plugin the list of "supposedly" indexed emails (UID), and the plugin shall purge the redundant

Fwd: [grosjo/fts-xapian] `doveadm fts rescan` removes all indices (#15)

2019-02-07 Thread Joan Moreau via dovecot
Hi, THis is a core problem in Dovecot in my understanding. In my opinion, the rescan in dovecot should send to the FTS plugin the list of "supposedly" indexed emails (UID), and the plugin shall purge the redundant UID (i..e UID present in the index but not in the list sent by dovecot) and

Re: Solr - complete setup (update)

2019-01-29 Thread Joan Moreau via dovecot
On 2019-01-30 07:33, Stephan Bosch wrote: (forgot to CC mailing list) Op 26/01/2019 om 20:07 schreef Joan Moreau via dovecot: *- Bugs so far* -> Line 620 of fts_solr dovecot plugin : the size oof header is improperly calculated ("huge header" warning for a simple email

Re: Solr - complete setup (update)

2019-01-26 Thread Joan Moreau via dovecot
*- Installation:* -> Create a clean install using the default, (at least in the Archlinux package), and do a "sudo -u solr solr create -c dovecot ". The config files are then in /opt/solr/server/solr/dovecot/conf and datafiles in /opt/solr/server/solr/dovecot/data On my system (Debian) these

[FTS Xapian] RC release

2019-01-24 Thread Joan Moreau via dovecot
Hi, FTS Xapian matches my targets for the plugins (replacing deprecated fts-squat in a production environment) https://github.com/grosjo/fts-xapian Please do not hesitate to add "issues" on github, if the case happen Hope it helps JM

Encoding input

2019-01-22 Thread Joan Moreau via dovecot
Hi Are data inputs of fts_backend_update_build_more(struct fts_backend_update_context *_ctx, const unsigned char *data, size_t size) already converted to UTF8 ? Thanks

Re: Solr -> Xapian ?

2019-01-22 Thread Joan Moreau via dovecot
9><99GeuQeANlt/AAAB>: Logged out in=20201 out=567147 deleted=0 expunged=0 trashed=0 hdr_count=200 hdr_bytes=62139 body_count=0 body_bytes=0 Jan 22 08:25:51 gjserver dovecot[20984]: indexer-worker(j...@grosjo.net)<20998>: Indexed 1 messages in Sent (UIDs 60585..60585) On 2019-01-08 04

Re: Solr - complete setup (update)

2019-01-18 Thread Joan Moreau via dovecot
Yes, the " -property update.autoCreateFields -value false " seems interesting However, we smash the created schema just after On 2019-01-14 23:25, Stephan Bosch wrote: Op 14/01/2019 om 07:44 schreef Joan Moreau via dovecot: Hi Stephan, What's up with that ? Thank you so much

Repetitive indexing nd LOCK WRITE error

2019-01-16 Thread Joan Moreau via dovecot
Hi, Monitoring the Xapian plugin, I notice that dovecot is re-indexing multiple times the same Inbox for no apparent reason (not the last email received but the whole mailbox, LAST UID being properly returned (with the confusion already discussed) ) : Why so ? Also, I get a LOCK on the

Re: [FTS Xapian] Beta release

2019-01-14 Thread Joan Moreau via dovecot
y or what??] On Mon, 14 Jan 2019 at 09:42, Joan Moreau via dovecot mailto:dovecot@dovecot.org>> wrote: Thank you Stephan. The version here shall be up and running : https://github.com/grosjo/fts-xapian On 2019-01-14 00:07, Stephan Bosch wrote: Op 13/01/2019 om 21:25 schreef Joan Moreau via dov

Re: [FTS Xapian] Beta release

2019-01-14 Thread Joan Moreau via dovecot
It is indeed better is you use the issue tracker of github: https://github.com/grosjo/fts-xapian/issues I updated the Readme accordingly On 2019-01-14 14:24, Stephan Bosch wrote: Op 14-1-2019 om 13:40 schreef Aki Tuomi: Just to remind that now that there is a github repo for fts-xapian,

Re: [FTS Xapian] Beta release

2019-01-14 Thread Joan Moreau via dovecot
that (usually some separate package). Regards, Stephan. On 14. Jan 2019, at 10:33, Joan Moreau via dovecot mailto:dovecot@dovecot.org>> wrote: Difficult to figure out without a coredump + gdb I have also battled quite a lot to make sure dovecot can core dump on my Archlinux servers

Re: [FTS Xapian] Beta release

2019-01-14 Thread Joan Moreau via dovecot
Debug: Mailbox sent: UID 3: Opened mail because: fts indexing Thank you! On 14. Jan 2019, at 10:11, Joan Moreau via dovecot wrote: Can you send the log part that includes the "init" of the plugins (something similar as below) ? WHich version of Xapian are you on ? Jan 14 09:10:04

Re: [FTS Xapian] Beta release

2019-01-14 Thread Joan Moreau via dovecot
to no avail). Custom Dovecot 2.3.4 on Debian Stretch. Thanks, Paul On 14. Jan 2019, at 07:42, Joan Moreau via dovecot wrote: Thank you Stephan. The version here shall be up and running : https://github.com/grosjo/fts-xapian On 2019-01-14 00:07, Stephan Bosch wrote: Op 13/01/2019 om 21:25 schreef

Re: [FTS Xapian] Beta release

2019-01-14 Thread Joan Moreau via dovecot
er?? Also in the part after cloning from git: ./configure --prefix=/usr --with-dovecot=/path/to/dovecot [ This /path/to/dovecot is not obvious. Is it the dovecot binary or what??] On Mon, 14 Jan 2019 at 09:42, Joan Moreau via dovecot wrote: Thank you Stephan. The version here shall be up a

Re: Solr - complete setup (update)

2019-01-13 Thread Joan Moreau via dovecot
Hi Stephan, What's up with that ? Thank you so much On 2019-01-05 02:04, Stephan Bosch wrote: Hi, Op 04/01/2019 om 05:36 schreef Joan Moreau via dovecot: Hi This is the summary of my work with SOLR-Dovecot, in my *quest to reproduce the previoulsy excellent work of fts_squat* @Aki

Re: [FTS Xapian] Beta release

2019-01-13 Thread Joan Moreau via dovecot
Thank you Stephan. The version here shall be up and running : https://github.com/grosjo/fts-xapian On 2019-01-14 00:07, Stephan Bosch wrote: Op 13/01/2019 om 21:25 schreef Joan Moreau via dovecot: I tried to combined it, the "autoreconf" errors are solved Now, when I type &qu

Re: [FTS Xapian] Beta release

2019-01-13 Thread Joan Moreau via dovecot
(or wherever dovecot installed its libraries) On 2019-01-13 21:25, Joan Moreau via dovecot wrote: I tried to combined it, the "autoreconf" errors are solved Now, when I type "make install", the lib is not pushed into dovecot folder, but somewhere in /usr/local/

Re: [FTS Xapian] Beta release

2019-01-13 Thread Joan Moreau via dovecot
h it in hte git (yes, everything is already done) On 2019-01-13 18:45, Aki Tuomi wrote: On 13 January 2019 at 17:05 Joan Moreau via dovecot wrote: Hi Please find attached the beta release of FTS Xapian, with the objective to replace fts_squat that is being deprecated. Configuration

Re: [FTS Xapian] Beta release

2019-01-13 Thread Joan Moreau via dovecot
/MAKEFILE.AM:11: WARNING: VARIABLE 'NOPLUGIN_LDFLAGS' IS DEFINED BUT NO PROGRAM OR SRC/MAKEFILE.AM:11: LIBRARY HAS 'NOPLUGIN' AS CANONICAL NAME (POSSIBLE TYPO) AUTORECONF: AUTOMAKE FAILED WITH EXIT STATUS: 1 On 2019-01-13 20:32, Joan Moreau via dovecot wrote: Please kindly check https://github.com

Re: [FTS Xapian] Beta release

2019-01-13 Thread Joan Moreau via dovecot
in configure.ac, makefile.am, etc.. in order to include it in the dovecot, and it compiles properly The only remaining point is to push it in hte git (yes, everything is already done) On 2019-01-13 18:45, Aki Tuomi wrote: On 13 January 2019 at 17:05 Joan Moreau via dovecot wrote: Hi Please find attached

Re: Solr -> Xapian ?

2019-01-13 Thread Joan Moreau via dovecot
writing a new FTS driver? If squat allegedly does everything you need it to do, why don't you just take that plugin and fix it up to do what you need? That seems way easier than trying to create a FTS driver from scratch. michael On January 7, 2019 at 7:05 AM Joan Moreau via dovecot wrote: Hi

Re: [FTS Xapian] Beta release

2019-01-13 Thread Joan Moreau via dovecot
remaining point is to push it in hte git (yes, everything is already done) On 2019-01-13 18:45, Aki Tuomi wrote: On 13 January 2019 at 17:05 Joan Moreau via dovecot wrote: Hi Please find attached the beta release of FTS Xapian, with the objective to replace fts_squat that is being deprecated

Re: [FTS Xapian] Beta release

2019-01-13 Thread Joan Moreau via dovecot
, Aki Tuomi wrote: On 13 January 2019 at 17:05 Joan Moreau via dovecot wrote: Hi Please find attached the beta release of FTS Xapian, with the objective to replace fts_squat that is being deprecated. Configuration is exactly the same as for fts_squat: plugin { plugin = fts fts_xapian

Re: [FTS Xapian] Beta release

2019-01-13 Thread Joan Moreau via dovecot
remaining point is to push it in hte git (yes, everything is already done) On 2019-01-13 18:45, Aki Tuomi wrote: On 13 January 2019 at 17:05 Joan Moreau via dovecot wrote: Hi Please find attached the beta release of FTS Xapian, with the objective to replace fts_squat that is being deprecated

Indexing paralelism

2019-01-13 Thread Joan Moreau via dovecot
Hi Observing the processes of FTS, I observe the following: 1 - For one user, indexer-wroker does not start several threads for each request. On teh contrary, it waits for the first request to finish before starting the second. How to make sure all requests (or a limited number of it, for

[FTS Xapian] Beta release

2019-01-13 Thread Joan Moreau via dovecot
Hi Please find attached the beta release of FTS Xapian, with the objective to replace fts_squat that is being deprecated. Configuration is exactly the same as for fts_squat: plugin { plugin = fts fts_xapian (...) fts = xapian fts_autoindex = yes fts_enforced = yes fts_xapian = partial=2

Re: Solr -> Xapian ?

2019-01-13 Thread Joan Moreau via dovecot
I found the solution o this using SEQ_RANGE_ARRAY_ADD(>DEFINITE_UIDS, UID); Now, I can see in the logs that several times, the dovecot calls the fts_backend_xapian_update_set_mailbox with box == NULL. WHy so ? THank you On 2019-01-12 21:40, Joan Moreau via dovecot wrote: I somehow fi

Re: Solr -> Xapian ?

2019-01-12 Thread Joan Moreau via dovecot
to say someting similar to result->definite_uids[1]=my_uid ? On 2019-01-12 10:25, Timo Sirainen wrote: On 11 Jan 2019, at 21.23, Joan Moreau via dovecot wrote: The below patch resolves the compilation error $ diff -p compat.h compat.h.joan *** compat.h 2019-01-11 20:

Re: Solr -> Xapian ?

2019-01-12 Thread Joan Moreau via dovecot
_t * How to put my UIDs into this "definite_uids" ? Obviously this is not a simple array/pointer. How to say someting similar to result->definite_uids[1]=my_uid ? On 2019-01-12 10:25, Timo Sirainen wrote: On 11 Jan 2019, at 21.23, Joan Moreau via dovecot wrote: The below patch resolve

Re: Solr -> Xapian ?

2019-01-12 Thread Joan Moreau via dovecot
to result->definite_uids[1]=my_uid ? On 2019-01-12 10:25, Timo Sirainen wrote: On 11 Jan 2019, at 21.23, Joan Moreau via dovecot wrote: The below patch resolves the compilation error $ diff -p compat.h compat.h.joan *** compat.h 2019-01-11 20:21:00.726625427 +0100 --- compat.h.joan 2019-01-11 20:14

Re: [FTS Xapian] Status & Questions

2019-01-12 Thread Joan Moreau via dovecot
;get_last_uid" is called without selecting a mailbox first, and therefore there is of course no uid. How come ? On 2019-01-12 10:50, Aki Tuomi wrote: Did you remember to load fts first? mail_plugins =$mail_plugins fts fts_xapian Aki On 12 January 2019 at 10:37 Joan Moreau via dovecot <

Re: [FTS Xapian] Status & Questions

2019-01-12 Thread Joan Moreau via dovecot
course no uid. How come ? On 2019-01-12 10:50, Aki Tuomi wrote: Did you remember to load fts first? mail_plugins =$mail_plugins fts fts_xapian Aki On 12 January 2019 at 10:37 Joan Moreau via dovecot < dovecot@dovecot.org> wrote: STATUS - Alpha code is written and compilin

[FTS Xapian] Status & Questions

2019-01-12 Thread Joan Moreau via dovecot
STATUS - Alpha code is written and compiling now. (attached) - I would like to start testing. However, there is an error when starting dovecot (git) : Error: Couldn't load required plugin /usr/lib/dovecot/lib21_fts_xapian_plugin.so: dlopen() failed:

Re: Solr -> Xapian ?

2019-01-11 Thread Joan Moreau via dovecot
n Moreau wrote: There is no point into a separate plugin, the purpose is to replace squat as the default fts (solr being a nightmare) On 2019-01-11 18:23, Aki Tuomi wrote: I would recommend making this a standalone plugin for now instead of trying to keep it in core fts. Aki On 11 January 201

Re: Solr -> Xapian ?

2019-01-11 Thread Joan Moreau via dovecot
Joan Moreau via dovecot < dovecot@dovecot.org> wrote: I managed to deal with the namespace issue (updated makefile.am) However, I reach : ../../../src/lib/compat.h:207:19: error: conflicting declaration of 'ssize_t i_my_pread(int, void*, size_t, __off_t)' with 'C' linkage # define

Re: Solr -> Xapian ?

2019-01-11 Thread Joan Moreau via dovecot
(whch it does not as there are no function for that) ? 4 - How to update configure.ac & additional files to add the "--with-xapian" wichi will test for libxapian presence and add it to the build ? Thank you On 2019-01-08 04:24, Timo Sirainen wrote: On 7 Jan 2019, at 16.05, Joan Moreau vi

Re: Solr -> Xapian ?

2019-01-11 Thread Joan Moreau via dovecot
'i_isspace'? namespace Xapian { Someone can bring me some light ? Thanks On 2019-01-09 09:58, Joan Moreau via dovecot wrote: Ok. Additional question : - for rescan : who is responsible for passing again the new email ? Is the Dovecot core sending again all the emails to index ? or the fts sh

Re: Solr -> Xapian ?

2019-01-09 Thread Joan Moreau via dovecot
e rebuilding a previous email, so *last* uid is something different than max. And how indexer does know whther there is a gap wihtout callong the fts backend (whch it does not as there are no function for that) ? On 2019-01-08 04:24, Timo Sirainen wrote: On 7 Jan 2019, at 16.05, Joan Moreau vi

Re: Solr -> Xapian ?

2019-01-07 Thread Joan Moreau via dovecot
ee below) ? THank you On 2019-01-06 16:50, Joan Moreau via dovecot wrote: and finally , for fts_backend__lookup_multi, why is that backend dependent ? Would- nt the below function below be the same for any backend ? Waiting fro your feedback on all those questions Thank

Re: Solr -> Xapian ?

2019-01-06 Thread Joan Moreau via dovecot
) { if(fts_backend_xapian_lookup(backend,box[i],args,flags,result->box_results[i])<0) return -1; i++; } return 0; } On 2019-01-06 16:31, Joan Moreau via dovecot wrote: for fts_backend_xxx_lookup, where is specidifed in which field (to, cc, subject, body, from, all) to lookup ? On 2019-01-06 16:03, Joan Moreau

Re: Solr -> Xapian ?

2019-01-06 Thread Joan Moreau via dovecot
:41, Joan Moreau wrote: also, for fts_backend_solr_update_set_build_key -> where is the data (of the hdr_name or the body) ? On 2019-01-06 14:10, Joan Moreau wrote: for the "last uid"-> this is not the last added, but the maximum of the UID in the indexed emails, right ? On 2019-01-06 11:53, Joan M

Re: Solr -> Xapian ?

2019-01-06 Thread Joan Moreau via dovecot
Moreau wrote: for the "last uid"-> this is not the last added, but the maximum of the UID in the indexed emails, right ? On 2019-01-06 11:53, Joan Moreau via dovecot wrote: Thank you I still don't get the "build_key" function. The email (body, hearders, .. and the uid) is the one (an

Re: Solr -> Xapian ?

2019-01-05 Thread Joan Moreau via dovecot
ra mails in the index. */ int fts_backend_rescan(struct fts_backend *backend); Regards, Stepham On January 5, 2019 14:23:10 Joan Moreau via dovecot wrote: Thank Stephan I basically need to know the role/description of each of the functions of the fts_backend: struct fts_backend fts_backend_xapian =

Re: Solr -> Xapian ?

2019-01-05 Thread Joan Moreau via dovecot
for the "last uid"-> this is not the last added, but the maximum of the UID in the indexed emails, right ? On 2019-01-06 11:53, Joan Moreau via dovecot wrote: Thank you I still don't get the "build_key" function. The email (body, hearders, .. and the uid) is the

Re: Solr -> Xapian ?

2019-01-05 Thread Joan Moreau via dovecot
nt fts_backend_rescan(struct fts_backend *backend); Regards, Stepham On January 5, 2019 14:23:10 Joan Moreau via dovecot wrote: Thank Stephan I basically need to know the role/description of each of the functions of the fts_backend: struct fts_backend fts_backend_xapian = { .name = "xapian

Re: Solr -> Xapian ?

2019-01-05 Thread Joan Moreau via dovecot
Anyone willing to explain those functions ? Most notably " get_last_uid" "set_build_key" "build_more" , what is refresh versus rescan ? On January 5, 2019 14:23:10 Joan Moreau via dovecot wrote: Thank Stephan I basically need to know the role/descri

Re: Solr -> Xapian ?

2019-01-05 Thread Joan Moreau via dovecot
Anyone willing to explain those functions ? On January 5, 2019 14:23:10 Joan Moreau via dovecot wrote: Thank Stephan I basically need to know the role/description of each of the functions of the fts_backend: struct fts_backend fts_backend_xapian = { .name = "xapian&quo

Re: Solr -> Xapian ?

2019-01-04 Thread Joan Moreau via dovecot
Op 04/01/2019 om 11:17 schreef Joan Moreau via dovecot: Why not, but please guide me about the core structure (mandatory funcitons, etc..) of a typical Dovecot FTS plugin The Dovecot API documentation is not exhaustive everywhere, but the basics are documented. The remaining questions ca

Re: Solr -> Xapian ?

2019-01-04 Thread Joan Moreau via dovecot
_backend_xapian_refresh, fts_backend_xapian_rescan, fts_backend_xapian_optimize, fts_backend_default_can_lookup, fts_backend_xapian_lookup, fts_backend_xapian_lookup_multi, fts_backend_xapian_lookup_done } }; On 2019-01-04 20:33, Joan Moreau via dovecot wrote: Yes but: 1 - is there a documenta

Re: Solr -> Xapian ?

2019-01-04 Thread Joan Moreau via dovecot
19-01-04 18:49, admin wrote: A starting point would be to have a look at the current FTS plugins: https://github.com/dovecot/core/tree/master/src/plugins/fts-solr and https://github.com/dovecot/core/tree/master/src/plugins/fts-squat -M Am Freitag, den 04.01.2019, 18:17 +0800 schrieb Joan More

Re: Solr -> Xapian ?

2019-01-04 Thread Joan Moreau via dovecot
feels like writing fts_xapian, go for it. Aki On 04 January 2019 at 08:20 Joan Moreau via dovecot wrote: What about consedering linking Dovecot with Xapian librairies instead of going to nightmare Solr ? https://xapian.org/features On 2019-01-02 17:10, John Tulp wrote: On Wed, 2019-01-02 at 0

Solr -> Xapian ?

2019-01-03 Thread Joan Moreau via dovecot
What about consedering linking Dovecot with Xapian librairies instead of going to nightmare Solr ? https://xapian.org/features On 2019-01-02 17:10, John Tulp wrote: On Wed, 2019-01-02 at 00:59 -0800, M. Balridge wrote: The main problem is : After some time of indexing from Dovecot, Dovecot

Solr - complete setup (update)

2019-01-03 Thread Joan Moreau via dovecot
Hi This is the summary of my work with SOLR-Dovecot, in my QUEST TO REPRODUCE THE PREVIOULSY EXCELLENT WORK OF FTS_SQUAT @Aki : Based on the time I have spent on this, I would love to see you updating the Wiki with those improvements, and adding my name somewhere @All : Hope it helps -

Solr - complete setup

2019-01-03 Thread Joan Moreau via dovecot
Hi This is the summary of my work with SOLR-Dovecot, in my QUEST TO REPRODUCE THE PREVIOULSY EXCELLENT WORK OF FTS_SQUAT @Aki : Based on the time I have spent on this, I would love to see you updating the Wiki with those improvements, and adding my name somewhere @All : Hope it helps -

Solr - schema.xml

2019-01-02 Thread Joan Moreau via dovecot
Hi, This is the best I can do for having a SCHEMA.XML matching the needs of an email user. Use case on Solr 7.5.0 (note : this does not solve any of the bugs previously mentionned, but at least, the logic of Solr is back on track) --- id

Re: Solr

2019-01-02 Thread Joan Moreau via dovecot
use of the errors of fts_solr, but maybe this will help On 2019-01-02 18:00, Joan Moreau via dovecot wrote: Another bug appearing today: Jan 02 09:59:08 indexer-worker(j...@grosjo.net)<6777>: Warning: FTS-SOLR(j...@grosjo.net): Mailbox XX UID=121635 HEADER SIZE IS HUGE, TRUNCATING h

Re: Solr

2019-01-02 Thread Joan Moreau via dovecot
Another bug appearing today: Jan 02 09:59:08 indexer-worker(j...@grosjo.net)<6777>: Warning: FTS-SOLR(j...@grosjo.net): Mailbox XX UID=121635 HEADER SIZE IS HUGE, TRUNCATING header of the said email has nothing of "huge" On 2019-01-02 15:22, Joan Moreau via dovecot wr

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot
The real main differecne seems coming from "diffconfig.xml" When I put yours, Solr delete (!) schema.xml and create a "manage-schema" and starts complaining about useless types (tdates, booleans, etc..) that are not needed for Mail fileds When I put mine (from standard distribution of Arch),

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot
Refinement of the schema.xml (below) THis however does not solve the "no results" and "Out of range" errors in Dovecot and Solr id

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot
ed an email add in RoundCube search box, using Dovecot as back end, using Solr as own backend) So many efforts for crappy results. Can't we really revive Squat ? It is 2 lines of config, and no single problems On January 2, 2019 08:16:33 Joan Moreau via dovecot wrote: and the first

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot
e problems On January 2, 2019 08:16:33 Joan Moreau via dovecot wrote: and the first line of the diff is : < this file, see http://wiki.apache.org/solr/SolrConfigXml. --- this file, see http://wiki.apache.org/solr/SolrConfigXml. 38c38 < 6.4.1 --- 7.5.0 So, are you running 6.4.1 or 7.5.0

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot
and the first line of the diff is : < this file, see http://wiki.apache.org/solr/SolrConfigXml. --- this file, see http://wiki.apache.org/solr/SolrConfigXml. 38c38 < 6.4.1 --- 7.5.0 So, are you running 6.4.1 or 7.5.0 On 2019-01-02 08:12, Joan Moreau wrote: The real main differecne

Re: Solr

2019-01-01 Thread Joan Moreau via dovecot
Hi Solr is a standard package in ArchLinux. ("pacman -S solr") . the systemd installation script is included (and it is launching /opt/solr/bin/solr.in.sh) Instance : sudo -u solr /opt/solr/bin/solr create -c dovecot -> this creates a separate folder with default solrconfig.xml, schema.xml,

Re: Solr

2018-12-29 Thread Joan Moreau via dovecot
tionally, is there a doc of fts-squat in order to adjust the code to new releases of dovect ? On December 12, 2018 4:44:10 PM Daniel Miller via dovecot wrote: On 12/11/2018 4:46 AM, Joan Moreau via dovecot wrote: I shared the errors already so many times (check this mailinling for "so

Re: Solr

2018-12-29 Thread Joan Moreau via dovecot
r returns properly for a few hours, then starts crashing or responding non-sense after some time Additionally, is there a doc of fts-squat in order to adjust the code to new releases of dovect ? On December 12, 2018 4:44:10 PM Daniel Miller via dovecot wrote: On 12/11/2018 4:46 AM, Joan Mor

Re: Solr

2018-12-21 Thread Joan Moreau via dovecot
after some time Additionally, is there a doc of fts-squat in order to adjust the code to new releases of dovect ? On December 12, 2018 4:44:10 PM Daniel Miller via dovecot wrote: On 12/11/2018 4:46 AM, Joan Moreau via dovecot wrote: I shared the errors already so many times (check this mailinlin

Re: Solr

2018-12-15 Thread Joan Moreau via dovecot
s crashing or responding non-sense after some time Additionally, is there a doc of fts-squat in order to adjust the code to new releases of dovect ? On December 12, 2018 4:44:10 PM Daniel Miller via dovecot wrote: On 12/11/2018 4:46 AM, Joan Moreau via dovecot wrote: I shared the errors already so

  1   2   >