Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-26 Thread Aki Tuomi
> On 25/08/2021 23:05 Steve Dondley wrote: > > > > The search time was no better with it on than off. > > > > So I'm thinking I got something misconfigured somewhere. It seems IMAP > > may not be using solr to fetch results. But this would be odd since I > > definitely do see a big

Re: What kind of search response time are you setting with solr full text search?

2021-08-26 Thread azurit
Citát Steve Dondley : On 2021-08-24 08:53 PM, Steve Dondley wrote: MY SETUP: I have apache solr full text search enabled with dovecot. I have an inbox with about 40 subfolders. I'm using the roundcube web-based mail client. The find command is showing 15823 email files and apache solr

Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
On 2021-08-24 08:53 PM, Steve Dondley wrote: MY SETUP: I have apache solr full text search enabled with dovecot. I have an inbox with about 40 subfolders. I'm using the roundcube web-based mail client. The find command is showing 15823 email files and apache solr reports the same number. I'm

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
On 2021-08-25 04:32 PM, Shawn Heisey wrote: On 8/25/2021 2:10 PM, Steve Dondley wrote: And it looks like I'm running into a major bug in the slightly dated version of dovecot debian uses: https://www.mail-archive.com/dovecot@dovecot.org/msg78825.html Recently I did a fairly major

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Shawn Heisey
On 8/25/2021 2:10 PM, Steve Dondley wrote: And it looks like I'm running into a major bug in the slightly dated version of dovecot debian uses: https://www.mail-archive.com/dovecot@dovecot.org/msg78825.html Recently I did a fairly major upgrade.  I had an older Ubuntu release with

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
I think this will be nailed once I figure out this issue. And it looks like I'm running into a major bug in the slightly dated version of dovecot debian uses: https://www.mail-archive.com/dovecot@dovecot.org/msg78825.html And this: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970692

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
On 2021-08-25 04:05 PM, Steve Dondley wrote: The search time was no better with it on than off. So I'm thinking I got something misconfigured somewhere. It seems IMAP may not be using solr to fetch results. But this would be odd since I definitely do see a big improvements in times with fts

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
The search time was no better with it on than off. So I'm thinking I got something misconfigured somewhere. It seems IMAP may not be using solr to fetch results. But this would be odd since I definitely do see a big improvements in times with fts plugins turned on when using roundcube. OK,

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
I'm inclined to believe the problem is not that high up the food chain. Because when I query IMAP on a single folder over telnet following the instructions found here: https://doc.dovecot.org/configuration_manual/fts/solr/, imap reports that it's taking 3 to 4 seconds to return results: a

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
Random guess... Buffering? Whatever is sending to the browser isn't sending enough bytes to flush the buffer so the data is left in limbo until enough time goes by the buffer gets flushed anyways. Maybe a apache/nginx thing, php thing or browser thing. Remember its solr > dovecot > php > web

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread dovecot
So this looks really good and fast. So I think we can say with confidence solr is doing its job. So why is roundcube/dovecot taking so long to show the results? Random guess... Buffering? Whatever is sending to the browser isn't sending enough bytes to flush the buffer so the data is left in

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
On 2021-08-25 02:05 PM, Steve Dondley wrote: Try this in on the commandline of the Solr server: time curl "http://localhost:YYY/solr/dovecot/select?q=maynez=edismax=body+to+subject+cc+from; OK I had to modify the query path slightly to get it to work with my core to: time curl

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
That query should search ALL emails that dovecot has indexed to Solr.  There is no restriction for mailbox or folder. OK. Try replacing "maynez" with something else that you know will be in the index. Did a search on "the". Still nothing. Very, very weird. What would explain why my

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Shawn Heisey
On 8/25/2021 12:05 PM, Steve Dondley wrote: OK I had to modify the query path slightly to get it to work with my core to: time curl http://localhost:8983/solr/dondley/select?q=maynez=edismax=body+to+subjec:t+ccfrom But it didn't return any results: I only have emails for this person

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
Try this in on the commandline of the Solr server: time curl "http://localhost:YYY/solr/dovecot/select?q=maynez=edismax=body+to+subject+cc+from; OK I had to modify the query path slightly to get it to work with my core to: time curl

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Shawn Heisey
On 8/25/2021 11:33 AM, Steve Dondley wrote: OK, I take this back. I did an imap search via telnet and solr reports the search takes about 3 to 4 seconds. Here's the output: a search text "maynez" Try this in on the commandline of the Solr server: time curl

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
This is a search for "a" which I had run several times, so Solr was serving it from its cache, and this time it only took 6 milliseconds.  It also shows what a facet can do.  The longest time I got for the "a" search was 15 milliseconds, before the query was in the cache. I think they

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
One other data point from my experimenting that might shed some light on the problem: If I limit a search to a single folder instead of across all folders, it still takes 5 or 6 seconds for the results to appear. So that kind of destroys my theory that the problem might be caused by having

[OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Shawn Heisey
On 8/24/2021 7:19 PM, Steve Dondley wrote: THE PROBLEM: When I do a full text search through all my inbox and all subfolders on a single word, search results are returned in about 10 to 15 seconds. This is better than the 40 seconds or so I'm getting when I turn off the fts and fts_solr plugins

Re: What kind of search response time are you setting with solr full text search?

2021-08-24 Thread Gedalya
On 8/25/21 9:19 AM, Steve Dondley wrote: > I did some experimenting. I noticed that if the word I'm searching on is > fairly rare, results will pop up quickly, like in around 3 to 5 seconds. > Words that don't exist at all in any email returns nothing almost instantly. > > But words that appear

Re: What kind of search response time are you setting with solr full text search?

2021-08-24 Thread Steve Dondley
THE PROBLEM: When I do a full text search through all my inbox and all subfolders on a single word, search results are returned in about 10 to 15 seconds. This is better than the 40 seconds or so I'm getting when I turn off the fts and fts_solr plugins but still a little disappointing. I did

What kind of search response time are you setting with solr full text search?

2021-08-24 Thread Steve Dondley
MY SETUP: I have apache solr full text search enabled with dovecot. I have an inbox with about 40 subfolders. I'm using the roundcube web-based mail client. The find command is showing 15823 email files and apache solr reports the same number. I'm running a dedicated mail server with a 1 GB of