Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-26 Thread Aki Tuomi
> On 25/08/2021 23:05 Steve Dondley wrote: > > > > The search time was no better with it on than off. > > > > So I'm thinking I got something misconfigured somewhere. It seems IMAP > > may not be using solr to fetch results. But this would be odd since I > > definitely do see a big improveme

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
On 2021-08-25 04:32 PM, Shawn Heisey wrote: On 8/25/2021 2:10 PM, Steve Dondley wrote: And it looks like I'm running into a major bug in the slightly dated version of dovecot debian uses: https://www.mail-archive.com/dovecot@dovecot.org/msg78825.html Recently I did a fairly major upgrade.

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Shawn Heisey
On 8/25/2021 2:10 PM, Steve Dondley wrote: And it looks like I'm running into a major bug in the slightly dated version of dovecot debian uses: https://www.mail-archive.com/dovecot@dovecot.org/msg78825.html Recently I did a fairly major upgrade.  I had an older Ubuntu release with Dovecot

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
I think this will be nailed once I figure out this issue. And it looks like I'm running into a major bug in the slightly dated version of dovecot debian uses: https://www.mail-archive.com/dovecot@dovecot.org/msg78825.html And this: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970692

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
On 2021-08-25 04:05 PM, Steve Dondley wrote: The search time was no better with it on than off. So I'm thinking I got something misconfigured somewhere. It seems IMAP may not be using solr to fetch results. But this would be odd since I definitely do see a big improvements in times with fts plug

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
The search time was no better with it on than off. So I'm thinking I got something misconfigured somewhere. It seems IMAP may not be using solr to fetch results. But this would be odd since I definitely do see a big improvements in times with fts plugins turned on when using roundcube. OK, I

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
I'm inclined to believe the problem is not that high up the food chain. Because when I query IMAP on a single folder over telnet following the instructions found here: https://doc.dovecot.org/configuration_manual/fts/solr/, imap reports that it's taking 3 to 4 seconds to return results: a sear

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
Random guess... Buffering? Whatever is sending to the browser isn't sending enough bytes to flush the buffer so the data is left in limbo until enough time goes by the buffer gets flushed anyways. Maybe a apache/nginx thing, php thing or browser thing. Remember its solr > dovecot > php > web se

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread dovecot
So this looks really good and fast. So I think we can say with confidence solr is doing its job. So why is roundcube/dovecot taking so long to show the results? Random guess... Buffering? Whatever is sending to the browser isn't sending enough bytes to flush the buffer so the data is left in li

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
On 2021-08-25 02:05 PM, Steve Dondley wrote: Try this in on the commandline of the Solr server: time curl "http://localhost:YYY/solr/dovecot/select?q=maynez&defType=edismax&qf=body+to+subject+cc+from"; OK I had to modify the query path slightly to get it to work with my core to: time curl h

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
That query should search ALL emails that dovecot has indexed to Solr.  There is no restriction for mailbox or folder. OK. Try replacing "maynez" with something else that you know will be in the index. Did a search on "the". Still nothing. Very, very weird. What would explain why my email

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Shawn Heisey
On 8/25/2021 12:05 PM, Steve Dondley wrote: OK I had to modify the query path slightly to get it to work with my core to: time curl http://localhost:8983/solr/dondley/select?q=maynez&defType=edismax&qf=body+to+subjec:t+ccfrom But it didn't return any results: I only have emails for th

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
Try this in on the commandline of the Solr server: time curl "http://localhost:YYY/solr/dovecot/select?q=maynez&defType=edismax&qf=body+to+subject+cc+from"; OK I had to modify the query path slightly to get it to work with my core to: time curl http://localhost:8983/solr/dondley/select?q

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Shawn Heisey
On 8/25/2021 11:33 AM, Steve Dondley wrote: OK, I take this back. I did an imap search via telnet and solr reports the search takes about 3 to 4 seconds. Here's the output: a search text "maynez" Try this in on the commandline of the Solr server: time curl "http://localhost:YYY/solr/doveco

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
This is a search for "a" which I had run several times, so Solr was serving it from its cache, and this time it only took 6 milliseconds.  It also shows what a facet can do.  The longest time I got for the "a" search was 15 milliseconds, before the query was in the cache. I think they querie

Re: [OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Steve Dondley
One other data point from my experimenting that might shed some light on the problem: If I limit a search to a single folder instead of across all folders, it still takes 5 or 6 seconds for the results to appear. So that kind of destroys my theory that the problem might be caused by having too

[OT] Re: What kind of search response time are you setting with solr full text search?

2021-08-25 Thread Shawn Heisey
On 8/24/2021 7:19 PM, Steve Dondley wrote: THE PROBLEM: When I do a full text search through all my inbox and all subfolders on a single word, search results are returned in about 10 to 15 seconds. This is better than the 40 seconds or so I'm getting when I turn off the fts and fts_solr plugins b