Hi Ere Please to be of service!
No I have not filed a JIRA ticket. I am new to interacting with the Solr
Community and only beginning to 'find my legs'. I am not too sure what JIRA
is I am afraid!
Regards
Matthew
Matthew Flowerday | Consultant | ULEAF
Unisys | 01908 774830| matthew.flower...@unisys.com
Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes | MK17
8LX
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is for use only by the intended recipient. If you received this
in error, please contact the sender and delete the e-mail and its
attachments from all devices.
-----Original Message-----
From: Ere Maijala <ere.maij...@helsinki.fi>
Sent: 01 March 2021 12:53
To: solr-user@lucene.apache.org
Subject: Re: Potential Slow searching for unified highlighting on Solr
8.8.0/8.8.1
EXTERNAL EMAIL - Be cautious of all links and attachments.
Hi,
Whoa, thanks for the heads-up! You may just have saved me from a whole lot
of trouble. Did you file a JIRA ticket already?
Thanks,
Ere
Flowerday, Matthew J kirjoitti 1.3.2021 klo 14.00:
> Hi There
>
> I just came across a situation where a unified highlighting search
> under solr 8.8.0/8.8.1 can take over 20 mins to run and eventually times
out.
> I resolved it by a config change but it can catch you out. Hence
> this email.
>
> With solr 8.8.0 a new unified highlighting parameter
> &hl.fragAlignRatio was implemented which if not set defaults to 0.5.
> This attempts to improve the high lighting so that highlighted text
> does not appear right at the left. This works well but if you have a
> search result with numerous occurrences of the word in question within
> the record performance goes right down!
>
> 2021-02-27 06:45:03.151 INFO (qtp762476028-20) [ x:uleaf]
> o.a.s.c.S.Request [uleaf] webapp=/solr path=/select
> params={hl.snippets=2&q=test&hl=on&hl.maxAnalyzedChars=1000000&fl=id,d
> escription,specification,score&start=20&hl.fl=*&rows=10&_=161440511913
> 4}
> hits=57008 status=0 QTime=1414320
>
> 2021-02-27 06:45:03.245 INFO (qtp762476028-20) [ x:uleaf]
> o.a.s.s.HttpSolrCall Unable to write response, client closed
> connection or we are shutting down =>
> org.eclipse.jetty.io.EofException
>
> at
> org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
>
> org.eclipse.jetty.io.EofException: null
>
> at
> org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:279)
> ~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
>
> at
> org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:422)
> ~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
>
> at
> org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:378)
> ~[jetty-io-9.4.34.v20201102.jar:9.4.34.v20201102]
>
> when I set &hl.fragAlignRatio=0.25 results came back much quicker
>
> 2021-02-27 14:59:57.189 INFO (qtp1291367132-24) [ x:holmes]
> o.a.s.c.S.Request [holmes] webapp=/solr path=/select
> params={hl.weightMatches=false&hl=on&fl=id,description,specification,s
> core&start=1&hl.fragAlignRatio=0.25&rows=100&hl.snippets=2&q=test&hl.m
> axAnalyzedChars=1000000&hl.fl=*&hl.method=unified&timeAllowed=90000&_=
> 1614430061690}
> hits=136939 status=0 QTime=87024
>
> And &hl.fragAlignRatio=0.1
>
> 2021-02-27 15:18:45.542 INFO (qtp1291367132-19) [ x:holmes]
> o.a.s.c.S.Request [holmes] webapp=/solr path=/select
> params={hl.weightMatches=false&hl=on&fl=id,description,specification,s
> core&start=1&hl.fragAlignRatio=0.1&rows=100&hl.snippets=2&q=test&hl.ma
> xAnalyzedChars=1000000&hl.fl=*&hl.method=unified&timeAllowed=90000&_=1
> 614430061690}
> hits=136939 status=0 QTime=69033
>
> And &hl.fragAlignRatio=0.0
>
> 2021-02-27 15:20:38.194 INFO (qtp1291367132-24) [ x:holmes]
> o.a.s.c.S.Request [holmes] webapp=/solr path=/select
> params={hl.weightMatches=false&hl=on&fl=id,description,specification,s
> core&start=1&hl.fragAlignRatio=0.0&rows=100&hl.snippets=2&q=test&hl.ma
> xAnalyzedChars=1000000&hl.fl=*&hl.method=unified&timeAllowed=90000&_=1
> 614430061690}
> hits=136939 status=0 QTime=2841
>
> I left our setting at 0.0 this presumably how it was in 7.7.1 (fully
> left aligned). I am not too sure as to how many time a word has to
> occur in a record for performance to go right down but if too many
> it can have a BIG impact.
>
> I also noticed that setting &timeAllowed=90000 did not break out of
> the query until it finished. Perhaps because the query finished
> quickly and what took the time was the highlighting. It might be an
> idea to get &timeAllowed to also cover any highlighting so that the
> query does not run until the jetty timeout is hit. The machine 100%
> one core for about
> 20 mins!.
>
> Hope this helps.
>
> Regards
>
> Matthew
>
> *Matthew Flowerday*| Consultant | ULEAF
>
> Unisys | 01908 774830| matthew.flower...@unisys.com
> <mailto:matthew.flower...@unisys.com>
>
> Address Enigma | Wavendon Business Park | Wavendon | Milton Keynes |
> MK17 8LX
>
> unisys_logo <http://www.unisys.com/>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE
> PROPRIETARY MATERIAL and is for use only by the intended recipient. If
> you received this in error, please contact the sender and delete the
> e-mail and its attachments from all devices.
>
> Grey_LI <http://www.linkedin.com/company/unisys>Grey_TW
> <http://twitter.com/unisyscorp>Grey_YT
> <http://www.youtube.com/theunisyschannel>Grey_FB
> <http://www.facebook.com/unisyscorp>Grey_Vimeo
> <https://vimeo.com/unisys>Grey_UB <http://blogs.unisys.com/>
>
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
smime.p7s
Description: S/MIME cryptographic signature
