Re: Slow highlighting on Solr 5.0.0
Thanks for the pointers. Using hl.usePhraseHighlighter=false does indeed
make it a lot faster. Obviously it's not really a solution, though,
since in 4.10 it wasn't a problem and turning it off has consequences.
I'm looking forward for the improvements in the next releases.
--Ere
8.5.2015, 19.06, Matt Hilt kirjoitti:
I¹ve been looking into this again. The phrase highlighter is much slower
than the default highlighter, so you might be able to add
hl.usePhraseHighlighter=false to your query to make it faster. Note that
web interface will NOT help here, because that param is true by default,
and the checkbox is basically broken in that respect. Also, the default
highlighter doesn¹t seem to work in all case the phrase highlighter does
though.
Also, the current development branch of 5x is much better than 5.1, but
not as good as 4.10. This ticket seems to be hitting on some of the issues
at hand:
https://issues.apache.org/jira/browse/SOLR-5855
I think this means they are getting there, but the performance is really
still much worse than 4.10, and its not obvious why.
On 5/5/15, 2:06 AM, Ere Maijala ere.maij...@helsinki.fi wrote:
I'm seeing the same with Solr 5.1.0 after upgrading from 4.10.2. Here
are my timings:
4.10.2:
process: 1432.0
highlight: 723.0
5.1.0:
process: 9570.0
highlight: 8790.0
schema.xml and solrconfig.xml are available at
https://github.com/NatLibFi/NDL-VuFind-Solr/tree/master/vufind/biblio/conf
.
A couple of jstack outputs taken when the query was executing are
available at http://pastebin.com/eJrEy2Wb
Any suggestions would be appreciated. Or would it make sense to just
file a JIRA issue?
--Ere
3.3.2015, 0.48, Matt Hilt kirjoitti:
Short form:
While testing Solr 5.0.0 within our staging environment, I noticed that
highlight enabled queries are much slower than I saw with 4.10. Are
there any obvious reasons why this might be the case? As far as I can
tell, nothing has changed with the default highlight search component or
its parameters.
A little more detail:
The bulk of the collection config set was stolen from the basic 4.X
example config set. I changed my schema.xml and solrconfig.xml just
enough to get 5.0 to create a new collection (removed non-trie fields,
some other deprecated response handler definitions, etc). I can provide
my version of the solr.HighlightComponent config, but it is identical to
the sample_techproducts_configs example in 5.0. Are there any other
config files I could provide that might be useful?
Number on ³much slower²:
I indexed a very small subset of my data into the new collection and
used the /select interface to do a simple debug query. Solr 4.10 gives
the following pertinent info:
response: { numFound: 72628,
...
debug: {
timing: { time: 95, process: { time: 94, query: { time: 6 },
highlight: { time: 84 }, debug: { time: 4 } }
---
Whereas solr 5.0 is:
response: { numFound: 1093,
...
debug: {
timing: { time: 6551, process: { time: 6549, query: { time:
0 }, highlight: { time: 6524 }, debug: { time: 25 }
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
Re: Slow highlighting on Solr 5.0.0
I¹ve been looking into this again. The phrase highlighter is much slower
than the default highlighter, so you might be able to add
hl.usePhraseHighlighter=false to your query to make it faster. Note that
web interface will NOT help here, because that param is true by default,
and the checkbox is basically broken in that respect. Also, the default
highlighter doesn¹t seem to work in all case the phrase highlighter does
though.
Also, the current development branch of 5x is much better than 5.1, but
not as good as 4.10. This ticket seems to be hitting on some of the issues
at hand:
https://issues.apache.org/jira/browse/SOLR-5855
I think this means they are getting there, but the performance is really
still much worse than 4.10, and its not obvious why.
On 5/5/15, 2:06 AM, Ere Maijala ere.maij...@helsinki.fi wrote:
I'm seeing the same with Solr 5.1.0 after upgrading from 4.10.2. Here
are my timings:
4.10.2:
process: 1432.0
highlight: 723.0
5.1.0:
process: 9570.0
highlight: 8790.0
schema.xml and solrconfig.xml are available at
https://github.com/NatLibFi/NDL-VuFind-Solr/tree/master/vufind/biblio/conf
.
A couple of jstack outputs taken when the query was executing are
available at http://pastebin.com/eJrEy2Wb
Any suggestions would be appreciated. Or would it make sense to just
file a JIRA issue?
--Ere
3.3.2015, 0.48, Matt Hilt kirjoitti:
Short form:
While testing Solr 5.0.0 within our staging environment, I noticed that
highlight enabled queries are much slower than I saw with 4.10. Are
there any obvious reasons why this might be the case? As far as I can
tell, nothing has changed with the default highlight search component or
its parameters.
A little more detail:
The bulk of the collection config set was stolen from the basic 4.X
example config set. I changed my schema.xml and solrconfig.xml just
enough to get 5.0 to create a new collection (removed non-trie fields,
some other deprecated response handler definitions, etc). I can provide
my version of the solr.HighlightComponent config, but it is identical to
the sample_techproducts_configs example in 5.0. Are there any other
config files I could provide that might be useful?
Number on ³much slower²:
I indexed a very small subset of my data into the new collection and
used the /select interface to do a simple debug query. Solr 4.10 gives
the following pertinent info:
response: { numFound: 72628,
...
debug: {
timing: { time: 95, process: { time: 94, query: { time: 6 },
highlight: { time: 84 }, debug: { time: 4 } }
---
Whereas solr 5.0 is:
response: { numFound: 1093,
...
debug: {
timing: { time: 6551, process: { time: 6549, query: { time:
0 }, highlight: { time: 6524 }, debug: { time: 25 }
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
smime.p7s
Description: S/MIME cryptographic signature
Re: Slow highlighting on Solr 5.0.0
I'm seeing the same with Solr 5.1.0 after upgrading from 4.10.2. Here
are my timings:
4.10.2:
process: 1432.0
highlight: 723.0
5.1.0:
process: 9570.0
highlight: 8790.0
schema.xml and solrconfig.xml are available at
https://github.com/NatLibFi/NDL-VuFind-Solr/tree/master/vufind/biblio/conf.
A couple of jstack outputs taken when the query was executing are
available at http://pastebin.com/eJrEy2Wb
Any suggestions would be appreciated. Or would it make sense to just
file a JIRA issue?
--Ere
3.3.2015, 0.48, Matt Hilt kirjoitti:
Short form:
While testing Solr 5.0.0 within our staging environment, I noticed that
highlight enabled queries are much slower than I saw with 4.10. Are
there any obvious reasons why this might be the case? As far as I can
tell, nothing has changed with the default highlight search component or
its parameters.
A little more detail:
The bulk of the collection config set was stolen from the basic 4.X
example config set. I changed my schema.xml and solrconfig.xml just
enough to get 5.0 to create a new collection (removed non-trie fields,
some other deprecated response handler definitions, etc). I can provide
my version of the solr.HighlightComponent config, but it is identical to
the sample_techproducts_configs example in 5.0. Are there any other
config files I could provide that might be useful?
Number on “much slower”:
I indexed a very small subset of my data into the new collection and
used the /select interface to do a simple debug query. Solr 4.10 gives
the following pertinent info:
response: { numFound: 72628,
...
debug: {
timing: { time: 95, process: { time: 94, query: { time: 6 },
highlight: { time: 84 }, debug: { time: 4 } }
---
Whereas solr 5.0 is:
response: { numFound: 1093,
...
debug: {
timing: { time: 6551, process: { time: 6549, query: { time:
0 }, highlight: { time: 6524 }, debug: { time: 25 }
--
Ere Maijala
Kansalliskirjasto / The National Library of Finland
