[
https://issues.apache.org/jira/browse/SOLR-10130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866806#comment-15866806
]
Ishan Chattopadhyaya commented on SOLR-10130:
---------------------------------------------
I could reproduce a 1.6x slowdown for prefix queries.
Benchmarking suite:
https://github.com/chatman/solr-upgrade-tests/blob/master/BENCHMARKS.md
Environment: packet.net, Type 0 server
(https://www.packet.net/bare-metal/servers/type-0/)
{code}
Prefix query times for 6.4.1
----------------------------
java -cp
target/org.apache.solr.tests.upgradetests-0.0.1-SNAPSHOT-jar-with-dependencies.jar:.
org.apache.solr.tests.upgradetests.SimpleBenchmarks -v
72f75b2503fa0aa4f0aff76d439874feb923bb0e -Nodes 1 -Shards 1 -Replicas 1
-numDocs 10000 -threads 4 -benchmarkType generalQuerying
Got results for prefix queries: 10000
Max time (prefix queries): 2156ms
Total time (prefix queries): 1324856ms
Prefix query times for 6.3.0
----------------------------
java -cp
target/org.apache.solr.tests.upgradetests-0.0.1-SNAPSHOT-jar-with-dependencies.jar:.
org.apache.solr.tests.upgradetests.SimpleBenchmarks -v
6fa26fe8553b7b65dee96da741f2c1adf4cb6216 -patchUrl
http://147.75.108.131/LUCENE-7651.patch -Nodes 1 -Shards 1 -Replicas 1 -numDocs
10000 -threads 4 -benchmarkType generalQuerying
Got results for prefix queries: 10000
Max time (prefix queries): 1358ms
Total time (prefix queries): 839534ms
{code}
Notes:
1. The -threads parameter here is for no. of indexing threads, and number of
querying threads is 4 times that, i.e. 16 in this case.
2. Total time is the sum of all times, as reported in the response header's
"QTime". Max time is the QTime for the worst performing query.
> Serious performance degradation in Solr 6.4.1 due to the new metrics
> collection
> -------------------------------------------------------------------------------
>
> Key: SOLR-10130
> URL: https://issues.apache.org/jira/browse/SOLR-10130
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: metrics
> Affects Versions: 6.4.1
> Environment: Centos 7, OpenJDK 1.8.0 update 111
> Reporter: Ere Maijala
> Assignee: Andrzej Bialecki
> Priority: Blocker
> Labels: perfomance
> Fix For: master (7.0), 6.4.2
>
> Attachments: SOLR-10130.patch, solr-8983-console-f1.log
>
>
> We've stumbled on serious performance issues after upgrading to Solr 6.4.1.
> Looks like the new metrics collection system in MetricsDirectoryFactory is
> causing a major slowdown. This happens with an index configuration that, as
> far as I can see, has no metrics specific configuration and uses
> luceneMatchVersion 5.5.0. In practice a moderate load will completely bog
> down the server with Solr threads constantly using up all CPU (600% on 6 core
> machine) capacity with a load that normally where we normally see an average
> load of < 50%.
> I took stack traces (I'll attach them) and noticed that the threads are
> spending time in com.codahale.metrics.Meter.mark. I tested building Solr
> 6.4.1 with the metrics collection disabled in MetricsDirectoryFactory getByte
> and getBytes methods and was unable to reproduce the issue.
> As far as I can see there are several issues:
> 1. Collecting metrics on every single byte read is slow.
> 2. Having it enabled by default is not a good idea.
> 3. The comment "enable coarse-grained metrics by default" at
> https://github.com/apache/lucene-solr/blob/branch_6x/solr/core/src/java/org/apache/solr/update/SolrIndexConfig.java#L104
> implies that only coarse-grained metrics should be enabled by default, and
> this contradicts with collecting metrics on every single byte read.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
