[
https://issues.apache.org/jira/browse/SOLR-6816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14382123#comment-14382123
]
Yonik Seeley commented on SOLR-6816:
------------------------------------
bq. I didn't think the lookupVersion would be that much of an overhead, but my
testing shows that it is, even when using docValues for the version field.
Can you tell what part is taking so much time? Looking up the ID to find the
internal lucene docid? Obtaining the docvalues reference? or using it to look
up the version? My guess would be that it's the external-id to internal-id
translation that is the slowest.
There's another way to safely & transparently speed up indexing for everyone:
maintain-highest version-per-bucket. It's been the plan since the beginning...
see VersionBucket:
{code}
public class VersionBucket {
public long highest;
public void updateHighest(long val) {
if (highest != 0) {
highest = Math.max(highest, Math.abs(val));
}
}
}
{code}
Right now, we start off with highest=0 (which essentially means "unknown"), so
we never update it.
If we could initialize the highest versions correctly (would need to get from
the index) then we would only need to check versions when a reorder happened on
the same bucket.
> Review SolrCloud Indexing Performance.
> --------------------------------------
>
> Key: SOLR-6816
> URL: https://issues.apache.org/jira/browse/SOLR-6816
> Project: Solr
> Issue Type: Task
> Components: SolrCloud
> Reporter: Mark Miller
> Priority: Critical
> Attachments: SolrBench.pdf
>
>
> We have never really focused on indexing performance, just correctness and
> low hanging fruit. We need to vet the performance and try to address any
> holes.
> Note: A common report is that adding any replication is very slow.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
