[
https://issues.apache.org/jira/browse/SOLR-18157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Salagnac updated SOLR-18157:
-----------------------------------
Description:
During massive indexing, I noticed GC on client/SolrJ was a top contributor in
performance bottlenecks.
A lot of time is spend in method {{JavaBinCodec.writeStr()}}. It seems we
allocate an internal buffer much too often. For a given indexing batch, each
time the provided string input is larger than the previous one, we allocate a
new buffer with the exact same size as the current input.
If we are very unlucky and each string is one char longer than the previous
one, we allocate a new buffer for each string!
We can easily reduce the number of allocations by increasing buffer size
exponentially.
> Optimize buffer allocation in JavaBinCodec
> ------------------------------------------
>
> Key: SOLR-18157
> URL: https://issues.apache.org/jira/browse/SOLR-18157
> Project: Solr
> Issue Type: Improvement
> Components: SolrJ
> Affects Versions: 9.10, 10.0
> Reporter: Pierre Salagnac
> Assignee: Pierre Salagnac
> Priority: Minor
>
> During massive indexing, I noticed GC on client/SolrJ was a top contributor
> in performance bottlenecks.
> A lot of time is spend in method {{JavaBinCodec.writeStr()}}. It seems we
> allocate an internal buffer much too often. For a given indexing batch, each
> time the provided string input is larger than the previous one, we allocate a
> new buffer with the exact same size as the current input.
> If we are very unlucky and each string is one char longer than the previous
> one, we allocate a new buffer for each string!
> We can easily reduce the number of allocations by increasing buffer size
> exponentially.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
