[
https://issues.apache.org/jira/browse/SOLR-2748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13100406#comment-13100406
]
Yonik Seeley commented on SOLR-2748:
------------------------------------
My 10M doc performance testing now shows the correct number of commits.
I also tried turning on soft autocommits at 1 sec, and that also resulted in
the correct number of soft commits being done.
Oddly enough the autoCommit + softAutoCommit test ran in 2:35 sec, while the
autoCommit only test ran in 3:33.
One explanation could be that DWPT doesn't necessarily seem optimal for older
(non-SSD) drives (Erick reported seeing trunk as slower than 3x on his system
with a spinning-magnets type drive), and the smaller segments avoided some of
this.
The other explanation (and this one actually makes more sense to me) is that
the CSV loader used is single-threaded. Adding the first 1000 documents to a
small segment is probably more efficient than adding the last 1000 to a larger
segment. Doing more soft commits means creating smaller segments and doing
more work in background merging using other CPU cores (basically, it increased
the parallelism).
> autocommit commits too many times
> ---------------------------------
>
> Key: SOLR-2748
> URL: https://issues.apache.org/jira/browse/SOLR-2748
> Project: Solr
> Issue Type: Bug
> Reporter: Yonik Seeley
> Attachments: SOLR-2748.patch, SOLR-2748.patch
>
>
> autocommit seems to commit more frequently than configured.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
