[
https://issues.apache.org/jira/browse/SOLR-2565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096774#comment-13096774
]
Yonik Seeley commented on SOLR-2565:
------------------------------------
I did some performance tests w/ autoCommit=60000 (60 sec) vs no commits at all
(10M small docs)
6 runs for each configuration, times in seconds.
no commits (not even at the end): 202,222,217,206,203,196 average=207
autoCommit=60000: 201,206,210,226,220,201 average=210
So committing every 60 seconds actually came out faster (probably due to chance
because of the high variability).
Aside: there could be a race issue w/ autocommit. I was checking the
segments_n files to verify that commits were actually taking place
periodically. On a new index, the first file is segments_1. All of the tests
ran around 3 minutes 30 seconds... which should have meant 4 more commits, or a
segments_5 file after 4 minutes. I only checked a few times, but one time I
saw a segments_6, and two other times I saw a segments_9 file. This suggests
that too many commits are happening for some reason.
> Prevent IW#close and cut over to IW#commit
> ------------------------------------------
>
> Key: SOLR-2565
> URL: https://issues.apache.org/jira/browse/SOLR-2565
> Project: Solr
> Issue Type: Improvement
> Components: update
> Affects Versions: 4.0
> Reporter: Simon Willnauer
> Assignee: Mark Miller
> Fix For: 4.0
>
> Attachments: SOLR-2565-revert.patch, SOLR-2565.patch,
> SOLR-2565.patch, SOLR-2565.patch, SOLR-2565__HuperDuperAutoCommitTest.patch,
> dump.txt, fix+hossmans-test.patch, slowtests.txt
>
>
> Spinnoff from SOLR-2193. We already have a branch to work on this issue here
> https://svn.apache.org/repos/asf/lucene/dev/branches/solr2193
> The main goal here is to prevent solr from closing the IW and use IW#commit
> instead. AFAIK the main issues here are:
> The update handler needs an overhaul.
> A few goals I think we might want to look at:
> 1. Expose the SolrIndexWriter in the api or add the proper abstractions to
> get done what we now do with special casing:
> 2. Stop closing the IndexWriter and start using commit (still lazy IW init
> though).
> 3. Drop iwAccess, iwCommit locks and sync mostly at the Lucene level.
> 4. Address the current issues we face because multiple original/'reloaded'
> cores can have a different IndexWriter on the same index.
> Eventually this is a preparation for NRT support in Solr which I will create
> a followup issue for.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
