[ https://issues.apache.org/jira/browse/SOLR-6776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14227062#comment-14227062 ]
Mark Miller commented on SOLR-6776: ----------------------------------- By default, the tlog doesnt fsync, it just flushes and leans on replicas. You can configure the sync level in solrconfig.xml. > Data lost when use SoftCommit and TLog > -------------------------------------- > > Key: SOLR-6776 > URL: https://issues.apache.org/jira/browse/SOLR-6776 > Project: Solr > Issue Type: Bug > Affects Versions: 4.10 > Reporter: yuanyun.cn > Labels: softCommit, updateLog > Fix For: 4.10.3 > > > We enabled update log and change autoCommit to some bigger value 10 mins. > After restart, we push one doc with softCommit=true > http://localhost:8983/solr/update?stream.body=<add><doc><field > name="id">id1</field></doc></add>&softCommit=true > Then we kill the java process after a min. > After restart, Tlog failed to replay with following exception, and there is > no data in solr. > 6245 [coreLoadExecutor-5-thread-1] ERROR org.apache.solr.update.UpdateLog รป > Failure to open existing log file (non fatal) > E:\jeffery\src\apache\solr\4.10.2\solr-4.10.2\example\solr\collection1\data\t > log\tlog.0000000000000000000:org.apache.solr.common.SolrException: > java.io.EOFException > at > org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:181) > at org.apache.solr.update.UpdateLog.init(UpdateLog.java:261) > at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:134) > at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:94) > at > org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:100) > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown > Source) > at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown > Source) > at java.lang.reflect.Constructor.newInstance(Unknown Source) > at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:550) > at > org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:620) > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:835) > at org.apache.solr.core.SolrCore.<init>(SolrCore.java:646) > at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491) > at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) > at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) > at java.util.concurrent.FutureTask.run(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at java.lang.Thread.run(Unknown Source) > Caused by: java.io.EOFException > at > org.apache.solr.common.util.FastInputStream.readUnsignedByte(FastInputStream.java:73) > at > org.apache.solr.common.util.FastInputStream.readInt(FastInputStream.java:216) > at > org.apache.solr.update.TransactionLog.readHeader(TransactionLog.java:268) > at > org.apache.solr.update.TransactionLog.<init>(TransactionLog.java:159) > ... 19 more > Check the code: seems this is related with: > org.apache.solr.update.processor.RunUpdateProcessor, in processCommit, it > sets changesSinceCommit=false(even we are using softCommit) > So in finish, updateLog.finish will not be called. > public void finish() throws IOException { > if (changesSinceCommit && updateHandler.getUpdateLog() != null) { > updateHandler.getUpdateLog().finish(null); > } > super.finish(); > } > To fix this issue: I have to change RunUpdateProcessor.processCommit like > below: > if (!cmd.softCommit) { > changesSinceCommit = false; > } -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org