[ https://issues.apache.org/jira/browse/IGNITE-10052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16681287#comment-16681287 ]
Ivan Artukhov edited comment on IGNITE-10052 at 11/9/18 11:26 AM: ------------------------------------------------------------------ Commit b2f697b shows another error when I go through the steps from description. Error on client's side: {code} 0: jdbc:ignite:thin://127.0.0.1/> insert into t1 values (1,1); Error: Failed to run update. Failed to update backup node: [localNodeId=a80dc9b8-6a28-4015-a288-0436e2916f98, remoteNodeId=56dadffc-0d9a-4c42-8d3b-55266e093fc8] (state=50000,code=1) {code} On server's side: {noformat} [14:08:44,202][SEVERE][jdbc-request-handler-worker-#48][DmlStatementsProcessor] Error during update [localNodeId=a80dc9b8-6a28-4015-a288-0436e2916f98] class org.apache.ignite.IgniteCheckedException: Failed to update backup node: [localNodeId=a80dc9b8-6a28-4015-a288-0436e2916f98, remoteNodeId=56dadffc-0d9a-4c42-8d3b-55266e093fc8] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.onResult(GridDhtTxAbstractEnlistFuture.java:987) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processDhtTxQueryEnlistResponse(GridDhtTransactionalCacheAdapter.java:2342) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.access$1200(GridDhtTransactionalCacheAdapter.java:112) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$18.apply(GridDhtTransactionalCacheAdapter.java:257) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$18.apply(GridDhtTransactionalCacheAdapter.java:255) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197) at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127) at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093) at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Caused by: class org.apache.ignite.IgniteCheckedException: Runtime failure on bounds: [lower=MvccMaxSearchRow [], upper=MvccMinSearchRow []] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1061) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:1939) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.mvccUpdate(GridCacheOffheapManager.java:1840) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:532) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.mvccSet(GridCacheMapEntry.java:1104) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.mvccEnlistBatch(IgniteTxHandler.java:1920) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processDhtTxQueryEnlistRequest(GridDhtTransactionalCacheAdapter.java:2305) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.access$1100(GridDhtTransactionalCacheAdapter.java:112) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$17.apply(GridDhtTransactionalCacheAdapter.java:250) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$17.apply(GridDhtTransactionalCacheAdapter.java:248) ... 13 more Caused by: class org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: Unexpected state: [state=0, rowVer=1541761538351:12:1, txVer=1541761538351:29:1, localNodeId=56dadffc-0d9a-4c42-8d3b-55266e093fc8] at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:317) at org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:304) at org.apache.ignite.internal.processors.cache.tree.mvcc.data.MvccUpdateDataRow.visit(MvccUpdateDataRow.java:413) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3021) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.init(BPlusTree.java:2990) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.notFound(BPlusTree.java:2946) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:306) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5615) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:271) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5600) at org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:159) at org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:334) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visitDown(BPlusTree.java:2585) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doVisit(BPlusTree.java:2552) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$8200(BPlusTree.java:90) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3148) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.access$5100(BPlusTree.java:2908) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1058) ... 22 more {noformat} *a80dc9b8-6a28-4015-a288-0436e2916f98* is the ID of the first server node. *56dadffc-0d9a-4c42-8d3b-55266e093fc8* is the ID of the second (restarted) server node. was (Author: artukhov): Commit b2f697b shows another error when I go through the steps from description. Error on client's side: {code} 0: jdbc:ignite:thin://127.0.0.1/> insert into t1 values (1,1); Error: Failed to run update. Failed to update backup node: [localNodeId=a80dc9b8-6a28-4015-a288-0436e2916f98, remoteNodeId=56dadffc-0d9a-4c42-8d3b-55266e093fc8] (state=50000,code=1) {code} On server's side: {code} [14:08:44,202][SEVERE][jdbc-request-handler-worker-#48][DmlStatementsProcessor] Error during update [localNodeId=a80dc9b8-6a28-4015-a288-0436e2916f98] class org.apache.ignite.IgniteCheckedException: Failed to update backup node: [localNodeId=a80dc9b8-6a28-4015-a288-0436e2916f98, remoteNodeId=56dadffc-0d9a-4c42-8d3b-55266e093fc8] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.onResult(GridDhtTxAbstractEnlistFuture.java:987) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processDhtTxQueryEnlistResponse(GridDhtTransactionalCacheAdapter.java:2342) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.access$1200(GridDhtTransactionalCacheAdapter.java:112) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$18.apply(GridDhtTransactionalCacheAdapter.java:257) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$18.apply(GridDhtTransactionalCacheAdapter.java:255) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197) at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127) at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093) at org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) Caused by: class org.apache.ignite.IgniteCheckedException: Runtime failure on bounds: [lower=MvccMaxSearchRow [], upper=MvccMinSearchRow []] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1061) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:1939) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.mvccUpdate(GridCacheOffheapManager.java:1840) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:532) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.mvccSet(GridCacheMapEntry.java:1104) at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.mvccEnlistBatch(IgniteTxHandler.java:1920) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processDhtTxQueryEnlistRequest(GridDhtTransactionalCacheAdapter.java:2305) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.access$1100(GridDhtTransactionalCacheAdapter.java:112) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$17.apply(GridDhtTransactionalCacheAdapter.java:250) at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$17.apply(GridDhtTransactionalCacheAdapter.java:248) ... 13 more {code} *a80dc9b8-6a28-4015-a288-0436e2916f98* is the ID of the first server node. *56dadffc-0d9a-4c42-8d3b-55266e093fc8* is the ID of the second (restarted) server node. > Restart node during TX causes vacuum error. > ------------------------------------------- > > Key: IGNITE-10052 > URL: https://issues.apache.org/jira/browse/IGNITE-10052 > Project: Ignite > Issue Type: Bug > Components: mvcc > Affects Versions: 2.7 > Reporter: Sergey Kozlov > Assignee: Andrew Mashenkov > Priority: Blocker > Fix For: 2.7 > > > 1. Start 2 nodes with PDS, activate , start {{sqlline}} > 2. Create table > {noformat} > 0: jdbc:ignite:thin://127.0.0.1/> create table t1 (a int, b int, primary > key(a)) with "atomicity=TRANSACTIONAL_SNAPSHOT, > backups=1"; > No rows affected (0,294 seconds) > {noformat} > 3. Open TX: > {noformat} > 0: jdbc:ignite:thin://127.0.0.1/> begin; > No rows affected (0,007 seconds) > 0: jdbc:ignite:thin://127.0.0.1/> insert into t1 values (1,1); > 1 row affected (0,112 seconds) > {noformat} > 4. Stop and then start 2nd node > 5. Rollback TX and check the table data (no rows added): > {noformat} > 0: jdbc:ignite:thin://127.0.0.1/> rollback; > No rows affected (0,011 seconds) > 0: jdbc:ignite:thin://127.0.0.1/> select * from t1; > +--------------------------------+--------------------------------+ > | A | B | > +--------------------------------+--------------------------------+ > +--------------------------------+--------------------------------+ > No rows selected (0,067 seconds) > {noformat} > 6. Start second TX that throws the exception: > {noformat} > 0: jdbc:ignite:thin://127.0.0.1/> begin; > No rows affected (0,001 seconds) > 0: jdbc:ignite:thin://127.0.0.1/> insert into t1 values (1,1); > Error: Failed to run update. Runtime failure on bounds: > [lower=MvccMaxSearchRow [], upper=MvccMinSearchRow []] (state=50 > 000,code=1) > java.sql.SQLException: Failed to run update. Runtime failure on bounds: > [lower=MvccMaxSearchRow [], upper=MvccMinSearchR > ow []] > at > org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:750) > at > org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:212) > at > org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:475) > at sqlline.Commands.execute(Commands.java:823) > at sqlline.Commands.sql(Commands.java:733) > at sqlline.SqlLine.dispatch(SqlLine.java:795) > at sqlline.SqlLine.begin(SqlLine.java:668) > at sqlline.SqlLine.start(SqlLine.java:373) > at sqlline.SqlLine.main(SqlLine.java:265) > 0: jdbc:ignite:thin://127.0.0.1/> > {noformat} > The server nodes print out: > {noformat} > [18:46:32,136][SEVERE][jdbc-request-handler-worker-#62][DmlStatementsProcessor] > Error during update [localNodeId=319a2fda-1315-4bdf-8647-7ddee2d3342e] > class org.apache.ignite.IgniteCheckedException: Runtime failure on bounds: > [lower=MvccMaxSearchRow [], upper=MvccMinSearchRow []] > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1061) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:1919) > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.mvccUpdate(GridCacheOffheapManager.java:1840) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:530) > at > org.apache.ignite.internal.processors.cache.GridCacheMapEntry.mvccSet(GridCacheMapEntry.java:1104) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.continueLoop(GridDhtTxAbstractEnlistFuture.java:460) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.init(GridDhtTxAbstractEnlistFuture.java:368) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearTxQueryResultsEnlistRequest(GridDhtTransactionalCacheAdapter.java:2000) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.access$700(GridDhtTransactionalCacheAdapter.java:112) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$12.apply(GridDhtTransactionalCacheAdapter.java:215) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$12.apply(GridDhtTransactionalCacheAdapter.java:213) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295) > at > org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569) > at > org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197) > at > org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127) > at > org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093) > at > org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505) > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > at java.lang.Thread.run(Thread.java:748) > Caused by: class > org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: > Unexpected state: [state=0, rowVer=1540827914710:6:1, > txVer=1540827914710:21:1, localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079] > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:297) > at > org.apache.ignite.internal.processors.cache.tree.mvcc.data.MvccUpdateDataRow.visit(MvccUpdateDataRow.java:413) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3021) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.init(BPlusTree.java:2990) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.notFound(BPlusTree.java:2946) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:306) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5615) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:271) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5600) > at > org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:159) > at > org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:334) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visitDown(BPlusTree.java:2585) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doVisit(BPlusTree.java:2552) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$8200(BPlusTree.java:90) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3148) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.access$5100(BPlusTree.java:2908) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1058) > ... 23 more > [18:46:32,138][SEVERE][jdbc-request-handler-worker-#62][JdbcRequestHandler] > Failed to execute SQL query [reqId=0, req=JdbcQueryExecuteRequest > [schemaName=PUBLIC, pageSize=1024, maxRows=0, sqlQry=insert into t1 values > (1,1), args=Object[] [], stmtType=ANY_STATEMENT_TYPE, autoCommit=true]] > class org.apache.ignite.internal.processors.query.IgniteSQLException: Failed > to run update. Runtime failure on bounds: [lower=MvccMaxSearchRow [], > upper=MvccMinSearchRow []] > at > org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.executeUpdateStatement(DmlStatementsProcessor.java:593) > at > org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFields(DmlStatementsProcessor.java:185) > at > org.apache.ignite.internal.processors.query.h2.DmlStatementsProcessor.updateSqlFieldsDistributed(DmlStatementsProcessor.java:358) > at > org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.doRunPrepared(IgniteH2Indexing.java:2239) > at > org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:2158) > at > org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2141) > at > org.apache.ignite.internal.processors.query.GridQueryProcessor$4.applyx(GridQueryProcessor.java:2136) > at > org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36) > at > org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2713) > at > org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2150) > at > org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.executeQuery(JdbcRequestHandler.java:511) > at > org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.doHandle(JdbcRequestHandler.java:245) > at > org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandlerWorker.body(JdbcRequestHandlerWorker.java:90) > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > at java.lang.Thread.run(Thread.java:748) > Caused by: class org.apache.ignite.IgniteCheckedException: Runtime failure on > bounds: [lower=MvccMaxSearchRow [], upper=MvccMinSearchRow []] > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1061) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:1919) > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.mvccUpdate(GridCacheOffheapManager.java:1840) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.mvccUpdate(IgniteCacheOffheapManagerImpl.java:530) > at > org.apache.ignite.internal.processors.cache.GridCacheMapEntry.mvccSet(GridCacheMapEntry.java:1104) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.continueLoop(GridDhtTxAbstractEnlistFuture.java:460) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxAbstractEnlistFuture.init(GridDhtTxAbstractEnlistFuture.java:368) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.processNearTxQueryResultsEnlistRequest(GridDhtTransactionalCacheAdapter.java:2000) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter.access$700(GridDhtTransactionalCacheAdapter.java:112) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$12.apply(GridDhtTransactionalCacheAdapter.java:215) > at > org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTransactionalCacheAdapter$12.apply(GridDhtTransactionalCacheAdapter.java:213) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1056) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:581) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:380) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:306) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:101) > at > org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:295) > at > org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1569) > at > org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1197) > at > org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:127) > at > org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1093) > at > org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:505) > ... 2 more > Caused by: class > org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: > Unexpected state: [state=0, rowVer=1540827914710:6:1, > txVer=1540827914710:21:1, localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079] > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:297) > at > org.apache.ignite.internal.processors.cache.tree.mvcc.data.MvccUpdateDataRow.visit(MvccUpdateDataRow.java:413) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3021) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.init(BPlusTree.java:2990) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.notFound(BPlusTree.java:2946) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:306) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5615) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run(BPlusTree.java:271) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:5600) > at > org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:159) > at > org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:334) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visitDown(BPlusTree.java:2585) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doVisit(BPlusTree.java:2552) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$8200(BPlusTree.java:90) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.visit(BPlusTree.java:3148) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$TreeVisitor.access$5100(BPlusTree.java:2908) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.visit(BPlusTree.java:1058) > ... 23 more > {noformat} > {noformat} > [18:46:20,439][SEVERE][vacuum-scheduler-#47][MvccProcessorImpl] Vacuum error. > class > org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: > Unexpected state: [state=0, rowVer=1540827914710:6:536870913, > localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079] > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:281) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3206) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3180) > at > org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:279) > at > org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:256) > at > org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.updateDataRow(AbstractFreeList.java:555) > at > org.apache.ignite.internal.processors.cache.persistence.RowStore.updateDataRow(RowStore.java:143) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxState(IgniteCacheOffheapManagerImpl.java:2449) > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.updateTxState(GridCacheOffheapManager.java:1917) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImpl.java:2415) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccProcessorImpl.java:2300) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2198) > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > at java.lang.Thread.run(Thread.java:748) > [18:46:20,582][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] > Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, > checkpointLockHoldTime=12ms, reason='timeout'] > [18:46:20,988][INFO][exchange-worker-#43][time] Started exchange init > [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], > mvccCrd=MvccCoordinator [nodeId=319a2fda-1315-4bdf-8647-7ddee2d3342e, > crdVer=1540827914710, topVer=AffinityTopologyVersion [topVer=4, > minorTopVer=0]], mvccCrdChange=false, crd=false, evt=DISCOVERY_CUSTOM_EVT, > evtNode=319a2fda-1315-4bdf-8647-7ddee2d3342e, > customEvt=CacheAffinityChangeMessage > [id=f4aff70c661-30eb588f-bc2f-4094-86c4-e7070c6fcb1e, > topVer=AffinityTopologyVersion [topVer=4, minorTopVer=0], exchId=null, > partsMsg=null, exchangeNeeded=true], allowMerge=false] > [18:46:20,998][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] > Finished waiting for partition release future [topVer=AffinityTopologyVersion > [topVer=4, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=DISTRIBUTED] > [18:46:21,004][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] > Finished waiting for partitions release latch: ClientLatch > [coordinator=TcpDiscoveryNode [id=319a2fda-1315-4bdf-8647-7ddee2d3342e, > addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], > discPort=47500, order=1, intOrder=1, lastExchangeTime=1540827969985, > loc=false, ver=2.7.1#20181029-sha1:83c27088, isClient=false], ackSent=true, > super=CompletableLatch [id=exchange, topVer=AffinityTopologyVersion > [topVer=4, minorTopVer=1]]] > [18:46:21,004][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] > Finished waiting for partition release future [topVer=AffinityTopologyVersion > [topVer=4, minorTopVer=1], waitTime=0ms, futInfo=NA, mode=LOCAL] > [18:46:21,005][INFO][exchange-worker-#43][GridCacheDatabaseSharedManager] > Logical recovery performed in 0 ms. > [18:46:21,015][INFO][exchange-worker-#43][GridDhtPartitionsExchangeFuture] > Sending Single Message performed in 3 ms. > [18:46:21,016][INFO][exchange-worker-#43][time] Finished exchange init > [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], crd=false] > [18:46:21,035][INFO][sys-#64][GridDhtPartitionsExchangeFuture] Received full > message, will finish exchange [node=319a2fda-1315-4bdf-8647-7ddee2d3342e, > resVer=AffinityTopologyVersion [topVer=4, minorTopVer=1]] > [18:46:21,039][INFO][sys-#64][GridDhtPartitionsExchangeFuture] Affinity > changes applied in 0 ms. > [18:46:21,044][INFO][sys-#64][GridDhtPartitionsExchangeFuture] Full map > updating for 2 groups performed in 2 ms. > [18:46:21,044][INFO][sys-#64][GridDhtPartitionsExchangeFuture] Finish > exchange future [startVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], > resVer=AffinityTopologyVersion [topVer=4, minorTopVer=1], err=null] > [18:46:21,050][INFO][sys-#64][GridDhtPartitionsExchangeFuture] Completed > partition exchange [localNode=4d66a9d5-4352-495c-9c64-405ccdf8a079, > exchange=GridDhtPartitionsExchangeFuture [topVer=AffinityTopologyVersion > [topVer=4, minorTopVer=1], evt=DISCOVERY_CUSTOM_EVT, evtNode=TcpDiscoveryNode > [id=319a2fda-1315-4bdf-8647-7ddee2d3342e, addrs=ArrayList [127.0.0.1], > sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, > lastExchangeTime=1540827969985, loc=false, ver=2.7.1#20181029-sha1:83c27088, > isClient=false], done=true], topVer=AffinityTopologyVersion [topVer=4, > minorTopVer=1], durationFromInit=66] > [18:46:21,052][INFO][exchange-worker-#43][GridCachePartitionExchangeManager] > Skipping rebalancing (no affinity changes) [top=AffinityTopologyVersion > [topVer=4, minorTopVer=1], rebTopVer=AffinityTopologyVersion [topVer=4, > minorTopVer=0], evt=DISCOVERY_CUSTOM_EVT, > evtNode=319a2fda-1315-4bdf-8647-7ddee2d3342e, client=false] > [18:46:21,586][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] > Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, > checkpointLockHoldTime=10ms, reason='timeout'] > [18:46:22,595][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] > Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, > checkpointLockHoldTime=9ms, reason='timeout'] > [18:46:24,602][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] > Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, > checkpointLockHoldTime=7ms, reason='timeout'] > [18:46:25,406][SEVERE][vacuum-scheduler-#47][MvccProcessorImpl] Vacuum error. > class > org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: > Unexpected state: [state=0, rowVer=1540827914710:6:536870913, > localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079] > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:281) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3206) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3180) > at > org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:279) > at > org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:256) > at > org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.updateDataRow(AbstractFreeList.java:555) > at > org.apache.ignite.internal.processors.cache.persistence.RowStore.updateDataRow(RowStore.java:143) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxState(IgniteCacheOffheapManagerImpl.java:2449) > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.updateTxState(GridCacheOffheapManager.java:1917) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImpl.java:2415) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccProcessorImpl.java:2300) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2198) > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > at java.lang.Thread.run(Thread.java:748) > [18:46:26,612][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] > Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, > checkpointLockHoldTime=5ms, reason='timeout'] > [18:46:29,632][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] > Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, > checkpointLockHoldTime=6ms, reason='timeout'] > [18:46:30,406][SEVERE][vacuum-scheduler-#47][MvccProcessorImpl] Vacuum error. > class > org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: > Unexpected state: [state=0, rowVer=1540827914710:6:536870913, > localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079] > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:281) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3206) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3180) > at > org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:279) > at > org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:256) > at > org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.updateDataRow(AbstractFreeList.java:555) > at > org.apache.ignite.internal.processors.cache.persistence.RowStore.updateDataRow(RowStore.java:143) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxState(IgniteCacheOffheapManagerImpl.java:2449) > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.updateTxState(GridCacheOffheapManager.java:1917) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImpl.java:2415) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccProcessorImpl.java:2300) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2198) > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > at java.lang.Thread.run(Thread.java:748) > [18:46:35,406][SEVERE][vacuum-scheduler-#47][MvccProcessorImpl] Vacuum error. > class > org.apache.ignite.internal.transactions.IgniteTxMvccVersionCheckedException: > Unexpected state: [state=0, rowVer=1540827914710:6:536870913, > localNodeId=4d66a9d5-4352-495c-9c64-405ccdf8a079] > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:310) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccUtils.unexpectedStateException(MvccUtils.java:281) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3206) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$MvccUpdateTxStateHintHandler.run(IgniteCacheOffheapManagerImpl.java:3180) > at > org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.writePage(PageHandler.java:279) > at > org.apache.ignite.internal.processors.cache.persistence.DataStructure.write(DataStructure.java:256) > at > org.apache.ignite.internal.processors.cache.persistence.freelist.AbstractFreeList.updateDataRow(AbstractFreeList.java:555) > at > org.apache.ignite.internal.processors.cache.persistence.RowStore.updateDataRow(RowStore.java:143) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.updateTxState(IgniteCacheOffheapManagerImpl.java:2449) > at > org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.updateTxState(GridCacheOffheapManager.java:1917) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.cleanup(MvccProcessorImpl.java:2415) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.processPartition(MvccProcessorImpl.java:2300) > at > org.apache.ignite.internal.processors.cache.mvcc.MvccProcessorImpl$VacuumWorker.body(MvccProcessorImpl.java:2198) > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > at java.lang.Thread.run(Thread.java:748) > [18:46:38,674][INFO][db-checkpoint-thread-#51][GridCacheDatabaseSharedManager] > Skipping checkpoint (no pages were modified) [checkpointLockWait=0ms, > checkpointLockHoldTime=4ms, reason='timeout'] > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)