[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357098#comment-17357098 ] Kirill Tkalenko commented on IGNITE-8719: - [~amashenkov] Please make code review. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 5h > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17357095#comment-17357095 ] Ignite TC Bot commented on IGNITE-8719: --- {panel:title=Branch: [pull/9090/head] Base: [master] : Possible Blockers (1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1} {color:#d04437}Platform .NET (Inspections)*{color} [[tests 0 Failure on metric |https://ci.ignite.apache.org/viewLog.html?buildId=6029650]] {panel} {panel:title=Branch: [pull/9090/head] Base: [master] : New Tests (8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1} {color:#8b}PDS (Indexing){color} [[tests 8|https://ci.ignite.apache.org/viewLog.html?buildId=6034392]] * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testRestartNodeFlowIndexRebuildStateStorage - PASSED{color} {panel} [TeamCity *--> Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=6029135&buildTypeId=IgniteTests24Java8_RunAll] > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 5h > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17354836#comment-17354836 ] Kirill Tkalenko commented on IGNITE-8719: - [~jooger] Please make code review. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17354498#comment-17354498 ] Ignite TC Bot commented on IGNITE-8719: --- {panel:title=Branch: [pull/9090/head] Base: [master] : Possible Blockers (1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1} {color:#d04437}Platform .NET (Inspections)*{color} [[tests 0 Failure on metric |https://ci.ignite.apache.org/viewLog.html?buildId=6029650]] {panel} {panel:title=Branch: [pull/9090/head] Base: [master] : New Tests (8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1} {color:#8b}PDS (Indexing){color} [[tests 8|https://ci.ignite.apache.org/viewLog.html?buildId=6029102]] * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testRestartNodeFlowIndexRebuildStateStorage - PASSED{color} {panel} [TeamCity *--> Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=6029135&buildTypeId=IgniteTests24Java8_RunAll] > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 4h 10m > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17345889#comment-17345889 ] Stanilovsky Evgeny commented on IGNITE-8719: [~ktkale...@gridgain.com] plz check one more comment from my side, overall looks good. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 3h 50m > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344961#comment-17344961 ] Ignite TC Bot commented on IGNITE-8719: --- {panel:title=Branch: [pull/9090/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/9090/head] Base: [master] : New Tests (8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1} {color:#8b}PDS (Indexing){color} [[tests 8|https://ci.ignite.apache.org/viewLog.html?buildId=6007815]] * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testRestartNodeFlowIndexRebuildStateStorage - PASSED{color} {panel} [TeamCity *--> Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=6007848&buildTypeId=IgniteTests24Java8_RunAll] > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 3h 40m > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343715#comment-17343715 ] Kirill Tkalenko commented on IGNITE-8719: - [~zstan] Please make code review. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 2.5h > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343714#comment-17343714 ] Ignite TC Bot commented on IGNITE-8719: --- {panel:title=Branch: [pull/9090/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/9090/head] Base: [master] : New Tests (8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1} {color:#8b}PDS (Indexing){color} [[tests 8|https://ci.ignite.apache.org/viewLog.html?buildId=6004413]] * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testRestartNodeFlowIndexRebuildStateStorage - PASSED{color} {panel} [TeamCity *--> Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=6004446&buildTypeId=IgniteTests24Java8_RunAll] > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 2.5h > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17343198#comment-17343198 ] Stanilovsky Evgeny commented on IGNITE-8719: [~ktkale...@gridgain.com] plz check my comments, if you ok with my proposals i would like to review it once more after fixes, thanks ! > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17342487#comment-17342487 ] Ignite TC Bot commented on IGNITE-8719: --- {panel:title=Branch: [pull/9090/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/9090/head] Base: [master] : New Tests (8)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1} {color:#8b}PDS (Indexing){color} [[tests 8|https://ci.ignite.apache.org/viewLog.html?buildId=6001403]] * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testRestartNodeFlowIndexRebuildStateStorage - PASSED{color} {panel} [TeamCity *--> Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=5999042&buildTypeId=IgniteTests24Java8_RunAll] > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17341353#comment-17341353 ] Ignite TC Bot commented on IGNITE-8719: --- {panel:title=Branch: [pull/9090/head] Base: [master] : No blockers found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel} {panel:title=Branch: [pull/9090/head] Base: [master] : New Tests (7)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1} {color:#8b}PDS (Indexing){color} [[tests 7|https://ci.ignite.apache.org/viewLog.html?buildId=5999055]] * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeRestart - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testTwoNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testDeleteIndexRebuildStateOnDestroyCache - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testSingleNodeReactivation - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testNormalFlowIndexRebuildStateStorage - PASSED{color} * {color:#013220}IgnitePdsWithIndexingTestSuite: ResumeRebuildIndexTest.testErrorFlowIndexRebuildStateStorage - PASSED{color} {panel} [TeamCity *--> Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=5999042&buildTypeId=IgniteTests24Java8_RunAll] > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug > Components: sql >Reporter: Alexey Goncharuk >Assignee: Kirill Tkalenko >Priority: Critical > Fix For: 2.11 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17243203#comment-17243203 ] Pavel Vinokurov commented on IGNITE-8719: - The issue still could be reproduced in both cases when an index is creating or rebuilding. Considering impact of this issue I suppose it could be fixed before implementation of IEP-28. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Priority: Critical > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16943412#comment-16943412 ] Maxim Muzafarov commented on IGNITE-8719: - Folks, This is a known issue and will be solved as a part of [IEP-28: Index rebuild|[https://cwiki.apache.org/confluence/display/IGNITE/IEP-28%3A+Cluster+peer-2-peer+balancing]] index rebuild procedure. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Priority: Critical > Fix For: 2.8 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16670124#comment-16670124 ] ASF GitHub Bot commented on IGNITE-8719: GitHub user antonovsergey93 opened a pull request: https://github.com/apache/ignite/pull/5223 IGNITE-8719 Store information about starting of index rebuild. You can merge this pull request into a Git repository by running: $ git pull https://github.com/gridgain/apache-ignite ignite-8719 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/ignite/pull/5223.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #5223 commit cfbd9ba1cc7c31fbe43bdd2e00bad81c8396a464 Author: Sergey Antonov Date: 2018-10-30T09:04:33Z IGNITE-8719 Try reproduce problem. commit b0c5d2740f9aa1814b6809ac7ad11b09d5d53250 Author: Sergey Antonov Date: 2018-10-31T13:34:11Z IGNITE-8719 First version of fix. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Sergey Antonov >Priority: Critical > Fix For: 2.8 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667382#comment-16667382 ] Sergey Antonov commented on IGNITE-8719: [~agoncharuk] I wrote test by your scena [^IndexRebuildAfterNodeCrashTest.java] rio. But the problem wasn't reproduced. What I did wrong? [^IndexRebuildAfterNodeCrashTest.java] > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Sergey Antonov >Priority: Critical > Fix For: 2.8 > > Attachments: IndexRebuildAfterNodeCrashTest.java, > IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16655040#comment-16655040 ] Sergey Antonov commented on IGNITE-8719: [~cyberdemon], Looks like the issue is abandoned but looks serious. Would you mind if I start working on it? > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.8 > > Attachments: IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623655#comment-16623655 ] Dmitriy Sorokin commented on IGNITE-8719: - [~NIzhikov], A chance is exists, but if is not critical, fix version can be moved to 2.8. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.7 > > Attachments: IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623549#comment-16623549 ] Nikolay Izhikov commented on IGNITE-8719: - [~cyberdemon] Do we have a chance to resolve this ticket until the code freeze of 2.7? > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.7 > > Attachments: IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16554095#comment-16554095 ] Stanislav Lukyanov commented on IGNITE-8719: There is a similar issue with an empty index tree being persisted. I can reproduce it by shutting node down forcibly (kill -9 or taskkill /f), but I guess the key is in shutting down in the middle of checkpoint or something like that. The reproducer is attached: [^IndexRebuildingTest.java]. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.7 > > Attachments: IndexRebuildingTest.java > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16549465#comment-16549465 ] Dmitriy Sorokin commented on IGNITE-8719: - [~agoncharuk], I wrote the reproducer working by your scheme, but so far no confirmation for case when partially built index can be used. Look at my patch [#4380|https://github.com/apache/ignite/pull/4380], please, and tell me your opinion. > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.7 > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (IGNITE-8719) Index left partially built if a node crashes during index create or rebuild
[ https://issues.apache.org/jira/browse/IGNITE-8719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16549150#comment-16549150 ] Stanilovsky Evgeny commented on IGNITE-8719: also, index creation can be crashed independently, for example with Error during parallel index create/rebuild. IgniteChackedException: maximum of retries 1000 reached. at org.apache.ignite.internal.processors.cache.persistence.tree.BplusTree$Get.checkLockRetry(BPlusTree.java:2565) > Index left partially built if a node crashes during index create or rebuild > --- > > Key: IGNITE-8719 > URL: https://issues.apache.org/jira/browse/IGNITE-8719 > Project: Ignite > Issue Type: Bug >Reporter: Alexey Goncharuk >Assignee: Dmitriy Sorokin >Priority: Critical > Fix For: 2.7 > > > Currently, we do not have any state associated with the index tree. Consider > the following scenario: > 1) Start node, put some data > 2) start CREATE INDEX operation > 3) Wait for a checkpoint and stop node before index create finished > 4) Restart node > Since the checkpoint finished, the new index tree will be persisted to the > disk, but not all data will be present in the index. > We should somehow store information about initializing index tree and mark it > valid only after all data is indexed. The state should be persisted as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)