[jira] [Commented] (CASSANDRA-12374) Can't rebuild SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427802#comment-15427802
]
Pavel Yaskevich commented on CASSANDRA-12374:
-
Sounds good! Thanks for helping out with SASI! :)
> Can't rebuild SASI index
>
>
> Key: CASSANDRA-12374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
> Fix For: 3.10
>
>
> There's been no real requirement for that so far.
> As [~beobal] has pointed out, it's not a big issue, since that only could be
> needed when index files are lost, data corruption on disk (hardware issue)
> has occurred or there was a bug that'd require an index rebuild.
> During {{rebuild_index}} task, indexes are only "marked" as removed with
> {{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}}
> is called. However, since SASI keeps track of SSTables for the index, it's
> going to filter them out with {{.filter((sstable) ->
> !sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.
> If I understand the logic correctly, we have to "invalidate" (drop data)
> right before we re-index them. This is also a blocker for [CASSANDRA-11990]
> since without it we can't have an upgrade path.
> I have a patch ready in branch, but since it's a bug, it's better to have it
> released earlier and for all branches affected.
> cc [~xedin]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12374) Can't rebuild SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427757#comment-15427757
]
Alex Petrov commented on CASSANDRA-12374:
-
Thanks [~xedin] for reviewing these patches so quickly! Really appreciate it.
I'm going to rebase 11990 and submit it shortly as well.
> Can't rebuild SASI index
>
>
> Key: CASSANDRA-12374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
> Fix For: 3.10
>
>
> There's been no real requirement for that so far.
> As [~beobal] has pointed out, it's not a big issue, since that only could be
> needed when index files are lost, data corruption on disk (hardware issue)
> has occurred or there was a bug that'd require an index rebuild.
> During {{rebuild_index}} task, indexes are only "marked" as removed with
> {{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}}
> is called. However, since SASI keeps track of SSTables for the index, it's
> going to filter them out with {{.filter((sstable) ->
> !sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.
> If I understand the logic correctly, we have to "invalidate" (drop data)
> right before we re-index them. This is also a blocker for [CASSANDRA-11990]
> since without it we can't have an upgrade path.
> I have a patch ready in branch, but since it's a bug, it's better to have it
> released earlier and for all branches affected.
> cc [~xedin]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12374) Can't rebuild SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427267#comment-15427267
]
Pavel Yaskevich commented on CASSANDRA-12374:
-
Test looks good now, I'm going to wait for CI to finish and commit everything,
thanks, [~ifesdjeen]!
> Can't rebuild SASI index
>
>
> Key: CASSANDRA-12374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>
> There's been no real requirement for that so far.
> As [~beobal] has pointed out, it's not a big issue, since that only could be
> needed when index files are lost, data corruption on disk (hardware issue)
> has occurred or there was a bug that'd require an index rebuild.
> During {{rebuild_index}} task, indexes are only "marked" as removed with
> {{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}}
> is called. However, since SASI keeps track of SSTables for the index, it's
> going to filter them out with {{.filter((sstable) ->
> !sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.
> If I understand the logic correctly, we have to "invalidate" (drop data)
> right before we re-index them. This is also a blocker for [CASSANDRA-11990]
> since without it we can't have an upgrade path.
> I have a patch ready in branch, but since it's a bug, it's better to have it
> released earlier and for all branches affected.
> cc [~xedin]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12374) Can't rebuild SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427265#comment-15427265
]
Pavel Yaskevich commented on CASSANDRA-12374:
-
bq. currently unqueriable indexes / are skipped.
Yes, that was intentional, because there is (and probably still isn't) a good
way to propagate exceptions in a meaningful way, so we've opted out to writing
to the log and returning 0 results instead.
> Can't rebuild SASI index
>
>
> Key: CASSANDRA-12374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>
> There's been no real requirement for that so far.
> As [~beobal] has pointed out, it's not a big issue, since that only could be
> needed when index files are lost, data corruption on disk (hardware issue)
> has occurred or there was a bug that'd require an index rebuild.
> During {{rebuild_index}} task, indexes are only "marked" as removed with
> {{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}}
> is called. However, since SASI keeps track of SSTables for the index, it's
> going to filter them out with {{.filter((sstable) ->
> !sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.
> If I understand the logic correctly, we have to "invalidate" (drop data)
> right before we re-index them. This is also a blocker for [CASSANDRA-11990]
> since without it we can't have an upgrade path.
> I have a patch ready in branch, but since it's a bug, it's better to have it
> released earlier and for all branches affected.
> cc [~xedin]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12374) Can't rebuild SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427175#comment-15427175
]
Alex Petrov commented on CASSANDRA-12374:
-
You're right, sleep would be not reliable there. I've changed test to just
write garbage to file and make sure that it gets overwritten plus index is
query-able after rebuild. So in fact the test without sleep turned out to be
much more useful.
I've triggered CI, will check results tomorrow morning, but local runs were
okay.
One question: currently unqueriable indexes / are skipped. For example, if
there's just one sstable, and it's index is corrupted, it's going to get
skipped and query will yield no results rather than an exception. That might be
desired though, just checking.
> Can't rebuild SASI index
>
>
> Key: CASSANDRA-12374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>
> There's been no real requirement for that so far.
> As [~beobal] has pointed out, it's not a big issue, since that only could be
> needed when index files are lost, data corruption on disk (hardware issue)
> has occurred or there was a bug that'd require an index rebuild.
> During {{rebuild_index}} task, indexes are only "marked" as removed with
> {{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}}
> is called. However, since SASI keeps track of SSTables for the index, it's
> going to filter them out with {{.filter((sstable) ->
> !sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.
> If I understand the logic correctly, we have to "invalidate" (drop data)
> right before we re-index them. This is also a blocker for [CASSANDRA-11990]
> since without it we can't have an upgrade path.
> I have a patch ready in branch, but since it's a bug, it's better to have it
> released earlier and for all branches affected.
> cc [~xedin]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12374) Can't rebuild SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427092#comment-15427092
]
Pavel Yaskevich commented on CASSANDRA-12374:
-
Is there any way to get rid of the Thread.sleep there? New test consistently
fails on my machine now...
> Can't rebuild SASI index
>
>
> Key: CASSANDRA-12374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>
> There's been no real requirement for that so far.
> As [~beobal] has pointed out, it's not a big issue, since that only could be
> needed when index files are lost, data corruption on disk (hardware issue)
> has occurred or there was a bug that'd require an index rebuild.
> During {{rebuild_index}} task, indexes are only "marked" as removed with
> {{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}}
> is called. However, since SASI keeps track of SSTables for the index, it's
> going to filter them out with {{.filter((sstable) ->
> !sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.
> If I understand the logic correctly, we have to "invalidate" (drop data)
> right before we re-index them. This is also a blocker for [CASSANDRA-11990]
> since without it we can't have an upgrade path.
> I have a patch ready in branch, but since it's a bug, it's better to have it
> released earlier and for all branches affected.
> cc [~xedin]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12374) Can't rebuild SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15426258#comment-15426258
]
Alex Petrov commented on CASSANDRA-12374:
-
Yes, sorry. I have only ran this test individually so overlooked it'd fail in
combination with other tests. Fixed, rebased and force-pushed, CI results are
good.
> Can't rebuild SASI index
>
>
> Key: CASSANDRA-12374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>
> There's been no real requirement for that so far.
> As [~beobal] has pointed out, it's not a big issue, since that only could be
> needed when index files are lost, data corruption on disk (hardware issue)
> has occurred or there was a bug that'd require an index rebuild.
> During {{rebuild_index}} task, indexes are only "marked" as removed with
> {{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}}
> is called. However, since SASI keeps track of SSTables for the index, it's
> going to filter them out with {{.filter((sstable) ->
> !sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.
> If I understand the logic correctly, we have to "invalidate" (drop data)
> right before we re-index them. This is also a blocker for [CASSANDRA-11990]
> since without it we can't have an upgrade path.
> I have a patch ready in branch, but since it's a bug, it's better to have it
> released earlier and for all branches affected.
> cc [~xedin]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12374) Can't rebuild SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15425255#comment-15425255
]
Pavel Yaskevich commented on CASSANDRA-12374:
-
Looks like the tests for rebuild are still broken with the latest changes, but
the changes look good, let me know when you fix the test, so I can commit this.
> Can't rebuild SASI index
>
>
> Key: CASSANDRA-12374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>
> There's been no real requirement for that so far.
> As [~beobal] has pointed out, it's not a big issue, since that only could be
> needed when index files are lost, data corruption on disk (hardware issue)
> has occurred or there was a bug that'd require an index rebuild.
> During {{rebuild_index}} task, indexes are only "marked" as removed with
> {{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}}
> is called. However, since SASI keeps track of SSTables for the index, it's
> going to filter them out with {{.filter((sstable) ->
> !sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.
> If I understand the logic correctly, we have to "invalidate" (drop data)
> right before we re-index them. This is also a blocker for [CASSANDRA-11990]
> since without it we can't have an upgrade path.
> I have a patch ready in branch, but since it's a bug, it's better to have it
> released earlier and for all branches affected.
> cc [~xedin]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12374) Can't rebuild SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-12374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15406928#comment-15406928
]
Pavel Yaskevich commented on CASSANDRA-12374:
-
This is definitely a porting bug, I'll be happy to review your changes,
[~ifesdjeen].
> Can't rebuild SASI index
>
>
> Key: CASSANDRA-12374
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12374
> Project: Cassandra
> Issue Type: Bug
>Reporter: Alex Petrov
>
> There's been no real requirement for that so far.
> As [~beobal] has pointed out, it's not a big issue, since that only could be
> needed when index files are lost, data corruption on disk (hardware issue)
> has occurred or there was a bug that'd require an index rebuild.
> During {{rebuild_index}} task, indexes are only "marked" as removed with
> {{SecondaryIndexManager::markIndexRemoved}} and then {{buildIndexesBlocking}}
> is called. However, since SASI keeps track of SSTables for the index, it's
> going to filter them out with {{.filter((sstable) ->
> !sasi.index.hasSSTable(sstable))}} in {{SASIIndexBuildingSupport}}.
> If I understand the logic correctly, we have to "invalidate" (drop data)
> right before we re-index them. This is also a blocker for [CASSANDRA-11990]
> since without it we can't have an upgrade path.
> I have a patch ready in branch, but since it's a bug, it's better to have it
> released earlier and for all branches affected.
> cc [~xedin]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
