[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[ https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376996#comment-14376996 ] Stefania commented on CASSANDRA-5174: - Thank you for your review and help! > expose nodetool scrub for 2Is > - > > Key: CASSANDRA-5174 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5174 > Project: Cassandra > Issue Type: Task > Components: Core, Tools >Reporter: Jason Brown >Assignee: Stefania >Priority: Minor > Fix For: 3.0 > > > Continuation of CASSANDRA-4464, where many other nodetool operations were > added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due > to the riskiness of deleting data on a bad bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[
https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14375269#comment-14375269
]
Stefania commented on CASSANDRA-5174:
-
I reverted the {{SecondaryIndex}} interface and moved {{loadsstables}} to the
base cfs data tracker. I figured that would be the best place to store this
flag.
I resorted to reflection in {{ScrubTest.java}} to change the values of
{{keyComparator}}, so that it can stay as a public final field. I did not want
to leave a public field that was not final.
https://github.com/apache/cassandra/commit/d3e0afc93b83fcd324d3944c5bccecf6dd7c7608
I also rebased and force pushed to my github branch. Let me know if I should
squash the commits as well.
> expose nodetool scrub for 2Is
> -
>
> Key: CASSANDRA-5174
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5174
> Project: Cassandra
> Issue Type: Task
> Components: Core, Tools
>Reporter: Jason Brown
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.0
>
>
> Continuation of CASSANDRA-4464, where many other nodetool operations were
> added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due
> to the riskiness of deleting data on a bad bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[
https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14371500#comment-14371500
]
Yuki Morishita commented on CASSANDRA-5174:
---
bq. since I know people can implement custom indexes, are the changes I made to
SecondaryIndex going to cause issues or is this expected in a major release?
Good point. I'd say it would be OK, though in this case, we can achive the goal
without changing {{SecondaryIndex}} API.
How about adding {{loadSSTable}} boolean property to {{SecondaryIndex}}, call
setter from {{SecondaryIndexManager#addIndexedColumn}} before {{#init()}}, use
that property in subclasses as they wish?
Also, leave static {{keyComparator}} as public for awhile.
> expose nodetool scrub for 2Is
> -
>
> Key: CASSANDRA-5174
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5174
> Project: Cassandra
> Issue Type: Task
> Components: Core, Tools
>Reporter: Jason Brown
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.0
>
>
> Continuation of CASSANDRA-4464, where many other nodetool operations were
> added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due
> to the riskiness of deleting data on a bad bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[
https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14370475#comment-14370475
]
Stefania commented on CASSANDRA-5174:
-
[~yukim] thank you for your review!
Here is the latest commit:
https://github.com/apache/cassandra/commit/b018430c17cf15915f9b2c2d4dd4e8689cf058b8
bq. I think we should log only when we are rebuilding index, and change log
level to WARN, since default exception handler will log in ERROR anyway.
Done, however I wanted this message to be logged before or together with the
"Rebuilding index..." message so I passed the {{throwable}} as an argument to
{{rebuildOnFailedScrub()}}, see if that's OK.
bq. Why don't you handle IllegalArgumentException in its own catch block?
Good point, done.
bq. dtest also looks good except coding styles (docstring, string format) but I
let QA team take a look when you do pull request.
This is the pull request, I explicitly asked them to pick up style issues:
https://github.com/riptano/cassandra-dtest/pull/200
I have one more question, since I know people can implement custom indexes, are
the changes I made to {{SecondaryIndex}} going to cause issues or is this
expected in a major release?
> expose nodetool scrub for 2Is
> -
>
> Key: CASSANDRA-5174
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5174
> Project: Cassandra
> Issue Type: Task
> Components: Core, Tools
>Reporter: Jason Brown
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.0
>
>
> Continuation of CASSANDRA-4464, where many other nodetool operations were
> added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due
> to the riskiness of deleting data on a bad bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[ https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14369929#comment-14369929 ] Yuki Morishita commented on CASSANDRA-5174: --- * I think we should log only when we are rebuilding index, and change log level to WARN, since default exception handler will log in ERROR anyway. (https://github.com/stef1927/cassandra/blob/5174/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1403) * Why don't you handle IllegalArgumentException in its own catch block? (https://github.com/stef1927/cassandra/blob/5174/src/java/org/apache/cassandra/tools/NodeTool.java#L1285) Otherwise, +1. dtest also looks good except coding styles (docstring, string format) but I let QA team take a look when you do pull request. > expose nodetool scrub for 2Is > - > > Key: CASSANDRA-5174 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5174 > Project: Cassandra > Issue Type: Task > Components: Core, Tools >Reporter: Jason Brown >Assignee: Stefania >Priority: Minor > Fix For: 3.0 > > > Continuation of CASSANDRA-4464, where many other nodetool operations were > added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due > to the riskiness of deleting data on a bad bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[ https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368422#comment-14368422 ] Jonathan Ellis commented on CASSANDRA-5174: --- Let's defer global index scrub to 6477 or a follow up ticket. > expose nodetool scrub for 2Is > - > > Key: CASSANDRA-5174 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5174 > Project: Cassandra > Issue Type: Task > Components: Core, Tools >Reporter: Jason Brown >Assignee: Stefania >Priority: Minor > Fix For: 3.0 > > > Continuation of CASSANDRA-4464, where many other nodetool operations were > added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due > to the riskiness of deleting data on a bad bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[ https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368385#comment-14368385 ] Stefania commented on CASSANDRA-5174: - [~yukim] the patch is ready for review: https://github.com/stef1927/cassandra/tree/5174 The dtests are here: https://github.com/stef1927/cassandra-dtest/tree/5174 [~jbellis] this patch is only for scrubbing local indexes, do you want to add support for CASSANDRA-6477 in this ticket as well? > expose nodetool scrub for 2Is > - > > Key: CASSANDRA-5174 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5174 > Project: Cassandra > Issue Type: Task > Components: Core, Tools >Reporter: Jason Brown >Assignee: Stefania >Priority: Minor > Fix For: 3.0 > > > Continuation of CASSANDRA-4464, where many other nodetool operations were > added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due > to the riskiness of deleting data on a bad bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[ https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362980#comment-14362980 ] Stefania commented on CASSANDRA-5174: - Thanks, still work in progress but the code is here: https://github.com/stef1927/cassandra/tree/5174. > expose nodetool scrub for 2Is > - > > Key: CASSANDRA-5174 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5174 > Project: Cassandra > Issue Type: Task > Components: Core, Tools >Reporter: Jason Brown >Assignee: Stefania >Priority: Minor > Fix For: 3.0 > > > Continuation of CASSANDRA-4464, where many other nodetool operations were > added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due > to the riskiness of deleting data on a bad bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[ https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358836#comment-14358836 ] Jonathan Ellis commented on CASSANDRA-5174: --- # I think we want to make those separate operations. The most common use case for scrub is "I saw a query error out because of a corrupt sstable" and I think we want scrub to touch only the sstables necessary to fix that. # skipCorrupted only applies to counters, which can't be indexed, so this shouldn't be an issue # Follow existing practice (I believe scrub and rebuild_index both block) # For local indexes, we should support standalone mode > expose nodetool scrub for 2Is > - > > Key: CASSANDRA-5174 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5174 > Project: Cassandra > Issue Type: Task > Components: Core, Tools >Reporter: Jason Brown >Assignee: Stefania >Priority: Minor > Fix For: 3.0 > > > Continuation of CASSANDRA-4464, where many other nodetool operations were > added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due > to the riskiness of deleting data on a bad bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[ https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14358410#comment-14358410 ] Stefania commented on CASSANDRA-5174: - Hello [~jbellis], a few questions: \\ # When scrubbing a non-index cf do we want to automatically scrub the associated secondary index cfs? # When we detect corruption whilst scrubbing an index cf, should we abort and rebuild the index regardless of the skipCorrupted parameter? # Do we block the nodetool client until the index is rebuilt in case of failure or do we let it run asynchronously? # Do we need to support the standalone scrubber as well? \\ \\ > expose nodetool scrub for 2Is > - > > Key: CASSANDRA-5174 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5174 > Project: Cassandra > Issue Type: Task > Components: Core, Tools >Reporter: Jason Brown >Assignee: Stefania >Priority: Minor > Fix For: 3.0 > > > Continuation of CASSANDRA-4464, where many other nodetool operations were > added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due > to the riskiness of deleting data on a bad bug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-5174) expose nodetool scrub for 2Is
[ https://issues.apache.org/jira/browse/CASSANDRA-5174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13827002#comment-13827002 ] Jonathan Ellis commented on CASSANDRA-5174: --- bq. This ticket supports scrub fo 2Is and is in its own ticket due to the riskiness of deleting data on a bad bug. Not sure what you meant here -- is there a reason scrub against 2i is more dangerous than against the base table? > expose nodetool scrub for 2Is > - > > Key: CASSANDRA-5174 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5174 > Project: Cassandra > Issue Type: Task > Components: Core, Tools >Reporter: Jason Brown >Assignee: Jason Brown >Priority: Minor > Fix For: 2.0.3 > > > Continuation of CASSANDRA-4464, where many other nodetool operations were > added for 2Is. This ticket supports scrub fo 2Is and is in its own ticket due > to the riskiness of deleting data on a bad bug. -- This message was sent by Atlassian JIRA (v6.1#6144)
