[jira] [Commented] (CASSANDRA-16969) Scrub does not detect invalid partition keys

2021-09-27 Thread Berenguer Blasi (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420716#comment-17420716
 ] 

Berenguer Blasi commented on CASSANDRA-16969:
-

Jenkins lgtm so +1 from me.

> Scrub does not detect invalid partition keys
> 
>
> Key: CASSANDRA-16969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16969
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.12, 4.0.2, 4.1
>
>
> The standalone scrubber [gets the 
> key|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/Scrubber.java#L202]
>  from the file but never validates it, and this will propagate to the new 
> sstable so it will be corrupted when read later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16969) Scrub does not detect invalid partition keys

2021-09-27 Thread Berenguer Blasi (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420495#comment-17420495
 ] 

Berenguer Blasi commented on CASSANDRA-16969:
-

I fired a 
[jenkins|https://ci-cassandra.apache.org/job/Cassandra-devbranch/1147/] one to 
be on the safe side as I've seen sometimes differences across the diff CIs and 
I have a suspicion it might be the case.

> Scrub does not detect invalid partition keys
> 
>
> Key: CASSANDRA-16969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16969
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.12, 4.0.2, 4.1
>
>
> The standalone scrubber [gets the 
> key|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/Scrubber.java#L202]
>  from the file but never validates it, and this will propagate to the new 
> sstable so it will be corrupted when read later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16969) Scrub does not detect invalid partition keys

2021-09-24 Thread Brandon Williams (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419751#comment-17419751
 ] 

Brandon Williams commented on CASSANDRA-16969:
--

The circle links 
[above|https://app.circleci.com/pipelines/github/driftx/cassandra?branch=CASSANDRA-16969]
 updated.

> Scrub does not detect invalid partition keys
> 
>
> Key: CASSANDRA-16969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16969
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.12, 4.0.2, 4.1
>
>
> The standalone scrubber [gets the 
> key|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/Scrubber.java#L202]
>  from the file but never validates it, and this will propagate to the new 
> sstable so it will be corrupted when read later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16969) Scrub does not detect invalid partition keys

2021-09-24 Thread Berenguer Blasi (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419615#comment-17419615
 ] 

Berenguer Blasi commented on CASSANDRA-16969:
-

Are there any jenkins CI links to see if this helps clear CI failures?

> Scrub does not detect invalid partition keys
> 
>
> Key: CASSANDRA-16969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16969
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.12, 4.0.2, 4.1
>
>
> The standalone scrubber [gets the 
> key|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/Scrubber.java#L202]
>  from the file but never validates it, and this will propagate to the new 
> sstable so it will be corrupted when read later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16969) Scrub does not detect invalid partition keys

2021-09-23 Thread Brandon Williams (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419150#comment-17419150
 ] 

Brandon Williams commented on CASSANDRA-16969:
--

Yes, this broke that :(

I've pushed updates to my branches that skip key validation for indexes; I'm 
not sure how to validate those, and it seems a little disingenuous of the api 
to give me the parent's validator without any indication.  We can go with this 
for now and tackle indexes later and still be better off than before.

> Scrub does not detect invalid partition keys
> 
>
> Key: CASSANDRA-16969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16969
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.12, 4.0.2, 4.1
>
>
> The standalone scrubber [gets the 
> key|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/Scrubber.java#L202]
>  from the file but never validates it, and this will propagate to the new 
> sstable so it will be corrupted when read later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16969) Scrub does not detect invalid partition keys

2021-09-23 Thread Benjamin Lerer (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419003#comment-17419003
 ] 

Benjamin Lerer commented on CASSANDRA-16969:


I have seen following failure of {{test_scrub_collections_table - 
scrub_test.TestScrubIndexes}} from {{scrub_test.py}} and wonder if it is not 
related to this patch.

{code}
test teardown failure
Unexpected error found in node logs (see stdout for full details). Errors: 
[ERROR [CompactionExecutor:1] 2021-09-22 17:50:31,020 CassandraDaemon.java:581 
- Exception in thread Thread[CompactionExecutor:1,1,main]
java.io.IOError: java.lang.IllegalArgumentException
at 
org.apache.cassandra.db.compaction.Scrubber.throwIfCannotContinue(Scrubber.java:449)
at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:272)
at 
org.apache.cassandra.db.compaction.CompactionManager.scrubOne(CompactionManager.java:1159)
at 
org.apache.cassandra.db.compaction.CompactionManager$3.execute(CompactionManager.java:459)
at 
org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:379)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at 
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: null
at java.nio.Buffer.limit(Buffer.java:275)
at 
org.apache.cassandra.db.marshal.ByteBufferAccessor.slice(ByteBufferAccessor.java:107)
at 
org.apache.cassandra.db.marshal.ByteBufferAccessor.slice(ByteBufferAccessor.java:39)
at 
org.apache.cassandra.serializers.CollectionSerializer.readValue(CollectionSerializer.java:127)
at 
org.apache.cassandra.serializers.ListSerializer.validateForNativeProtocol(ListSerializer.java:76)
at 
org.apache.cassandra.serializers.CollectionSerializer.validate(CollectionSerializer.java:66)
at 
org.apache.cassandra.db.marshal.AbstractType.validate(AbstractType.java:193)
at 
org.apache.cassandra.db.marshal.AbstractType.validate(AbstractType.java:188)
at org.apache.cassandra.db.compaction.Scrubber.scrub(Scrubber.java:263)
... 8 common frames omitted, WARN  [RMI TCP Connection(8)-127.0.0.1] 
2021-09-22 17:50:31,037 ColumnFamilyStore.java:1560 - Rebuilding index for 
users.user_uuids_idx because of 
{code}

> Scrub does not detect invalid partition keys
> 
>
> Key: CASSANDRA-16969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16969
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.12, 4.0.2, 4.1
>
>
> The standalone scrubber [gets the 
> key|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/Scrubber.java#L202]
>  from the file but never validates it, and this will propagate to the new 
> sstable so it will be corrupted when read later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16969) Scrub does not detect invalid partition keys

2021-09-22 Thread Benjamin Lerer (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17418468#comment-17418468
 ] 

Benjamin Lerer commented on CASSANDRA-16969:


+1

> Scrub does not detect invalid partition keys
> 
>
> Key: CASSANDRA-16969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16969
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>
> The standalone scrubber [gets the 
> key|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/Scrubber.java#L202]
>  from the file but never validates it, and this will propagate to the new 
> sstable so it will be corrupted when read later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16969) Scrub does not detect invalid partition keys

2021-09-20 Thread Brandon Williams (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17417758#comment-17417758
 ] 

Brandon Williams commented on CASSANDRA-16969:
--

||Branch||CI||
|[3.11|https://github.com/driftx/cassandra/tree/CASSANDRA-16969]|[circle|https://app.circleci.com/pipelines/github/driftx/cassandra?branch=CASSANDRA-16969],
 
[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/1126/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1126/pipeline]|
|[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-16969-4.0]|[circle|https://app.circleci.com/pipelines/github/driftx/cassandra?branch=CASSANDRA-16969-4.0],
 
[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/1127/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1127/pipeline]|
|[trunk|https://github.com/driftx/cassandra/tree/CASSANDRA-16969-trunk]|[circle|https://app.circleci.com/pipelines/github/driftx/cassandra?branch=CASSANDRA-16969-trunk],
 
[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/1128/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1128/pipeline]|

> Scrub does not detect invalid partition keys
> 
>
> Key: CASSANDRA-16969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16969
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>
> The standalone scrubber [gets the 
> key|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/Scrubber.java#L202]
>  from the file but never validates it, and this will propagate to the new 
> sstable so it will be corrupted when read later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16969) Scrub does not detect invalid partition keys

2021-09-17 Thread Brandon Williams (Jira) 


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416946#comment-17416946
 ] 

Brandon Williams commented on CASSANDRA-16969:
--

For now, [here|https://github.com/driftx/cassandra/tree/CASSANDRA-16969] is a 
branch against 3.11.


> Scrub does not detect invalid partition keys
> 
>
> Key: CASSANDRA-16969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16969
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/sstable
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>
> The standalone scrubber [gets the 
> key|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/compaction/Scrubber.java#L202]
>  from the file but never validates it, and this will propagate to the new 
> sstable so it will be corrupted when read later.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org