[jira] [Comment Edited] (CASSANDRA-14365) Commit log replay failure for static columns with collections in clustering keys
[ https://issues.apache.org/jira/browse/CASSANDRA-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984658#comment-16984658 ] Michael Semb Wever edited comment on CASSANDRA-14365 at 3/9/20, 2:33 PM: - With new tests… (test against trunk also needed a rewrite bc {{`TableMetadata.Builder`}} ||branch||circleci||jenkins pipeline|| |[cassandra_2.2_14365|https://github.com/apache/cassandra/compare/cassandra-2.2...thelastpickle:mck/cassandra-2.2_14365]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-2.2_14365]|[!https://builds.apache.org/job/Cassandra-devbranch/40/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/40]| |[cassandra_3.0_14365|https://github.com/apache/cassandra/compare/cassandra-3.0...thelastpickle:mck/cassandra-3.0_14365]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.0_14365]|[!https://builds.apache.org/job/Cassandra-devbranch/41/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/41]| |[cassandra_3.11_14365|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_14365]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.11_14365]|[!https://builds.apache.org/job/Cassandra-devbranch/42/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/42]| |[trunk_14365|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_14365]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Ftrunk_14365]|[!https://builds.apache.org/job/Cassandra-devbranch/43/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/43]| was (Author: michaelsembwever): With new tests… (test against trunk also needed a rewrite bc {{`TableMetadata.Builder`}} ||branch||circleci||jenkins pipeline|| |[cassandra_3.0_14365|https://github.com/apache/cassandra/compare/cassandra-3.0...thelastpickle:mck/cassandra-3.0_14365]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.0_14365]|[!https://builds.apache.org/job/Cassandra-devbranch/41/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/41]| |[cassandra_3.11_14365|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_14365]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.11_14365]|[!https://builds.apache.org/job/Cassandra-devbranch/42/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/42]| |[trunk_14365|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_14365]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Ftrunk_14365]|[!https://builds.apache.org/job/Cassandra-devbranch/43/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/43]| > Commit log replay failure for static columns with collections in clustering > keys > > > Key: CASSANDRA-14365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14365 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Vincent White >Assignee: Vincent White >Priority: Normal > > In the old storage engine, static cells with a collection as part of the > clustering key fail to validate because a 0 byte collection (like in the cell > name of a static cell) isn't valid. > To reproduce: > 1. > {code:java} > CREATE TABLE test.x ( > id int, > id2 frozen>, > st int static, > PRIMARY KEY (id, id2) > ); > INSERT INTO test.x (id, st) VALUES (1, 2); > {code} > 2. > Kill the cassandra process > 3. > Restart cassandra to replay the commitlog > Outcome: > {noformat} > ERROR [main] 2018-04-05 04:58:23,741 JVMStabilityInspector.java:99 - Exiting > due to error while processing commit log during initialization. > org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: > Unexpected error deserializing mutation; saved to > /tmp/mutation3825739904516830950dat. This may be caused by replaying a > mutation against a table with the same name but incompatible schema. > Exception follows: org.apache.cassandra.serializers.MarshalException: Not > enough bytes to read a set > at > org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:638) > [main/:na] > at >
[jira] [Comment Edited] (CASSANDRA-14365) Commit log replay failure for static columns with collections in clustering keys
[ https://issues.apache.org/jira/browse/CASSANDRA-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984658#comment-16984658 ] Michael Semb Wever edited comment on CASSANDRA-14365 at 3/9/20, 2:32 PM: - With new tests… (test against trunk also needed a rewrite bc {{`TableMetadata.Builder`}} ||branch||circleci||jenkins pipeline|| |[cassandra_3.0_14365|https://github.com/apache/cassandra/compare/cassandra-3.0...thelastpickle:mck/cassandra-3.0_14365]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.0_14365]|[!https://builds.apache.org/job/Cassandra-devbranch/41/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/41]| |[cassandra_3.11_14365|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_14365]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.11_14365]|[!https://builds.apache.org/job/Cassandra-devbranch/42/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/42]| |[trunk_14365|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_14365]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Ftrunk_14365]|[!https://builds.apache.org/job/Cassandra-devbranch/43/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/43]| was (Author: michaelsembwever): With new tests… (test against trunk also needed a rewrite bc {{`TableMetadata.Builder`}} ||branch||circleci||asf jenkins tests||asf jenkins dtests|| |[cassandra-2.2_14365|https://github.com/apache/cassandra/compare/cassandra-2.2...thelastpickle:mck/cassandra-2.2_14365]|[circleci|https://circleci.com/workflow-run/d500cc5f-1d87-4beb-815e-9931f8e84d95]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/29//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/29/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/703//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/703]| |[cassandra-3.0_14365|https://github.com/apache/cassandra/compare/cassandra-3.0...thelastpickle:mck/cassandra-3.0_14365]|[circleci|https://circleci.com/workflow-run/747730de-573a-4e80-98f0-4defa14db909]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/33//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/33/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/706//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/706]| |[cassandra-3.11_14365|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_14365]|[circleci|https://circleci.com/workflow-run/86ca8a61-5cc2-40db-84a4-1210cf44f285]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/34//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/34/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/707//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/707]| |[trunk_14365|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_14365]|[circleci|https://circleci.com/workflow-run/a034a6b1-a7d7-43cd-b1ab-14769799b30e]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/35//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/35/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/707//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/707]| > Commit log replay failure for static columns with collections in clustering > keys > > > Key: CASSANDRA-14365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14365 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Vincent White >Assignee: Vincent White >Priority: Normal > > In the old storage engine, static cells with a collection as part of the > clustering key fail to validate because a 0 byte collection (like in the cell > name of a static cell) isn't valid. > To reproduce: > 1. > {code:java} > CREATE TABLE test.x ( > id int, > id2 frozen>, > st int static, > PRIMARY KEY (id, id2) > ); > INSERT INTO test.x (id, st) VALUES (1, 2); > {code} > 2. > Kill the cassandra process > 3. >
[jira] [Comment Edited] (CASSANDRA-14365) Commit log replay failure for static columns with collections in clustering keys
[ https://issues.apache.org/jira/browse/CASSANDRA-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975056#comment-16975056 ] Michael Semb Wever edited comment on CASSANDRA-14365 at 11/15/19 1:24 PM: -- ||branch||circleci||asf jenkins tests||asf jenkins dtests|| |[cassandra-2.2_14365|https://github.com/apache/cassandra/compare/trunk...vincewhite:zero_length_collection]|[circleci|https://circleci.com/workflow-run/d500cc5f-1d87-4beb-815e-9931f8e84d95]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/29//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/29/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/703//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/703]| [~VincentWhite], can you update the unit test on a cassandra-3.0 based branch as well… you see can [here|https://github.com/thelastpickle/cassandra/commit/af2ca79b1001150d39a6b97cfadc0ce97f4e99b4] the api the tests use have changed a bit. was (Author: michaelsembwever): ||branch||circleci||asf jenkins tests||asf jenkins dtests|| |[cassandra-2.2_14365|https://github.com/apache/cassandra/compare/trunk...vincewhite:zero_length_collection]|[circleci|https://circleci.com/workflow-run/d500cc5f-1d87-4beb-815e-9931f8e84d95]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/29//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-pipeline/29/]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/703//badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/703]| | wip… | | | | | wip… | | | | | wip… | | | | > Commit log replay failure for static columns with collections in clustering > keys > > > Key: CASSANDRA-14365 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14365 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Vincent White >Assignee: Vincent White >Priority: Normal > > In the old storage engine, static cells with a collection as part of the > clustering key fail to validate because a 0 byte collection (like in the cell > name of a static cell) isn't valid. > To reproduce: > 1. > {code:java} > CREATE TABLE test.x ( > id int, > id2 frozen>, > st int static, > PRIMARY KEY (id, id2) > ); > INSERT INTO test.x (id, st) VALUES (1, 2); > {code} > 2. > Kill the cassandra process > 3. > Restart cassandra to replay the commitlog > Outcome: > {noformat} > ERROR [main] 2018-04-05 04:58:23,741 JVMStabilityInspector.java:99 - Exiting > due to error while processing commit log during initialization. > org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: > Unexpected error deserializing mutation; saved to > /tmp/mutation3825739904516830950dat. This may be caused by replaying a > mutation against a table with the same name but incompatible schema. > Exception follows: org.apache.cassandra.serializers.MarshalException: Not > enough bytes to read a set > at > org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:638) > [main/:na] > at > org.apache.cassandra.db.commitlog.CommitLogReplayer.replayMutation(CommitLogReplayer.java:565) > [main/:na] > at > org.apache.cassandra.db.commitlog.CommitLogReplayer.replaySyncSection(CommitLogReplayer.java:517) > [main/:na] > at > org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:397) > [main/:na] > at > org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:143) > [main/:na] > at > org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:181) > [main/:na] > at > org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:161) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:284) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:533) > [main/:na] > at > org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:642) > [main/:na] > {noformat} > I haven't investigated if there are other more subtle issues caused by these > cells failing to validate other places in the code, but I believe the fix for > this is to check for 0 byte length collections and accept them as valid as we > do with other types. > I haven't had a chance for any extensive testing but this naive patch seems > to have the desired affect. > ||Patch|| > |[2.2 >