[jira] [Created] (CASSANDRA-14929) twcs sstables gets merged following node removal
Gil Ganz created CASSANDRA-14929: Summary: twcs sstables gets merged following node removal Key: CASSANDRA-14929 URL: https://issues.apache.org/jira/browse/CASSANDRA-14929 Project: Cassandra Issue Type: Bug Components: Compaction Environment: cassandra 3.0.17 Reporter: Gil Ganz after removing a node from the cluster, a table that is defined as twcs, has sstables from different time windows merged together, making old and new data sit in the same sstable. CREATE KEYSPACE gil_test WITH replication = \{'class': 'NetworkTopologyStrategy', 'DC1': '2'} AND durable_writes = true; CREATE TABLE gil_test.my_test ( id int, creation_time timestamp, name text, PRIMARY KEY (id, creation_time) ) WITH CLUSTERING ORDER BY (creation_time ASC) AND bloom_filter_fp_chance = 0.01 AND caching = \{'keys': 'ALL', 'rows_per_partition': 'NONE'} AND comment = '' AND compaction = \{'compaction_window_unit': 'HOURS', 'compaction_window_size': '2', 'class': 'TimeWindowCompactionStrategy'} AND compression = \{'chunk_length_kb': '4', 'sstable_compression': 'org.apache.cassandra.io.compress.LZ4Compressor'} AND dclocal_read_repair_chance = 0.0 AND default_time_to_live = 0 AND gc_grace_seconds = 3600 AND max_index_interval = 2048 AND memtable_flush_period_in_ms = 0 AND min_index_interval = 128 AND read_repair_chance = 0.0 AND speculative_retry = 'NONE'; 3 nodes cluster before removing node number 3 - directory listing drwxr-xr-x 2 cassandra cassandra 4096 Dec 10 20:28 backups -rw-r--r-- 1 cassandra cassandra 51 Dec 10 22:10 mc-16-big-CompressionInfo.db -rw-r--r-- 1 cassandra cassandra 2044 Dec 10 22:10 mc-16-big-Data.db -rw-r--r-- 1 cassandra cassandra 9 Dec 10 22:10 mc-16-big-Digest.crc32 -rw-r--r-- 1 cassandra cassandra 64 Dec 10 22:10 mc-16-big-Filter.db -rw-r--r-- 1 cassandra cassandra 375 Dec 10 22:10 mc-16-big-Index.db -rw-r--r-- 1 cassandra cassandra 4805 Dec 10 22:10 mc-16-big-Statistics.db -rw-r--r-- 1 cassandra cassandra 56 Dec 10 22:10 mc-16-big-Summary.db -rw-r--r-- 1 cassandra cassandra 92 Dec 10 22:10 mc-16-big-TOC.txt -rw-r--r-- 1 cassandra cassandra 51 Dec 11 00:00 mc-31-big-CompressionInfo.db -rw-r--r-- 1 cassandra cassandra 2343 Dec 11 00:00 mc-31-big-Data.db -rw-r--r-- 1 cassandra cassandra 10 Dec 11 00:00 mc-31-big-Digest.crc32 -rw-r--r-- 1 cassandra cassandra 72 Dec 11 00:00 mc-31-big-Filter.db -rw-r--r-- 1 cassandra cassandra 411 Dec 11 00:00 mc-31-big-Index.db -rw-r--r-- 1 cassandra cassandra 4816 Dec 11 00:00 mc-31-big-Statistics.db -rw-r--r-- 1 cassandra cassandra 56 Dec 11 00:00 mc-31-big-Summary.db -rw-r--r-- 1 cassandra cassandra 92 Dec 11 00:00 mc-31-big-TOC.txt -rw-r--r-- 1 cassandra cassandra 51 Dec 11 02:00 mc-47-big-CompressionInfo.db -rw-r--r-- 1 cassandra cassandra 2732 Dec 11 02:00 mc-47-big-Data.db -rw-r--r-- 1 cassandra cassandra 9 Dec 11 02:00 mc-47-big-Digest.crc32 -rw-r--r-- 1 cassandra cassandra 80 Dec 11 02:00 mc-47-big-Filter.db -rw-r--r-- 1 cassandra cassandra 475 Dec 11 02:00 mc-47-big-Index.db -rw-r--r-- 1 cassandra cassandra 4835 Dec 11 02:00 mc-47-big-Statistics.db -rw-r--r-- 1 cassandra cassandra 56 Dec 11 02:00 mc-47-big-Summary.db -rw-r--r-- 1 cassandra cassandra 92 Dec 11 02:00 mc-47-big-TOC.txt -rw-r--r-- 1 cassandra cassandra 51 Dec 11 04:00 mc-63-big-CompressionInfo.db -rw-r--r-- 1 cassandra cassandra 2504 Dec 11 04:00 mc-63-big-Data.db -rw-r--r-- 1 cassandra cassandra 10 Dec 11 04:00 mc-63-big-Digest.crc32 -rw-r--r-- 1 cassandra cassandra 72 Dec 11 04:00 mc-63-big-Filter.db -rw-r--r-- 1 cassandra cassandra 430 Dec 11 04:00 mc-63-big-Index.db -rw-r--r-- 1 cassandra cassandra 4823 Dec 11 04:00 mc-63-big-Statistics.db -rw-r--r-- 1 cassandra cassandra 56 Dec 11 04:00 mc-63-big-Summary.db -rw-r--r-- 1 cassandra cassandra 92 Dec 11 04:00 mc-63-big-TOC.txt -rw-r--r-- 1 cassandra cassandra 43 Dec 11 06:10 mc-79-big-CompressionInfo.db -rw-r--r-- 1 cassandra cassandra 317 Dec 11 06:10 mc-79-big-Data.db -rw-r--r-- 1 cassandra cassandra 10 Dec 11 06:10 mc-79-big-Digest.crc32 -rw-r--r-- 1 cassandra cassandra 24 Dec 11 06:10 mc-79-big-Filter.db -rw-r--r-- 1 cassandra cassandra 69 Dec 11 06:10 mc-79-big-Index.db -rw-r--r-- 1 cassandra cassandra 4698 Dec 11 06:10 mc-79-big-Statistics.db -rw-r--r-- 1 cassandra cassandra 56 Dec 11 06:10 mc-79-big-Summary.db -rw-r--r-- 1 cassandra cassandra 92 Dec 11 06:10 mc-79-big-TOC.txt -rw-r--r-- 1 cassandra cassandra 51 Dec 11 06:10 mc-80-big-CompressionInfo.db -rw-r--r-- 1 cassandra cassandra 2327 Dec 11 06:10 mc-80-big-Data.db -rw-r--r-- 1 cassandra cassandra 9 Dec 11 06:10 mc-80-big-Digest.crc32 -rw-r--r-- 1 cassandra cassandra 72 Dec 11 06:10 mc-80-big-Filter.db -rw-r--r-- 1 cassandra cassandra 430 Dec 11 06:10 mc-80-big-Index.db -rw-r--r-- 1 cassandra cassandra 4821 Dec 11 06:10 mc-80-big-Statistics.db -rw-r--r-- 1 cassandra cassandra 56 Dec 11 06:10 mc-80-big-Summary.db -rw-r--r-- 1 cassandr
[jira] [Commented] (CASSANDRA-14880) drop table and materialized view frequently get error over time
[ https://issues.apache.org/jira/browse/CASSANDRA-14880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16716105#comment-16716105 ] jhon commented on CASSANDRA-14880: -- in fact,before delete materialized views ,I will add some data to table and sleep some time to wait it delete success.I find some delete operation will take 10 seconds if delete materialized views frequently, and I set cassandra operation timeout from 10s to 30s, it seems no problems. > drop table and materialized view frequently get error over time > --- > > Key: CASSANDRA-14880 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14880 > Project: Cassandra > Issue Type: Bug > Components: Materialized Views >Reporter: jhon >Assignee: jhon >Priority: Major > > when i create table and materialized view,then i drop it, if i drop it > frequently it will got error: "no response received from cassandra within > timeout period",for example: > for i=0; i<100; i++ { > create table; > create materialized view; > drop materialized view; > drop table; > how can i solve it ? > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14928) MigrationManager attempts to pull schema from different major version nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715633#comment-16715633 ] Ariel Weisberg commented on CASSANDRA-14928: ||Branch|CircleCI| |[2.1|https://github.com/apache/cassandra/compare/cassandra-2.1...aweisberg:14928-2.1?expand=1]|[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/14928-2%2E1]| |[2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...aweisberg:14928-2.2?expand=1]|[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/14928-2%2E2]| |[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...aweisberg:14928-3.0?expand=1]|[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/14928-3%2E0]| |[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...aweisberg:14928-3.11?expand=1]|[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/14928-3%2E11]| |[trunk|https://github.com/apache/cassandra/compare/cassandra-2.2...aweisberg:14928-trunk?expand=1]|[CircleCI|https://circleci.com/gh/aweisberg/cassandra/tree/14928-trunk]| > MigrationManager attempts to pull schema from different major version nodes > --- > > Key: CASSANDRA-14928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14928 > Project: Cassandra > Issue Type: Bug > Components: 4.0, Distributed Metadata >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Blocker > Fix For: 4.0, 3.0.x, 3.11.x > > > MigrationManager will do the version check against nodes it hasn't connected > to yet so it doesn't know their messaging service version. We should also > check the version in gossip as an additional layer of protection. > This causes many of the upgrade tests to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14923) Java8 forEach cannot be used in UDF
[ https://issues.apache.org/jira/browse/CASSANDRA-14923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas resolved CASSANDRA-14923. -- Resolution: Duplicate > Java8 forEach cannot be used in UDF > --- > > Key: CASSANDRA-14923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14923 > Project: Cassandra > Issue Type: Bug > Environment: FreeBSD 11.2 > Cassandra 3.11.3 > OpenJDK 8.181.13 >Reporter: Lapo Luchini >Priority: Major > > I get the following error: > {noformat} > cqlsh:test> CREATE OR REPLACE FUNCTION sumMap (state map, val > map) > ... RETURNS NULL ON NULL INPUT > ... RETURNS map > ... LANGUAGE java AS > ... $$ > ... val.forEach((k, v) -> { > ... Integer cur = state.get(k); > ... state.put(k, (cur == null) ? v : cur + v); > ... }); > ... return state; > ... $$; > InvalidRequest: Error from server: code=2200 [Invalid query] message="Java > source compilation failed: > Line 2: The type java.util.function.BiConsumer cannot be resolved. It is > indirectly referenced from required .class files > Line 2: The method forEach(BiConsumer) from > the type Map refers to the missing type BiConsumer > Line 2: The target type of this expression must be a functional interface > " > {noformat} > on the other hand, this compiles correctly: > {noformat} > CREATE OR REPLACE FUNCTION sumMap (state map, val map) > RETURNS NULL ON NULL INPUT > RETURNS map > LANGUAGE java AS > $$ > for (Map.Entry e : val.entrySet()) { > String k = e.getKey(); > Integer v = e.getValue(); > Integer cur = state.get(k); > state.put(k, (cur == null) ? v : cur + v); > }; > return state; > $$; > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14923) Java8 forEach cannot be used in UDF
[ https://issues.apache.org/jira/browse/CASSANDRA-14923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715623#comment-16715623 ] C. Scott Andreas commented on CASSANDRA-14923: -- Thanks for filing, [~l...@lapo.it]! This is tracked under [CASSANDRA-11052|https://issues.apache.org/jira/browse/CASSANDRA-11052]. > Java8 forEach cannot be used in UDF > --- > > Key: CASSANDRA-14923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14923 > Project: Cassandra > Issue Type: Bug > Environment: FreeBSD 11.2 > Cassandra 3.11.3 > OpenJDK 8.181.13 >Reporter: Lapo Luchini >Priority: Major > > I get the following error: > {noformat} > cqlsh:test> CREATE OR REPLACE FUNCTION sumMap (state map, val > map) > ... RETURNS NULL ON NULL INPUT > ... RETURNS map > ... LANGUAGE java AS > ... $$ > ... val.forEach((k, v) -> { > ... Integer cur = state.get(k); > ... state.put(k, (cur == null) ? v : cur + v); > ... }); > ... return state; > ... $$; > InvalidRequest: Error from server: code=2200 [Invalid query] message="Java > source compilation failed: > Line 2: The type java.util.function.BiConsumer cannot be resolved. It is > indirectly referenced from required .class files > Line 2: The method forEach(BiConsumer) from > the type Map refers to the missing type BiConsumer > Line 2: The target type of this expression must be a functional interface > " > {noformat} > on the other hand, this compiles correctly: > {noformat} > CREATE OR REPLACE FUNCTION sumMap (state map, val map) > RETURNS NULL ON NULL INPUT > RETURNS map > LANGUAGE java AS > $$ > for (Map.Entry e : val.entrySet()) { > String k = e.getKey(); > Integer v = e.getValue(); > Integer cur = state.get(k); > state.put(k, (cur == null) ? v : cur + v); > }; > return state; > $$; > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14928) MigrationManager attempts to pull schema from different major version nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14928: --- Description: MigrationManager will do the version check against nodes it hasn't connected to yet so it doesn't know their messaging service version. We should also check the version in gossip as an additional layer of protection. This causes many of the upgrade tests to fail. was: MigrationManager will do the version check against nodes it hasn't connected to yet so it doesn't know their messaging service version. We should also check the version in gossip as an additional layer of protection. This causes many of the upgrade tests to fail as the exceptions from the failed schema pull are detected. > MigrationManager attempts to pull schema from different major version nodes > --- > > Key: CASSANDRA-14928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14928 > Project: Cassandra > Issue Type: Bug > Components: 4.0, Distributed Metadata >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Blocker > Fix For: 4.0, 3.0.x, 3.11.x > > > MigrationManager will do the version check against nodes it hasn't connected > to yet so it doesn't know their messaging service version. We should also > check the version in gossip as an additional layer of protection. > This causes many of the upgrade tests to fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14928) MigrationManager attempts to pull schema from different major version nodes
[ https://issues.apache.org/jira/browse/CASSANDRA-14928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ariel Weisberg updated CASSANDRA-14928: --- Description: MigrationManager will do the version check against nodes it hasn't connected to yet so it doesn't know their messaging service version. We should also check the version in gossip as an additional layer of protection. This causes many of the upgrade tests to fail as the exceptions from the failed schema pull are detected. was:MigrationManager will do the version check against nodes it hasn't connected to yet so it doesn't know their messaging service version. We should also check the version in gossip as an additional layer of protection. > MigrationManager attempts to pull schema from different major version nodes > --- > > Key: CASSANDRA-14928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14928 > Project: Cassandra > Issue Type: Bug > Components: 4.0, Distributed Metadata >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Blocker > Fix For: 4.0, 3.0.x, 3.11.x > > > MigrationManager will do the version check against nodes it hasn't connected > to yet so it doesn't know their messaging service version. We should also > check the version in gossip as an additional layer of protection. > This causes many of the upgrade tests to fail as the exceptions from the > failed schema pull are detected. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14928) MigrationManager attempts to pull schema from different major version nodes
Ariel Weisberg created CASSANDRA-14928: -- Summary: MigrationManager attempts to pull schema from different major version nodes Key: CASSANDRA-14928 URL: https://issues.apache.org/jira/browse/CASSANDRA-14928 Project: Cassandra Issue Type: Bug Components: 4.0, Distributed Metadata Reporter: Ariel Weisberg Assignee: Ariel Weisberg Fix For: 4.0, 3.0.x, 3.11.x MigrationManager will do the version check against nodes it hasn't connected to yet so it doesn't know their messaging service version. We should also check the version in gossip as an additional layer of protection. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14927) During data migration from 7 node to 21 node cluster using sstableloader, new data is being populated on the new tables & data is being duplicated on user type table
[ https://issues.apache.org/jira/browse/CASSANDRA-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KALYAN CHAKRAVARTHY KANCHARLA updated CASSANDRA-14927: -- Priority: Blocker (was: Major) > During data migration from 7 node to 21 node cluster using sstableloader, new > data is being populated on the new tables & data is being duplicated on user > type tables > --- > > Key: CASSANDRA-14927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14927 > Project: Cassandra > Issue Type: Test >Reporter: KALYAN CHAKRAVARTHY KANCHARLA >Priority: Blocker > Labels: test > Fix For: 2.1.13 > > > I'm trying to migrate data from 7 node (single DC) cluster to a 21 node (3 > DC) cluster using sstableloader. > We have same versions on both old and new clusters. > *cqlsh 5.0.1* > *Cassandra 2.1.13* > *CQL spec 3.2.1* > Old and New clusters are in different networks. So we opened the following > ports between them. > 7000- storage port > 7001- ssl storage port > 7199- JMX port > 9042- client port > 9160- Thrift client port > We use vnodes in the clusters. > We made sure cassandra.yaml file on the new cluster is set correct by > changing following options, > > {{cluster_name: 'MyCassandraCluster' }} > {{num_tokens: 256 }} > {{seed_provider: - }} > {{class_name: org.apache.cassandra.locator.SimpleSeedProvider }} > {{parameters: - }} > {{seeds: "10.168.66.41,10.176.170.59" }} > {{listen_address: localhost}} > {{endpoint_snitch: GossipingPropertyFileSnitch}} > And also changes in cassaandra-rackdc-properties for each DC by specifying > respective DC and rack. > while creating keyspaces, changed Replication to NetworkTopologyStratagy. > > cluster looks healthy, all the node are UP and NORMAL. > > {color:#FF}*I was able to get the data from old cluster to new cluster. > But, along with the data from old cluster, I see some new rows being > populated in the tables on new cluster and data is being duplicated in the > tables with user type*. {color} > {color:#33}We have used the following steps to migrate data:{color} > # Took snapshorts for all the keyspaces that we want to migrate. (9 > keyspaces). Used the _nodetool snapshot_ command on source nodes to take > snapshot of required keyspace/table by specifying _hostname, jmx port_ and > _keyspace_ > __ > _/a/cassandra/bin/nodetool -u $(sudo su - company -c "cat > /a/cassandra/jmxremote.password" | awk '\{print $1}') -pw $(sudo su - company > -c "cat /a/cassandra/jmxremote.password" | awk '\{print $2}')_ *_-h > localhost -p 7199 snapshot keyspace_name_* > # After taking snapshots, move these snapshot directory from source nodes to > target node. > > → Create a tar file on source node for the snapshot directory that we want to > move on to target node. > tar -cvf file.tar snapshot_name > → Move this file.tar from source node to local machine. > scp -S gwsh root@192.168.64.99:/a/cassandra/data/file.tar . > → Now move this file.tar from local machine to a new directory(example: test) > in the target node. > scp -S gwsh file.tar root@192.168.58.41:/a/cassandra/data/test/. > # Now untar this file.tar in test directory in target node. > # The path of the sstables must be same in both source and target. > # To bulk load these files using _sstableloader,run sstableloader on source > node, indicate one or more nodes in the destination Cluster with -d flag, > which can accept comma-separated list of IP addresses or hostnames, and > specify the path to sstables in the source node._ __ > _/a/Cassandra/bin/_ *_./sstableloader -d host_IP path_to_sstables_* > *_Example:_* > [/a/cassandra/bin#|mailto:root@sqa-cassandra03.sqaextranet:/a/cassandra/bin] > sstableloader -d 192.168.58.41 -u popps -pw *** -tf > org.apache.cassandra.thrift.SSLTransportFactory -ts > /a/cassandra/ssl/truststore.jks -tspw test123 -ks > /a/cassandra/ssl/keystore.jks -kspw test123 -f > /a/cassandra/conf/cassandra.yaml > /a/cassandra/data/app_properties/_admins-58524140431511e8bbb6357f562e11ca_/ > Summary statistics: > Connections per host: : 1 > Total files transferred: : 9 > Total bytes transferred: : 1787893 > Total duration (ms): : 2936 > Average transfer rate (MB/s): : 0 > Peak transfer rate (MB/s): : 0 > > Performed these steps on all the tables. And checked the row count in old and > new tables using CQLSH > cqlsh> SELECT count(*) FROM keyspace.table; > example for a single table: > count on new table: 341 > count on old table: 303 > > And we are also able to identify the difference in tables by using 'sdiff' > command. Followed the following steps: > * created .
[jira] [Commented] (CASSANDRA-14910) Don't skip entire sstables when reading backwards with mixed clustering column order
[ https://issues.apache.org/jira/browse/CASSANDRA-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715312#comment-16715312 ] Aleksey Yeschenko commented on CASSANDRA-14910: --- Committed to 2.2 as [afa4563864889c78569e29466047b411cd866b38|https://github.com/apache/cassandra/commit/afa4563864889c78569e29466047b411cd866b38] and merged (the unit tests) upwards. Thanks! > Don't skip entire sstables when reading backwards with mixed clustering > column order > > > Key: CASSANDRA-14910 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14910 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Major > Fix For: 2.2.14 > > > In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that > doesn’t have any static rows in it will be skipped altogether when iterated > in reverse. > This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting > any empty {{ByteBuffer}} after non-empty values due to the way > {{ReversedType}} operates. In case that empty {{ByteBuffer}} is coming from a > static {{Composite}}, however, the logic breaks down. Static {{Composite}} > components must *always* sort before any non-empty value, no matter the > table’s comparator. > 2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice > intersection logic entirely if static rows are present in the filter. > Introduced by CASSANDRA-8502. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14910) Don't skip entire sstables when reading backwards with mixed clustering column order
[ https://issues.apache.org/jira/browse/CASSANDRA-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14910: -- Resolution: Fixed Fix Version/s: (was: 2.2.x) 2.2.14 Reproduced In: 2.2.13, 2.1.20 (was: 2.1.20, 2.2.13) Status: Resolved (was: Ready to Commit) > Don't skip entire sstables when reading backwards with mixed clustering > column order > > > Key: CASSANDRA-14910 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14910 > Project: Cassandra > Issue Type: Bug > Components: Local Write-Read Paths >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Major > Fix For: 2.2.14 > > > In 2.x, if a table has clustering columns in {{DESC}} order, any SSTable that > doesn’t have any static rows in it will be skipped altogether when iterated > in reverse. > This occurs due to the logic in {{ColumnSlice.compare()}} errorneusly sorting > any empty {{ByteBuffer}} after non-empty values due to the way > {{ReversedType}} operates. In case that empty {{ByteBuffer}} is coming from a > static {{Composite}}, however, the logic breaks down. Static {{Composite}} > components must *always* sort before any non-empty value, no matter the > table’s comparator. > 2.0, 2.1, and 2.2 are all affected. 3.0 isn’t, but only because we skip slice > intersection logic entirely if static rows are present in the filter. > Introduced by CASSANDRA-8502. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[03/10] cassandra git commit: Don't skip entire sstables when reading backwards with mixed clustering column order
Don't skip entire sstables when reading backwards with mixed clustering column order patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14910 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/afa45638 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/afa45638 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/afa45638 Branch: refs/heads/cassandra-3.11 Commit: afa4563864889c78569e29466047b411cd866b38 Parents: cf6f792 Author: Aleksey Yeshchenko Authored: Thu Nov 22 14:59:22 2018 + Committer: Aleksey Yeshchenko Committed: Tue Nov 27 21:03:56 2018 + -- CHANGES.txt | 2 + .../apache/cassandra/db/filter/ColumnSlice.java | 5 +- .../operations/SelectOrderByTest.java | 78 3 files changed, 84 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index bca036d..b989f3c 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,6 @@ 2.2.14 + * Don't skip entire sstables when reading backwards with mixed clustering column order + (CASSANDRA-14910) * Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order (CASSANDRA-14899) * Fix incorrect cqlsh results when selecting same columns multiple times (CASSANDRA-13262) http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/src/java/org/apache/cassandra/db/filter/ColumnSlice.java -- diff --git a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java index 1cc348c..316226d 100644 --- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java +++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java @@ -68,7 +68,10 @@ public class ColumnSlice Composite sStart = reversed ? finish : start; Composite sEnd = reversed ? start : finish; -if (compare(sStart, maxCellNames, comparator, true) > 0 || compare(sEnd, minCellNames, comparator, false) < 0) +// don't compare static slice bounds with min/max cell names to determine intersection - that can yield unexpected +// results, in particular with ReverseType comparators; see CASSANDRA-14910 for more context. +if ((!sStart.isStatic() && compare(sStart, maxCellNames, comparator, true) > 0) + || (!sEnd.isStatic() && compare(sEnd, minCellNames, comparator, false) < 0)) return false; // We could safely return true here, but there's a minor optimization: if the first component is restricted http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- diff --git a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java index 32d800a..e21074b 100644 --- a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java +++ b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java @@ -732,6 +732,84 @@ public class SelectOrderByTest extends CQLTester } } +@Test +public void testSelectWithReversedTypeInReverseOrderWithStaticColumnsWithoutStaticRow() throws Throwable +{ +createTable("CREATE TABLE %s (a int, b int, c int, d int static, PRIMARY KEY (a, b)) WITH CLUSTERING ORDER BY (b DESC);"); + +execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 1);"); +execute("INSERT INTO %s (a, b, c) VALUES (1, 2, 2);"); +execute("INSERT INTO %s (a, b, c) VALUES (1, 3, 3);"); + +// read in comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"), + row(3, 3), + row(2, 2), + row(1, 1)); + +// read in reverse comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b ASC;"), + row(1, 1), + row(2, 2), + row(3, 3)); + +/* + * Flush the sstable. We *should* see the same results when reading in both directions, but prior to CASSANDRA-14910 + * fix this would now have returned an empty result set when reading in reverse comparator order. + */ +flush(); + +// read in comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"), + row(3, 3), +
[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b871decb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b871decb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b871decb Branch: refs/heads/trunk Commit: b871decba274be126fc94ab3ab48f75a95be9d94 Parents: 33eada0 2edd6fd Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:23:39 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:23:39 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/b871decb/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- diff --cc test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java index fd44c50,06aa2fd..8a3ae03 --- a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java +++ b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java @@@ -639,132 -702,84 +639,210 @@@ public class SelectOrderByTest extends } } +/** + * Test that ORDER BY columns allow skipping equality-restricted clustering columns, see CASSANDRA-10271. + */ +@Test +public void testAllowSkippingEqualityAndSingleValueInRestrictedClusteringColumns() throws Throwable +{ +createTable("CREATE TABLE %s (a int, b int, c int, d int, PRIMARY KEY (a, b, c))"); +execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 0, 0); +execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 1, 1); +execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 2, 2); +execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 0, 3); +execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 1, 4); +execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 2, 5); + +assertInvalidMessage("Order by is currently only supported on the clustered columns of the PRIMARY KEY, got d", + "SELECT * FROM %s WHERE a=? ORDER BY d DESC", 0); + +assertInvalidMessage("Order by is currently only supported on the clustered columns of the PRIMARY KEY, got d", + "SELECT * FROM %s WHERE a=? ORDER BY b ASC, c ASC, d ASC", 0); + +String errorMsg = "Order by currently only supports the ordering of columns following their declared order in the PRIMARY KEY"; + +assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? ORDER BY c", 0, 0), + row(0, 0, 0, 0), + row(0, 0, 1, 1), + row(0, 0, 2, 2) +); + +assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? ORDER BY c ASC", 0, 0), + row(0, 0, 0, 0), + row(0, 0, 1, 1), + row(0, 0, 2, 2) +); + +assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? ORDER BY c DESC", 0, 0), + row(0, 0, 2, 2), + row(0, 0, 1, 1), + row(0, 0, 0, 0) +); + +assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? AND c>=? ORDER BY c ASC", 0, 0, 1), + row(0, 0, 1, 1), + row(0, 0, 2, 2)); + +assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? AND c>=? ORDER BY c DESC", 0, 0, 1), + row(0, 0, 2, 2), + row(0, 0, 1, 1)); + +assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? AND c IN (?, ?) ORDER BY c ASC", 0, 0, 1, 2), + row(0, 0, 1, 1), + row(0, 0, 2, 2)); + +assertRows(execute("SELECT * FROM %s WHERE a=? AND b=? AND c IN (?, ?) ORDER BY c DESC", 0, 0, 1, 2), + row(0, 0, 2, 2), + row(0, 0, 1, 1)); + +assertInvalidMessage(errorMsg, "SELECT * FROM %s WHERE a=? AND b (?, ?) ORDER BY c", 0, 0, 0); +assertInvalidMessage(errorMsg, "SELECT * FROM %s WHERE a=? AND (b, c) >= (?, ?) ORDER BY c", 0, 0, 0); +assertInvalidMessage(errorMsg, "SELECT * FROM %s WHERE a=? AND (b, c) < (?, ?) ORDER BY c", 0, 0, 0); +assertInvalidMessage(errorMsg, "SELECT * FROM %s WHERE a=? AND (b, c) <= (?, ?) ORDER BY c", 0, 0, 0); +assertRows(execute("SELECT * FROM %s WHERE a=? AND (b, c) = (?, ?) ORDER BY c ASC", 0, 0, 0), + row(0, 0, 0, 0)); +assertRows(execute("SELECT * FROM %s WHERE a=? AND (b, c) = (?, ?) ORDER BY c DESC", 0, 0, 0), + row(0, 0, 0, 0)); + +
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4f3a6feb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4f3a6feb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4f3a6feb Branch: refs/heads/cassandra-3.0 Commit: 4f3a6feb22f844a99e1c20c20ae17b2427f81307 Parents: 84ffcb8 afa4563 Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:14:14 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:14:14 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f3a6feb/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[04/10] cassandra git commit: Don't skip entire sstables when reading backwards with mixed clustering column order
Don't skip entire sstables when reading backwards with mixed clustering column order patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14910 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/afa45638 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/afa45638 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/afa45638 Branch: refs/heads/trunk Commit: afa4563864889c78569e29466047b411cd866b38 Parents: cf6f792 Author: Aleksey Yeshchenko Authored: Thu Nov 22 14:59:22 2018 + Committer: Aleksey Yeshchenko Committed: Tue Nov 27 21:03:56 2018 + -- CHANGES.txt | 2 + .../apache/cassandra/db/filter/ColumnSlice.java | 5 +- .../operations/SelectOrderByTest.java | 78 3 files changed, 84 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index bca036d..b989f3c 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,6 @@ 2.2.14 + * Don't skip entire sstables when reading backwards with mixed clustering column order + (CASSANDRA-14910) * Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order (CASSANDRA-14899) * Fix incorrect cqlsh results when selecting same columns multiple times (CASSANDRA-13262) http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/src/java/org/apache/cassandra/db/filter/ColumnSlice.java -- diff --git a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java index 1cc348c..316226d 100644 --- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java +++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java @@ -68,7 +68,10 @@ public class ColumnSlice Composite sStart = reversed ? finish : start; Composite sEnd = reversed ? start : finish; -if (compare(sStart, maxCellNames, comparator, true) > 0 || compare(sEnd, minCellNames, comparator, false) < 0) +// don't compare static slice bounds with min/max cell names to determine intersection - that can yield unexpected +// results, in particular with ReverseType comparators; see CASSANDRA-14910 for more context. +if ((!sStart.isStatic() && compare(sStart, maxCellNames, comparator, true) > 0) + || (!sEnd.isStatic() && compare(sEnd, minCellNames, comparator, false) < 0)) return false; // We could safely return true here, but there's a minor optimization: if the first component is restricted http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- diff --git a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java index 32d800a..e21074b 100644 --- a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java +++ b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java @@ -732,6 +732,84 @@ public class SelectOrderByTest extends CQLTester } } +@Test +public void testSelectWithReversedTypeInReverseOrderWithStaticColumnsWithoutStaticRow() throws Throwable +{ +createTable("CREATE TABLE %s (a int, b int, c int, d int static, PRIMARY KEY (a, b)) WITH CLUSTERING ORDER BY (b DESC);"); + +execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 1);"); +execute("INSERT INTO %s (a, b, c) VALUES (1, 2, 2);"); +execute("INSERT INTO %s (a, b, c) VALUES (1, 3, 3);"); + +// read in comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"), + row(3, 3), + row(2, 2), + row(1, 1)); + +// read in reverse comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b ASC;"), + row(1, 1), + row(2, 2), + row(3, 3)); + +/* + * Flush the sstable. We *should* see the same results when reading in both directions, but prior to CASSANDRA-14910 + * fix this would now have returned an empty result set when reading in reverse comparator order. + */ +flush(); + +// read in comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"), + row(3, 3), +
[02/10] cassandra git commit: Don't skip entire sstables when reading backwards with mixed clustering column order
Don't skip entire sstables when reading backwards with mixed clustering column order patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14910 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/afa45638 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/afa45638 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/afa45638 Branch: refs/heads/cassandra-3.0 Commit: afa4563864889c78569e29466047b411cd866b38 Parents: cf6f792 Author: Aleksey Yeshchenko Authored: Thu Nov 22 14:59:22 2018 + Committer: Aleksey Yeshchenko Committed: Tue Nov 27 21:03:56 2018 + -- CHANGES.txt | 2 + .../apache/cassandra/db/filter/ColumnSlice.java | 5 +- .../operations/SelectOrderByTest.java | 78 3 files changed, 84 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index bca036d..b989f3c 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,6 @@ 2.2.14 + * Don't skip entire sstables when reading backwards with mixed clustering column order + (CASSANDRA-14910) * Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order (CASSANDRA-14899) * Fix incorrect cqlsh results when selecting same columns multiple times (CASSANDRA-13262) http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/src/java/org/apache/cassandra/db/filter/ColumnSlice.java -- diff --git a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java index 1cc348c..316226d 100644 --- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java +++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java @@ -68,7 +68,10 @@ public class ColumnSlice Composite sStart = reversed ? finish : start; Composite sEnd = reversed ? start : finish; -if (compare(sStart, maxCellNames, comparator, true) > 0 || compare(sEnd, minCellNames, comparator, false) < 0) +// don't compare static slice bounds with min/max cell names to determine intersection - that can yield unexpected +// results, in particular with ReverseType comparators; see CASSANDRA-14910 for more context. +if ((!sStart.isStatic() && compare(sStart, maxCellNames, comparator, true) > 0) + || (!sEnd.isStatic() && compare(sEnd, minCellNames, comparator, false) < 0)) return false; // We could safely return true here, but there's a minor optimization: if the first component is restricted http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- diff --git a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java index 32d800a..e21074b 100644 --- a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java +++ b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java @@ -732,6 +732,84 @@ public class SelectOrderByTest extends CQLTester } } +@Test +public void testSelectWithReversedTypeInReverseOrderWithStaticColumnsWithoutStaticRow() throws Throwable +{ +createTable("CREATE TABLE %s (a int, b int, c int, d int static, PRIMARY KEY (a, b)) WITH CLUSTERING ORDER BY (b DESC);"); + +execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 1);"); +execute("INSERT INTO %s (a, b, c) VALUES (1, 2, 2);"); +execute("INSERT INTO %s (a, b, c) VALUES (1, 3, 3);"); + +// read in comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"), + row(3, 3), + row(2, 2), + row(1, 1)); + +// read in reverse comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b ASC;"), + row(1, 1), + row(2, 2), + row(3, 3)); + +/* + * Flush the sstable. We *should* see the same results when reading in both directions, but prior to CASSANDRA-14910 + * fix this would now have returned an empty result set when reading in reverse comparator order. + */ +flush(); + +// read in comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"), + row(3, 3), +
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2edd6fd2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2edd6fd2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2edd6fd2 Branch: refs/heads/trunk Commit: 2edd6fd2e02b9e2769e4ea751c1030428f70ce47 Parents: 732c43b 4f3a6fe Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:19:31 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:19:31 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2edd6fd2/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2edd6fd2 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2edd6fd2 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2edd6fd2 Branch: refs/heads/cassandra-3.11 Commit: 2edd6fd2e02b9e2769e4ea751c1030428f70ce47 Parents: 732c43b 4f3a6fe Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:19:31 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:19:31 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2edd6fd2/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4f3a6feb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4f3a6feb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4f3a6feb Branch: refs/heads/trunk Commit: 4f3a6feb22f844a99e1c20c20ae17b2427f81307 Parents: 84ffcb8 afa4563 Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:14:14 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:14:14 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f3a6feb/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[01/10] cassandra git commit: Don't skip entire sstables when reading backwards with mixed clustering column order
Repository: cassandra Updated Branches: refs/heads/cassandra-2.2 cf6f7920f -> afa456386 refs/heads/cassandra-3.0 84ffcb82a -> 4f3a6feb2 refs/heads/cassandra-3.11 732c43b0f -> 2edd6fd2e refs/heads/trunk 33eada06a -> b871decba Don't skip entire sstables when reading backwards with mixed clustering column order patch by Aleksey Yeschenko; reviewed by Alex Petrov for CASSANDRA-14910 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/afa45638 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/afa45638 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/afa45638 Branch: refs/heads/cassandra-2.2 Commit: afa4563864889c78569e29466047b411cd866b38 Parents: cf6f792 Author: Aleksey Yeshchenko Authored: Thu Nov 22 14:59:22 2018 + Committer: Aleksey Yeshchenko Committed: Tue Nov 27 21:03:56 2018 + -- CHANGES.txt | 2 + .../apache/cassandra/db/filter/ColumnSlice.java | 5 +- .../operations/SelectOrderByTest.java | 78 3 files changed, 84 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index bca036d..b989f3c 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,6 @@ 2.2.14 + * Don't skip entire sstables when reading backwards with mixed clustering column order + (CASSANDRA-14910) * Cannot perform slice reads in reverse direction against tables with clustering columns in mixed order (CASSANDRA-14899) * Fix incorrect cqlsh results when selecting same columns multiple times (CASSANDRA-13262) http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/src/java/org/apache/cassandra/db/filter/ColumnSlice.java -- diff --git a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java index 1cc348c..316226d 100644 --- a/src/java/org/apache/cassandra/db/filter/ColumnSlice.java +++ b/src/java/org/apache/cassandra/db/filter/ColumnSlice.java @@ -68,7 +68,10 @@ public class ColumnSlice Composite sStart = reversed ? finish : start; Composite sEnd = reversed ? start : finish; -if (compare(sStart, maxCellNames, comparator, true) > 0 || compare(sEnd, minCellNames, comparator, false) < 0) +// don't compare static slice bounds with min/max cell names to determine intersection - that can yield unexpected +// results, in particular with ReverseType comparators; see CASSANDRA-14910 for more context. +if ((!sStart.isStatic() && compare(sStart, maxCellNames, comparator, true) > 0) + || (!sEnd.isStatic() && compare(sEnd, minCellNames, comparator, false) < 0)) return false; // We could safely return true here, but there's a minor optimization: if the first component is restricted http://git-wip-us.apache.org/repos/asf/cassandra/blob/afa45638/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- diff --git a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java index 32d800a..e21074b 100644 --- a/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java +++ b/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java @@ -732,6 +732,84 @@ public class SelectOrderByTest extends CQLTester } } +@Test +public void testSelectWithReversedTypeInReverseOrderWithStaticColumnsWithoutStaticRow() throws Throwable +{ +createTable("CREATE TABLE %s (a int, b int, c int, d int static, PRIMARY KEY (a, b)) WITH CLUSTERING ORDER BY (b DESC);"); + +execute("INSERT INTO %s (a, b, c) VALUES (1, 1, 1);"); +execute("INSERT INTO %s (a, b, c) VALUES (1, 2, 2);"); +execute("INSERT INTO %s (a, b, c) VALUES (1, 3, 3);"); + +// read in comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b DESC;"), + row(3, 3), + row(2, 2), + row(1, 1)); + +// read in reverse comparator order +assertRows(execute("SELECT b, c FROM %s WHERE a = 1 ORDER BY b ASC;"), + row(1, 1), + row(2, 2), + row(3, 3)); + +/* + * Flush the sstable. We *should* see the same results when reading in both directions, but prior to CASSANDRA-14910 + * fix this would now have returned an empty result s
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4f3a6feb Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4f3a6feb Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4f3a6feb Branch: refs/heads/cassandra-3.11 Commit: 4f3a6feb22f844a99e1c20c20ae17b2427f81307 Parents: 84ffcb8 afa4563 Author: Aleksey Yeshchenko Authored: Mon Dec 10 18:14:14 2018 + Committer: Aleksey Yeshchenko Committed: Mon Dec 10 18:14:14 2018 + -- .../operations/SelectOrderByTest.java | 78 1 file changed, 78 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4f3a6feb/test/unit/org/apache/cassandra/cql3/validation/operations/SelectOrderByTest.java -- - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14927) During data migration from 7 node to 21 node cluster using sstableloader, new data is being populated on the new tables & data is being duplicated on user type table
[ https://issues.apache.org/jira/browse/CASSANDRA-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] KALYAN CHAKRAVARTHY KANCHARLA updated CASSANDRA-14927: -- Issue Type: Test (was: Task) > During data migration from 7 node to 21 node cluster using sstableloader, new > data is being populated on the new tables & data is being duplicated on user > type tables > --- > > Key: CASSANDRA-14927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14927 > Project: Cassandra > Issue Type: Test >Reporter: KALYAN CHAKRAVARTHY KANCHARLA >Priority: Major > Labels: test > Fix For: 2.1.13 > > > I'm trying to migrate data from 7 node (single DC) cluster to a 21 node (3 > DC) cluster using sstableloader. > We have same versions on both old and new clusters. > *cqlsh 5.0.1* > *Cassandra 2.1.13* > *CQL spec 3.2.1* > Old and New clusters are in different networks. So we opened the following > ports between them. > 7000- storage port > 7001- ssl storage port > 7199- JMX port > 9042- client port > 9160- Thrift client port > We use vnodes in the clusters. > We made sure cassandra.yaml file on the new cluster is set correct by > changing following options, > > {{cluster_name: 'MyCassandraCluster' }} > {{num_tokens: 256 }} > {{seed_provider: - }} > {{class_name: org.apache.cassandra.locator.SimpleSeedProvider }} > {{parameters: - }} > {{seeds: "10.168.66.41,10.176.170.59" }} > {{listen_address: localhost}} > {{endpoint_snitch: GossipingPropertyFileSnitch}} > And also changes in cassaandra-rackdc-properties for each DC by specifying > respective DC and rack. > while creating keyspaces, changed Replication to NetworkTopologyStratagy. > > cluster looks healthy, all the node are UP and NORMAL. > > {color:#FF}*I was able to get the data from old cluster to new cluster. > But, along with the data from old cluster, I see some new rows being > populated in the tables on new cluster and data is being duplicated in the > tables with user type*. {color} > {color:#33}We have used the following steps to migrate data:{color} > # Took snapshorts for all the keyspaces that we want to migrate. (9 > keyspaces). Used the _nodetool snapshot_ command on source nodes to take > snapshot of required keyspace/table by specifying _hostname, jmx port_ and > _keyspace_ > __ > _/a/cassandra/bin/nodetool -u $(sudo su - company -c "cat > /a/cassandra/jmxremote.password" | awk '\{print $1}') -pw $(sudo su - company > -c "cat /a/cassandra/jmxremote.password" | awk '\{print $2}')_ *_-h > localhost -p 7199 snapshot keyspace_name_* > # After taking snapshots, move these snapshot directory from source nodes to > target node. > > → Create a tar file on source node for the snapshot directory that we want to > move on to target node. > tar -cvf file.tar snapshot_name > → Move this file.tar from source node to local machine. > scp -S gwsh root@192.168.64.99:/a/cassandra/data/file.tar . > → Now move this file.tar from local machine to a new directory(example: test) > in the target node. > scp -S gwsh file.tar root@192.168.58.41:/a/cassandra/data/test/. > # Now untar this file.tar in test directory in target node. > # The path of the sstables must be same in both source and target. > # To bulk load these files using _sstableloader,run sstableloader on source > node, indicate one or more nodes in the destination Cluster with -d flag, > which can accept comma-separated list of IP addresses or hostnames, and > specify the path to sstables in the source node._ __ > _/a/Cassandra/bin/_ *_./sstableloader -d host_IP path_to_sstables_* > *_Example:_* > [/a/cassandra/bin#|mailto:root@sqa-cassandra03.sqaextranet:/a/cassandra/bin] > sstableloader -d 192.168.58.41 -u popps -pw *** -tf > org.apache.cassandra.thrift.SSLTransportFactory -ts > /a/cassandra/ssl/truststore.jks -tspw test123 -ks > /a/cassandra/ssl/keystore.jks -kspw test123 -f > /a/cassandra/conf/cassandra.yaml > /a/cassandra/data/app_properties/_admins-58524140431511e8bbb6357f562e11ca_/ > Summary statistics: > Connections per host: : 1 > Total files transferred: : 9 > Total bytes transferred: : 1787893 > Total duration (ms): : 2936 > Average transfer rate (MB/s): : 0 > Peak transfer rate (MB/s): : 0 > > Performed these steps on all the tables. And checked the row count in old and > new tables using CQLSH > cqlsh> SELECT count(*) FROM keyspace.table; > example for a single table: > count on new table: 341 > count on old table: 303 > > And we are also able to identify the difference in tables by using 'sdiff' > command. Followed the following steps: > * created .txt/
[jira] [Created] (CASSANDRA-14927) During data migration from 7 node to 21 node cluster using sstableloader, new data is being populated on the new tables & data is being duplicated on user type table
KALYAN CHAKRAVARTHY KANCHARLA created CASSANDRA-14927: - Summary: During data migration from 7 node to 21 node cluster using sstableloader, new data is being populated on the new tables & data is being duplicated on user type tables Key: CASSANDRA-14927 URL: https://issues.apache.org/jira/browse/CASSANDRA-14927 Project: Cassandra Issue Type: Task Reporter: KALYAN CHAKRAVARTHY KANCHARLA Fix For: 2.1.13 I'm trying to migrate data from 7 node (single DC) cluster to a 21 node (3 DC) cluster using sstableloader. We have same versions on both old and new clusters. *cqlsh 5.0.1* *Cassandra 2.1.13* *CQL spec 3.2.1* Old and New clusters are in different networks. So we opened the following ports between them. 7000- storage port 7001- ssl storage port 7199- JMX port 9042- client port 9160- Thrift client port We use vnodes in the clusters. We made sure cassandra.yaml file on the new cluster is set correct by changing following options, {{cluster_name: 'MyCassandraCluster' }} {{num_tokens: 256 }} {{seed_provider: - }} {{class_name: org.apache.cassandra.locator.SimpleSeedProvider }} {{parameters: - }} {{seeds: "10.168.66.41,10.176.170.59" }} {{listen_address: localhost}} {{endpoint_snitch: GossipingPropertyFileSnitch}} And also changes in cassaandra-rackdc-properties for each DC by specifying respective DC and rack. while creating keyspaces, changed Replication to NetworkTopologyStratagy. cluster looks healthy, all the node are UP and NORMAL. {color:#FF}*I was able to get the data from old cluster to new cluster. But, along with the data from old cluster, I see some new rows being populated in the tables on new cluster and data is being duplicated in the tables with user type*. {color} {color:#33}We have used the following steps to migrate data:{color} # Took snapshorts for all the keyspaces that we want to migrate. (9 keyspaces). Used the _nodetool snapshot_ command on source nodes to take snapshot of required keyspace/table by specifying _hostname, jmx port_ and _keyspace_ __ _/a/cassandra/bin/nodetool -u $(sudo su - company -c "cat /a/cassandra/jmxremote.password" | awk '\{print $1}') -pw $(sudo su - company -c "cat /a/cassandra/jmxremote.password" | awk '\{print $2}')_ *_-h localhost -p 7199 snapshot keyspace_name_* # After taking snapshots, move these snapshot directory from source nodes to target node. → Create a tar file on source node for the snapshot directory that we want to move on to target node. tar -cvf file.tar snapshot_name → Move this file.tar from source node to local machine. scp -S gwsh root@192.168.64.99:/a/cassandra/data/file.tar . → Now move this file.tar from local machine to a new directory(example: test) in the target node. scp -S gwsh file.tar root@192.168.58.41:/a/cassandra/data/test/. # Now untar this file.tar in test directory in target node. # The path of the sstables must be same in both source and target. # To bulk load these files using _sstableloader,run sstableloader on source node, indicate one or more nodes in the destination Cluster with -d flag, which can accept comma-separated list of IP addresses or hostnames, and specify the path to sstables in the source node._ __ _/a/Cassandra/bin/_ *_./sstableloader -d host_IP path_to_sstables_* *_Example:_* [/a/cassandra/bin#|mailto:root@sqa-cassandra03.sqaextranet:/a/cassandra/bin] sstableloader -d 192.168.58.41 -u popps -pw *** -tf org.apache.cassandra.thrift.SSLTransportFactory -ts /a/cassandra/ssl/truststore.jks -tspw test123 -ks /a/cassandra/ssl/keystore.jks -kspw test123 -f /a/cassandra/conf/cassandra.yaml /a/cassandra/data/app_properties/_admins-58524140431511e8bbb6357f562e11ca_/ Summary statistics: Connections per host: : 1 Total files transferred: : 9 Total bytes transferred: : 1787893 Total duration (ms): : 2936 Average transfer rate (MB/s): : 0 Peak transfer rate (MB/s): : 0 Performed these steps on all the tables. And checked the row count in old and new tables using CQLSH cqlsh> SELECT count(*) FROM keyspace.table; example for a single table: count on new table: 341 count on old table: 303 And we are also able to identify the difference in tables by using 'sdiff' command. Followed the following steps: * created .txt/.csv files for tables in old and new clusters. * compared them using sdiff command *So I request someone can help me to know the cause behind the population of new data in the new tables.* Please let me know if you need more info. PS: After migrating the data for the first time and saw these issues, we have TRUNCATED all the tables and DROPPED tables with user 'type' and recreated the dropped tables. And did the same procedure for migrating data again. Still we see the same issues. -- This message was
[jira] [Commented] (CASSANDRA-14924) Cassandra nodes becomes unreachable to each other
[ https://issues.apache.org/jira/browse/CASSANDRA-14924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715238#comment-16715238 ] C. Scott Andreas commented on CASSANDRA-14924: -- Hi [~venksta], thanks for your report. This bug tracker is primarily used by contributors of the Apache Cassandra project toward development of the database itself. Can you reach out to the user's list or public IRC channel for support? A member of the community may be able to help. Here's a page with information on the best channels for support: http://cassandra.apache.org/community/ > Cassandra nodes becomes unreachable to each other > - > > Key: CASSANDRA-14924 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14924 > Project: Cassandra > Issue Type: Bug >Reporter: ventsislav >Priority: Critical > > I have 3 nodes of elassandra running in docker containers. > Containers created like: > {code:java} > > Host 10.0.0.1 : docker run --name elassandra-node-1 --net=host -e > > CASSANDRA_SEEDS="10.0.0.1" -e CASSANDRA_CLUSTER_NAME="BD Storage" -e > > CASSANDRA_DC="DC1" -e CASSANDRA_RACK="r1" -d > > strapdata/elassandra:latest{code} > {code:java} > > Host 10.0.0.2 : docker run --name elassandra-node-2 --net=host -e > > CASSANDRA_SEEDS="10.0.0.1,10.0.0.2" -e CASSANDRA_CLUSTER_NAME="BD Storage" > > -e CASSANDRA_DC="DC1" -e CASSANDRA_RACK="r1" -d > > strapdata/elassandra:latest{code} > {code:java} > > Host 10.0.0.3 : docker run --name elassandra-node-3 --net=host -e > > CASSANDRA_SEEDS="10.0.0.1,10.0.0.2,10.0.0.3" -e CASSANDRA_CLUSTER_NAME="BD > > Storage" -e CASSANDRA_DC="DC1" -e CASSANDRA_RACK="r1" -d > > strapdata/elassandra:latest{code} > Cluster was working fine for a couple of days since created, elastic, > cassandra all was perfect. > Currently however all cassandra nodes became unreachable to each other: > Nodetool status on all nodes is like > {code:java} > > Datacenter: DC1 > > === > > Status=Up/Down > > |/ State=Normal/Leaving/Joining/Moving > > – Address Load Tokens Owns (effective) Host ID Rack > > DN 10.0.0.3 11.95 GiB 8 100.0% 7652f66e-194e-4886-ac10-0fc21ac8afeb r1 > > DN 10.0.0.2 11.92 GiB 8 100.0% b91fa129-1dd0-4cf8-be96-9c06b23daac6 r1 > > UN 10.0.0.1 11.9 GiB 8 100.0% 5c1afcff-b0aa-4985-a3cc-7f932056c08f r1{code} > Where the UN is the current host 10.0.0.1 > Same on all other nodes. > Nodetool describecluster on 10.0.0.1 is like > {code:java} > > Cluster Information: > > Name: BD Storage > > Snitch: org.apache.cassandra.locator.GossipingPropertyFileSnitch > > DynamicEndPointSnitch: enabled > > Partitioner: org.apache.cassandra.dht.Murmur3Partitioner > > Schema versions: > > 24fa5e55-3935-3c0e-9808-99ce502fe98d: [10.0.0.1] > > > > UNREACHABLE: [10.0.0.2,10.0.0.3]{code} > When attached to the first node its only repeating these infos: > {code:java} > > 2018-12-09 07:47:32,927 WARN [OptionalTasks:1] > > org.apache.cassandra.auth.CassandraRoleManager.setupDefaultRole(CassandraRoleManager.java:361) > > CassandraRoleManager skipped default role setup: some nodes were not ready > > 2018-12-09 07:47:32,927 INFO [OptionalTasks:1] > org.apache.cassandra.auth.CassandraRoleManager$4.run(CassandraRoleManager.java:400) > Setup task failed with error, rescheduling > > 2018-12-09 07:47:32,980 INFO [HANDSHAKE-/10.0.0.2] > org.apache.cassandra.net.OutboundTcpConnection.lambda$handshakeVersion$1(OutboundTcpConnection.java:561) > Handshaking version with /10.0.0.2 > > 2018-12-09 07:47:32,980 INFO [HANDSHAKE-/10.0.0.3] > org.apache.cassandra.net.OutboundTcpConnection.lambda$handshakeVersion$1(OutboundTcpConnection.java:561) > Handshaking version with /10.0.0.3{code} > After a while when some node is restarted: > {code:java} > > 2018-12-09 07:52:21,972 WARN [MigrationStage:1] > > org.apache.cassandra.service.MigrationTask.runMayThrow(MigrationTask.java:67) > > Can't send schema pull request: node /10.0.0.2 is down.{code} > Tried so far: > Restarting all containers at the same time > Restarting all containers one after another > Restarting cassandra in all containers like : service cassandra restart > Nodetool disablegossip then enable it > Nodetool repair : Repair command #1 failed with error Endpoint not alive: > /10.0.0.2 > Seems that all node schemas are different, but I still dont understand why > they are marked as down to each other. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14871) Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
[ https://issues.apache.org/jira/browse/CASSANDRA-14871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714681#comment-16714681 ] Robert Stupp edited comment on CASSANDRA-14871 at 12/10/18 3:59 PM: I'm fine with using {{Verify}}, but it should include the string that yielded {{null}}. Like {{Verify.verify(type != null, "Parsing %s yielded null, which is a bug", str)}} and move that after the assignment of {{type}}. That way we don't need the {{cache.get(str)}} in the synchronized block, because {{type}} is then guaranteed to be non-null. EDIT: So instead of {code:java} synchronized (TypeParser.class) { if (!cache.containsKey(str)) { ImmutableMap.Builder> builder = ImmutableMap.builder(); builder.putAll(cache).put(str, type); cache = builder.build(); } AbstractType rtype = cache.get(str); Verify.verify(rtype != null); return rtype; } {code} like this: {code:java} if (!isEOS(str, i) && str.charAt(i) == '(') type = getAbstractType(name, new TypeParser(str, i)); else type = getAbstractType(name); Verify.verify(type != null, "Parsing %s yielded null, which is a bug", str); // synchronized (TypeParser.class) { if (!cache.containsKey(str)) { ImmutableMap.Builder> builder = ImmutableMap.builder(); builder.putAll(cache).put(str, type); cache = builder.build(); } return type; } {code} was (Author: snazy): I'm fine with using {{Verify}}, but it should include the string that yielded {{null}}. Like {{Verify.verify(type != null, "Parsing %s yielded null, which is a bug", type)}} and move that after the assignment of {{type}}. That way we don't need the {{cache.get(str)}} in the synchronized block, because {{type}} is then guaranteed to be non-null. {code:java} synchronized (TypeParser.class) { if (!cache.containsKey(str)) { ImmutableMap.Builder> builder = ImmutableMap.builder(); builder.putAll(cache).put(str, type); cache = builder.build(); } AbstractType rtype = cache.get(str); Verify.verify(rtype != null); return rtype; } {code} > Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser > --- > > Key: CASSANDRA-14871 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14871 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Critical > Fix For: 4.0, 3.0.x, 3.11.x > > > There are a couple of places in the code base that do not respect that > j.u.HashMap + related classes are not thread safe and some parts rely on > internals of the implementation of HM, which can change. > We have observed failures like {{NullPointerException}} and > {{ConcurrentModificationException}} as well as wrong behavior. > Affected areas in the code base: > * {{SizeTieredCompactionStrategy}} > * {{DateTieredCompactionStrategy}} > * {{TimeWindowCompactionStrategy}} > * {{TokenMetadata.Topology}} > * {{TypeParser}} > * streaming / concurrent access to {{LifecycleTransaction}} (handled in > CASSANDRA-14554) > While the patches for the compaction strategies + {{TypeParser}} are pretty > straight forward, the patch for {{TokenMetadata.Topology}} requires it to be > made immutable. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14482) ZSTD Compressor support in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714966#comment-16714966 ] Sushma A Devendrappa commented on CASSANDRA-14482: -- Thanks for all the feedback! [~aweisberg] we have seen similar results as [~jolynch], several usecases show better results than LZ4 and Deflate, ZSTD has good speed for most usecases given same compression ratio. Current code changes does not affect any testing neither touches any other code part of cassandra itself, apart from adding a new compression. So this should be safe changes for any experimentation and also real usage in production. Dictionary based compression is a very good feature of ZSTD, that can be enabled and code changes for that can go in a separate patch in other upcoming releases. Current patch gives a good start for introducing ZSTD as a compression variant and later add more feature. > ZSTD Compressor support in Cassandra > > > Key: CASSANDRA-14482 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14482 > Project: Cassandra > Issue Type: New Feature > Components: Compression, Libraries >Reporter: Sushma A Devendrappa >Assignee: Sushma A Devendrappa >Priority: Major > Labels: performance, pull-request-available > Fix For: 4.x > > Time Spent: 1h 10m > Remaining Estimate: 0h > > ZStandard has a great speed and compression ratio tradeoff. > ZStandard is open source compression from Facebook. > More about ZSTD > [https://github.com/facebook/zstd] > https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/ > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Resolved] (CASSANDRA-14554) Streaming needs to synchronise access to LifecycleTransaction
[ https://issues.apache.org/jira/browse/CASSANDRA-14554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict resolved CASSANDRA-14554. -- Resolution: Fixed Fix Version/s: (was: 3.11.x) (was: 3.0.x) 3.11.4 3.0.18 Reviewers: Benedict, Robert Stupp (was: Benedict, Dinesh Joshi) > Streaming needs to synchronise access to LifecycleTransaction > - > > Key: CASSANDRA-14554 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14554 > Project: Cassandra > Issue Type: Bug > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Stefania >Priority: Major > Fix For: 3.0.18, 3.11.4, 4.0 > > > When LifecycleTransaction is used in a multi-threaded context, we encounter > this exception - > {quote}java.util.ConcurrentModificationException: null > at > java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719) > at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:742) > at java.lang.Iterable.forEach(Iterable.java:74) > at > org.apache.cassandra.db.lifecycle.LogReplicaSet.maybeCreateReplica(LogReplicaSet.java:78) > at org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:320) > at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:285) > at > org.apache.cassandra.db.lifecycle.LogTransaction.trackNew(LogTransaction.java:136) > at > org.apache.cassandra.db.lifecycle.LifecycleTransaction.trackNew(LifecycleTransaction.java:529) > {quote} > During streaming we create a reference to a {{LifeCycleTransaction}} and > share it between threads - > [https://github.com/apache/cassandra/blob/5cc68a87359dd02412bdb70a52dfcd718d44a5ba/src/java/org/apache/cassandra/db/streaming/CassandraStreamReader.java#L156] > This is used in a multi-threaded context inside {{CassandraIncomingFile}} > which is an {{IncomingStreamMessage}}. This is being deserialized in parallel. > {{LifecycleTransaction}} is not meant to be used in a multi-threaded context > and this leads to streaming failures due to object sharing. On trunk, this > object is shared across all threads that transfer sstables in parallel for > the given {{TableId}} in a {{StreamSession}}. There are two options to solve > this - make {{LifecycleTransaction}} and the associated objects thread safe, > scope the transaction to a single {{CassandraIncomingFile}}. The consequences > of the latter option is that if we experience streaming failure we may have > redundant SSTables on disk. This is ok as compaction should clean this up. A > third option is we synchronize access in the streaming infrastructure. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14926) OpenJDK - Using 1.8.0 "adoptopenjdk" throws error from command line
Aaron Ploetz created CASSANDRA-14926: Summary: OpenJDK - Using 1.8.0 "adoptopenjdk" throws error from command line Key: CASSANDRA-14926 URL: https://issues.apache.org/jira/browse/CASSANDRA-14926 Project: Cassandra Issue Type: Bug Components: Configuration Reporter: Aaron Ploetz Assignee: Aaron Ploetz Fix For: 4.x When testing with a recent build of 4.0, I noticed that Cassandra commands from my Mac (nodetool, bin/cassandra) was throwing a Bash syntax error: {{bin/cassandra.in.sh: line 115: [: 1.8.0: integer expression expected}} The offending line (115 in cassandra.in.sh) is this: {{if [ "$JVM_PATCH_VERSION" -lt 151 ] ; then}} Essentially, the patch version of the JDK is being parsed and then verified to be higher than 1.8.0_151. The value of $JVM_PATCH_VERSION is assumed to be numeric. Further investigation revealed this to be specific to the OpenJDK version on my Mac (our test CentOS clusters are fine). On CentOS, the JDK version reads like this: {{}}{{$ java -version openjdk version "1.8.0_191" OpenJDK Runtime Environment (build 1.8.0_191-b12) OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)}} {{}}But when I check the JDK version on my Mac, I see this: {{$ java -version openjdk version "1.8.0-adoptopenjdk" OpenJDK Runtime Environment (build 1.8.0-adoptopenjdk-jenkins_2018_05_19_02_01-b00) OpenJDK 64-Bit Server VM (build 25.71-b00, mixed mode)}} This output causes the $JVM_PATCH_VERSION to resolve to "1.8.0." Obviously not a valid integer. I messed around with adding an extra if-check prior to the line in-question: {{if [[ "$JVM_PATCH_VERSION" =~ ^[0-9]+$ ]] ; then}} This suppresses the error, but doesn't really validate an adequate patch level (which is the true goal here). The problem with this OpenJDK version, is that we'd have to take a different approach in obtaining the patch version. I wonder how prevalent the "adoptopenjdk" versions are out there to begin with, and if simply suppressing this error is fine for this edge case? Or is it worth the extra digging to pull in the patch version in this case? Note: This is not an issue prior to 4.0, due to the changes made in the cassandra.in.sh file. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[11/11] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/33eada06 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/33eada06 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/33eada06 Branch: refs/heads/trunk Commit: 33eada06a6dd3529da644377dba180795f522176 Parents: 4dd7faa 732c43b Author: Benedict Elliott Smith Authored: Mon Dec 10 15:07:18 2018 + Committer: Benedict Elliott Smith Committed: Mon Dec 10 15:07:18 2018 + -- CHANGES.txt | 1 + .../apache/cassandra/db/ColumnFamilyStore.java | 8 ++-- .../compaction/AbstractCompactionStrategy.java | 5 ++- .../db/compaction/AbstractStrategyHolder.java | 3 +- .../db/compaction/CompactionStrategyHolder.java | 5 ++- .../compaction/CompactionStrategyManager.java | 5 ++- .../db/compaction/PendingRepairHolder.java | 5 ++- .../db/lifecycle/LifecycleNewTracker.java | 47 .../db/lifecycle/LifecycleTransaction.java | 7 ++- .../apache/cassandra/db/lifecycle/LogFile.java | 24 -- .../cassandra/db/lifecycle/LogTransaction.java | 4 +- .../CassandraEntireSSTableStreamReader.java | 5 ++- .../db/streaming/CassandraStreamReader.java | 5 ++- .../db/streaming/CassandraStreamReceiver.java | 47 .../io/sstable/SimpleSSTableMultiWriter.java| 16 +++ .../sstable/format/RangeAwareSSTableWriter.java | 12 ++--- .../io/sstable/format/SSTableWriter.java| 20 - .../io/sstable/format/big/BigFormat.java| 6 +-- .../io/sstable/format/big/BigTableWriter.java | 6 +-- .../format/big/BigTableZeroCopyWriter.java | 5 ++- .../locator/AbstractReplicationStrategy.java| 2 +- .../cassandra/streaming/StreamReceiveTask.java | 2 +- .../unit/org/apache/cassandra/db/ScrubTest.java | 5 ++- 23 files changed, 166 insertions(+), 79 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/33eada06/CHANGES.txt -- diff --cc CHANGES.txt index 75b05ea,20da1fa..b8410b8 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -338,7 -2,9 +338,8 @@@ * Make stop-server.bat wait for Cassandra to terminate (CASSANDRA-14829) * Correct sstable sorting for garbagecollect and levelled compaction (CASSANDRA-14870) Merged from 3.0: + * Streaming needs to synchronise access to LifecycleTransaction (CASSANDRA-14554) * Fix cassandra-stress write hang with default options (CASSANDRA-14616) - * Differentiate between slices and RTs when decoding legacy bounds (CASSANDRA-14919) * Netty epoll IOExceptions caused by unclean client disconnects being logged at INFO (CASSANDRA-14909) * Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty (CASSANDRA-14588) * RangeTombstoneList doesn't properly clean up mergeable or superseded rts in some cases (CASSANDRA-14894) http://git-wip-us.apache.org/repos/asf/cassandra/blob/33eada06/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java index c5149cf,700c1cc..cc6af6f --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@@ -497,15 -507,15 +497,15 @@@ public class ColumnFamilyStore implemen return directories; } - public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, UUID pendingRepair, boolean isTransient, int sstableLevel, SerializationHeader header, LifecycleTransaction txn) -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) ++public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, UUID pendingRepair, boolean isTransient, int sstableLevel, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { -MetadataCollector collector = new MetadataCollector(metadata.comparator).sstableLevel(sstableLevel); -return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, lifecycleNewTracker); +MetadataCollector collector = new MetadataCollector(metadata().comparator).sstableLevel(sstableLevel); - return createSSTableMultiWriter(descriptor, keyCount, repairedAt, pendingRepair, isTransient, collector, header, txn); ++return createSSTableMultiWriter(descriptor, keyCount, repairedAt, pendingRepair, isTransient, collector, header, lifecycleNewTracker); } -
[jira] [Commented] (CASSANDRA-14554) Streaming needs to synchronise access to LifecycleTransaction
[ https://issues.apache.org/jira/browse/CASSANDRA-14554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714873#comment-16714873 ] Benedict commented on CASSANDRA-14554: -- Thanks [~Stefania]. I've committed to [84ffcb82a74667b957201f2cdae2d6b308956549|https://github.com/apache/cassandra/commit/84ffcb82a74667b957201f2cdae2d6b308956549] on 3.0 and merged upwards. > Streaming needs to synchronise access to LifecycleTransaction > - > > Key: CASSANDRA-14554 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14554 > Project: Cassandra > Issue Type: Bug > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Stefania >Priority: Major > Fix For: 4.0, 3.0.x, 3.11.x > > > When LifecycleTransaction is used in a multi-threaded context, we encounter > this exception - > {quote}java.util.ConcurrentModificationException: null > at > java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719) > at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:742) > at java.lang.Iterable.forEach(Iterable.java:74) > at > org.apache.cassandra.db.lifecycle.LogReplicaSet.maybeCreateReplica(LogReplicaSet.java:78) > at org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:320) > at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:285) > at > org.apache.cassandra.db.lifecycle.LogTransaction.trackNew(LogTransaction.java:136) > at > org.apache.cassandra.db.lifecycle.LifecycleTransaction.trackNew(LifecycleTransaction.java:529) > {quote} > During streaming we create a reference to a {{LifeCycleTransaction}} and > share it between threads - > [https://github.com/apache/cassandra/blob/5cc68a87359dd02412bdb70a52dfcd718d44a5ba/src/java/org/apache/cassandra/db/streaming/CassandraStreamReader.java#L156] > This is used in a multi-threaded context inside {{CassandraIncomingFile}} > which is an {{IncomingStreamMessage}}. This is being deserialized in parallel. > {{LifecycleTransaction}} is not meant to be used in a multi-threaded context > and this leads to streaming failures due to object sharing. On trunk, this > object is shared across all threads that transfer sstables in parallel for > the given {{TableId}} in a {{StreamSession}}. There are two options to solve > this - make {{LifecycleTransaction}} and the associated objects thread safe, > scope the transaction to a single {{CassandraIncomingFile}}. The consequences > of the latter option is that if we experience streaming failure we may have > redundant SSTables on disk. This is ok as compaction should clean this up. A > third option is we synchronize access in the streaming infrastructure. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[05/11] cassandra git commit: Streaming needs to synchronise access to LifecycleTransaction
Streaming needs to synchronise access to LifecycleTransaction patch by Stefania Alborghetti and Benedict; reviewed by Robert Stupp for CASSANDRA-14554 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84ffcb82 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84ffcb82 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84ffcb82 Branch: refs/heads/cassandra-3.11 Commit: 84ffcb82a74667b957201f2cdae2d6b308956549 Parents: bbf7dac Author: Stefania Alborghetti Authored: Wed Nov 7 14:07:12 2018 +0800 Committer: Benedict Elliott Smith Committed: Mon Dec 10 15:00:00 2018 + -- CHANGES.txt | 1 + .../apache/cassandra/db/ColumnFamilyStore.java | 8 ++-- .../compaction/AbstractCompactionStrategy.java | 6 +-- .../compaction/CompactionStrategyManager.java | 8 ++-- .../db/lifecycle/LifecycleNewTracker.java | 47 .../db/lifecycle/LifecycleTransaction.java | 7 ++- .../apache/cassandra/db/lifecycle/LogFile.java | 24 -- .../cassandra/db/lifecycle/LogTransaction.java | 2 +- .../io/sstable/SimpleSSTableMultiWriter.java| 6 +-- .../io/sstable/format/SSTableWriter.java| 24 +- .../io/sstable/format/big/BigFormat.java| 6 +-- .../io/sstable/format/big/BigTableWriter.java | 6 +-- .../cassandra/streaming/StreamReader.java | 6 +-- .../cassandra/streaming/StreamReceiveTask.java | 37 +-- .../cassandra/streaming/StreamSession.java | 4 +- .../unit/org/apache/cassandra/db/ScrubTest.java | 9 ++-- 16 files changed, 140 insertions(+), 61 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index f9b59df..01d4789 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.18 + * Streaming needs to synchronise access to LifecycleTransaction (CASSANDRA-14554) * Fix cassandra-stress write hang with default options (CASSANDRA-14616) * Differentiate between slices and RTs when decoding legacy bounds (CASSANDRA-14919) * CommitLogReplayer.handleReplayError should print stack traces (CASSANDRA-14589) http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 4c7bc46..c455c4c 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -475,15 +475,15 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean return directories; } -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleTransaction txn) +public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { MetadataCollector collector = new MetadataCollector(metadata.comparator).sstableLevel(sstableLevel); -return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, txn); +return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, lifecycleNewTracker); } -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleTransaction txn) +public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { -return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, txn); +return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, lifecycleNewTracker); } public boolean supportsEarlyOpen() http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java index a80a6f4..9f07691 100644 --- a/src/java/org/apache/ca
[03/11] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0918ba01 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0918ba01 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0918ba01 Branch: refs/heads/cassandra-3.11 Commit: 0918ba01176392deadf8655a61dad44979a49ee5 Parents: 6a449b8 3539a07 Author: Benedict Elliott Smith Authored: Fri Dec 7 12:27:08 2018 + Committer: Benedict Elliott Smith Committed: Fri Dec 7 12:27:08 2018 + -- .../apache/cassandra/db/ColumnFamilyStore.java | 8 ++-- .../compaction/AbstractCompactionStrategy.java | 5 ++- .../compaction/CompactionStrategyManager.java | 14 +++--- .../db/lifecycle/LifecycleNewTracker.java | 47 .../db/lifecycle/LifecycleTransaction.java | 7 ++- .../apache/cassandra/db/lifecycle/LogFile.java | 24 -- .../cassandra/db/lifecycle/LogTransaction.java | 2 +- .../io/sstable/SimpleSSTableMultiWriter.java| 16 +++ .../sstable/format/RangeAwareSSTableWriter.java | 12 ++--- .../io/sstable/format/SSTableWriter.java| 24 +- .../io/sstable/format/big/BigFormat.java| 6 +-- .../io/sstable/format/big/BigTableWriter.java | 6 +-- .../cassandra/streaming/StreamReader.java | 6 +-- .../cassandra/streaming/StreamReceiveTask.java | 35 ++- .../cassandra/streaming/StreamSession.java | 4 +- .../unit/org/apache/cassandra/db/ScrubTest.java | 5 ++- 16 files changed, 150 insertions(+), 71 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/0918ba01/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 13f0280,c455c4c..700c1cc --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@@ -507,15 -475,15 +507,15 @@@ public class ColumnFamilyStore implemen return directories; } - public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleTransaction txn) + public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { MetadataCollector collector = new MetadataCollector(metadata.comparator).sstableLevel(sstableLevel); - return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, txn); + return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, lifecycleNewTracker); } - public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleTransaction txn) + public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { - return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, indexManager.listIndexes(), txn); -return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, lifecycleNewTracker); ++return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, indexManager.listIndexes(), lifecycleNewTracker); } public boolean supportsEarlyOpen() http://git-wip-us.apache.org/repos/asf/cassandra/blob/0918ba01/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java -- diff --cc src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java index 8454147,9f07691..3d7800d --- a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java +++ b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java @@@ -26,7 -26,8 +26,8 @@@ import com.google.common.collect.Iterab import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.SerializationHeader; +import org.apache.cassandra.index.Index; + import org.apache.cassandra.db.lifecycle.LifecycleNewTracker; import org.apache.cassandra.io.sstable.Descriptor; import org.apache.cassandra.io.sstable.SSTableMultiWriter; import org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter; @@@ -571,20 -514,9 +572,2
[08/11] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/732c43b0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/732c43b0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/732c43b0 Branch: refs/heads/trunk Commit: 732c43b0f2dbc085c6e1da6498966fd656f32c2f Parents: 0918ba0 84ffcb8 Author: Benedict Elliott Smith Authored: Mon Dec 10 15:03:23 2018 + Committer: Benedict Elliott Smith Committed: Mon Dec 10 15:03:23 2018 + -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/732c43b0/CHANGES.txt -- diff --cc CHANGES.txt index 369d96e,01d4789..20da1fa --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,9 -1,8 +1,10 @@@ -3.0.18 +3.11.4 + * Make stop-server.bat wait for Cassandra to terminate (CASSANDRA-14829) + * Correct sstable sorting for garbagecollect and levelled compaction (CASSANDRA-14870) +Merged from 3.0: + * Streaming needs to synchronise access to LifecycleTransaction (CASSANDRA-14554) * Fix cassandra-stress write hang with default options (CASSANDRA-14616) * Differentiate between slices and RTs when decoding legacy bounds (CASSANDRA-14919) - * CommitLogReplayer.handleReplayError should print stack traces (CASSANDRA-14589) * Netty epoll IOExceptions caused by unclean client disconnects being logged at INFO (CASSANDRA-14909) * Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty (CASSANDRA-14588) * RangeTombstoneList doesn't properly clean up mergeable or superseded rts in some cases (CASSANDRA-14894) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[06/11] cassandra git commit: Streaming needs to synchronise access to LifecycleTransaction
Streaming needs to synchronise access to LifecycleTransaction patch by Stefania Alborghetti and Benedict; reviewed by Robert Stupp for CASSANDRA-14554 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84ffcb82 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84ffcb82 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84ffcb82 Branch: refs/heads/cassandra-3.0 Commit: 84ffcb82a74667b957201f2cdae2d6b308956549 Parents: bbf7dac Author: Stefania Alborghetti Authored: Wed Nov 7 14:07:12 2018 +0800 Committer: Benedict Elliott Smith Committed: Mon Dec 10 15:00:00 2018 + -- CHANGES.txt | 1 + .../apache/cassandra/db/ColumnFamilyStore.java | 8 ++-- .../compaction/AbstractCompactionStrategy.java | 6 +-- .../compaction/CompactionStrategyManager.java | 8 ++-- .../db/lifecycle/LifecycleNewTracker.java | 47 .../db/lifecycle/LifecycleTransaction.java | 7 ++- .../apache/cassandra/db/lifecycle/LogFile.java | 24 -- .../cassandra/db/lifecycle/LogTransaction.java | 2 +- .../io/sstable/SimpleSSTableMultiWriter.java| 6 +-- .../io/sstable/format/SSTableWriter.java| 24 +- .../io/sstable/format/big/BigFormat.java| 6 +-- .../io/sstable/format/big/BigTableWriter.java | 6 +-- .../cassandra/streaming/StreamReader.java | 6 +-- .../cassandra/streaming/StreamReceiveTask.java | 37 +-- .../cassandra/streaming/StreamSession.java | 4 +- .../unit/org/apache/cassandra/db/ScrubTest.java | 9 ++-- 16 files changed, 140 insertions(+), 61 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index f9b59df..01d4789 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.18 + * Streaming needs to synchronise access to LifecycleTransaction (CASSANDRA-14554) * Fix cassandra-stress write hang with default options (CASSANDRA-14616) * Differentiate between slices and RTs when decoding legacy bounds (CASSANDRA-14919) * CommitLogReplayer.handleReplayError should print stack traces (CASSANDRA-14589) http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 4c7bc46..c455c4c 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -475,15 +475,15 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean return directories; } -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleTransaction txn) +public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { MetadataCollector collector = new MetadataCollector(metadata.comparator).sstableLevel(sstableLevel); -return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, txn); +return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, lifecycleNewTracker); } -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleTransaction txn) +public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { -return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, txn); +return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, lifecycleNewTracker); } public boolean supportsEarlyOpen() http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java index a80a6f4..9f07691 100644 --- a/src/java/org/apache/cas
[01/11] cassandra git commit: Prevent concurrent access to lifecycle txn when streaming
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 bbf7dac87 -> 84ffcb82a refs/heads/cassandra-3.11 6a449b88d -> 732c43b0f refs/heads/trunk 4dd7faa75 -> 33eada06a Prevent concurrent access to lifecycle txn when streaming patch by Stefania Alborghetti; reviewed by Robert Stupp for CASSANDRA-14554 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3539a078 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3539a078 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3539a078 Branch: refs/heads/cassandra-3.11 Commit: 3539a07884d0a94c1432d7dda0ec17676195cc53 Parents: bbf7dac Author: Stefania Alborghetti Authored: Wed Nov 7 14:07:12 2018 +0800 Committer: Benedict Elliott Smith Committed: Fri Dec 7 12:18:05 2018 + -- .../apache/cassandra/db/ColumnFamilyStore.java | 8 ++-- .../compaction/AbstractCompactionStrategy.java | 6 +-- .../compaction/CompactionStrategyManager.java | 8 ++-- .../db/lifecycle/LifecycleNewTracker.java | 47 .../db/lifecycle/LifecycleTransaction.java | 7 ++- .../apache/cassandra/db/lifecycle/LogFile.java | 24 -- .../cassandra/db/lifecycle/LogTransaction.java | 2 +- .../io/sstable/SimpleSSTableMultiWriter.java| 6 +-- .../io/sstable/format/SSTableWriter.java| 24 +- .../io/sstable/format/big/BigFormat.java| 6 +-- .../io/sstable/format/big/BigTableWriter.java | 6 +-- .../cassandra/streaming/StreamReader.java | 6 +-- .../cassandra/streaming/StreamReceiveTask.java | 37 +-- .../cassandra/streaming/StreamSession.java | 4 +- .../unit/org/apache/cassandra/db/ScrubTest.java | 9 ++-- 15 files changed, 139 insertions(+), 61 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3539a078/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 4c7bc46..c455c4c 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -475,15 +475,15 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean return directories; } -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleTransaction txn) +public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { MetadataCollector collector = new MetadataCollector(metadata.comparator).sstableLevel(sstableLevel); -return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, txn); +return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, lifecycleNewTracker); } -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleTransaction txn) +public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { -return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, txn); +return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, lifecycleNewTracker); } public boolean supportsEarlyOpen() http://git-wip-us.apache.org/repos/asf/cassandra/blob/3539a078/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java index a80a6f4..9f07691 100644 --- a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java +++ b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java @@ -27,7 +27,7 @@ import com.google.common.util.concurrent.RateLimiter; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.SerializationHeader; -import org.apache.cassandra.db.lifecycle.SSTableSet; +import org.apache.cassandra.db.lifecycle.LifecycleNewTracker; import org.apache.cassandra.io.sstable.Descriptor; import org.apache.ca
[02/11] cassandra git commit: Prevent concurrent access to lifecycle txn when streaming
Prevent concurrent access to lifecycle txn when streaming patch by Stefania Alborghetti; reviewed by Robert Stupp for CASSANDRA-14554 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3539a078 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3539a078 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3539a078 Branch: refs/heads/trunk Commit: 3539a07884d0a94c1432d7dda0ec17676195cc53 Parents: bbf7dac Author: Stefania Alborghetti Authored: Wed Nov 7 14:07:12 2018 +0800 Committer: Benedict Elliott Smith Committed: Fri Dec 7 12:18:05 2018 + -- .../apache/cassandra/db/ColumnFamilyStore.java | 8 ++-- .../compaction/AbstractCompactionStrategy.java | 6 +-- .../compaction/CompactionStrategyManager.java | 8 ++-- .../db/lifecycle/LifecycleNewTracker.java | 47 .../db/lifecycle/LifecycleTransaction.java | 7 ++- .../apache/cassandra/db/lifecycle/LogFile.java | 24 -- .../cassandra/db/lifecycle/LogTransaction.java | 2 +- .../io/sstable/SimpleSSTableMultiWriter.java| 6 +-- .../io/sstable/format/SSTableWriter.java| 24 +- .../io/sstable/format/big/BigFormat.java| 6 +-- .../io/sstable/format/big/BigTableWriter.java | 6 +-- .../cassandra/streaming/StreamReader.java | 6 +-- .../cassandra/streaming/StreamReceiveTask.java | 37 +-- .../cassandra/streaming/StreamSession.java | 4 +- .../unit/org/apache/cassandra/db/ScrubTest.java | 9 ++-- 15 files changed, 139 insertions(+), 61 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3539a078/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 4c7bc46..c455c4c 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -475,15 +475,15 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean return directories; } -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleTransaction txn) +public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { MetadataCollector collector = new MetadataCollector(metadata.comparator).sstableLevel(sstableLevel); -return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, txn); +return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, lifecycleNewTracker); } -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleTransaction txn) +public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { -return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, txn); +return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, lifecycleNewTracker); } public boolean supportsEarlyOpen() http://git-wip-us.apache.org/repos/asf/cassandra/blob/3539a078/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java index a80a6f4..9f07691 100644 --- a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java +++ b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java @@ -27,7 +27,7 @@ import com.google.common.util.concurrent.RateLimiter; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.SerializationHeader; -import org.apache.cassandra.db.lifecycle.SSTableSet; +import org.apache.cassandra.db.lifecycle.LifecycleNewTracker; import org.apache.cassandra.io.sstable.Descriptor; import org.apache.cassandra.io.sstable.SSTableMultiWriter; import org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter; @@ -514,9 +514,9 @@ public abstract class AbstractCompactionStrategy return grou
[10/11] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
http://git-wip-us.apache.org/repos/asf/cassandra/blob/33eada06/src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java -- diff --cc src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java index cca59cf,e320f30..f54bc03 --- a/src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java +++ b/src/java/org/apache/cassandra/io/sstable/format/SSTableWriter.java @@@ -104,10 -96,10 +104,10 @@@ public abstract class SSTableWriter ext MetadataCollector metadataCollector, SerializationHeader header, Collection indexes, -LifecycleTransaction txn) +LifecycleNewTracker lifecycleNewTracker) { Factory writerFactory = descriptor.getFormat().getWriterFactory(); - return writerFactory.open(descriptor, keyCount, repairedAt, pendingRepair, isTransient, metadata, metadataCollector, header, observers(descriptor, indexes, txn.opType()), txn); -return writerFactory.open(descriptor, keyCount, repairedAt, metadata, metadataCollector, header, observers(descriptor, indexes, lifecycleNewTracker.opType()), lifecycleNewTracker); ++return writerFactory.open(descriptor, keyCount, repairedAt, pendingRepair, isTransient, metadata, metadataCollector, header, observers(descriptor, indexes, lifecycleNewTracker.opType()), lifecycleNewTracker); } public static SSTableWriter create(Descriptor descriptor, @@@ -118,13 -108,13 +118,13 @@@ int sstableLevel, SerializationHeader header, Collection indexes, -LifecycleTransaction txn) +LifecycleNewTracker lifecycleNewTracker) { -CFMetaData metadata = Schema.instance.getCFMetaData(descriptor); -return create(metadata, descriptor, keyCount, repairedAt, sstableLevel, header, indexes, lifecycleNewTracker); +TableMetadataRef metadata = Schema.instance.getTableMetadataRef(descriptor); - return create(metadata, descriptor, keyCount, repairedAt, pendingRepair, isTransient, sstableLevel, header, indexes, txn); ++return create(metadata, descriptor, keyCount, repairedAt, pendingRepair, isTransient, sstableLevel, header, indexes, lifecycleNewTracker); } -public static SSTableWriter create(CFMetaData metadata, +public static SSTableWriter create(TableMetadataRef metadata, Descriptor descriptor, long keyCount, long repairedAt, @@@ -133,26 -121,36 +133,26 @@@ int sstableLevel, SerializationHeader header, Collection indexes, -LifecycleTransaction txn) +LifecycleNewTracker lifecycleNewTracker) { -MetadataCollector collector = new MetadataCollector(metadata.comparator).sstableLevel(sstableLevel); -return create(descriptor, keyCount, repairedAt, metadata, collector, header, indexes, lifecycleNewTracker); -} - -public static SSTableWriter create(String filename, - long keyCount, - long repairedAt, - int sstableLevel, - SerializationHeader header, - Collection indexes, - LifecycleNewTracker lifecycleNewTracker) -{ -return create(Descriptor.fromFilename(filename), keyCount, repairedAt, sstableLevel, header, indexes, lifecycleNewTracker); +MetadataCollector collector = new MetadataCollector(metadata.get().comparator).sstableLevel(sstableLevel); - return create(descriptor, keyCount, repairedAt, pendingRepair, isTransient, metadata, collector, header, indexes, txn); ++return create(descriptor, keyCount, repairedAt, pendingRepair, isTransient, metadata, collector, header, indexes, lifecycleNewTracker); } @VisibleForTesting -public static SSTableWriter create(String filename, +public static SSTableWriter create(Descriptor descriptor, long keyCount, long repairedAt, + UUID pendingRepair, + boolean isTransient, SerializationHeader header,
[07/11] cassandra git commit: Streaming needs to synchronise access to LifecycleTransaction
Streaming needs to synchronise access to LifecycleTransaction patch by Stefania Alborghetti and Benedict; reviewed by Robert Stupp for CASSANDRA-14554 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/84ffcb82 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/84ffcb82 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/84ffcb82 Branch: refs/heads/trunk Commit: 84ffcb82a74667b957201f2cdae2d6b308956549 Parents: bbf7dac Author: Stefania Alborghetti Authored: Wed Nov 7 14:07:12 2018 +0800 Committer: Benedict Elliott Smith Committed: Mon Dec 10 15:00:00 2018 + -- CHANGES.txt | 1 + .../apache/cassandra/db/ColumnFamilyStore.java | 8 ++-- .../compaction/AbstractCompactionStrategy.java | 6 +-- .../compaction/CompactionStrategyManager.java | 8 ++-- .../db/lifecycle/LifecycleNewTracker.java | 47 .../db/lifecycle/LifecycleTransaction.java | 7 ++- .../apache/cassandra/db/lifecycle/LogFile.java | 24 -- .../cassandra/db/lifecycle/LogTransaction.java | 2 +- .../io/sstable/SimpleSSTableMultiWriter.java| 6 +-- .../io/sstable/format/SSTableWriter.java| 24 +- .../io/sstable/format/big/BigFormat.java| 6 +-- .../io/sstable/format/big/BigTableWriter.java | 6 +-- .../cassandra/streaming/StreamReader.java | 6 +-- .../cassandra/streaming/StreamReceiveTask.java | 37 +-- .../cassandra/streaming/StreamSession.java | 4 +- .../unit/org/apache/cassandra/db/ScrubTest.java | 9 ++-- 16 files changed, 140 insertions(+), 61 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index f9b59df..01d4789 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 3.0.18 + * Streaming needs to synchronise access to LifecycleTransaction (CASSANDRA-14554) * Fix cassandra-stress write hang with default options (CASSANDRA-14616) * Differentiate between slices and RTs when decoding legacy bounds (CASSANDRA-14919) * CommitLogReplayer.handleReplayError should print stack traces (CASSANDRA-14589) http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 4c7bc46..c455c4c 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -475,15 +475,15 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean return directories; } -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleTransaction txn) +public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { MetadataCollector collector = new MetadataCollector(metadata.comparator).sstableLevel(sstableLevel); -return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, txn); +return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, lifecycleNewTracker); } -public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleTransaction txn) +public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { -return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, txn); +return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, lifecycleNewTracker); } public boolean supportsEarlyOpen() http://git-wip-us.apache.org/repos/asf/cassandra/blob/84ffcb82/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java -- diff --git a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java index a80a6f4..9f07691 100644 --- a/src/java/org/apache/cassandra/d
[09/11] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/732c43b0 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/732c43b0 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/732c43b0 Branch: refs/heads/cassandra-3.11 Commit: 732c43b0f2dbc085c6e1da6498966fd656f32c2f Parents: 0918ba0 84ffcb8 Author: Benedict Elliott Smith Authored: Mon Dec 10 15:03:23 2018 + Committer: Benedict Elliott Smith Committed: Mon Dec 10 15:03:23 2018 + -- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/732c43b0/CHANGES.txt -- diff --cc CHANGES.txt index 369d96e,01d4789..20da1fa --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,9 -1,8 +1,10 @@@ -3.0.18 +3.11.4 + * Make stop-server.bat wait for Cassandra to terminate (CASSANDRA-14829) + * Correct sstable sorting for garbagecollect and levelled compaction (CASSANDRA-14870) +Merged from 3.0: + * Streaming needs to synchronise access to LifecycleTransaction (CASSANDRA-14554) * Fix cassandra-stress write hang with default options (CASSANDRA-14616) * Differentiate between slices and RTs when decoding legacy bounds (CASSANDRA-14919) - * CommitLogReplayer.handleReplayError should print stack traces (CASSANDRA-14589) * Netty epoll IOExceptions caused by unclean client disconnects being logged at INFO (CASSANDRA-14909) * Unfiltered.isEmpty conflicts with Row extends AbstractCollection.isEmpty (CASSANDRA-14588) * RangeTombstoneList doesn't properly clean up mergeable or superseded rts in some cases (CASSANDRA-14894) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[04/11] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0918ba01 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0918ba01 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0918ba01 Branch: refs/heads/trunk Commit: 0918ba01176392deadf8655a61dad44979a49ee5 Parents: 6a449b8 3539a07 Author: Benedict Elliott Smith Authored: Fri Dec 7 12:27:08 2018 + Committer: Benedict Elliott Smith Committed: Fri Dec 7 12:27:08 2018 + -- .../apache/cassandra/db/ColumnFamilyStore.java | 8 ++-- .../compaction/AbstractCompactionStrategy.java | 5 ++- .../compaction/CompactionStrategyManager.java | 14 +++--- .../db/lifecycle/LifecycleNewTracker.java | 47 .../db/lifecycle/LifecycleTransaction.java | 7 ++- .../apache/cassandra/db/lifecycle/LogFile.java | 24 -- .../cassandra/db/lifecycle/LogTransaction.java | 2 +- .../io/sstable/SimpleSSTableMultiWriter.java| 16 +++ .../sstable/format/RangeAwareSSTableWriter.java | 12 ++--- .../io/sstable/format/SSTableWriter.java| 24 +- .../io/sstable/format/big/BigFormat.java| 6 +-- .../io/sstable/format/big/BigTableWriter.java | 6 +-- .../cassandra/streaming/StreamReader.java | 6 +-- .../cassandra/streaming/StreamReceiveTask.java | 35 ++- .../cassandra/streaming/StreamSession.java | 4 +- .../unit/org/apache/cassandra/db/ScrubTest.java | 5 ++- 16 files changed, 150 insertions(+), 71 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/0918ba01/src/java/org/apache/cassandra/db/ColumnFamilyStore.java -- diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 13f0280,c455c4c..700c1cc --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@@ -507,15 -475,15 +507,15 @@@ public class ColumnFamilyStore implemen return directories; } - public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleTransaction txn) + public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, int sstableLevel, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { MetadataCollector collector = new MetadataCollector(metadata.comparator).sstableLevel(sstableLevel); - return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, txn); + return createSSTableMultiWriter(descriptor, keyCount, repairedAt, collector, header, lifecycleNewTracker); } - public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleTransaction txn) + public SSTableMultiWriter createSSTableMultiWriter(Descriptor descriptor, long keyCount, long repairedAt, MetadataCollector metadataCollector, SerializationHeader header, LifecycleNewTracker lifecycleNewTracker) { - return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, indexManager.listIndexes(), txn); -return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, lifecycleNewTracker); ++return getCompactionStrategyManager().createSSTableMultiWriter(descriptor, keyCount, repairedAt, metadataCollector, header, indexManager.listIndexes(), lifecycleNewTracker); } public boolean supportsEarlyOpen() http://git-wip-us.apache.org/repos/asf/cassandra/blob/0918ba01/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java -- diff --cc src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java index 8454147,9f07691..3d7800d --- a/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java +++ b/src/java/org/apache/cassandra/db/compaction/AbstractCompactionStrategy.java @@@ -26,7 -26,8 +26,8 @@@ import com.google.common.collect.Iterab import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.SerializationHeader; +import org.apache.cassandra.index.Index; + import org.apache.cassandra.db.lifecycle.LifecycleNewTracker; import org.apache.cassandra.io.sstable.Descriptor; import org.apache.cassandra.io.sstable.SSTableMultiWriter; import org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter; @@@ -571,20 -514,9 +572,20 @@@ pub
[jira] [Updated] (CASSANDRA-14554) Streaming needs to synchronise access to LifecycleTransaction
[ https://issues.apache.org/jira/browse/CASSANDRA-14554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-14554: - Summary: Streaming needs to synchronise access to LifecycleTransaction (was: LifecycleTransaction encounters ConcurrentModificationException when used in multi-threaded context) > Streaming needs to synchronise access to LifecycleTransaction > - > > Key: CASSANDRA-14554 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14554 > Project: Cassandra > Issue Type: Bug > Components: Streaming and Messaging >Reporter: Dinesh Joshi >Assignee: Stefania >Priority: Major > Fix For: 4.0, 3.0.x, 3.11.x > > > When LifecycleTransaction is used in a multi-threaded context, we encounter > this exception - > {quote}java.util.ConcurrentModificationException: null > at > java.util.LinkedHashMap$LinkedHashIterator.nextNode(LinkedHashMap.java:719) > at java.util.LinkedHashMap$LinkedKeyIterator.next(LinkedHashMap.java:742) > at java.lang.Iterable.forEach(Iterable.java:74) > at > org.apache.cassandra.db.lifecycle.LogReplicaSet.maybeCreateReplica(LogReplicaSet.java:78) > at org.apache.cassandra.db.lifecycle.LogFile.makeRecord(LogFile.java:320) > at org.apache.cassandra.db.lifecycle.LogFile.add(LogFile.java:285) > at > org.apache.cassandra.db.lifecycle.LogTransaction.trackNew(LogTransaction.java:136) > at > org.apache.cassandra.db.lifecycle.LifecycleTransaction.trackNew(LifecycleTransaction.java:529) > {quote} > During streaming we create a reference to a {{LifeCycleTransaction}} and > share it between threads - > [https://github.com/apache/cassandra/blob/5cc68a87359dd02412bdb70a52dfcd718d44a5ba/src/java/org/apache/cassandra/db/streaming/CassandraStreamReader.java#L156] > This is used in a multi-threaded context inside {{CassandraIncomingFile}} > which is an {{IncomingStreamMessage}}. This is being deserialized in parallel. > {{LifecycleTransaction}} is not meant to be used in a multi-threaded context > and this leads to streaming failures due to object sharing. On trunk, this > object is shared across all threads that transfer sstables in parallel for > the given {{TableId}} in a {{StreamSession}}. There are two options to solve > this - make {{LifecycleTransaction}} and the associated objects thread safe, > scope the transaction to a single {{CassandraIncomingFile}}. The consequences > of the latter option is that if we experience streaming failure we may have > redundant SSTables on disk. This is ok as compaction should clean this up. A > third option is we synchronize access in the streaming infrastructure. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14871) Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser
[ https://issues.apache.org/jira/browse/CASSANDRA-14871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714681#comment-16714681 ] Robert Stupp commented on CASSANDRA-14871: -- I'm fine with using {{Verify}}, but it should include the string that yielded {{null}}. Like {{Verify.verify(type != null, "Parsing %s yielded null, which is a bug", type)}} and move that after the assignment of {{type}}. That way we don't need the {{cache.get(str)}} in the synchronized block, because {{type}} is then guaranteed to be non-null. {code:java} synchronized (TypeParser.class) { if (!cache.containsKey(str)) { ImmutableMap.Builder> builder = ImmutableMap.builder(); builder.putAll(cache).put(str, type); cache = builder.build(); } AbstractType rtype = cache.get(str); Verify.verify(rtype != null); return rtype; } {code} > Severe concurrency issues in STCS,DTCS,TWCS,TMD.Topology,TypeParser > --- > > Key: CASSANDRA-14871 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14871 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: Robert Stupp >Assignee: Robert Stupp >Priority: Critical > Fix For: 4.0, 3.0.x, 3.11.x > > > There are a couple of places in the code base that do not respect that > j.u.HashMap + related classes are not thread safe and some parts rely on > internals of the implementation of HM, which can change. > We have observed failures like {{NullPointerException}} and > {{ConcurrentModificationException}} as well as wrong behavior. > Affected areas in the code base: > * {{SizeTieredCompactionStrategy}} > * {{DateTieredCompactionStrategy}} > * {{TimeWindowCompactionStrategy}} > * {{TokenMetadata.Topology}} > * {{TypeParser}} > * streaming / concurrent access to {{LifecycleTransaction}} (handled in > CASSANDRA-14554) > While the patches for the compaction strategies + {{TypeParser}} are pretty > straight forward, the patch for {{TokenMetadata.Topology}} requires it to be > made immutable. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14925) DecimalSerializer.toString() can be used as OOM attack
[ https://issues.apache.org/jira/browse/CASSANDRA-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714659#comment-16714659 ] ZhaoYang commented on CASSANDRA-14925: -- {quote}Can't we just use BigDecimal.toString() all the time as save ourselves the trouble of adding yet one more runtime parameter that no user will probably ever modify?{quote} Make sense.. updated the patch to use `toString()` only. |patch|circle-ci| |[3.0|https://github.com/jasonstack/cassandra/commits/decimal-tostring-3.0]|[unit|https://circleci.com/gh/jasonstack/cassandra/747?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link]| |[3.11|https://github.com/jasonstack/cassandra/commits/decimal-tostring-3.11]|[unit|https://circleci.com/gh/jasonstack/cassandra/752?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link]| |[trunk|https://github.com/jasonstack/cassandra/commits/decimal-tostring-trunk]|[unit|https://circleci.com/gh/jasonstack/cassandra/751?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link]| > DecimalSerializer.toString() can be used as OOM attack > --- > > Key: CASSANDRA-14925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14925 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Minor > > Currently, in {{DecimalSerializer.toString(value)}}, it uses > {{BigDecimal.toPlainString()}} which generates huge string for large scale > values. > > {code:java} > BigDecimal d = new BigDecimal("1e-" + (Integer.MAX_VALUE - 6)); > d.toPlainString(); // oom{code} > > Propose to use {{BigDecimal.toString()}} when scale is larger than 100 which > is configurable via {{-Dcassandra.decimal.maxscaleforstring}} > > | patch | circle-ci | > | [3.0|https://github.com/jasonstack/cassandra/commits/decimal-tostring-3.0] > | > [unit|https://circleci.com/gh/jasonstack/cassandra/747?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link] > | > The code should apply cleanly to 3.0+. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14482) ZSTD Compressor support in Cassandra
[ https://issues.apache.org/jira/browse/CASSANDRA-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714590#comment-16714590 ] Benedict commented on CASSANDRA-14482: -- [~jolynch]: I know we've talked about this before, but once 4.0 freeze is over I'd be happy to help you get this dictionary approach merged into mainline (and help you figure out any issues with your interactions with early-open - presumably the issue is with sharing ownership of the new dictionary). I'm not sure it would strictly be necessary to wait until 5.0 for this, as while it certainly breaks _my_ interpretation of the freeze, it's probably not going to be a _dramatic_ change. It could be quite localised, and might be acceptable for a patch release, depending on how we approach our future release processes. > ZSTD Compressor support in Cassandra > > > Key: CASSANDRA-14482 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14482 > Project: Cassandra > Issue Type: New Feature > Components: Compression, Libraries >Reporter: Sushma A Devendrappa >Assignee: Sushma A Devendrappa >Priority: Major > Labels: performance, pull-request-available > Fix For: 4.x > > Time Spent: 1h 10m > Remaining Estimate: 0h > > ZStandard has a great speed and compression ratio tradeoff. > ZStandard is open source compression from Facebook. > More about ZSTD > [https://github.com/facebook/zstd] > https://code.facebook.com/posts/1658392934479273/smaller-and-faster-data-compression-with-zstandard/ > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14925) DecimalSerializer.toString() can be used as OOM attack
[ https://issues.apache.org/jira/browse/CASSANDRA-14925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714525#comment-16714525 ] Sylvain Lebresne commented on CASSANDRA-14925: -- Can't we just use {{BigDecimal.toString()}} all the time as save ourselves the trouble of adding yet one more runtime parameter that no user will probably ever modify? > DecimalSerializer.toString() can be used as OOM attack > --- > > Key: CASSANDRA-14925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14925 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: ZhaoYang >Assignee: ZhaoYang >Priority: Minor > > Currently, in {{DecimalSerializer.toString(value)}}, it uses > {{BigDecimal.toPlainString()}} which generates huge string for large scale > values. > > {code:java} > BigDecimal d = new BigDecimal("1e-" + (Integer.MAX_VALUE - 6)); > d.toPlainString(); // oom{code} > > Propose to use {{BigDecimal.toString()}} when scale is larger than 100 which > is configurable via {{-Dcassandra.decimal.maxscaleforstring}} > > | patch | circle-ci | > | [3.0|https://github.com/jasonstack/cassandra/commits/decimal-tostring-3.0] > | > [unit|https://circleci.com/gh/jasonstack/cassandra/747?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link] > | > The code should apply cleanly to 3.0+. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org