[jira] [Assigned] (CASSANDRA-15607) Incorrect Outcome returned when acquiring capacity for incoming message
[
https://issues.apache.org/jira/browse/CASSANDRA-15607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko reassigned CASSANDRA-15607:
-
Assignee: Aleksey Yeschenko
> Incorrect Outcome returned when acquiring capacity for incoming message
> ---
>
> Key: CASSANDRA-15607
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15607
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Sam Tunnicliffe
>Assignee: Aleksey Yeschenko
>Priority: Normal
>
> When acquiring capacity to process an inbound internode message, a failure to
> allocate from the endpoint-specific reserve returns the wrong {{Outcome}}.
> This means we only ever register with {{globalWaitQueue}}, although this
> probably doesn't actually ever get detected as the global and endpoint queues
> are always signalled together.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15271) CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE"
[ https://issues.apache.org/jira/browse/CASSANDRA-15271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15271: -- Fix Version/s: (was: 4.0) 4.0-alpha Since Version: 4.0-alpha Source Control Link: [1f242c900bb087770a4cffeade6986ef19ba303d|https://github.com/apache/cassandra/commit/1f242c900bb087770a4cffeade6986ef19ba303d] Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed with some minor touch-ups and nomenclature correction (we don't have clustering key or clustering key columns anymore, the preferred term is just "clustering columns"). > CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE" > > > Key: CASSANDRA-15271 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15271 > Project: Cassandra > Issue Type: Bug > Components: CQL/Syntax >Reporter: Jeremiah Jordan >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-alpha > > Attachments: CASSANDRA-15271-testall.txt, CASSANDRA-15271.zip, Screen > Shot 2020-02-26 at 11.25.29 AM.png > > > CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE", > it now complains if you don't specify all the columns. > It was nice that previously you could just specify to make the first > clustering DESC and leave the rest ASC without needing to specify them. Also > it would be nice I think to avoid breaking changes to the CREATE TABLE syntax. > We should either update NEWS.txt to call out the breaking change there, or > update the new code to be able to default the columns which were not > mentioned. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15271) CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE"
[ https://issues.apache.org/jira/browse/CASSANDRA-15271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15271: -- Status: Ready to Commit (was: Review In Progress) > CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE" > > > Key: CASSANDRA-15271 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15271 > Project: Cassandra > Issue Type: Bug > Components: CQL/Syntax >Reporter: Jeremiah Jordan >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0 > > Attachments: CASSANDRA-15271-testall.txt, CASSANDRA-15271.zip, Screen > Shot 2020-02-26 at 11.25.29 AM.png > > > CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE", > it now complains if you don't specify all the columns. > It was nice that previously you could just specify to make the first > clustering DESC and leave the rest ASC without needing to specify them. Also > it would be nice I think to avoid breaking changes to the CREATE TABLE syntax. > We should either update NEWS.txt to call out the breaking change there, or > update the new code to be able to default the columns which were not > mentioned. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15271) CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE"
[ https://issues.apache.org/jira/browse/CASSANDRA-15271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15271: -- Reviewers: Aleksey Yeschenko, Aleksey Yeschenko (was: Aleksey Yeschenko) Aleksey Yeschenko, Aleksey Yeschenko (was: Aleksey Yeschenko) Status: Review In Progress (was: Patch Available) > CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE" > > > Key: CASSANDRA-15271 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15271 > Project: Cassandra > Issue Type: Bug > Components: CQL/Syntax >Reporter: Jeremiah Jordan >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0 > > Attachments: CASSANDRA-15271-testall.txt, CASSANDRA-15271.zip, Screen > Shot 2020-02-26 at 11.25.29 AM.png > > > CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE", > it now complains if you don't specify all the columns. > It was nice that previously you could just specify to make the first > clustering DESC and leave the rest ASC without needing to specify them. Also > it would be nice I think to avoid breaking changes to the CREATE TABLE syntax. > We should either update NEWS.txt to call out the breaking change there, or > update the new code to be able to default the columns which were not > mentioned. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15271) CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE"
[ https://issues.apache.org/jira/browse/CASSANDRA-15271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15271: -- Reviewers: Aleksey Yeschenko > CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE" > > > Key: CASSANDRA-15271 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15271 > Project: Cassandra > Issue Type: Bug > Components: CQL/Syntax >Reporter: Jeremiah Jordan >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0 > > Attachments: CASSANDRA-15271-testall.txt, CASSANDRA-15271.zip, Screen > Shot 2020-02-26 at 11.25.29 AM.png > > > CASSANDRA-13426 changed the behavior of "CLUSTERING ORDER" on "CREATE TABLE", > it now complains if you don't specify all the columns. > It was nice that previously you could just specify to make the first > clustering DESC and leave the rest ASC without needing to specify them. Also > it would be nice I think to avoid breaking changes to the CREATE TABLE syntax. > We should either update NEWS.txt to call out the breaking change there, or > update the new code to be able to default the columns which were not > mentioned. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15598) cassandra-stress in user profile mode fails on handling set> columns
[ https://issues.apache.org/jira/browse/CASSANDRA-15598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko reassigned CASSANDRA-15598: - Assignee: (was: Aleksey Yeschenko) > cassandra-stress in user profile mode fails on handling set> columns > - > > Key: CASSANDRA-15598 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15598 > Project: Cassandra > Issue Type: Bug >Reporter: Dmitry Kropachev >Priority: Normal > Attachments: cs_no_mv_basic_profile.zip > > > Reason: > c-s reads metadata from database, and build column generator from it. > But this logic is build in a way so that it does not support generator > recursion, which is necessary to support embedded collections. > Fix: > This fix is simple exclude this type of columns from being processed. > Steps to reproduce: > # docker run -d --name cassandra cassandra:latest > # cassandra-stress user profile=cs_no_mv_basic_profile.yaml ops'(insert=1)' > cl=ONE n=20 -pop seq=1..1000 -port jmx=6868 -mode cql3 native -rate > threads=2 -node 172.17.0.2 > # docker exec -ti cassandra cqlsh -e 'ALTER TABLE mview.users ADD ( > asdasdasd set>);' > # cassandra-stress user profile=cs_no_mv_basic_profile.yaml ops'(insert=1)' > cl=ONE n=20 -pop seq=1..1000 -port jmx=6868 -mode cql3 native -rate > threads=2 -node 172.17.0.2 > Result: > Extra Schema Definitions: null > Generator Configs: > password: Size: Fixed: key=80; > last_name: Size: Uniform: min=1,max=32; > id: Size: Uniform: min=1,max=10; > first_name: Size: Fixed: key=16; > email: Size: Uniform: min=16,max=50; > username: Size: Uniform: min=10,max=30; > Query Definitions: > read1: CQL:select * from mview.users where id = ? LIMIT 10;Fields:samerow; > Token Range Queries: > Insert Settings: > partitions: fixed(1) > batchtype: UNLOGGED > Connected to cluster: , max pending requests per connection 128, max > connections per host 8 > Datacenter: datacenter1; Host: /172.17.0.2:9042; Rack: rack1 > Created schema. Sleeping 1s for propagation. > java.lang.NullPointerException > at > org.apache.cassandra.stress.StressProfile$ColumnInfo.getGenerator(StressProfile.java:760) > at > org.apache.cassandra.stress.StressProfile$ColumnInfo.getGenerator(StressProfile.java:800) > at > org.apache.cassandra.stress.StressProfile$ColumnInfo.getGenerator(StressProfile.java:800) > at > org.apache.cassandra.stress.StressProfile$ColumnInfo.getGenerator(StressProfile.java:755) > at > org.apache.cassandra.stress.StressProfile$GeneratorFactory.get(StressProfile.java:733) > at > org.apache.cassandra.stress.StressProfile$GeneratorFactory.newGenerator(StressProfile.java:726) > at > org.apache.cassandra.stress.StressProfile.newGenerator(StressProfile.java:696) > at > org.apache.cassandra.stress.StressProfile.printSettings(StressProfile.java:126) > at > org.apache.cassandra.stress.settings.StressSettings.lambda$printSettings$1(StressSettings.java:311) > at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684) > at > org.apache.cassandra.stress.settings.StressSettings.printSettings(StressSettings.java:311) > at org.apache.cassandra.stress.Stress.run(Stress.java:95) > at org.apache.cassandra.stress.Stress.main(Stress.java:62) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021353#comment-17021353
]
Aleksey Yeschenko commented on CASSANDRA-15499:
---
Committed as
[0e3a90698a94772e57df39e7461efe6b7e09d678|https://github.com/apache/cassandra/commit/0e3a90698a94772e57df39e7461efe6b7e09d678],
cheers.
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.x
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15499:
--
Fix Version/s: (was: 4.x)
4.0-alpha
Since Version: 4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/0e3a90698a94772e57df39e7461efe6b7e09d678
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021248#comment-17021248
]
Aleksey Yeschenko commented on CASSANDRA-15499:
---
+1 from me. When and if David approves, I'll commit for you guys.
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.x
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[ https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021091#comment-17021091 ] Aleksey Yeschenko commented on CASSANDRA-14781: --- I'm perfectly fine with this level of duplication. Don't mind a separate ticket, either. Although FWIW that other ticket is the important one - we should never ever get to the point when an oversized mutation makes it to the commitlog at all, outside of potentially boundary mixed mode conditions - when a mutation validates for the current messaging version but ends up being slightly over the size for legacy. > Log message when mutation passed to CommitLog#add(Mutation) is too large is > not descriptive enough > -- > > Key: CASSANDRA-14781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14781 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Hints, Local/Commit Log, Messaging/Client >Reporter: Jordan West >Assignee: Tom Petracca >Priority: Normal > Labels: protocolv5 > Fix For: 4.0-beta > > Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch, > CASSANDRA-14781_3.11.patch > > > When hitting > [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257], > the log message produced does not help the operator track down what data is > being written. At a minimum the keyspace and cfIds involved would be useful > (and are available) – more detail might not be reasonable to include. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13938) Default repair is broken, crashes other nodes participating in repair (in trunk)
[
https://issues.apache.org/jira/browse/CASSANDRA-13938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17017397#comment-17017397
]
Aleksey Yeschenko commented on CASSANDRA-13938:
---
Thanks [~djoshi]!
Addressed the nits and committed as
[602a5eef177ac65020470cb0fcf8d88d820ab888|https://github.com/apache/cassandra/commit/602a5eef177ac65020470cb0fcf8d88d820ab888].
CI results
[here|https://circleci.com/workflow-run/43acfc46-861a-478f-adf7-f2d145d43446]
and
[here|https://circleci.com/workflow-run/99a21fd3-cfc8-49bb-99a8-0b904a20669a].
The existing failures are unrelated and known. One that was suspiciously close
is {{org.apache.cassandra.distributed.test.FailingRepairTest}} in both 8 and 11
in-JVM tests, but, again, it's not related, and also reliably passes locally. I
will be looking into it, though.
> Default repair is broken, crashes other nodes participating in repair (in
> trunk)
>
>
> Key: CASSANDRA-13938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13938
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Nate McCall
>Assignee: Aleksey Yeschenko
>Priority: Urgent
> Fix For: 4.0-alpha
>
> Attachments: 13938.yaml, test.sh
>
>
> Running through a simple scenario to test some of the new repair features, I
> was not able to make a repair command work. Further, the exception seemed to
> trigger a nasty failure state that basically shuts down the netty connections
> for messaging *and* CQL on the nodes transferring back data to the node being
> repaired. The following steps reproduce this issue consistently.
> Cassandra stress profile (probably not necessary, but this one provides a
> really simple schema and consistent data shape):
> {noformat}
> keyspace: standard_long
> keyspace_definition: |
> CREATE KEYSPACE standard_long WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> table: test_data
> table_definition: |
> CREATE TABLE test_data (
> key text,
> ts bigint,
> val text,
> PRIMARY KEY (key, ts)
> ) WITH COMPACT STORAGE AND
> CLUSTERING ORDER BY (ts DESC) AND
> bloom_filter_fp_chance=0.01 AND
> caching={'keys':'ALL', 'rows_per_partition':'NONE'} AND
> comment='' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.00 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> columnspec:
> - name: key
> population: uniform(1..5000) # 50 million records available
> - name: ts
> cluster: gaussian(1..50) # Up to 50 inserts per record
> - name: val
> population: gaussian(128..1024) # varrying size of value data
> insert:
> partitions: fixed(1) # only one insert per batch for individual partitions
> select: fixed(1)/1 # each insert comes in one at a time
> batchtype: UNLOGGED
> queries:
> single:
> cql: select * from test_data where key = ? and ts = ? limit 1;
> series:
> cql: select key,ts,val from test_data where key = ? limit 10;
> {noformat}
> The commands to build and run:
> {noformat}
> ccm create 4_0_test -v git:trunk -n 3 -s
> ccm stress user profile=./histo-test-schema.yml
> ops\(insert=20,single=1,series=1\) duration=15s -rate threads=4
> # flush the memtable just to get everything on disk
> ccm node1 nodetool flush
> ccm node2 nodetool flush
> ccm node3 nodetool flush
> # disable hints for nodes 2 and 3
> ccm node2 nodetool disablehandoff
> ccm node3 nodetool disablehandoff
> # stop node1
> ccm node1 stop
> ccm stress user profile=./histo-test-schema.yml
> ops\(insert=20,single=1,series=1\) duration=45s -rate threads=4
> # wait 10 seconds
> ccm node1 start
> # Note that we are local to ccm's nodetool install 'cause repair preview is
> not reported yet
> node1/bin/nodetool repair --preview
> node1/bin/nodetool repair standard_long test_data
> {noformat}
> The error outputs from the last repair command follow. First, this is stdout
> from node1:
> {noformat}
> $ node1/bin/nodetool repair standard_long test_data
> objc[47876]: Class JavaLaunchHelper is implemented in both
> /Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/bin/java
> (0x10274d4c0) and
> /Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre/lib/libinstrument.dylib
> (0x1047b64e0). One of the two will be used. Which one is undefined.
> [2017-10-05 14:31:52,425] Starting repair command #4
> (7e1a9150-a98e-11e7-ad86-cbd2801b8de2), repairing keyspace standard_long with
> repair options (parallelism: parallel, primary range: false, incremental:
> true, job threads: 1, ColumnFamilies: [test_data], dataCenters: [], hosts:
> [], previewKind: NONE, # of ranges: 3,
[jira] [Updated] (CASSANDRA-13938) Default repair is broken, crashes other nodes participating in repair (in trunk)
[
https://issues.apache.org/jira/browse/CASSANDRA-13938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-13938:
--
Since Version: 4.0-alpha
Source Control Link:
[602a5eef177ac65020470cb0fcf8d88d820ab888|https://github.com/apache/cassandra/commit/602a5eef177ac65020470cb0fcf8d88d820ab888]
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Default repair is broken, crashes other nodes participating in repair (in
> trunk)
>
>
> Key: CASSANDRA-13938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13938
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Nate McCall
>Assignee: Aleksey Yeschenko
>Priority: Urgent
> Fix For: 4.0-alpha
>
> Attachments: 13938.yaml, test.sh
>
>
> Running through a simple scenario to test some of the new repair features, I
> was not able to make a repair command work. Further, the exception seemed to
> trigger a nasty failure state that basically shuts down the netty connections
> for messaging *and* CQL on the nodes transferring back data to the node being
> repaired. The following steps reproduce this issue consistently.
> Cassandra stress profile (probably not necessary, but this one provides a
> really simple schema and consistent data shape):
> {noformat}
> keyspace: standard_long
> keyspace_definition: |
> CREATE KEYSPACE standard_long WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> table: test_data
> table_definition: |
> CREATE TABLE test_data (
> key text,
> ts bigint,
> val text,
> PRIMARY KEY (key, ts)
> ) WITH COMPACT STORAGE AND
> CLUSTERING ORDER BY (ts DESC) AND
> bloom_filter_fp_chance=0.01 AND
> caching={'keys':'ALL', 'rows_per_partition':'NONE'} AND
> comment='' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.00 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> columnspec:
> - name: key
> population: uniform(1..5000) # 50 million records available
> - name: ts
> cluster: gaussian(1..50) # Up to 50 inserts per record
> - name: val
> population: gaussian(128..1024) # varrying size of value data
> insert:
> partitions: fixed(1) # only one insert per batch for individual partitions
> select: fixed(1)/1 # each insert comes in one at a time
> batchtype: UNLOGGED
> queries:
> single:
> cql: select * from test_data where key = ? and ts = ? limit 1;
> series:
> cql: select key,ts,val from test_data where key = ? limit 10;
> {noformat}
> The commands to build and run:
> {noformat}
> ccm create 4_0_test -v git:trunk -n 3 -s
> ccm stress user profile=./histo-test-schema.yml
> ops\(insert=20,single=1,series=1\) duration=15s -rate threads=4
> # flush the memtable just to get everything on disk
> ccm node1 nodetool flush
> ccm node2 nodetool flush
> ccm node3 nodetool flush
> # disable hints for nodes 2 and 3
> ccm node2 nodetool disablehandoff
> ccm node3 nodetool disablehandoff
> # stop node1
> ccm node1 stop
> ccm stress user profile=./histo-test-schema.yml
> ops\(insert=20,single=1,series=1\) duration=45s -rate threads=4
> # wait 10 seconds
> ccm node1 start
> # Note that we are local to ccm's nodetool install 'cause repair preview is
> not reported yet
> node1/bin/nodetool repair --preview
> node1/bin/nodetool repair standard_long test_data
> {noformat}
> The error outputs from the last repair command follow. First, this is stdout
> from node1:
> {noformat}
> $ node1/bin/nodetool repair standard_long test_data
> objc[47876]: Class JavaLaunchHelper is implemented in both
> /Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/bin/java
> (0x10274d4c0) and
> /Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre/lib/libinstrument.dylib
> (0x1047b64e0). One of the two will be used. Which one is undefined.
> [2017-10-05 14:31:52,425] Starting repair command #4
> (7e1a9150-a98e-11e7-ad86-cbd2801b8de2), repairing keyspace standard_long with
> repair options (parallelism: parallel, primary range: false, incremental:
> true, job threads: 1, ColumnFamilies: [test_data], dataCenters: [], hosts:
> [], previewKind: NONE, # of ranges: 3, pull repair: false, force repair:
> false)
> [2017-10-05 14:32:07,045] Repair session 7e2e8e80-a98e-11e7-ad86-cbd2801b8de2
> for range [(3074457345618258602,-9223372036854775808],
> (-9223372036854775808,-3074457345618258603],
> (-3074457345618258603,3074457345618258602]] failed with error Stream failed
> [2017-10-05 14:32:07,048] null
> [2017-10-05 14:32:07,050] Repair command #4 finished in
[jira] [Commented] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016940#comment-17016940
]
Aleksey Yeschenko commented on CASSANDRA-15499:
---
It would help if you showed the code from your other upcoming patch. The
general idea is/was that for out messages you should be using a variant of
{{Message#out*}} methods, which all add the header for you.
[~dcapwell] In that scenario (forwardTo with mixed ids), the original tracing
headers are preserved when the builder copied params field from the original
message which has them, so there is no bug currently.
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.x
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15216) Cross node message creation times are disabled by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016915#comment-17016915 ] Aleksey Yeschenko commented on CASSANDRA-15216: --- Looks good to me (except you want to trim your CHANGES.txt entry to one shortish line preferably). If Brandon is a bit busy, I can +1 and commit myself - just lmk on Slack or here. Cheers. > Cross node message creation times are disabled by default > - > > Key: CASSANDRA-15216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15216 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-alpha > > Attachments: Screen Shot 2020-01-15 at 5.01.06 PM.png, Screen Shot > 2020-01-15 at 5.01.33 PM.png, Screen Shot 2020-01-15 at 5.02.22 PM.png > > > This can cause a lot of wasted work for messages that have timed out on the > coordinator. We should generally assume that our users have setup NTP on > their clusters, and that clocks are modestly in sync, since it’s a > requirement for general correctness of last write wins. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15499) Internode message builder does not add trace header
[
https://issues.apache.org/jira/browse/CASSANDRA-15499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17016193#comment-17016193
]
Aleksey Yeschenko commented on CASSANDRA-15499:
---
The code looks fine, but conceptually I would prefer it to be the
responsibility of {{withParams()}} caller to build the correct map, and not
have {{build()}} reach out to global state to set it implicitly if avoidable.
Add a helper for other callers if needed?
> Internode message builder does not add trace header
> ---
>
> Key: CASSANDRA-15499
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15499
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.x
>
>
> The messages built with the {{Builder}}
> ({{org.apache.cassandra.net.Message.Builder}}) do not have the trace header
> when tracing is enabled.
> Consequently, no tracing session gets propagated to other nodes, and the
> tracing function is broken.
> The set of static {{out*}} methods provided (to create an out-bounding
> message) in Message do not have the issue. They can properly add the trace
> header when necessary.
> To be clear, only the {{Builder}} missed adding the tracing header and it
> should be fixed to be consistent with the {{out*}} methods.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[
https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012915#comment-17012915
]
Aleksey Yeschenko commented on CASSANDRA-14781:
---
Could you flip the size fields to {{int}} from {{long}}? (not a full review,
just a super quick skim related to my only previous point)
> Log message when mutation passed to CommitLog#add(Mutation) is too large is
> not descriptive enough
> --
>
> Key: CASSANDRA-14781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14781
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints, Local/Commit Log, Messaging/Client
>Reporter: Jordan West
>Assignee: Tom Petracca
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-beta
>
> Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch,
> CASSANDRA-14781_3.11.patch
>
>
> When hitting
> [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257],
> the log message produced does not help the operator track down what data is
> being written. At a minimum the keyspace and cfIds involved would be useful
> (and are available) – more detail might not be reasonable to include.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15216) Cross node message creation times are disabled by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17012910#comment-17012910 ] Aleksey Yeschenko commented on CASSANDRA-15216: --- [~e.dimitrova] go for it. We could have a wider discussion, but it's hard for me to imagine why someone would object to this default switch. Just make sure to update NEWS.txt to note that the change has taken place. > Cross node message creation times are disabled by default > - > > Key: CASSANDRA-15216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15216 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-alpha > > > This can cause a lot of wasted work for messages that have timed out on the > coordinator. We should generally assume that our users have setup NTP on > their clusters, and that clocks are modestly in sync, since it’s a > requirement for general correctness of last write wins. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[
https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009694#comment-17009694
]
Aleksey Yeschenko commented on CASSANDRA-14781:
---
For what it's worth, I would strongly prefer the memoised fields to be inlined
into {{Mutation}}, same way they are in {{Message}}. It's a very common object,
and the bloat of the overhead of an extra {{SizeHolder}} object vs. just the 3
inlined fields matters.
We do want the memoisation itself, as otherwise we would be performing this
calculation at least twice - once when validation, once when calculating
message size for internode.
> Log message when mutation passed to CommitLog#add(Mutation) is too large is
> not descriptive enough
> --
>
> Key: CASSANDRA-14781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14781
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints, Local/Commit Log, Messaging/Client
>Reporter: Jordan West
>Assignee: Tom Petracca
>Priority: Normal
> Labels: protocolv5
> Fix For: 4.0-beta
>
> Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch,
> CASSANDRA-14781_3.11.patch
>
>
> When hitting
> [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257],
> the log message produced does not help the operator track down what data is
> being written. At a minimum the keyspace and cfIds involved would be useful
> (and are available) – more detail might not be reasonable to include.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13938) Default repair is broken, crashes other nodes participating in repair (in trunk)
[
https://issues.apache.org/jira/browse/CASSANDRA-13938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17000203#comment-17000203
]
Aleksey Yeschenko commented on CASSANDRA-13938:
---
As multiple folks have noticed, {{current}} tracking is indeed busted. And,
also, as noticed, we don’t really need to track {{current}} - it can always be
inferred from the current chunk offset into the partition position, plus
position in the current buffer. So what we need to track is only that offset of
the actual buffer itself.
Tracking it is actually rather trivial, since there are two distinct cases when
we move to the next chunk:
1. A {{reBuffer()}} call upon exhaustion of the previous buffer, while still
not done with the current partition position range. In this case we are moving
to the adjacent compressed chunk, and we should bump offset by uncompressed
chunk length, which is a fixed value for a given session;
2. We are skipping to the next partition position range, via {{position(long
position)}} call; it might be in the current chunk, or it might skip 1..n
compressed chunks. In either case, the new offset and buffer position are
derived from the new {{position}} argument only
There is another bug in the current implementation: if the compressed buffer
exceeds length of {{info.parameters.chunkLength()}} (if data was poorly
compressible, for example, and upon compression blowed up in size instead of
shrinking), then read code in {{Reader#runMayThrow()}} wouldn’t be able to read
that chunk fully into the temporary byte array.
Speaking of that array: the slow-path copy happens to be the one we use, and it
involves a redundant copy into this temporary array, followed by a copy of that
array into the destination {{ByteBuffer}}. That can be trivially eliminated by
adding a {{readFully(ByteBuffer)}} method to {{RebufferingInputStream}}.
I’ve also realised that for no good reason we have an extra thread whose only
job is to read the chunks off input into chunk-size byte buffers and put the on
a queue for {{CompressedInputStream}} to later consume. There is absolutely no
reason for that and the complexity it introduces. It’s not the place to handle
prefetching, nor does it make the input stream non-blocking, nor is it an issue
if it were, given that streaming utilises a separate event loop group from
messaging.
It’s also very much unnecessary to allocate a whole new direct {{ByteBuffer}}
for every chunk. A single {{ByteBuffer}} for the compressed chunk, reused, is
all we need.
Also, we’ve had no test coverage for {{min_compress_ratio}}, introduced by
CASSANDRA-10520.
And, one last thing/nit: while looking at the write side, I spotted some
unnecessary garbage creation in
{{CassandraCompressedStreamWriter#getTransferSections()}} when extending
current section, that can and should be easily avoided. Also the use of
{{SSTableReader.PartitionPositionBounds}} class in place of {{Pair}}, when we
did that refactor recently, is semantically incorrect:
{{PartitionPositionBounds}} as a class represents partition bound in the
uncompressed stream, and not chunk bounds in the compressed one.
I’ve addressed all these points and a bit more in [this
branch|https://github.com/iamaleksey/cassandra/commits/13938-4.0].
> Default repair is broken, crashes other nodes participating in repair (in
> trunk)
>
>
> Key: CASSANDRA-13938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13938
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Nate McCall
>Assignee: Aleksey Yeschenko
>Priority: Urgent
> Fix For: 4.0-alpha
>
> Attachments: 13938.yaml, test.sh
>
>
> Running through a simple scenario to test some of the new repair features, I
> was not able to make a repair command work. Further, the exception seemed to
> trigger a nasty failure state that basically shuts down the netty connections
> for messaging *and* CQL on the nodes transferring back data to the node being
> repaired. The following steps reproduce this issue consistently.
> Cassandra stress profile (probably not necessary, but this one provides a
> really simple schema and consistent data shape):
> {noformat}
> keyspace: standard_long
> keyspace_definition: |
> CREATE KEYSPACE standard_long WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> table: test_data
> table_definition: |
> CREATE TABLE test_data (
> key text,
> ts bigint,
> val text,
> PRIMARY KEY (key, ts)
> ) WITH COMPACT STORAGE AND
> CLUSTERING ORDER BY (ts DESC) AND
> bloom_filter_fp_chance=0.01 AND
> caching={'keys':'ALL', 'rows_per_partition':'NONE'} AND
> comment='' AND
> dclocal_read_repair_chance=0.00
[jira] [Updated] (CASSANDRA-15454) Update generations and update comments for altered distributed system keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15454:
--
Fix Version/s: (was: 4.0)
4.0-alpha
Since Version: 4.0-alpha
Source Control Link:
[8202845facd741f01fbfbbec93d6a3c8e6078644|https://github.com/apache/cassandra/commit/8202845facd741f01fbfbbec93d6a3c8e6078644]
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Update generations and update comments for altered distributed system
> keyspaces
> ---
>
> Key: CASSANDRA-15454
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15454
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> In 4.0, we set {{memtable_flush_period_in_ms}} to 0 (from 1 hour previously),
> and changed default chunk length from 64KiB to 16KiB. On startup we would try
> to override existing schema definitions for {{system_traces}},
> {{system_distributed}}, and {{system_auth}} to reflect these changes, but
> because these are changes to smaller values, while timestamps remain the
> same, these will never override existing, pre-upgrade values from 3.0 or
> 3.11. As a result, a schema migration request will be pushed by a node on
> every bounce to every replica, causing unnecessary and ultimately fruitless
> work.
> Bumping the generations fixes the issue, and documenting the changes doesn't
> hurt either.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15454) Update generations and update comments for altered distributed system keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997483#comment-16997483
]
Aleksey Yeschenko commented on CASSANDRA-15454:
---
Cheers, committed to 4.0 as
[8202845facd741f01fbfbbec93d6a3c8e6078644|https://github.com/apache/cassandra/commit/8202845facd741f01fbfbbec93d6a3c8e6078644]
> Update generations and update comments for altered distributed system
> keyspaces
> ---
>
> Key: CASSANDRA-15454
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15454
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0
>
>
> In 4.0, we set {{memtable_flush_period_in_ms}} to 0 (from 1 hour previously),
> and changed default chunk length from 64KiB to 16KiB. On startup we would try
> to override existing schema definitions for {{system_traces}},
> {{system_distributed}}, and {{system_auth}} to reflect these changes, but
> because these are changes to smaller values, while timestamps remain the
> same, these will never override existing, pre-upgrade values from 3.0 or
> 3.11. As a result, a schema migration request will be pushed by a node on
> every bounce to every replica, causing unnecessary and ultimately fruitless
> work.
> Bumping the generations fixes the issue, and documenting the changes doesn't
> hurt either.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15454) Update generations and update comments for altered distributed system keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15454:
--
Test and Documentation Plan: N/A
Status: Patch Available (was: Open)
> Update generations and update comments for altered distributed system
> keyspaces
> ---
>
> Key: CASSANDRA-15454
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15454
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0
>
>
> In 4.0, we set {{memtable_flush_period_in_ms}} to 0 (from 1 hour previously),
> and changed default chunk length from 64KiB to 16KiB. On startup we would try
> to override existing schema definitions for {{system_traces}},
> {{system_distributed}}, and {{system_auth}} to reflect these changes, but
> because these are changes to smaller values, while timestamps remain the
> same, these will never override existing, pre-upgrade values from 3.0 or
> 3.11. As a result, a schema migration request will be pushed by a node on
> every bounce to every replica, causing unnecessary and ultimately fruitless
> work.
> Bumping the generations fixes the issue, and documenting the changes doesn't
> hurt either.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15454) Update generations and update comments for altered distributed system keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16997177#comment-16997177
]
Aleksey Yeschenko commented on CASSANDRA-15454:
---
Code: https://github.com/iamaleksey/cassandra/commits/15454-4.0
> Update generations and update comments for altered distributed system
> keyspaces
> ---
>
> Key: CASSANDRA-15454
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15454
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0
>
>
> In 4.0, we set {{memtable_flush_period_in_ms}} to 0 (from 1 hour previously),
> and changed default chunk length from 64KiB to 16KiB. On startup we would try
> to override existing schema definitions for {{system_traces}},
> {{system_distributed}}, and {{system_auth}} to reflect these changes, but
> because these are changes to smaller values, while timestamps remain the
> same, these will never override existing, pre-upgrade values from 3.0 or
> 3.11. As a result, a schema migration request will be pushed by a node on
> every bounce to every replica, causing unnecessary and ultimately fruitless
> work.
> Bumping the generations fixes the issue, and documenting the changes doesn't
> hurt either.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15454) Update generations and update comments for altered distributed system keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15454:
--
Bug Category: Parent values: Degradation(12984)
Complexity: Low Hanging Fruit
Discovered By: Code Inspection
Fix Version/s: 4.0
Reviewers: Sam Tunnicliffe
Severity: Low
Status: Open (was: Triage Needed)
> Update generations and update comments for altered distributed system
> keyspaces
> ---
>
> Key: CASSANDRA-15454
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15454
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0
>
>
> In 4.0, we set {{memtable_flush_period_in_ms}} to 0 (from 1 hour previously),
> and changed default chunk length from 64KiB to 16KiB. On startup we would try
> to override existing schema definitions for {{system_traces}},
> {{system_distributed}}, and {{system_auth}} to reflect these changes, but
> because these are changes to smaller values, while timestamps remain the
> same, these will never override existing, pre-upgrade values from 3.0 or
> 3.11. As a result, a schema migration request will be pushed by a node on
> every bounce to every replica, causing unnecessary and ultimately fruitless
> work.
> Bumping the generations fixes the issue, and documenting the changes doesn't
> hurt either.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15454) Update generations and update comments for altered distributed system keyspaces
Aleksey Yeschenko created CASSANDRA-15454:
-
Summary: Update generations and update comments for altered
distributed system keyspaces
Key: CASSANDRA-15454
URL: https://issues.apache.org/jira/browse/CASSANDRA-15454
Project: Cassandra
Issue Type: Bug
Components: Cluster/Schema
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
In 4.0, we set {{memtable_flush_period_in_ms}} to 0 (from 1 hour previously),
and changed default chunk length from 64KiB to 16KiB. On startup we would try
to override existing schema definitions for {{system_traces}},
{{system_distributed}}, and {{system_auth}} to reflect these changes, but
because these are changes to smaller values, while timestamps remain the same,
these will never override existing, pre-upgrade values from 3.0 or 3.11. As a
result, a schema migration request will be pushed by a node on every bounce to
every replica, causing unnecessary and ultimately fruitless work.
Bumping the generations fixes the issue, and documenting the changes doesn't
hurt either.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13938) Default repair is broken, crashes other nodes participating in repair (in trunk)
[
https://issues.apache.org/jira/browse/CASSANDRA-13938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko reassigned CASSANDRA-13938:
-
Assignee: Aleksey Yeschenko (was: Alex Petrov)
> Default repair is broken, crashes other nodes participating in repair (in
> trunk)
>
>
> Key: CASSANDRA-13938
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13938
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Nate McCall
>Assignee: Aleksey Yeschenko
>Priority: Urgent
> Fix For: 4.0-alpha
>
> Attachments: 13938.yaml, test.sh
>
>
> Running through a simple scenario to test some of the new repair features, I
> was not able to make a repair command work. Further, the exception seemed to
> trigger a nasty failure state that basically shuts down the netty connections
> for messaging *and* CQL on the nodes transferring back data to the node being
> repaired. The following steps reproduce this issue consistently.
> Cassandra stress profile (probably not necessary, but this one provides a
> really simple schema and consistent data shape):
> {noformat}
> keyspace: standard_long
> keyspace_definition: |
> CREATE KEYSPACE standard_long WITH replication = {'class':'SimpleStrategy',
> 'replication_factor':3};
> table: test_data
> table_definition: |
> CREATE TABLE test_data (
> key text,
> ts bigint,
> val text,
> PRIMARY KEY (key, ts)
> ) WITH COMPACT STORAGE AND
> CLUSTERING ORDER BY (ts DESC) AND
> bloom_filter_fp_chance=0.01 AND
> caching={'keys':'ALL', 'rows_per_partition':'NONE'} AND
> comment='' AND
> dclocal_read_repair_chance=0.00 AND
> gc_grace_seconds=864000 AND
> read_repair_chance=0.00 AND
> compaction={'class': 'SizeTieredCompactionStrategy'} AND
> compression={'sstable_compression': 'LZ4Compressor'};
> columnspec:
> - name: key
> population: uniform(1..5000) # 50 million records available
> - name: ts
> cluster: gaussian(1..50) # Up to 50 inserts per record
> - name: val
> population: gaussian(128..1024) # varrying size of value data
> insert:
> partitions: fixed(1) # only one insert per batch for individual partitions
> select: fixed(1)/1 # each insert comes in one at a time
> batchtype: UNLOGGED
> queries:
> single:
> cql: select * from test_data where key = ? and ts = ? limit 1;
> series:
> cql: select key,ts,val from test_data where key = ? limit 10;
> {noformat}
> The commands to build and run:
> {noformat}
> ccm create 4_0_test -v git:trunk -n 3 -s
> ccm stress user profile=./histo-test-schema.yml
> ops\(insert=20,single=1,series=1\) duration=15s -rate threads=4
> # flush the memtable just to get everything on disk
> ccm node1 nodetool flush
> ccm node2 nodetool flush
> ccm node3 nodetool flush
> # disable hints for nodes 2 and 3
> ccm node2 nodetool disablehandoff
> ccm node3 nodetool disablehandoff
> # stop node1
> ccm node1 stop
> ccm stress user profile=./histo-test-schema.yml
> ops\(insert=20,single=1,series=1\) duration=45s -rate threads=4
> # wait 10 seconds
> ccm node1 start
> # Note that we are local to ccm's nodetool install 'cause repair preview is
> not reported yet
> node1/bin/nodetool repair --preview
> node1/bin/nodetool repair standard_long test_data
> {noformat}
> The error outputs from the last repair command follow. First, this is stdout
> from node1:
> {noformat}
> $ node1/bin/nodetool repair standard_long test_data
> objc[47876]: Class JavaLaunchHelper is implemented in both
> /Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/bin/java
> (0x10274d4c0) and
> /Library/Java/JavaVirtualMachines/jdk1.8.0_101.jdk/Contents/Home/jre/lib/libinstrument.dylib
> (0x1047b64e0). One of the two will be used. Which one is undefined.
> [2017-10-05 14:31:52,425] Starting repair command #4
> (7e1a9150-a98e-11e7-ad86-cbd2801b8de2), repairing keyspace standard_long with
> repair options (parallelism: parallel, primary range: false, incremental:
> true, job threads: 1, ColumnFamilies: [test_data], dataCenters: [], hosts:
> [], previewKind: NONE, # of ranges: 3, pull repair: false, force repair:
> false)
> [2017-10-05 14:32:07,045] Repair session 7e2e8e80-a98e-11e7-ad86-cbd2801b8de2
> for range [(3074457345618258602,-9223372036854775808],
> (-9223372036854775808,-3074457345618258603],
> (-3074457345618258603,3074457345618258602]] failed with error Stream failed
> [2017-10-05 14:32:07,048] null
> [2017-10-05 14:32:07,050] Repair command #4 finished in 14 seconds
> error: Repair job has failed with the error message: [2017-10-05
> 14:32:07,048] null
> -- StackTrace --
> java.lang.RuntimeException: Repair job has failed with the error message:
> [2017-10-05
[jira] [Updated] (CASSANDRA-15441) Bump generations and document changes to system_distributed and system_traces in 3.0, 3.11
[ https://issues.apache.org/jira/browse/CASSANDRA-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15441: -- Fix Version/s: (was: 3.11.x) (was: 3.0.x) 3.11.6 3.0.20 Since Version: 3.0.0 Source Control Link: [3e30f9c1ba59fffa485da1f21b5ea58d6c115c57|https://github.com/apache/cassandra/commit/3e30f9c1ba59fffa485da1f21b5ea58d6c115c57] Resolution: Fixed Status: Resolved (was: Ready to Commit) Thanks, committed to 3.0 as [3e30f9c1ba59fffa485da1f21b5ea58d6c115c57|https://github.com/apache/cassandra/commit/3e30f9c1ba59fffa485da1f21b5ea58d6c115c57] and merged up > Bump generations and document changes to system_distributed and system_traces > in 3.0, 3.11 > -- > > Key: CASSANDRA-15441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15441 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.20, 3.11.6 > > > We should document all the changes to distributed system keyspaces and assign > unique generations to them. In 3.0 and 3.11 this is just a documentation > issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15441) Bump generations and document changes to system_distributed and system_traces in 3.0, 3.11
[ https://issues.apache.org/jira/browse/CASSANDRA-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15441: -- Test and Documentation Plan: Documentation Status: Patch Available (was: In Progress) > Bump generations and document changes to system_distributed and system_traces > in 3.0, 3.11 > -- > > Key: CASSANDRA-15441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15441 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > We should document all the changes to distributed system keyspaces and assign > unique generations to them. In 3.0 and 3.11 this is just a documentation > issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15441) Bump generations and document changes to system_distributed and system_traces in 3.0, 3.11
[ https://issues.apache.org/jira/browse/CASSANDRA-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986178#comment-16986178 ] Aleksey Yeschenko commented on CASSANDRA-15441: --- 3.0: https://github.com/apache/cassandra/commits/15441-3.0 3.11: https://github.com/apache/cassandra/commits/15441-3.11 > Bump generations and document changes to system_distributed and system_traces > in 3.0, 3.11 > -- > > Key: CASSANDRA-15441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15441 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > We should document all the changes to distributed system keyspaces and assign > unique generations to them. In 3.0 and 3.11 this is just a documentation > issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15441) Bump generations and document changes to system_distributed and system_traces in 3.0, 3.11
[ https://issues.apache.org/jira/browse/CASSANDRA-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15441: -- Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear Impact(13164) Complexity: Low Hanging Fruit Component/s: Cluster/Schema Discovered By: Code Inspection Fix Version/s: 3.11.x 3.0.x Reviewers: Sam Tunnicliffe Severity: Low Assignee: Aleksey Yeschenko Status: Open (was: Triage Needed) > Bump generations and document changes to system_distributed and system_traces > in 3.0, 3.11 > -- > > Key: CASSANDRA-15441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15441 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > We should document all the changes to distributed system keyspaces and assign > unique generations to them. In 3.0 and 3.11 this is just a documentation > issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15441) Bump generations and document changes to system_distributed and system_traces in 3.0, 3.11
[ https://issues.apache.org/jira/browse/CASSANDRA-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15441: -- Description: We should document all the changes to distributed system keyspaces and assign unique generations to them. In 3.0 and 3.11 this is just a documentation issue. > Bump generations and document changes to system_distributed and system_traces > in 3.0, 3.11 > -- > > Key: CASSANDRA-15441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15441 > Project: Cassandra > Issue Type: Bug >Reporter: Aleksey Yeschenko >Priority: Normal > > We should document all the changes to distributed system keyspaces and assign > unique generations to them. In 3.0 and 3.11 this is just a documentation > issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15441) Bump generations and document changes to system_distributed and system_traces in 3.0, 3.11
[ https://issues.apache.org/jira/browse/CASSANDRA-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15441: -- Summary: Bump generations and document changes to system_distributed and system_traces in 3.0, 3.11 (was: TBD (3.11, trivial, boring)) > Bump generations and document changes to system_distributed and system_traces > in 3.0, 3.11 > -- > > Key: CASSANDRA-15441 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15441 > Project: Cassandra > Issue Type: Bug >Reporter: Aleksey Yeschenko >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15441) TBD (3.11, trivial, boring)
Aleksey Yeschenko created CASSANDRA-15441: - Summary: TBD (3.11, trivial, boring) Key: CASSANDRA-15441 URL: https://issues.apache.org/jira/browse/CASSANDRA-15441 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15398) Fix system_traces creation timestamp; optimise system keyspace upgrades
[ https://issues.apache.org/jira/browse/CASSANDRA-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15398: -- Fix Version/s: (was: 3.11.x) (was: 3.0.x) 3.11.6 3.0.20 Since Version: 3.0.0 Source Control Link: [7e878c1eb61b180227d6f1b70c4223e3ee71a754|https://github.com/apache/cassandra/commit/7e878c1eb61b180227d6f1b70c4223e3ee71a754] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Fix system_traces creation timestamp; optimise system keyspace upgrades > --- > > Key: CASSANDRA-15398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15398 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.20, 3.11.6, 4.0 > > > We have introduced changes to system_traces tables in 3.0 (removal of > default_time_to_live, lowering of bloom_filter_fp_chance). We did not, > however, bump the timestamp with which we add the tables to schema, still > defaulting to 0. As a result, for clusters that upgraded from 2.1/2.2, on > bounce we would always detect a mismatch between actual and desired table > definitions, always try to reconcile it by issuing migration tasks, but have > them never override the existing definitions in place. > Additionally, prior to 2.0.2 (CASSANDRA-6016) we’d use a ‘real’ timestamp, so > for clusters that started on even earlier versions of C* (say, 1.2), a bump > to the timestamp by 1 would be insufficient, and a larger generation is > necessary (I picked Jan 1 2020 as cut-off date). > The patch also optimises the process of upgrading replicated system tables. > Instead of issuing a migration task for every table that changed for every > node, we batch all changes into a single schema migration task. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15398) Fix system_traces creation timestamp; optimise system keyspace upgrades
[ https://issues.apache.org/jira/browse/CASSANDRA-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986035#comment-16986035 ] Aleksey Yeschenko commented on CASSANDRA-15398: --- Cheers, committed to 3.0 as [7e878c1eb61b180227d6f1b70c4223e3ee71a754|https://github.com/apache/cassandra/commit/7e878c1eb61b180227d6f1b70c4223e3ee71a754] and merged up. > Fix system_traces creation timestamp; optimise system keyspace upgrades > --- > > Key: CASSANDRA-15398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15398 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 4.0, 3.0.x, 3.11.x > > > We have introduced changes to system_traces tables in 3.0 (removal of > default_time_to_live, lowering of bloom_filter_fp_chance). We did not, > however, bump the timestamp with which we add the tables to schema, still > defaulting to 0. As a result, for clusters that upgraded from 2.1/2.2, on > bounce we would always detect a mismatch between actual and desired table > definitions, always try to reconcile it by issuing migration tasks, but have > them never override the existing definitions in place. > Additionally, prior to 2.0.2 (CASSANDRA-6016) we’d use a ‘real’ timestamp, so > for clusters that started on even earlier versions of C* (say, 1.2), a bump > to the timestamp by 1 would be insufficient, and a larger generation is > necessary (I picked Jan 1 2020 as cut-off date). > The patch also optimises the process of upgrading replicated system tables. > Instead of issuing a migration task for every table that changed for every > node, we batch all changes into a single schema migration task. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16977596#comment-16977596
]
Aleksey Yeschenko commented on CASSANDRA-15410:
---
Committed to {{trunk}} as
[647bdd6a11970f80666d7f20b53af76fbda4ff14|https://github.com/apache/cassandra/commit/647bdd6a11970f80666d7f20b53af76fbda4ff14],
thanks. Made just one tiny tweak: switched encoding of UDT names and field
types to ascii strings as well.
> Avoid over-allocation of bytes for UTF8 string serialization
> -
>
> Key: CASSANDRA-15410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15410
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0
>
>
> In the current message encoding implementation, it first calculates the
> `encodeSize` and allocates the bytebuffer with that size.
> However, during encoding, it assumes the worst case of writing UTF8 string to
> allocate bytes, i.e. assuming each letter takes 3 bytes.
> The over-estimation further leads to resizing the underlying array and data
> copy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15410: -- Source Control Link: [647bdd6a11970f80666d7f20b53af76fbda4ff14|https://github.com/apache/cassandra/commit/647bdd6a11970f80666d7f20b53af76fbda4ff14] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Avoid over-allocation of bytes for UTF8 string serialization > - > > Key: CASSANDRA-15410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15410 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0 > > > In the current message encoding implementation, it first calculates the > `encodeSize` and allocates the bytebuffer with that size. > However, during encoding, it assumes the worst case of writing UTF8 string to > allocate bytes, i.e. assuming each letter takes 3 bytes. > The over-estimation further leads to resizing the underlying array and data > copy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15410: -- Status: Ready to Commit (was: Changes Suggested) > Avoid over-allocation of bytes for UTF8 string serialization > - > > Key: CASSANDRA-15410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15410 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0 > > > In the current message encoding implementation, it first calculates the > `encodeSize` and allocates the bytebuffer with that size. > However, during encoding, it assumes the worst case of writing UTF8 string to > allocate bytes, i.e. assuming each letter takes 3 bytes. > The over-estimation further leads to resizing the underlying array and data > copy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15410: -- Reviewers: Aleksey Yeschenko (was: Aleksey Yeschenko, Dinesh Joshi) > Avoid over-allocation of bytes for UTF8 string serialization > - > > Key: CASSANDRA-15410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15410 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0 > > > In the current message encoding implementation, it first calculates the > `encodeSize` and allocates the bytebuffer with that size. > However, during encoding, it assumes the worst case of writing UTF8 string to > allocate bytes, i.e. assuming each letter takes 3 bytes. > The over-estimation further leads to resizing the underlying array and data > copy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15432) The "read defragmentation" optimization does not work
[
https://issues.apache.org/jira/browse/CASSANDRA-15432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16977563#comment-16977563
]
Aleksey Yeschenko commented on CASSANDRA-15432:
---
Never liked that piece of code, and IIRC [~jbellis] suggested removing it at
least once, too.
bq. The only question might be in which versions? This impact all versions, but
this isn't a correction bug either, "just" a performance one. So do we want 4.0
only or is there appetite for earlier?
It's borderline, if not clearly, a bug. I would go for 3.0, 3.11, and 4.0.
> The "read defragmentation" optimization does not work
> -
>
> Key: CASSANDRA-15432
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15432
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sylvain Lebresne
>Priority: Normal
>
> The so-called "read defragmentation" that has been added way back with
> CASSANDRA-2503 actually does not work, and never has. That is, the
> defragmentation writes do happen, but they only additional load on the nodes
> without helping anything, and are thus a clear negative.
> The "read defragmentation" (which only impact so-called "names queries")
> kicks in when a read hits "too many" sstables (> 4 by default), and when it
> does, it writes down the result of that read. The assumption being that the
> next read for that data would only read the newly written data, which if not
> still in memtable would at least be in a single sstable, thus speeding that
> next read.
> Unfortunately, this is not how this work. When we defrag and write the result
> of our original read, we do so with the timestamp of the data read (as we
> should, changing the timestamp would be plain wrong). And as a result,
> following reads will read that data first, but will have no way to tell that
> no more sstables should be read. Technically, the
> [{{reduceFilter}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/SinglePartitionReadCommand.java#L830]
> call will not return {{null}} because the {{currentMaxTs}} will be higher
> than at least some of the data in the result, and this until we've read from
> as many sstables than in the original read.
> I see no easy way to fix this. It might be possible to make it work with
> additional per-sstable metadata, but nothing sufficiently simple and cheap to
> be worth it comes to mind. And I thus suggest simply removing that code.
> For the record, I'll note that there is actually a 2nd problem with that
> code: currently, we "defrag" a read even if we didn't got data for everything
> that the query requests. This also is "wrong" even if we ignore the first
> issue: a following read that would read the defragmented data would also have
> no way to know to not read more sstables to try to get the missing parts.
> This problem would be fixeable, but is obviously overshadowed by the previous
> one anyway.
> Anyway, as mentioned, I suggest to just remove the "optimization" (which
> again, never optimized anything) altogether, and happy to provide the simple
> patch.
> The only question might be in which versions? This impact all versions, but
> this isn't a correction bug either, "just" a performance one. So do we want
> 4.0 only or is there appetite for earlier?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-2848) Make the Client API support passing down timeouts
[ https://issues.apache.org/jira/browse/CASSANDRA-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-2848: - Component/s: Messaging/Internode Messaging/Client > Make the Client API support passing down timeouts > - > > Key: CASSANDRA-2848 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2848 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client, Messaging/Internode >Reporter: Chris Goffinet >Assignee: Yifan Cai >Priority: Low > Fix For: 3.11.x > > Attachments: 2848-trunk-v2.txt, 2848-trunk.txt > > > Having a max server RPC timeout is good for worst case, but many applications > that have middleware in front of Cassandra, might have higher timeout > requirements. In a fail fast environment, if my application starting at say > the front-end, only has 20ms to process a request, and it must connect to X > services down the stack, by the time it hits Cassandra, we might only have > 10ms. I propose we provide the ability to specify the timeout on each call we > do optionally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-2848) Make the Client API support passing down timeouts
[ https://issues.apache.org/jira/browse/CASSANDRA-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-2848: - Labels: protocolv5 (was: ) > Make the Client API support passing down timeouts > - > > Key: CASSANDRA-2848 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2848 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client, Messaging/Internode >Reporter: Chris Goffinet >Assignee: Yifan Cai >Priority: Low > Labels: protocolv5 > Fix For: 4.0-beta > > Attachments: 2848-trunk-v2.txt, 2848-trunk.txt > > > Having a max server RPC timeout is good for worst case, but many applications > that have middleware in front of Cassandra, might have higher timeout > requirements. In a fail fast environment, if my application starting at say > the front-end, only has 20ms to process a request, and it must connect to X > services down the stack, by the time it hits Cassandra, we might only have > 10ms. I propose we provide the ability to specify the timeout on each call we > do optionally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-2848) Make the Client API support passing down timeouts
[ https://issues.apache.org/jira/browse/CASSANDRA-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-2848: - Fix Version/s: (was: 3.11.x) 4.0-beta > Make the Client API support passing down timeouts > - > > Key: CASSANDRA-2848 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2848 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client, Messaging/Internode >Reporter: Chris Goffinet >Assignee: Yifan Cai >Priority: Low > Fix For: 4.0-beta > > Attachments: 2848-trunk-v2.txt, 2848-trunk.txt > > > Having a max server RPC timeout is good for worst case, but many applications > that have middleware in front of Cassandra, might have higher timeout > requirements. In a fail fast environment, if my application starting at say > the front-end, only has 20ms to process a request, and it must connect to X > services down the stack, by the time it hits Cassandra, we might only have > 10ms. I propose we provide the ability to specify the timeout on each call we > do optionally. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976583#comment-16976583
]
Aleksey Yeschenko commented on CASSANDRA-15410:
---
While you are at it, maybe update {{encodedSize()}} implementation as well to
use the faster {{sizeOfAsciiString()}} - if not for performance then for
symmetry?
> Avoid over-allocation of bytes for UTF8 string serialization
> -
>
> Key: CASSANDRA-15410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15410
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0
>
>
> In the current message encoding implementation, it first calculates the
> `encodeSize` and allocates the bytebuffer with that size.
> However, during encoding, it assumes the worst case of writing UTF8 string to
> allocate bytes, i.e. assuming each letter takes 3 bytes.
> The over-estimation further leads to resizing the underlying array and data
> copy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975077#comment-16975077
]
Aleksey Yeschenko commented on CASSANDRA-15410:
---
I don't want to trust {{encodedSize()}} implementations, really (we had
mismatch bugs in those calculations before).
Now, if you inspect the usages of {{CBUtil#writeString()}} you'll notice that
absolute most of those calls are performed strictly on ASCII strings. Keyspace
names, table names, type names, function names, field names, enums - that can
*only* be ASCII due to our grammar limitations.
Of the strings in messages C* server writes - majority being in
{{ResultSet.ResultMetadata}} - none of the strings ever go beyond ASCII. So we
could introduce a special-case {{CBUtil#writeASCIIString()}} to use
{{ByteBufUtil#writeAscii()}} and use that throughout, that should resolve the
issue. And you'll get a further speed bump from the loop removal in ascii
string size calculation in all the {{encodedSize()}} methods.
> Avoid over-allocation of bytes for UTF8 string serialization
> -
>
> Key: CASSANDRA-15410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15410
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0
>
>
> In the current message encoding implementation, it first calculates the
> `encodeSize` and allocates the bytebuffer with that size.
> However, during encoding, it assumes the worst case of writing UTF8 string to
> allocate bytes, i.e. assuming each letter takes 3 bytes.
> The over-estimation further leads to resizing the underlying array and data
> copy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15410: -- Status: Changes Suggested (was: Review In Progress) > Avoid over-allocation of bytes for UTF8 string serialization > - > > Key: CASSANDRA-15410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15410 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0 > > > In the current message encoding implementation, it first calculates the > `encodeSize` and allocates the bytebuffer with that size. > However, during encoding, it assumes the worst case of writing UTF8 string to > allocate bytes, i.e. assuming each letter takes 3 bytes. > The over-estimation further leads to resizing the underlying array and data > copy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[
https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16974246#comment-16974246
]
Aleksey Yeschenko commented on CASSANDRA-15410:
---
I welcome this change in principle, but it makes me feel a little uneasy.
Specifically, it relies implicitly on how the method is used two levels of
abstraction up, and feels slightly brittle to me.
I wonder if you'd be open instead to a slightly less efficient but,
importantly, still resize-avoiding alternative: modify {{CBUtil#writeString()}}
(and {{CBUtil#writeLongString()}} while at it, just for consistency sake) to
explicitly calculate the precise size of encoded string using
{{TypeSizes#encodedUTF8Length()}} method, and then pass that value to
{{ByteBufUtil#reserveAndWriteUtf8()}}. This way, yes, you would effectively
calculate the size of the string twice, but the abstraction won't leak
assumptions, and will still avoid unnecessary resizing.
> Avoid over-allocation of bytes for UTF8 string serialization
> -
>
> Key: CASSANDRA-15410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15410
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0
>
>
> In the current message encoding implementation, it first calculates the
> `encodeSize` and allocates the bytebuffer with that size.
> However, during encoding, it assumes the worst case of writing UTF8 string to
> allocate bytes, i.e. assuming each letter takes 3 bytes.
> The over-estimation further leads to resizing the underlying array and data
> copy.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[ https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14781: -- Component/s: (was: Legacy/Local Write-Read Paths) Messaging/Client Local/Commit Log Consistency/Hints > Log message when mutation passed to CommitLog#add(Mutation) is too large is > not descriptive enough > -- > > Key: CASSANDRA-14781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14781 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Hints, Local/Commit Log, Messaging/Client >Reporter: Jordan West >Assignee: Tom Petracca >Priority: Normal > Labels: protocolv5 > Fix For: 4.0-beta > > Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch, > CASSANDRA-14781_3.11.patch > > > When hitting > [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257], > the log message produced does not help the operator track down what data is > being written. At a minimum the keyspace and cfIds involved would be useful > (and are available) – more detail might not be reasonable to include. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[ https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14781: -- Labels: protocolv5 (was: ) > Log message when mutation passed to CommitLog#add(Mutation) is too large is > not descriptive enough > -- > > Key: CASSANDRA-14781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14781 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Jordan West >Assignee: Tom Petracca >Priority: Normal > Labels: protocolv5 > Fix For: 4.0-beta > > Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch, > CASSANDRA-14781_3.11.patch > > > When hitting > [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257], > the log message produced does not help the operator track down what data is > being written. At a minimum the keyspace and cfIds involved would be useful > (and are available) – more detail might not be reasonable to include. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[ https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14781: -- Fix Version/s: (was: 3.11.x) (was: 4.x) (was: 3.0.x) 4.0-beta > Log message when mutation passed to CommitLog#add(Mutation) is too large is > not descriptive enough > -- > > Key: CASSANDRA-14781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14781 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Jordan West >Assignee: Tom Petracca >Priority: Normal > Fix For: 4.0-beta > > Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch, > CASSANDRA-14781_3.11.patch > > > When hitting > [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257], > the log message produced does not help the operator track down what data is > being written. At a minimum the keyspace and cfIds involved would be useful > (and are available) – more detail might not be reasonable to include. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15399) Add ability to track state in repair
[
https://issues.apache.org/jira/browse/CASSANDRA-15399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15399:
--
Reviewers: Aleksey Yeschenko
> Add ability to track state in repair
>
>
> Key: CASSANDRA-15399
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15399
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> To enhance the visibility in repair, we should expose internal state via
> virtual tables; the state should include coordinator as well as participant
> state (validation, sync, etc.)
> I propose the following tables:
> repairs - high level summary of the global state of repair; this should be
> called on the coordinator.
> {code:sql}
> CREATE TABLE repairs (
> id uuid,
> keyspace_name text,
> table_names frozen>,
> ranges frozen>,
> coordinator text,
> participants frozen>,
> state text,
> progress_percentage float,
> last_updated_at_millis bigint,
> duration_micro bigint,
> failure_cause text,
> PRIMARY KEY ( (id) )
> )
> {code}
> repair_tasks - represents RepairJob and participants state. This will show
> if validations are running on participants and the progress they are making;
> this should be called on the coordinator.
> {code:sql}
> CREATE TABLE repair_tasks (
> id uuid,
> session_id uuid,
> keyspace_name text,
> table_name text,
> ranges frozen>,
> coordinator text,
> participant text,
> state text,
> state_description text,
> progress_percentage float, -- between 0.0 and 100.0
> last_updated_at_millis bigint,
> duration_micro bigint,
> failure_cause text,
> PRIMARY KEY ( (id), session_id, table_name, participant )
> )
> {code}
> repair_validations - shows the state of the validation task and updated
> periodically while validation is running; this should be called on the
> participants.
> {code:sql}
> CREATE TABLE repair_validations (
> id uuid,
> session_id uuid,
> ranges frozen>,
> keyspace_name text,
> table_name text,
> initiator text,
> state text,
> progress_percentage float,
> queue_duration_ms bigint,
> runtime_duration_ms bigint,
> total_duration_ms bigint,
> estimated_partitions bigint,
> partitions_processed bigint,
> estimated_total_bytes bigint,
> failure_cause text,
> PRIMARY KEY ( (id), session_id, table_name )
> )
> {code}
> The main reason for exposing virtual tables rather than exposing through
> durable tables is to make sure what is exposed is accurate. In cases of
> write failures or node failures, the durable tables could become in-accurate
> and could add edge cases where the repair is not running but the tables say
> it is; by relying on repair's internal in-memory bookkeeping, these problems
> go away.
> This jira does not try to solve the following:
> 1) repair resiliency - there are edge cases where repair hits an error and
> runs forever (at least from nodetool's perspective).
> 2) repair stream tracking - I have not learned the streaming side yet and
> what I see is multiple implementations exist, so seems like high scope. My
> hope is to punt from this jira and tackle separately.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15408) Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
[
https://issues.apache.org/jira/browse/CASSANDRA-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15408:
--
Fix Version/s: (was: 3.11.x)
(was: 3.0.x)
3.11.6
3.0.20
Since Version: 3.0.0
Source Control Link:
https://github.com/apache/cassandra/commit/01b52de41e742d4c9a27d9d907839f1082af95a2
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
> --
>
> Key: CASSANDRA-15408
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15408
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Fix For: 3.0.20, 3.11.6
>
> Attachments: CASSANDRA-15408.patch
>
>
> In [this
> refactor|https://github.com/apache/cassandra/commit/b31845c4a7982358a7c5bfd9bcf572fda6c1bfa9#diff-826a67bf1ae2e45372a35a6a2a6f3f3cL74]
> of CFPropDefs to TableAttributes for CASSANDRA-9712, three obsolete keywords
> were removed:
> {code:java}
> obsoleteKeywords.add("index_interval");
> obsoleteKeywords.add("replicate_on_write");
> obsoleteKeywords.add("populate_io_cache_on_flush");
> {code}
>
> The Thrift API continues to reference these keywords as deprecated, so it's
> not clear that they are actually unsupported.
> Could we either add them back as obsoleteKeywords, or add a change log that
> statements with these properties will fail (There is already a changelog
> about "index_interval" but not the other two)? I understand that the Thrift
> API is totally deprecated so I don't feel strongly about cleaning it up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15408) Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
[
https://issues.apache.org/jira/browse/CASSANDRA-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15408:
--
Reviewers: Aleksey Yeschenko, Aleksey Yeschenko (was: Aleksey Yeschenko)
Aleksey Yeschenko, Aleksey Yeschenko
Status: Review In Progress (was: Patch Available)
> Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
> --
>
> Key: CASSANDRA-15408
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15408
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
> Attachments: CASSANDRA-15408.patch
>
>
> In [this
> refactor|https://github.com/apache/cassandra/commit/b31845c4a7982358a7c5bfd9bcf572fda6c1bfa9#diff-826a67bf1ae2e45372a35a6a2a6f3f3cL74]
> of CFPropDefs to TableAttributes for CASSANDRA-9712, three obsolete keywords
> were removed:
> {code:java}
> obsoleteKeywords.add("index_interval");
> obsoleteKeywords.add("replicate_on_write");
> obsoleteKeywords.add("populate_io_cache_on_flush");
> {code}
>
> The Thrift API continues to reference these keywords as deprecated, so it's
> not clear that they are actually unsupported.
> Could we either add them back as obsoleteKeywords, or add a change log that
> statements with these properties will fail (There is already a changelog
> about "index_interval" but not the other two)? I understand that the Thrift
> API is totally deprecated so I don't feel strongly about cleaning it up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15408) Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
[
https://issues.apache.org/jira/browse/CASSANDRA-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15408:
--
Status: Ready to Commit (was: Review In Progress)
> Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
> --
>
> Key: CASSANDRA-15408
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15408
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
> Attachments: CASSANDRA-15408.patch
>
>
> In [this
> refactor|https://github.com/apache/cassandra/commit/b31845c4a7982358a7c5bfd9bcf572fda6c1bfa9#diff-826a67bf1ae2e45372a35a6a2a6f3f3cL74]
> of CFPropDefs to TableAttributes for CASSANDRA-9712, three obsolete keywords
> were removed:
> {code:java}
> obsoleteKeywords.add("index_interval");
> obsoleteKeywords.add("replicate_on_write");
> obsoleteKeywords.add("populate_io_cache_on_flush");
> {code}
>
> The Thrift API continues to reference these keywords as deprecated, so it's
> not clear that they are actually unsupported.
> Could we either add them back as obsoleteKeywords, or add a change log that
> statements with these properties will fail (There is already a changelog
> about "index_interval" but not the other two)? I understand that the Thrift
> API is totally deprecated so I don't feel strongly about cleaning it up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15408) Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
[
https://issues.apache.org/jira/browse/CASSANDRA-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16973512#comment-16973512
]
Aleksey Yeschenko commented on CASSANDRA-15408:
---
Thanks! Committed to 3.0 and merged upwards.
> Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
> --
>
> Key: CASSANDRA-15408
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15408
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
> Attachments: CASSANDRA-15408.patch
>
>
> In [this
> refactor|https://github.com/apache/cassandra/commit/b31845c4a7982358a7c5bfd9bcf572fda6c1bfa9#diff-826a67bf1ae2e45372a35a6a2a6f3f3cL74]
> of CFPropDefs to TableAttributes for CASSANDRA-9712, three obsolete keywords
> were removed:
> {code:java}
> obsoleteKeywords.add("index_interval");
> obsoleteKeywords.add("replicate_on_write");
> obsoleteKeywords.add("populate_io_cache_on_flush");
> {code}
>
> The Thrift API continues to reference these keywords as deprecated, so it's
> not clear that they are actually unsupported.
> Could we either add them back as obsoleteKeywords, or add a change log that
> statements with these properties will fail (There is already a changelog
> about "index_interval" but not the other two)? I understand that the Thrift
> API is totally deprecated so I don't feel strongly about cleaning it up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15410: -- Test and Documentation Plan: JMH benchmark included Status: Patch Available (was: Open) > Avoid over-allocation of bytes for UTF8 string serialization > - > > Key: CASSANDRA-15410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15410 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0 > > > In the current message encoding implementation, it first calculates the > `encodeSize` and allocates the bytebuffer with that size. > However, during encoding, it assumes the worst case of writing UTF8 string to > allocate bytes, i.e. assuming each letter takes 3 bytes. > The over-estimation further leads to resizing the underlying array and data > copy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15410: -- Reviewers: Aleksey Yeschenko, Aleksey Yeschenko (was: Aleksey Yeschenko) Aleksey Yeschenko, Aleksey Yeschenko (was: Aleksey Yeschenko) Status: Review In Progress (was: Patch Available) > Avoid over-allocation of bytes for UTF8 string serialization > - > > Key: CASSANDRA-15410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15410 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0 > > > In the current message encoding implementation, it first calculates the > `encodeSize` and allocates the bytebuffer with that size. > However, during encoding, it assumes the worst case of writing UTF8 string to > allocate bytes, i.e. assuming each letter takes 3 bytes. > The over-estimation further leads to resizing the underlying array and data > copy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko reassigned CASSANDRA-15410: - Assignee: Yifan Cai (was: Abhishek Singh) > Avoid over-allocation of bytes for UTF8 string serialization > - > > Key: CASSANDRA-15410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15410 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0 > > > In the current message encoding implementation, it first calculates the > `encodeSize` and allocates the bytebuffer with that size. > However, during encoding, it assumes the worst case of writing UTF8 string to > allocate bytes, i.e. assuming each letter takes 3 bytes. > The over-estimation further leads to resizing the underlying array and data > copy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15410) Avoid over-allocation of bytes for UTF8 string serialization
[ https://issues.apache.org/jira/browse/CASSANDRA-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15410: -- Change Category: Performance Complexity: Low Hanging Fruit Fix Version/s: 4.0 Reviewers: Aleksey Yeschenko Status: Open (was: Triage Needed) > Avoid over-allocation of bytes for UTF8 string serialization > - > > Key: CASSANDRA-15410 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15410 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Client >Reporter: Yifan Cai >Assignee: Abhishek Singh >Priority: Normal > Fix For: 4.0 > > > In the current message encoding implementation, it first calculates the > `encodeSize` and allocates the bytebuffer with that size. > However, during encoding, it assumes the worst case of writing UTF8 string to > allocate bytes, i.e. assuming each letter takes 3 bytes. > The over-estimation further leads to resizing the underlying array and data > copy. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[
https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972601#comment-16972601
]
Aleksey Yeschenko commented on CASSANDRA-14781:
---
I would also not bother with listing individual tables; keyspace and partition
key should hopefully be sufficient enough, and memoise calculated {{Mutation}}
size in the {{Mutation}} object (see {{serializedSize*}} fields in {{Message}}
in 4.0) to prevent redundant calculations by subsequent stages.
> Log message when mutation passed to CommitLog#add(Mutation) is too large is
> not descriptive enough
> --
>
> Key: CASSANDRA-14781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14781
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Jordan West
>Assignee: Tom Petracca
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch,
> CASSANDRA-14781_3.11.patch
>
>
> When hitting
> [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257],
> the log message produced does not help the operator track down what data is
> being written. At a minimum the keyspace and cfIds involved would be useful
> (and are available) – more detail might not be reasonable to include.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[
https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16972590#comment-16972590
]
Aleksey Yeschenko commented on CASSANDRA-14781:
---
I'd say this is a little insufficient. You have the exact same situation with
writing hints in {{HintsBuffer.allocate()}}.
What you want is to add an extra validation downstream all the way to
{{ModificationStatement}}, so that you can return a meaningful exception to the
client immediately - rather than ending up timing out the response.
> Log message when mutation passed to CommitLog#add(Mutation) is too large is
> not descriptive enough
> --
>
> Key: CASSANDRA-14781
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14781
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Jordan West
>Assignee: Tom Petracca
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.x
>
> Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch,
> CASSANDRA-14781_3.11.patch
>
>
> When hitting
> [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257],
> the log message produced does not help the operator track down what data is
> being written. At a minimum the keyspace and cfIds involved would be useful
> (and are available) – more detail might not be reasonable to include.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15074) Allow table property defaults (e.g. compaction, compression) to be specified for a cluster/keyspace
[ https://issues.apache.org/jira/browse/CASSANDRA-15074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16971715#comment-16971715 ] Aleksey Yeschenko commented on CASSANDRA-15074: --- Sounds like a good idea to me. And not that hard to implement either. Would need to make some changes to params to only persist explicit values in the template tables, and would have to think about how we want to handle authz, but it's definitely both valuable and doable. > Allow table property defaults (e.g. compaction, compression) to be specified > for a cluster/keyspace > --- > > Key: CASSANDRA-15074 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15074 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Joey Lynch >Priority: Low > Fix For: 4.x > > > During an IRC discussion in > [cassandra-dev|https://wilderness.apache.org/channels/?f=cassandra-dev/2019-04-02#1554224083] > it was proposed that we could have table property defaults stored on a > Keyspace or globally within the cluster. For example, this would allow users > to specify "All new tables on this cluster should default to LCS with SSTable > size of 320MiB" or "all new tables in Keyspace XYZ should have Zstd > commpression with a 8 KiB block size" or "default_time_to_live should default > to 3 days" etc ... This way operators can choose the default that makes sense > for their organization once (e.g. LCS if they are running on fast SSDs), > rather than requiring developers creating the Keyspaces/Tables to make the > decision on every creation (often without context of which choices are right). > A few implementation options were discussed including: > * A YAML option > * Schema provided at the Keyspace level that would be inherited by any > tables automatically > * Schema provided at the Cluster level that would be inherited by any > Keyspaces or Tables automatically > In IRC it appears that rough consensus was found in having global -> keyspace > -> table defaults which would be stored in schema (no YAML configuration > since this isn't node level really, it's a cluster level config). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15408) Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
[
https://issues.apache.org/jira/browse/CASSANDRA-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15408:
--
Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear
Impact(13164)
Complexity: Low Hanging Fruit
Component/s: Documentation/NEWS.txt
Discovered By: User Report
Fix Version/s: 3.11.x
3.0.x
Severity: Low
Status: Open (was: Triage Needed)
> Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
> --
>
> Key: CASSANDRA-15408
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15408
> Project: Cassandra
> Issue Type: Bug
> Components: Documentation/NEWS.txt
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
> Attachments: CASSANDRA-15408.patch
>
>
> In [this
> refactor|https://github.com/apache/cassandra/commit/b31845c4a7982358a7c5bfd9bcf572fda6c1bfa9#diff-826a67bf1ae2e45372a35a6a2a6f3f3cL74]
> of CFPropDefs to TableAttributes for CASSANDRA-9712, three obsolete keywords
> were removed:
> {code:java}
> obsoleteKeywords.add("index_interval");
> obsoleteKeywords.add("replicate_on_write");
> obsoleteKeywords.add("populate_io_cache_on_flush");
> {code}
>
> The Thrift API continues to reference these keywords as deprecated, so it's
> not clear that they are actually unsupported.
> Could we either add them back as obsoleteKeywords, or add a change log that
> statements with these properties will fail (There is already a changelog
> about "index_interval" but not the other two)? I understand that the Thrift
> API is totally deprecated so I don't feel strongly about cleaning it up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15408) Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
[
https://issues.apache.org/jira/browse/CASSANDRA-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16971685#comment-16971685
]
Aleksey Yeschenko commented on CASSANDRA-15408:
---
Hi Leon; I'm inclined to agree that the omission of {{replicate_on_write}} and
{{populate_io_cache_on_flush}} in 3.0 (and perhaps 3.11) is a documentation
bug. {{index_interval}}, luckily, is mentioned.
So if you cook up a tiny patch to update NEWS.txt, upgrading section, to
mention them, I'll be happy to commit.
> Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
> --
>
> Key: CASSANDRA-15408
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15408
> Project: Cassandra
> Issue Type: Bug
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Attachments: CASSANDRA-15408.patch
>
>
> In [this
> refactor|https://github.com/apache/cassandra/commit/b31845c4a7982358a7c5bfd9bcf572fda6c1bfa9#diff-826a67bf1ae2e45372a35a6a2a6f3f3cL74]
> of CFPropDefs to TableAttributes for CASSANDRA-9712, three obsolete keywords
> were removed:
> {code:java}
> obsoleteKeywords.add("index_interval");
> obsoleteKeywords.add("replicate_on_write");
> obsoleteKeywords.add("populate_io_cache_on_flush");
> {code}
>
> The Thrift API continues to reference these keywords as deprecated, so it's
> not clear that they are actually unsupported.
> Could we either add them back as obsoleteKeywords, or add a change log that
> statements with these properties will fail (There is already a changelog
> about "index_interval" but not the other two)? I understand that the Thrift
> API is totally deprecated so I don't feel strongly about cleaning it up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15408) Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
[
https://issues.apache.org/jira/browse/CASSANDRA-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16971577#comment-16971577
]
Aleksey Yeschenko commented on CASSANDRA-15408:
---
They have no effect anymore since 2.1, and were deprecated there. And, per our
policies, removed in the next major release after deprecation.
Not to mention Thrift is completely gone in 4.0 now. I would prefer to let
these go personally.
> Cassandra throws SyntaxException for obsolete keywords that Thrift API permits
> --
>
> Key: CASSANDRA-15408
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15408
> Project: Cassandra
> Issue Type: Bug
>Reporter: Leon Zaruvinsky
>Priority: Normal
> Attachments: CASSANDRA-15408.patch
>
>
> In [this
> refactor|https://github.com/apache/cassandra/commit/b31845c4a7982358a7c5bfd9bcf572fda6c1bfa9#diff-826a67bf1ae2e45372a35a6a2a6f3f3cL74]
> of CFPropDefs to TableAttributes for CASSANDRA-9712, three obsolete keywords
> were removed:
> {code:java}
> obsoleteKeywords.add("index_interval");
> obsoleteKeywords.add("replicate_on_write");
> obsoleteKeywords.add("populate_io_cache_on_flush");
> {code}
>
> The Thrift API continues to reference these keywords as deprecated, so it's
> not clear that they are actually unsupported.
> Could we either add them back as obsoleteKeywords, or add a change log that
> statements with these properties will fail (There is already a changelog
> about "index_interval" but not the other two)? I understand that the Thrift
> API is totally deprecated so I don't feel strongly about cleaning it up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15405) Mixed mode reads on compact storage tables can return incomplete results
[ https://issues.apache.org/jira/browse/CASSANDRA-15405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15405: -- Status: Ready to Commit (was: Changes Suggested) > Mixed mode reads on compact storage tables can return incomplete results > > > Key: CASSANDRA-15405 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15405 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > In mixed mode (2.1/3.0), when coordinating a read on a 2.1 node, reading data > from 3.0 nodes, we [incorrectly > trim|https://github.com/apache/cassandra/blob/53f604dc1789a800dbcbc3c8aee77f8f36b8b5db/src/java/org/apache/cassandra/db/LegacyLayout.java#L529] > the result (if it has tombstones) when preparing it for the 2.1 node, this > is then [interpreted by the 2.1 > node|https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/pager/AbstractQueryPager.java#L110] > as the pager has been exhausted. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15405) Mixed mode reads on compact storage tables can return incomplete results
[ https://issues.apache.org/jira/browse/CASSANDRA-15405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970100#comment-16970100 ] Aleksey Yeschenko commented on CASSANDRA-15405: --- LGTM as is to me. Minor inefficiency, I think: we are potentially oversending some tombstones if they follow the last live cell that's still within limit. I think it's safe to not do that. > Mixed mode reads on compact storage tables can return incomplete results > > > Key: CASSANDRA-15405 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15405 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > In mixed mode (2.1/3.0), when coordinating a read on a 2.1 node, reading data > from 3.0 nodes, we [incorrectly > trim|https://github.com/apache/cassandra/blob/53f604dc1789a800dbcbc3c8aee77f8f36b8b5db/src/java/org/apache/cassandra/db/LegacyLayout.java#L529] > the result (if it has tombstones) when preparing it for the 2.1 node, this > is then [interpreted by the 2.1 > node|https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/pager/AbstractQueryPager.java#L110] > as the pager has been exhausted. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15398) Fix system_traces creation timestamp; optimise system keyspace upgrades
[ https://issues.apache.org/jira/browse/CASSANDRA-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15398: -- Test and Documentation Plan: Unit tests added Status: Patch Available (was: Open) > Fix system_traces creation timestamp; optimise system keyspace upgrades > --- > > Key: CASSANDRA-15398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15398 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 4.0, 3.0.x, 3.11.x > > > We have introduced changes to system_traces tables in 3.0 (removal of > default_time_to_live, lowering of bloom_filter_fp_chance). We did not, > however, bump the timestamp with which we add the tables to schema, still > defaulting to 0. As a result, for clusters that upgraded from 2.1/2.2, on > bounce we would always detect a mismatch between actual and desired table > definitions, always try to reconcile it by issuing migration tasks, but have > them never override the existing definitions in place. > Additionally, prior to 2.0.2 (CASSANDRA-6016) we’d use a ‘real’ timestamp, so > for clusters that started on even earlier versions of C* (say, 1.2), a bump > to the timestamp by 1 would be insufficient, and a larger generation is > necessary (I picked Jan 1 2020 as cut-off date). > The patch also optimises the process of upgrading replicated system tables. > Instead of issuing a migration task for every table that changed for every > node, we batch all changes into a single schema migration task. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15398) Fix system_traces creation timestamp; optimise system keyspace upgrades
[ https://issues.apache.org/jira/browse/CASSANDRA-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15398: -- Reviewers: Sam Tunnicliffe > Fix system_traces creation timestamp; optimise system keyspace upgrades > --- > > Key: CASSANDRA-15398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15398 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 4.0, 3.0.x, 3.11.x > > > We have introduced changes to system_traces tables in 3.0 (removal of > default_time_to_live, lowering of bloom_filter_fp_chance). We did not, > however, bump the timestamp with which we add the tables to schema, still > defaulting to 0. As a result, for clusters that upgraded from 2.1/2.2, on > bounce we would always detect a mismatch between actual and desired table > definitions, always try to reconcile it by issuing migration tasks, but have > them never override the existing definitions in place. > Additionally, prior to 2.0.2 (CASSANDRA-6016) we’d use a ‘real’ timestamp, so > for clusters that started on even earlier versions of C* (say, 1.2), a bump > to the timestamp by 1 would be insufficient, and a larger generation is > necessary (I picked Jan 1 2020 as cut-off date). > The patch also optimises the process of upgrading replicated system tables. > Instead of issuing a migration task for every table that changed for every > node, we batch all changes into a single schema migration task. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15398) Fix system_traces creation timestamp; optimise system keyspace upgrades
[ https://issues.apache.org/jira/browse/CASSANDRA-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969475#comment-16969475 ] Aleksey Yeschenko commented on CASSANDRA-15398: --- 3.0: [code|https://github.com/iamaleksey/cassandra/commits/15398-3.0], [CI|https://circleci.com/workflow-run/67c87af7-3421-4e0e-8104-58bbb2bdfa67] 3.11: [code|https://github.com/iamaleksey/cassandra/commits/15398-3.11], [CI|https://circleci.com/workflow-run/8035e900-0589-46ad-88cf-cd64c6d2d2af] 4.0: [code|https://github.com/iamaleksey/cassandra/commits/15398-4.0], [CI|https://circleci.com/workflow-run/66c01c80-e97b-420a-b94b-598f8702d318] > Fix system_traces creation timestamp; optimise system keyspace upgrades > --- > > Key: CASSANDRA-15398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15398 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 4.0, 3.0.x, 3.11.x > > > We have introduced changes to system_traces tables in 3.0 (removal of > default_time_to_live, lowering of bloom_filter_fp_chance). We did not, > however, bump the timestamp with which we add the tables to schema, still > defaulting to 0. As a result, for clusters that upgraded from 2.1/2.2, on > bounce we would always detect a mismatch between actual and desired table > definitions, always try to reconcile it by issuing migration tasks, but have > them never override the existing definitions in place. > Additionally, prior to 2.0.2 (CASSANDRA-6016) we’d use a ‘real’ timestamp, so > for clusters that started on even earlier versions of C* (say, 1.2), a bump > to the timestamp by 1 would be insufficient, and a larger generation is > necessary (I picked Jan 1 2020 as cut-off date). > The patch also optimises the process of upgrading replicated system tables. > Instead of issuing a migration task for every table that changed for every > node, we batch all changes into a single schema migration task. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15398) Fix system_traces creation timestamp; optimise system keyspace upgrades
[ https://issues.apache.org/jira/browse/CASSANDRA-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15398: -- Bug Category: Parent values: Code(13163) Complexity: Normal Discovered By: Code Inspection Severity: Low Status: Open (was: Triage Needed) > Fix system_traces creation timestamp; optimise system keyspace upgrades > --- > > Key: CASSANDRA-15398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15398 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 4.0, 3.0.x, 3.11.x > > > We have introduced changes to system_traces tables in 3.0 (removal of > default_time_to_live, lowering of bloom_filter_fp_chance). We did not, > however, bump the timestamp with which we add the tables to schema, still > defaulting to 0. As a result, for clusters that upgraded from 2.1/2.2, on > bounce we would always detect a mismatch between actual and desired table > definitions, always try to reconcile it by issuing migration tasks, but have > them never override the existing definitions in place. > Additionally, prior to 2.0.2 (CASSANDRA-6016) we’d use a ‘real’ timestamp, so > for clusters that started on even earlier versions of C* (say, 1.2), a bump > to the timestamp by 1 would be insufficient, and a larger generation is > necessary (I picked Jan 1 2020 as cut-off date). > The patch also optimises the process of upgrading replicated system tables. > Instead of issuing a migration task for every table that changed for every > node, we batch all changes into a single schema migration task. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15398) Fix system_traces creation timestamp; optimise system keyspace upgrades
[ https://issues.apache.org/jira/browse/CASSANDRA-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15398: -- Fix Version/s: 3.11.x 3.0.x 4.0 > Fix system_traces creation timestamp; optimise system keyspace upgrades > --- > > Key: CASSANDRA-15398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15398 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 4.0, 3.0.x, 3.11.x > > > We have introduced changes to system_traces tables in 3.0 (removal of > default_time_to_live, lowering of bloom_filter_fp_chance). We did not, > however, bump the timestamp with which we add the tables to schema, still > defaulting to 0. As a result, for clusters that upgraded from 2.1/2.2, on > bounce we would always detect a mismatch between actual and desired table > definitions, always try to reconcile it by issuing migration tasks, but have > them never override the existing definitions in place. > Additionally, prior to 2.0.2 (CASSANDRA-6016) we’d use a ‘real’ timestamp, so > for clusters that started on even earlier versions of C* (say, 1.2), a bump > to the timestamp by 1 would be insufficient, and a larger generation is > necessary (I picked Jan 1 2020 as cut-off date). > The patch also optimises the process of upgrading replicated system tables. > Instead of issuing a migration task for every table that changed for every > node, we batch all changes into a single schema migration task. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15398) Fix system_traces creation timestamp; optimise system keyspace upgrades
[ https://issues.apache.org/jira/browse/CASSANDRA-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15398: -- Description: We have introduced changes to system_traces tables in 3.0 (removal of default_time_to_live, lowering of bloom_filter_fp_chance). We did not, however, bump the timestamp with which we add the tables to schema, still defaulting to 0. As a result, for clusters that upgraded from 2.1/2.2, on bounce we would always detect a mismatch between actual and desired table definitions, always try to reconcile it by issuing migration tasks, but have them never override the existing definitions in place. Additionally, prior to 2.0.2 (CASSANDRA-6016) we’d use a ‘real’ timestamp, so for clusters that started on even earlier versions of C* (say, 1.2), a bump to the timestamp by 1 would be insufficient, and a larger generation is necessary (I picked Jan 1 2020 as cut-off date). The patch also optimises the process of upgrading replicated system tables. Instead of issuing a migration task for every table that changed for every node, we batch all changes into a single schema migration task. > Fix system_traces creation timestamp; optimise system keyspace upgrades > --- > > Key: CASSANDRA-15398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15398 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > > We have introduced changes to system_traces tables in 3.0 (removal of > default_time_to_live, lowering of bloom_filter_fp_chance). We did not, > however, bump the timestamp with which we add the tables to schema, still > defaulting to 0. As a result, for clusters that upgraded from 2.1/2.2, on > bounce we would always detect a mismatch between actual and desired table > definitions, always try to reconcile it by issuing migration tasks, but have > them never override the existing definitions in place. > Additionally, prior to 2.0.2 (CASSANDRA-6016) we’d use a ‘real’ timestamp, so > for clusters that started on even earlier versions of C* (say, 1.2), a bump > to the timestamp by 1 would be insufficient, and a larger generation is > necessary (I picked Jan 1 2020 as cut-off date). > The patch also optimises the process of upgrading replicated system tables. > Instead of issuing a migration task for every table that changed for every > node, we batch all changes into a single schema migration task. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15398) Fix system_traces creation timestamp; optimise system keyspace upgrades
[ https://issues.apache.org/jira/browse/CASSANDRA-15398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15398: -- Summary: Fix system_traces creation timestamp; optimise system keyspace upgrades (was: TBD (minor, boring)) > Fix system_traces creation timestamp; optimise system keyspace upgrades > --- > > Key: CASSANDRA-15398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15398 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15398) TBD (minor, boring)
Aleksey Yeschenko created CASSANDRA-15398: - Summary: TBD (minor, boring) Key: CASSANDRA-15398 URL: https://issues.apache.org/jira/browse/CASSANDRA-15398 Project: Cassandra Issue Type: Bug Components: Cluster/Schema Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15385) Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15385: -- Fix Version/s: (was: 3.11.x) (was: 3.0.x) 3.11.6 3.0.20 Since Version: 4.0-alpha Source Control Link: [9b1f3796a65db46f15f2f2ad8af4180f71e3f53f|https://github.com/apache/cassandra/commit/9b1f3796a65db46f15f2f2ad8af4180f71e3f53f] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default > -- > > Key: CASSANDRA-15385 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15385 > Project: Cassandra > Issue Type: Bug > Components: Observability/Tracing >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.20, 3.11.6 > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15385) Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16966880#comment-16966880 ] Aleksey Yeschenko commented on CASSANDRA-15385: --- Cheers - committed to 3.0 as [9b1f3796a65db46f15f2f2ad8af4180f71e3f53f|https://github.com/apache/cassandra/commit/9b1f3796a65db46f15f2f2ad8af4180f71e3f53f] and merged upwards. > Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default > -- > > Key: CASSANDRA-15385 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15385 > Project: Cassandra > Issue Type: Bug > Components: Observability/Tracing >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15385) Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15385: -- Reviewers: Sam Tunnicliffe > Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default > -- > > Key: CASSANDRA-15385 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15385 > Project: Cassandra > Issue Type: Bug > Components: Observability/Tracing >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15385) Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15385: -- Test and Documentation Plan: Use existing upgrade test that covers it, remove whitelisted error logs. Status: Patch Available (was: In Progress) > Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default > -- > > Key: CASSANDRA-15385 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15385 > Project: Cassandra > Issue Type: Bug > Components: Observability/Tracing >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15385) Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964996#comment-16964996 ] Aleksey Yeschenko commented on CASSANDRA-15385: --- 3.0: [code|https://github.com/iamaleksey/cassandra/commits/15385-3.0], [CI|https://circleci.com/workflow-run/e202c2ef-6bba-453b-ac05-c8125b643683] 3.11: [code|https://github.com/iamaleksey/cassandra/commits/15385-3.11], [CI|https://circleci.com/workflow-run/a0ce5ec1-b3c7-417e-97c9-7387a30d0ab5] 4.0: [code|https://github.com/iamaleksey/cassandra/commits/15385-4.0] (only updates NEWS.txt) dtest changes: [here|https://github.com/iamaleksey/cassandra-dtest/commits/15385] We do have a dtest that covers the issue - it's just that at the moment it white-lists these exceptions and ignores them in the logs. The commit to dtest repo forces the test to fail without the suggested changes to C*, but the test passes cleanly with them, without any disconnects, message drops, and errors in the logs. Running the test is a little involved, however, since running upgrade tests is currently broken on Circle. I did run them locally however. > Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default > -- > > Key: CASSANDRA-15385 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15385 > Project: Cassandra > Issue Type: Bug > Components: Observability/Tracing >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15385) Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15385: -- Bug Category: Parent values: Availability(12983) Complexity: Normal Discovered By: Code Inspection Severity: Normal Status: Open (was: Triage Needed) > Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default > -- > > Key: CASSANDRA-15385 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15385 > Project: Cassandra > Issue Type: Bug > Components: Observability/Tracing >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15385) Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
[
https://issues.apache.org/jira/browse/CASSANDRA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964009#comment-16964009
]
Aleksey Yeschenko commented on CASSANDRA-15385:
---
CASSANDRA-7544 allowed storage port to be configurable per node, and among its
changes introduced several new columns to some of the distributed system tables:
1. {{coordinator_port}} and {{participants_v2}} to
{{system_distributed.repair_history}}
2. {{coordinator_port}} to {{system_traces.sessions}}
3. {{source_port}} to {{system_traces.events}}
(1) is not a huge deal, since we don't support repair in mixed mode clusters;
(2) and (3), however, are. And while CASSANDRA-14841 modifies tracing logic to
not write those new added columns while still in mixed mode, this is still a
problem for reads - which in case of tracing will be issued automatically by
the drivers.
CASSANDRA-14897 gives advice to add some of these columns manually (though not
tracing) while still on 3.x, but there is a reason why such alters are
explicitly forbidden and require a workaround: distributed pseudo-system tables
are evolved programmatically, by bumping internal generation timestamp. Mixing
manual alters and real-world timestamps and those surrogate timestamps simply
prevents us from deterministically evolving those schemas in future versions.
The patches to be linked pre-add the columns on 3.0/3.11 side automatically,
preserving the correct timestamps, so that tracing can go on uninhibited,
without crashing the internode connections and losing the enqueued messages in
the process.
> Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
> --
>
> Key: CASSANDRA-15385
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15385
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Tracing
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15385) Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15385: -- Component/s: Observability/Tracing > Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default > -- > > Key: CASSANDRA-15385 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15385 > Project: Cassandra > Issue Type: Bug > Components: Observability/Tracing >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15385) Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15385: -- Fix Version/s: 3.11.x 3.0.x > Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default > -- > > Key: CASSANDRA-15385 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15385 > Project: Cassandra > Issue Type: Bug >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > Fix For: 3.0.x, 3.11.x > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15385) Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default
[ https://issues.apache.org/jira/browse/CASSANDRA-15385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15385: -- Summary: Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default (was: TBD (minor; boring)) > Ensure that tracing doesn't break connections in 3.x/4.0 mixed mode by default > -- > > Key: CASSANDRA-15385 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15385 > Project: Cassandra > Issue Type: Bug >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15385) TBD (minor; boring)
Aleksey Yeschenko created CASSANDRA-15385: - Summary: TBD (minor; boring) Key: CASSANDRA-15385 URL: https://issues.apache.org/jira/browse/CASSANDRA-15385 Project: Cassandra Issue Type: Bug Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14629) Abstract Virtual Table for very large result sets
[
https://issues.apache.org/jira/browse/CASSANDRA-14629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961186#comment-16961186
]
Aleksey Yeschenko commented on CASSANDRA-14629:
---
Looks ok to me overall, but I do have two questions (two variations of the same
question, really):
1. {{select(DecoratedKey partitionKey, ClusteringIndexFilter clusteringFilter,
ColumnFilter columnFilter)}} involves invoking {{hasKey(DecoratedKey
partitionKey)}} and then {{getRows(DecoratedKey key, ClusteringIndexFilter
clusteringFilter, ColumnFilter columnFilter)}}. Depending on the underlying
implementation, this could mean a lot of extra work - up to doubling the amount
of work needed. As an illustration, a common {{Map}} usage antipattern comes to
mind: doing a {{contains()}} followed by {{get()}} instead of just doing get
and checking for {{null}}. I know that one of the use cases you have in mind
for this code is exposing the raw content of sstables, and I can see this
overhead being relatively significant there potentially. I would suggest
getting rid of {{hasKey()}} entirely and of the related check.
2. Similarly, {{select(DataRange dataRange, ColumnFilter columnFilter)}} and
{{getPartitionKeys(DataRange dataRange)}} invocation could maybe also be
remodelled to permit a single underlying iterator?
It's possible that I'm missing something here, so these aren't demands for
changes - just a conversation starter.
P.S. You can suppress the redundant suppression warnings themselves like this:
{code:java}
@SuppressWarnings({"resource", "RedundantSuppression"})
{code}
> Abstract Virtual Table for very large result sets
> -
>
> Key: CASSANDRA-14629
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14629
> Project: Cassandra
> Issue Type: New Feature
> Components: Legacy/CQL, Legacy/Observability
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Low
> Labels: pull-request-available, virtual-tables
> Time Spent: 4h
> Remaining Estimate: 0h
>
> For virtual tables that are very large we cannot use existing
> abstractvirtualtable since it would OOM the node possibly. An example would
> be a table to view the internal cache contents or to view contents of
> sstables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15363) Read repair in mixed mode between 2.1 and 3.0 on COMPACT STORAGE tables causes unreadable sstables after upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15363:
--
Status: Ready to Commit (was: Review In Progress)
LGTM, +1
> Read repair in mixed mode between 2.1 and 3.0 on COMPACT STORAGE tables
> causes unreadable sstables after upgrade
>
>
> Key: CASSANDRA-15363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15363
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> if we have a table like this:
> {{CREATE TABLE tbl (pk ascii, b boolean, v blob, PRIMARY KEY (pk)) WITH
> COMPACT STORAGE}}
> with a cluster where node1 is 2.1 and node2 is 3.0 (during upgrade):
> * node2 coordinates a delete {{DELETE FROM tbl WHERE pk = 'something'}} which
> node1 does not get
> * node1 coordinates a quorum read {{SELECT * FROM tbl WHERE id =
> 'something'}} which causes a read repair
> * this makes node1 flush an sstable like this:
> {code}
> [
> {"key": "something",
> "metadata": {"deletionInfo":
> {"markedForDeleteAt":1571388944364000,"localDeletionTime":1571388944}},
> "cells": [["b","b",1571388944364000,"t",1571388944],
>["v","v",1571388944364000,"t",1571388944]]}
> ]
> {code}
> (It has range tombstones which are covered by the partition deletion)
> Then, when we upgrade this node to 3.0 and try to read or run
> upgradesstables, we get this:
> {code}
> ERROR [node1_CompactionExecutor:1] node1 2019-10-18 10:44:11,325
> DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
> java.lang.UnsupportedOperationException: null
> at
> org.apache.cassandra.db.LegacyLayout.extractStaticColumns(LegacyLayout.java:779)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:120)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:57)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:362)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:65)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:103)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:94)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:442)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:144)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:92)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:227)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:190)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:675)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[dtest-3.0.19.jar:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_121]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_121]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_121]
> at
>
[jira] [Updated] (CASSANDRA-15363) Read repair in mixed mode between 2.1 and 3.0 on COMPACT STORAGE tables causes unreadable sstables after upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15363:
--
Reviewers: Aleksey Yeschenko, Benedict Elliott Smith, Aleksey Yeschenko
(was: Aleksey Yeschenko, Benedict Elliott Smith)
Aleksey Yeschenko, Benedict Elliott Smith, Aleksey Yeschenko
(was: Aleksey Yeschenko, Benedict Elliott Smith)
Status: Review In Progress (was: Patch Available)
> Read repair in mixed mode between 2.1 and 3.0 on COMPACT STORAGE tables
> causes unreadable sstables after upgrade
>
>
> Key: CASSANDRA-15363
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15363
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> if we have a table like this:
> {{CREATE TABLE tbl (pk ascii, b boolean, v blob, PRIMARY KEY (pk)) WITH
> COMPACT STORAGE}}
> with a cluster where node1 is 2.1 and node2 is 3.0 (during upgrade):
> * node2 coordinates a delete {{DELETE FROM tbl WHERE pk = 'something'}} which
> node1 does not get
> * node1 coordinates a quorum read {{SELECT * FROM tbl WHERE id =
> 'something'}} which causes a read repair
> * this makes node1 flush an sstable like this:
> {code}
> [
> {"key": "something",
> "metadata": {"deletionInfo":
> {"markedForDeleteAt":1571388944364000,"localDeletionTime":1571388944}},
> "cells": [["b","b",1571388944364000,"t",1571388944],
>["v","v",1571388944364000,"t",1571388944]]}
> ]
> {code}
> (It has range tombstones which are covered by the partition deletion)
> Then, when we upgrade this node to 3.0 and try to read or run
> upgradesstables, we get this:
> {code}
> ERROR [node1_CompactionExecutor:1] node1 2019-10-18 10:44:11,325
> DebuggableThreadPoolExecutor.java:242 - Error in ThreadPoolExecutor
> java.lang.UnsupportedOperationException: null
> at
> org.apache.cassandra.db.LegacyLayout.extractStaticColumns(LegacyLayout.java:779)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$OldFormatIterator.readStaticRow(SSTableSimpleIterator.java:120)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.(SSTableIdentityIterator.java:57)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.io.sstable.format.big.BigTableScanner$KeyScanningIterator$1.initializeIterator(BigTableScanner.java:362)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.maybeInit(LazilyInitializedUnfilteredRowIterator.java:48)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.isReverseOrder(LazilyInitializedUnfilteredRowIterator.java:65)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:103)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$1.reduce(UnfilteredPartitionIterators.java:94)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.utils.MergeIterator$OneToOne.computeNext(MergeIterator.java:442)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$2.hasNext(UnfilteredPartitionIterators.java:144)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:92)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:227)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:190)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:89)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:61)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.db.compaction.CompactionManager$8.runMayThrow(CompactionManager.java:675)
> ~[dtest-3.0.19.jar:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[dtest-3.0.19.jar:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_121]
> at
[jira] [Updated] (CASSANDRA-14629) Abstract Virtual Table for very large result sets
[ https://issues.apache.org/jira/browse/CASSANDRA-14629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14629: -- Reviewers: Aleksey Yeschenko, Dinesh Joshi, Vinay Chella, Aleksey Yeschenko (was: Aleksey Yeschenko, Dinesh Joshi, Vinay Chella) Aleksey Yeschenko, Dinesh Joshi, Vinay Chella, Aleksey Yeschenko (was: Aleksey Yeschenko, Dinesh Joshi, Vinay Chella) Status: Review In Progress (was: Patch Available) > Abstract Virtual Table for very large result sets > - > > Key: CASSANDRA-14629 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14629 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/CQL, Legacy/Observability >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Low > Labels: pull-request-available, virtual-tables > Time Spent: 3h 40m > Remaining Estimate: 0h > > For virtual tables that are very large we cannot use existing > abstractvirtualtable since it would OOM the node possibly. An example would > be a table to view the internal cache contents or to view contents of > sstables. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15193) Add ability to cap max negotiable protocol version
[
https://issues.apache.org/jira/browse/CASSANDRA-15193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945877#comment-16945877
]
Aleksey Yeschenko commented on CASSANDRA-15193:
---
The changes look good to me. I figure we don't really need to have
{{SystemKeyspace::loadPeerVersions}} be a map, and it could just be a set, but
it doesn't really matter or affect anything.
+1
> Add ability to cap max negotiable protocol version
> --
>
> Key: CASSANDRA-15193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15193
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> 3.0 and native protocol V4 introduced a change to how PagingState is
> serialized. Unfortunately that can break requests during upgrades: since
> paging states are opaque, it's possible for a client to receive a paging
> state encoded as V3 on a 2.1 node, and then send it to a 3.0 node on a V4
> session. The version of the current session will be used to deserialize the
> paging state, instead of the actual version used to serialize it, and the
> request will fail.
> CASSANDRA-15176 solves half of this problem by enabling 3.0 nodes to
> serialize mis-versioned PagingStates. To address the other side of the issue,
> 2.1 nodes receiving V4 PagingStates, we can introduce a property to cap the
> max native protocol version that the 3.0 nodes will negotiate with clients.
> If we cap this to V3 during upgrades, no V4 connections will be established
> and so no incompatible PagingStates will be sent to clients.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15193) Add ability to cap max negotiable protocol version
[ https://issues.apache.org/jira/browse/CASSANDRA-15193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15193: -- Status: Ready to Commit (was: Review In Progress) > Add ability to cap max negotiable protocol version > -- > > Key: CASSANDRA-15193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15193 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > 3.0 and native protocol V4 introduced a change to how PagingState is > serialized. Unfortunately that can break requests during upgrades: since > paging states are opaque, it's possible for a client to receive a paging > state encoded as V3 on a 2.1 node, and then send it to a 3.0 node on a V4 > session. The version of the current session will be used to deserialize the > paging state, instead of the actual version used to serialize it, and the > request will fail. > CASSANDRA-15176 solves half of this problem by enabling 3.0 nodes to > serialize mis-versioned PagingStates. To address the other side of the issue, > 2.1 nodes receiving V4 PagingStates, we can introduce a property to cap the > max native protocol version that the 3.0 nodes will negotiate with clients. > If we cap this to V3 during upgrades, no V4 connections will be established > and so no incompatible PagingStates will be sent to clients. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15193) Add ability to cap max negotiable protocol version
[ https://issues.apache.org/jira/browse/CASSANDRA-15193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-15193: -- Reviewers: Aleksey Yeschenko, Alex Petrov, Aleksey Yeschenko (was: Aleksey Yeschenko, Alex Petrov) Aleksey Yeschenko, Alex Petrov, Aleksey Yeschenko (was: Aleksey Yeschenko, Alex Petrov) Status: Review In Progress (was: Patch Available) > Add ability to cap max negotiable protocol version > -- > > Key: CASSANDRA-15193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15193 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > 3.0 and native protocol V4 introduced a change to how PagingState is > serialized. Unfortunately that can break requests during upgrades: since > paging states are opaque, it's possible for a client to receive a paging > state encoded as V3 on a 2.1 node, and then send it to a 3.0 node on a V4 > session. The version of the current session will be used to deserialize the > paging state, instead of the actual version used to serialize it, and the > request will fail. > CASSANDRA-15176 solves half of this problem by enabling 3.0 nodes to > serialize mis-versioned PagingStates. To address the other side of the issue, > 2.1 nodes receiving V4 PagingStates, we can introduce a property to cap the > max native protocol version that the 3.0 nodes will negotiate with clients. > If we cap this to V3 during upgrades, no V4 connections will be established > and so no incompatible PagingStates will be sent to clients. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936759#comment-16936759
]
Aleksey Yeschenko commented on CASSANDRA-15299:
---
bq. Can you share what you had in mind?
Of course. We would be reusing the framing protocols introduced by the
messaging rewrite (see the links to code in the description above), including
for when neither checksumming or compression are enabled
({{FrameDecoderUnprotected}} in messaging).
bq. Are you thinking some kind of window for flushing messages?
We already have that implicitly. The only difference will be that instead of
emitting one frame for one message, multiple messages (if we have several to
encode) will often map to a single frame - with per-frame CRC32 and
compression, when either is enabled.
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Labels: client-impacting, protocolv5
> Fix For: 4.0-beta
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> parcicular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15163) Untangle RepairMessage sub-hierarchy of messages, use new messaging (more) correctly
[
https://issues.apache.org/jira/browse/CASSANDRA-15163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16934805#comment-16934805
]
Aleksey Yeschenko commented on CASSANDRA-15163:
---
Committed as
[f9ff88437742675db5c53f5834884b43f8937e00|https://github.com/apache/cassandra/commit/f9ff88437742675db5c53f5834884b43f8937e00]
to trunk, cheers.
> Untangle RepairMessage sub-hierarchy of messages, use new messaging (more)
> correctly
>
>
> Key: CASSANDRA-15163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15163
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Internode
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0
>
>
> There is currently a nested sub-hierarchy of messages used by repair - in
> {{RepairMessage}} - all sharing one verb, all sharing one deserialiser, all
> sharing one verb handler. The first two can and should be addressed, and it’s
> mostly a mechanical task to do so. The last one should be tackled separately,
> when we get to refactor repair some day (and it could definitely use some
> love).
> The proposed patch follows-up the work done in CASSANDRA-15066 and makes
> repair use the new messaging service in a more correct way, making verbs and
> deserialization more explicit. It wasn't included into the original commit so
> that to not make the single commit larger that it needed to be.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15163) Untangle RepairMessage sub-hierarchy of messages, use new messaging (more) correctly
[
https://issues.apache.org/jira/browse/CASSANDRA-15163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15163:
--
Source Control Link:
https://github.com/apache/cassandra/commit/f9ff88437742675db5c53f5834884b43f8937e00
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Untangle RepairMessage sub-hierarchy of messages, use new messaging (more)
> correctly
>
>
> Key: CASSANDRA-15163
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15163
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Internode
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0
>
>
> There is currently a nested sub-hierarchy of messages used by repair - in
> {{RepairMessage}} - all sharing one verb, all sharing one deserialiser, all
> sharing one verb handler. The first two can and should be addressed, and it’s
> mostly a mechanical task to do so. The last one should be tackled separately,
> when we get to refactor repair some day (and it could definitely use some
> love).
> The proposed patch follows-up the work done in CASSANDRA-15066 and makes
> repair use the new messaging service in a more correct way, making verbs and
> deserialization more explicit. It wasn't included into the original commit so
> that to not make the single commit larger that it needed to be.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924320#comment-16924320
]
Aleksey Yeschenko commented on CASSANDRA-14825:
---
I think I've been on all three sides of this argument over the past years.
Ever since I rewrote the schema system tables for 3.0 in CASSANDRA-6717, the
code to generate 3.0+ from system tables has been relatively easy. It will
become even easier in 4.0 with {{COMPACT STORAGE}} gone. So I've never felt
strongly about a need to have a server-side {{DESCRIBE}} variant in any shape
or form.
But, much like [~slebresne], I've come around since then. There is value in a
server-side {{DESCRIBE}} implementation. Less than there was before, and will
be even less once 4.0 is out, but there still is.
Now, I love that we have virtual tables now, and I can see this functionality
implemented using them. But my preference would be slightly in favour of moving
{{cqlsh}} {{DESCRIBE}} into its set of standalone CQL statements, on balance.
It's not at all invasive, IMO, but would allow us to have nice tailored
grammar, more flexible than a generic {{SELECT}} statement could ever be.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14772) Fix issues in audit / full query log interactions
[
https://issues.apache.org/jira/browse/CASSANDRA-14772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921337#comment-16921337
]
Aleksey Yeschenko commented on CASSANDRA-14772:
---
Good stuff.
1. Would prefer if {{QueryEvents#startTime()}} weren’t conditional on listeners
size; this, coupled with the {{time > 0}} check conflates the two concerns
(current timestamp and presence of listereners)
2. Following up on that, would prefer if the {{time > 0}} check was gone from
{{QueryEvents#notify*}} methods; instead, to prevent potential allocation of a
capturing lambda with empty listeners, replace {{listeners#forEach()}} calls
with for-each loops
3. Minor, but I don’t see why {{getInstance()}} method exists in
{{QueryEvents}} and {{AuthEvents}}. It has no value and goes against our
guidelines (prefer public final fields to getter methods) and precedent (public
static {{INSTANCE}} singletons)
4. {{AuditLogManager#executeFailure()}} - there are to {{toString()}} calls for
{{setOperation()}}; Seem to not be intentional to me?
5. {{IAuditLogger#path()}} method should be removed; nit: rename {{enabled()}}
to a proper predicate name like {{isEnabled()}}?
> Fix issues in audit / full query log interactions
> -
>
> Key: CASSANDRA-14772
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14772
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL, Legacy/Tools
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0, 4.0-rc
>
>
> There are some problems with the audit + full query log code that need to be
> resolved before 4.0 is released:
> * Fix performance regression in FQL that makes it less usable than it should
> be.
> * move full query log specific code to a separate package
> * do some audit log class renames (I keep reading {{BinLogAuditLogger}} vs
> {{BinAuditLogger}} wrong for example)
> * avoid parsing the CQL queries twice in {{QueryMessage}} when audit log is
> enabled.
> * add a new tool to dump audit logs (ie, let fqltool be full query log
> specific). fqltool crashes when pointed to them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15299:
--
Authors: Aleksey Yeschenko, Sam Tunnicliffe (was: Aleksey Yeschenko)
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> parcicular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15299) CASSANDRA-13304 follow-up: improve checksumming and compression in protocol v5-beta
[
https://issues.apache.org/jira/browse/CASSANDRA-15299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-15299:
--
Change Category: Semantic
Complexity: Normal
Status: Open (was: Triage Needed)
> CASSANDRA-13304 follow-up: improve checksumming and compression in protocol
> v5-beta
> ---
>
> Key: CASSANDRA-15299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15299
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-13304 made an important improvement to our native protocol: it
> introduced checksumming/CRC32 to request and response bodies. It’s an
> important step forward, but it doesn’t cover the entire stream. In
> parcicular, the message header is not covered by a checksum or a crc, which
> poses a correctness issue if, for example, {{streamId}} gets corrupted.
> Additionally, we aren’t quite using CRC32 correctly, in two ways:
> 1. We are calculating the CRC32 of the *decompressed* value instead of
> computing the CRC32 on the bytes written on the wire - losing the properties
> of the CRC32. In some cases, due to this sequencing, attempting to decompress
> a corrupt stream can cause a segfault by LZ4.
> 2. When using CRC32, the CRC32 value is written in the incorrect byte order,
> also losing some of the protections.
> See https://users.ece.cmu.edu/~koopman/pubs/KoopmanCRCWebinar9May2012.pdf for
> explanation for the two points above.
> Separately, there are some long-standing issues with the protocol - since
> *way* before CASSANDRA-13304. Importantly, both checksumming and compression
> operate on individual message bodies rather than frames of multiple complete
> messages. In reality, this has several important additional downsides. To
> name a couple:
> # For compression, we are getting poor compression ratios for smaller
> messages - when operating on tiny sequences of bytes. In reality, for most
> small requests and responses we are discarding the compressed value as it’d
> be smaller than the uncompressed one - incurring both redundant allocations
> and compressions.
> # For checksumming and CRC32 we pay a high overhead price for small messages.
> 4 bytes extra is *a lot* for an empty write response, for example.
> To address the correctness issue of {{streamId}} not being covered by the
> checksum/CRC32 and the inefficiency in compression and checksumming/CRC32, we
> should switch to a framing protocol with multiple messages in a single frame.
> I suggest we reuse the framing protocol recently implemented for internode
> messaging in CASSANDRA-15066 to the extent that its logic can be borrowed,
> and that we do it before native protocol v5 graduates from beta. See
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderCrc.java
> and
> https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/FrameDecoderLZ4.java.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
