[jira] [Commented] (CASSANDRA-14664) Allow providing and overriding nowInSeconds via native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597576#comment-16597576 ] Aleksey Yeschenko commented on CASSANDRA-14664: --- Additionally spotted during review, CASSANDRA-14680, which we'll hopefully address at some point in all the branches. > Allow providing and overriding nowInSeconds via native protocol > --- > > Key: CASSANDRA-14664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14664 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: client-impacting, fqltool, protocolv5 > Fix For: 4.0.x > > > For FQL replay testing, to allow for deterministic and repeatable workload > replay comparisons, we need to be able to set custom nowInSeconds via native > protocol - primarily to control TTL expiration, both on read and write paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14680) Built-in 2i implementation applies updates non-deterministically
Aleksey Yeschenko created CASSANDRA-14680:
-
Summary: Built-in 2i implementation applies updates
non-deterministically
Key: CASSANDRA-14680
URL: https://issues.apache.org/jira/browse/CASSANDRA-14680
Project: Cassandra
Issue Type: Bug
Reporter: Chris Lohfink
Fix For: 3.0.x, 3.11.x, 4.0.x
Spotted by [~cnlwsu] during CASSANDRA-14664 review, and confirmed by me and
[~beobal] separately.
{{Keyspace.applyInternal()}} generates {{nowInSeconds}} from local time at the
time of mutation application - which can happen at quite a delay from that
mutation creation (think streaming path, hints, batchlog replay). That
{{nowInSeconds}} value is later used by {{CassandraIndex}} {{Indexer}} to
determine liveness of cells and also used for some of generated tombstones.
Depending on when {{Keyspace.applyInternal()}} call happens, you'll see varying
results in the internal 2i table, which sounds problematic. The values should
be derived from the cells and liveness info in the partition updates
exclusively.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14664) Allow providing and overriding nowInSeconds via native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592161#comment-16592161 ] Aleksey Yeschenko edited comment on CASSANDRA-14664 at 8/30/18 2:45 PM: 4.0 branch [here|https://github.com/iamaleksey/cassandra/commits/14664-4.0], CI [here|https://circleci.com/workflow-run/626a23e8-efc6-46f4-9c71-ca3af4b93159]. For end-to-end testing I used [these changes to the driver|https://github.com/iamaleksey/java-driver/commits/14664], which I cannot contribute, yet, for CLA reasons, but they are sufficient to use for verification purposes - in addition to provided unit tests. was (Author: iamaleksey): 4.0 branch [here|https://github.com/iamaleksey/cassandra/commits/14664-4.0], CI [here|https://circleci.com/workflow-run/806f605f-a63d-44c2-a03b-9883b217612f]. For end-to-end testing I used [these changes to the driver|https://github.com/iamaleksey/java-driver/commits/14664], which I cannot contribute, yet, for CLA reasons, but they are sufficient to use for verification purposes - in addition to provided unit tests. > Allow providing and overriding nowInSeconds via native protocol > --- > > Key: CASSANDRA-14664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14664 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: client-impacting, fqltool, protocolv5 > Fix For: 4.0.x > > > For FQL replay testing, to allow for deterministic and repeatable workload > replay comparisons, we need to be able to set custom nowInSeconds via native > protocol - primarily to control TTL expiration, both on read and write paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14664) Allow providing and overriding nowInSeconds via native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597532#comment-16597532 ] Aleksey Yeschenko commented on CASSANDRA-14664: --- [~cnlwsu] Good catch, thanks. Addressed in a pushed commit. > Allow providing and overriding nowInSeconds via native protocol > --- > > Key: CASSANDRA-14664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14664 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: client-impacting, fqltool, protocolv5 > Fix For: 4.0.x > > > For FQL replay testing, to allow for deterministic and repeatable workload > replay comparisons, we need to be able to set custom nowInSeconds via native > protocol - primarily to control TTL expiration, both on read and write paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14677) Clean up Message.Request implementations
[
https://issues.apache.org/jira/browse/CASSANDRA-14677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597357#comment-16597357
]
Aleksey Yeschenko commented on CASSANDRA-14677:
---
[~cnlwsu] Pushed a commit to further get rid of some state ({{shouldTrace}}
boolean field) and rename perform/execute, to the same branch.
> Clean up Message.Request implementations
>
>
> Key: CASSANDRA-14677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14677
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> First tracing support, many years ago, then most recently audit log, made a
> big mess out of {{Message.Request.execute()}} implementations.
> This patch tries to clean up some of it by removing tracing logic from
> {{QueryState}} and moving shared tracing functionality to
> {{Message.Request.perform()}}. It also moves out tracing and audit log boiler
> plate into their own small methods instead of polluting {{execute()}}
> implementations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14673) Removing user defined type column results in ReadFailure due to CorruptSSTableException or IllegalStateException
[
https://issues.apache.org/jira/browse/CASSANDRA-14673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597337#comment-16597337
]
Aleksey Yeschenko commented on CASSANDRA-14673:
---
[~umcanl] Yep. Until we can implement it safely, we should reject it in 3.0,
unfortunately. Alternative would be to forbid dropping a UDT if it was ever
referenced by a column that was later dropped.
> Removing user defined type column results in ReadFailure due to
> CorruptSSTableException or IllegalStateException
>
>
> Key: CASSANDRA-14673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14673
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Horia Mocioi
>Priority: Minor
> Attachments: script
>
>
> Steps to reproduce:
> # create keyspace
> # create user defined type
> # create a table that would use the udt
> # insert some data
> # drop the udt column
> # restart cassandra
> # query the table
> See the attached script for steps 1-5.
> When querying the table in cqlsh I got the following error:
> {code:java}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read]
> message="Operation failed - received 0 responses and 1 failures"
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1,
> 'consistency': 'ONE'}{code}
> On system.log I get errors. The errors are different, meaning that some times
> I get:
> {code:java}
> WARN [ReadStage-1] 2018-08-29 10:22:30,229
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
> Thread[ReadStage-1,10,main]: {}
> java.lang.RuntimeException:
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
> /.ccm/3113/node1/data0/my_ks/my_table-90f73700ab6411e8a72e45b164590124/mc-1-big-Data.db
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2601)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_131]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
> [apache-cassandra-3.11.3.jar:3.11.3]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
> [apache-cassandra-3.11.3.jar:3.11.3]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException:
> Corrupted:
> /.ccm/3113/node1/data0/my_ks/my_table-90f73700ab6411e8a72e45b164590124/mc-1-big-Data.db
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:391)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:258)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:136)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:92)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:79)
>
[jira] [Commented] (CASSANDRA-14677) Clean up Message.Request implementations
[
https://issues.apache.org/jira/browse/CASSANDRA-14677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597260#comment-16597260
]
Aleksey Yeschenko commented on CASSANDRA-14677:
---
bq. From my reading of your patch, you have refactored code into small, concise
blocks which is great. For future reference it would be helpful for the whole
community if we have a set of best practices that we should all follow. Perhaps
they could be part of the contribution guidelines. It will help all reviewers
enforce these best practices. I would be happy to work with you and other folks
to draft this up and review it with a wider audience. It will only help us
deliver better quality patches, consistently. WDYT?
I think this is a good idea, [~djoshi3]. There are some aspects potentially
specific to Cassandra that would go into such a guide that aren't necessarily
covered by books like Effective Java, and the expectations the community has -
or should have - about a good patch and good review. It didn't used to be
necessary before, but now that we have more contributors of varying levels
around, we could use such a document.
> Clean up Message.Request implementations
>
>
> Key: CASSANDRA-14677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14677
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> First tracing support, many years ago, then most recently audit log, made a
> big mess out of {{Message.Request.execute()}} implementations.
> This patch tries to clean up some of it by removing tracing logic from
> {{QueryState}} and moving shared tracing functionality to
> {{Message.Request.perform()}}. It also moves out tracing and audit log boiler
> plate into their own small methods instead of polluting {{execute()}}
> implementations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14677) Clean up Message.Request implementations
[
https://issues.apache.org/jira/browse/CASSANDRA-14677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597228#comment-16597228
]
Aleksey Yeschenko commented on CASSANDRA-14677:
---
bq. I think it would be helpful for committers and for future reviewers to have
a better understanding of what is meant by "big mess". Perhaps you could update
the description to better outline the specific issues with the execute() method
implementations. This would also make the changes in the patch more clear for
review.
It would be, and normally I would. Just running out of hours and days in the
week before the freeze :\
In this case it should be relatively obvious: the implementations got fat, and
had chunks of logic that were clearly movable into smaller blocks, as
[~djoshi3] well noted. That applies to both tracing and audit log changes.
Additionally, there was unnecessary logical duplication in tracing logic, and
in addition to that, confusing indirection due to tracing state living in
{{QueryState}}, for no apparent reason (anymore). That could be and was remade.
Additionally there was some field bloat, both due to audit log changes, and due
to old tracing changes. In the latter case with the session id being a field
where it could just be a local variable. I addressed both.
> Clean up Message.Request implementations
>
>
> Key: CASSANDRA-14677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14677
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> First tracing support, many years ago, then most recently audit log, made a
> big mess out of {{Message.Request.execute()}} implementations.
> This patch tries to clean up some of it by removing tracing logic from
> {{QueryState}} and moving shared tracing functionality to
> {{Message.Request.perform()}}. It also moves out tracing and audit log boiler
> plate into their own small methods instead of polluting {{execute()}}
> implementations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14677) Clean up Message.Request implementations
[
https://issues.apache.org/jira/browse/CASSANDRA-14677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597222#comment-16597222
]
Aleksey Yeschenko commented on CASSANDRA-14677:
---
bq. Also, Request.perform() is an unexpected naming choice, and doesn't seem
typical of how we usually name things. You should add a comment that perform()
is now the main entry point for running the Request, and perhaps make execute()
protected (instead of public).
I don't know, seems quite typical to me. Made them protected but checked out
before force-pushing - did it now, that I agree with.
> Clean up Message.Request implementations
>
>
> Key: CASSANDRA-14677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14677
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> First tracing support, many years ago, then most recently audit log, made a
> big mess out of {{Message.Request.execute()}} implementations.
> This patch tries to clean up some of it by removing tracing logic from
> {{QueryState}} and moving shared tracing functionality to
> {{Message.Request.perform()}}. It also moves out tracing and audit log boiler
> plate into their own small methods instead of polluting {{execute()}}
> implementations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14677) Clean up Message.Request implementations
[
https://issues.apache.org/jira/browse/CASSANDRA-14677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16597219#comment-16597219
]
Aleksey Yeschenko commented on CASSANDRA-14677:
---
bq. Previously, we had memoized auditLogEnabled in the parent Request class in
order read the volatile auditLogManager.isAuditingEnabled() only once per
instance. With this refactor, you are calling
auditLogManager.isAuditingEnabled() every time you need to check if logging is
still enabled, which references a volatile variable
(AuditLogManager.isAuditLogEnabled). You might consider memoizing the value
again.
I can only sigh at this. It seems to me like the author of the patch didn't
really know the cost of reading a volatile field, if they chose to add entire
three fields to every request object just to avoid it. Hint: the cost is
negligible in comparison, it might merely prevent some reordering. Additionally:
1. {{AuditLogManager.instance}} itself is a final field, why was it memoized?
2. Instead of hoisting them, the original author could still just read them
once as local variables in {{execute()}} implementations, as they aren't used
outside.
> Clean up Message.Request implementations
>
>
> Key: CASSANDRA-14677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14677
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> First tracing support, many years ago, then most recently audit log, made a
> big mess out of {{Message.Request.execute()}} implementations.
> This patch tries to clean up some of it by removing tracing logic from
> {{QueryState}} and moving shared tracing functionality to
> {{Message.Request.perform()}}. It also moves out tracing and audit log boiler
> plate into their own small methods instead of polluting {{execute()}}
> implementations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14677) Clean up Message.Request implementations
[
https://issues.apache.org/jira/browse/CASSANDRA-14677?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14677:
--
Reviewer: Chris Lohfink
Status: Patch Available (was: Open)
> Clean up Message.Request implementations
>
>
> Key: CASSANDRA-14677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14677
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> First tracing support, many years ago, then most recently audit log, made a
> big mess out of {{Message.Request.execute()}} implementations.
> This patch tries to clean up some of it by removing tracing logic from
> {{QueryState}} and moving shared tracing functionality to
> {{Message.Request.perform()}}. It also moves out tracing and audit log boiler
> plate into their own small methods instead of polluting {{execute()}}
> implementations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14677) Clean up Message.Request implementations
[
https://issues.apache.org/jira/browse/CASSANDRA-14677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596885#comment-16596885
]
Aleksey Yeschenko commented on CASSANDRA-14677:
---
Code [here|https://github.com/iamaleksey/cassandra/commits/14677-4.0], CI
[here|https://circleci.com/workflow-run/f02f947b-f709-4b11-a71b-69aabe21b1c8].
> Clean up Message.Request implementations
>
>
> Key: CASSANDRA-14677
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14677
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> First tracing support, many years ago, then most recently audit log, made a
> big mess out of {{Message.Request.execute()}} implementations.
> This patch tries to clean up some of it by removing tracing logic from
> {{QueryState}} and moving shared tracing functionality to
> {{Message.Request.perform()}}. It also moves out tracing and audit log boiler
> plate into their own small methods instead of polluting {{execute()}}
> implementations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14677) Clean up Message.Request implementations
Aleksey Yeschenko created CASSANDRA-14677:
-
Summary: Clean up Message.Request implementations
Key: CASSANDRA-14677
URL: https://issues.apache.org/jira/browse/CASSANDRA-14677
Project: Cassandra
Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Fix For: 4.0.x
First tracing support, many years ago, then most recently audit log, made a big
mess out of {{Message.Request.execute()}} implementations.
This patch tries to clean up some of it by removing tracing logic from
{{QueryState}} and moving shared tracing functionality to
{{Message.Request.perform()}}. It also moves out tracing and audit log boiler
plate into their own small methods instead of polluting {{execute()}}
implementations.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14675) Log the actual (if server-generated) timestamp used by queries in full query log
[ https://issues.apache.org/jira/browse/CASSANDRA-14675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14675: -- Reviewer: Chris Lohfink > Log the actual (if server-generated) timestamp used by queries in full query > log > > > Key: CASSANDRA-14675 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14675 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Major > Fix For: 4.0.x > > > FQL doesn't currently use the actual timestamp - in microseconds - if it's > been server generated, used for request execution. It needs to, to allow for > - in conjunction with CASSANDRA-14664 and CASSANDRA-14671 - deterministic > playback tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14675) Log the actual (if server-generated) timestamp used by queries in full query log
Aleksey Yeschenko created CASSANDRA-14675: - Summary: Log the actual (if server-generated) timestamp used by queries in full query log Key: CASSANDRA-14675 URL: https://issues.apache.org/jira/browse/CASSANDRA-14675 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 4.0.x FQL doesn't currently use the actual timestamp - in microseconds - if it's been server generated, used for request execution. It needs to, to allow for - in conjunction with CASSANDRA-14664 and CASSANDRA-14671 - deterministic playback tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14673) Removing user defined type column results in ReadFailure due to CorruptSSTableException or IllegalStateException
[
https://issues.apache.org/jira/browse/CASSANDRA-14673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16596603#comment-16596603
]
Aleksey Yeschenko commented on CASSANDRA-14673:
---
Yep. CASSANDRA-13426, in 4.0, just forbids this operation. It's currently
unsafe unfortunately, and we should modify 3.0/3.11 to reject it as well.
> Removing user defined type column results in ReadFailure due to
> CorruptSSTableException or IllegalStateException
>
>
> Key: CASSANDRA-14673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14673
> Project: Cassandra
> Issue Type: Bug
> Components: Local Write-Read Paths
>Reporter: Horia Mocioi
>Priority: Minor
> Attachments: script
>
>
> Steps to reproduce:
> # create keyspace
> # create user defined type
> # create a table that would use the udt
> # insert some data
> # drop the udt column
> # restart cassandra
> # query the table
> See the attached script for steps 1-5.
> When querying the table in cqlsh I got the following error:
> {code:java}
> ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read]
> message="Operation failed - received 0 responses and 1 failures"
> info={'failures': 1, 'received_responses': 0, 'required_responses': 1,
> 'consistency': 'ONE'}{code}
> On system.log I get errors. The errors are different, meaning that some times
> I get:
> {code:java}
> WARN [ReadStage-1] 2018-08-29 10:22:30,229
> AbstractLocalAwareExecutorService.java:167 - Uncaught exception on thread
> Thread[ReadStage-1,10,main]: {}
> java.lang.RuntimeException:
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
> /.ccm/3113/node1/data0/my_ks/my_table-90f73700ab6411e8a72e45b164590124/mc-1-big-Data.db
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2601)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_131]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:134)
> [apache-cassandra-3.11.3.jar:3.11.3]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109)
> [apache-cassandra-3.11.3.jar:3.11.3]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException:
> Corrupted:
> /.ccm/3113/node1/data0/my_ks/my_table-90f73700ab6411e8a72e45b164590124/mc-1-big-Data.db
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:391)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:258)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:136)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:92)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:79)
> ~[apache-cassandra-3.11.3.jar:3.11.3]
> at
>
[jira] [Comment Edited] (CASSANDRA-14671) Log the actual nowInSeconds used by queries in full query log
[
https://issues.apache.org/jira/browse/CASSANDRA-14671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595630#comment-16595630
]
Aleksey Yeschenko edited comment on CASSANDRA-14671 at 8/29/18 4:25 PM:
Code [here|https://github.com/iamaleksey/cassandra/commits/14671-4.0], CI
[here|https://circleci.com/workflow-run/84fb21c1-b677-4b2f-a6b1-de3e606f2359].
was (Author: iamaleksey):
Code [here|https://github.com/iamaleksey/cassandra/commits/14671-4.0], CI
[here|https://circleci.com/workflow-run/4fbe10d2-96a3-4927-a3ef-54b9ed713fb3].
> Log the actual nowInSeconds used by queries in full query log
> -
>
> Key: CASSANDRA-14671
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14671
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> FQL doesn't currently use the actual {{nowInSeconds}} value used for request
> execution. It needs to, to allow for - in conjunction with CASSANDRA-14664 -
> less indeterministic playback tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14671) Log the actual nowInSeconds used by queries in full query log
[
https://issues.apache.org/jira/browse/CASSANDRA-14671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595630#comment-16595630
]
Aleksey Yeschenko edited comment on CASSANDRA-14671 at 8/29/18 11:47 AM:
-
Code [here|https://github.com/iamaleksey/cassandra/commits/14671-4.0], CI
[here|https://circleci.com/workflow-run/4fbe10d2-96a3-4927-a3ef-54b9ed713fb3].
was (Author: iamaleksey):
Code [here|https://github.com/iamaleksey/cassandra/commits/14671-4.0], CI
[here|https://circleci.com/workflow-run/ce1008e6-7a5f-4fa3-b242-b61c2af8ab51].
> Log the actual nowInSeconds used by queries in full query log
> -
>
> Key: CASSANDRA-14671
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14671
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> FQL doesn't currently use the actual {{nowInSeconds}} value used for request
> execution. It needs to, to allow for - in conjunction with CASSANDRA-14664 -
> less indeterministic playback tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14664) Allow providing and overriding nowInSeconds via native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14664: -- Fix Version/s: (was: 4.0) 4.0.x > Allow providing and overriding nowInSeconds via native protocol > --- > > Key: CASSANDRA-14664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14664 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: client-impacting, protocolv5 > Fix For: 4.0.x > > > For FQL replay testing, to allow for deterministic and repeatable workload > replay comparisons, we need to be able to set custom nowInSeconds via native > protocol - primarily to control TTL expiration, both on read and write paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14671) Log the actual nowInSeconds used by queries in full query log
[
https://issues.apache.org/jira/browse/CASSANDRA-14671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14671:
--
Status: Patch Available (was: Open)
> Log the actual nowInSeconds used by queries in full query log
> -
>
> Key: CASSANDRA-14671
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14671
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> FQL doesn't currently use the actual {{nowInSeconds}} value used for request
> execution. It needs to, to allow for - in conjunction with CASSANDRA-14664 -
> less indeterministic playback tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14671) Log the actual nowInSeconds used by queries in full query log
[
https://issues.apache.org/jira/browse/CASSANDRA-14671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595630#comment-16595630
]
Aleksey Yeschenko commented on CASSANDRA-14671:
---
Code [here|https://github.com/iamaleksey/cassandra/commits/14671-4.0], CI
[here|https://circleci.com/workflow-run/ce1008e6-7a5f-4fa3-b242-b61c2af8ab51].
> Log the actual nowInSeconds used by queries in full query log
> -
>
> Key: CASSANDRA-14671
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14671
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> FQL doesn't currently use the actual {{nowInSeconds}} value used for request
> execution. It needs to, to allow for - in conjunction with CASSANDRA-14664 -
> less indeterministic playback tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14671) Log the actual nowInSeconds used by queries in full query log
[
https://issues.apache.org/jira/browse/CASSANDRA-14671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14671:
--
Reviewer: Chris Lohfink
> Log the actual nowInSeconds used by queries in full query log
> -
>
> Key: CASSANDRA-14671
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14671
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 4.0.x
>
>
> FQL doesn't currently use the actual {{nowInSeconds}} value used for request
> execution. It needs to, to allow for - in conjunction with CASSANDRA-14664 -
> less indeterministic playback tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14664) Allow providing and overriding nowInSeconds via native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14664: -- Reviewer: Chris Lohfink > Allow providing and overriding nowInSeconds via native protocol > --- > > Key: CASSANDRA-14664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14664 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: client-impacting, protocolv5 > Fix For: 4.0 > > > For FQL replay testing, to allow for deterministic and repeatable workload > replay comparisons, we need to be able to set custom nowInSeconds via native > protocol - primarily to control TTL expiration, both on read and write paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14568) Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages
[
https://issues.apache.org/jira/browse/CASSANDRA-14568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595319#comment-16595319
]
Aleksey Yeschenko commented on CASSANDRA-14568:
---
Looks good to me as well. Sorry for failing to spot this in the first place.
Old memories are fading.
> Static collection deletions are corrupted in 3.0 -> 2.{1,2} messages
>
>
> Key: CASSANDRA-14568
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14568
> Project: Cassandra
> Issue Type: Bug
>Reporter: Benedict
>Assignee: Benedict
>Priority: Critical
> Fix For: 3.0.17, 3.11.3
>
>
> In 2.1 and 2.2, row and complex deletions were represented as range
> tombstones. LegacyLayout is our compatibility layer, that translates the
> relevant RT patterns in 2.1/2.2 to row/complex deletions in 3.0, and vice
> versa. Unfortunately, it does not handle the special case of static row
> deletions, they are treated as regular row deletions. Since static rows are
> themselves never directly deleted, the only issue is with collection
> deletions.
> Collection deletions in 2.1/2.2 were encoded as a range tombstone, consisting
> of a sequence of the clustering keys’ data for the affected row, followed by
> the bytes representing the name of the collection column. STATIC_CLUSTERING
> contains zero clusterings, so by treating the deletion as for a regular row,
> zero clusterings are written to precede the column name of the erased
> collection, so the column name is written at position zero.
> This can exhibit itself in at least two ways:
> # If the type of your first clustering key is a variable width type, new
> deletes will begin appearing covering the clustering key represented by the
> column name.
> ** If you have multiple clustering keys, you will receive a RT covering all
> those rows with a matching first clustering key.
> ** This RT will be valid as far as the system is concerned, and go
> undetected unless there are outside data quality checks in place.
> # Otherwise, an invalid size of data will be written to the clustering and
> sent over the network to the 2.1 node.
> ** The 2.1/2.2 node will handle this just fine, even though the record is
> junk. Since it is a deletion covering impossible data, there will be no
> user-API visible effect. But if received as a write from a 3.0 node, it will
> dutifully persist the junk record.
> ** The 3.0 node that originally sent this junk, may later coordinate a read
> of the partition, and will notice a digest mismatch, read-repair and
> serialize the junk to disk
> ** The sstable containing this record is now corrupt; the deserialization
> expects fixed-width data, but it encounters too many (or too few) bytes, and
> is now at an incorrect position to read its structural information
> ** (Alternatively when the 2.1 node is upgraded this will occur on eventual
> compaction)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14671) Log the actual nowInSeconds used by queries in full query log
Aleksey Yeschenko created CASSANDRA-14671:
-
Summary: Log the actual nowInSeconds used by queries in full query
log
Key: CASSANDRA-14671
URL: https://issues.apache.org/jira/browse/CASSANDRA-14671
Project: Cassandra
Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Fix For: 4.0.x
FQL doesn't currently use the actual {{nowInSeconds}} value used for request
execution. It needs to, to allow for - in conjunction with CASSANDRA-14664 -
less indeterministic playback tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14664) Allow providing and overriding nowInSeconds via native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592161#comment-16592161 ] Aleksey Yeschenko edited comment on CASSANDRA-14664 at 8/24/18 9:04 PM: 4.0 branch [here|https://github.com/iamaleksey/cassandra/commits/14664-4.0], CI [here|https://circleci.com/workflow-run/806f605f-a63d-44c2-a03b-9883b217612f]. For end-to-end testing I used [these changes to the driver|https://github.com/iamaleksey/java-driver/commits/14664], which I cannot contribute, yet, for CLA reasons, but they are sufficient to use for verification purposes - in addition to provided unit tests. was (Author: iamaleksey): 4.0 branch [here|https://github.com/iamaleksey/cassandra/commits/14664-4.0], CI [here|https://circleci.com/workflow-run/806f605f-a63d-44c2-a03b-9883b217612f]. For end-to-end testing I used [these changed to the driver|https://github.com/iamaleksey/java-driver/commits/14664], which I cannot contribute, yet, for CLA reasons, but they are sufficient to use for verification purposes - in addition to provided unit tests. > Allow providing and overriding nowInSeconds via native protocol > --- > > Key: CASSANDRA-14664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14664 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: protocolv5 > Fix For: 4.0 > > > For FQL replay testing, to allow for deterministic and repeatable workload > replay comparisons, we need to be able to set custom nowInSeconds via native > protocol - primarily to control TTL expiration, both on read and write paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14664) Allow providing and overriding nowInSeconds via native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592161#comment-16592161 ] Aleksey Yeschenko commented on CASSANDRA-14664: --- 4.0 branch [here|https://github.com/iamaleksey/cassandra/commits/14664-4.0], CI [here|https://circleci.com/workflow-run/806f605f-a63d-44c2-a03b-9883b217612f]. For end-to-end testing I used [these changed to the driver|https://github.com/iamaleksey/java-driver/commits/14664], which I cannot contribute, yet, for CLA reasons, but they are sufficient to use for verification purposes - in addition to provided unit tests. > Allow providing and overriding nowInSeconds via native protocol > --- > > Key: CASSANDRA-14664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14664 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: protocolv5 > Fix For: 4.0 > > > For FQL replay testing, to allow for deterministic and repeatable workload > replay comparisons, we need to be able to set custom nowInSeconds via native > protocol - primarily to control TTL expiration, both on read and write paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14664) Allow providing and overriding nowInSeconds via native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14664: -- Status: Patch Available (was: Open) > Allow providing and overriding nowInSeconds via native protocol > --- > > Key: CASSANDRA-14664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14664 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: protocolv5 > Fix For: 4.0 > > > For FQL replay testing, to allow for deterministic and repeatable workload > replay comparisons, we need to be able to set custom nowInSeconds via native > protocol - primarily to control TTL expiration, both on read and write paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14664) Allow providing and overriding nowInSeconds via native protocol
Aleksey Yeschenko created CASSANDRA-14664: - Summary: Allow providing and overriding nowInSeconds via native protocol Key: CASSANDRA-14664 URL: https://issues.apache.org/jira/browse/CASSANDRA-14664 Project: Cassandra Issue Type: New Feature Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 4.0 For FQL replay testing, to allow for deterministic and repeatable workload replay comparisons, we need to be able to set custom nowInSeconds via native protocol - primarily to control TTL expiration, both on read and write paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14664) Allow providing and overriding nowInSeconds via native protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-14664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14664: -- Labels: protocolv5 (was: ) > Allow providing and overriding nowInSeconds via native protocol > --- > > Key: CASSANDRA-14664 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14664 > Project: Cassandra > Issue Type: New Feature >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: protocolv5 > Fix For: 4.0 > > > For FQL replay testing, to allow for deterministic and repeatable workload > replay comparisons, we need to be able to set custom nowInSeconds via native > protocol - primarily to control TTL expiration, both on read and write paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14635) Support table level configuration of monotonic reads
[
https://issues.apache.org/jira/browse/CASSANDRA-14635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590476#comment-16590476
]
Aleksey Yeschenko commented on CASSANDRA-14635:
---
{{SchemaKeyspace.getReadRepairStrategy()}} can be {{private}},
{{ReadRepairVerbHandler}} changes can now be rolled back beacause {{ASYNC}} is
gone. Both can be fixed on commit, LGTM so long as CI is happy.
> Support table level configuration of monotonic reads
>
>
> Key: CASSANDRA-14635
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14635
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Ariel Weisberg
>Assignee: Blake Eggleston
>Priority: Major
> Fix For: 4.0
>
>
> In CASSANDRA-10726 it was discussed that allowing expert users to forgo
> monotonic reads might be desirable. It is practical to control monotonicity
> of reads at a fine grained level because it involves changing the behavior of
> read repair on a per read basis.
> Per CASSANDRA-14593 we already don't preserve update atomicity down to the
> column level. You could read the key out of a row and read repair the key,
> pass the key to another process which attempts to read the value, but finds
> the value is null because read repair only repairs the data (including
> columns) that is part of the read. IMO it's a stretch to say that reads are
> monotonic. It is technically correct, the best kind of correct, but far from
> as useful as it should be.
> An initial implementation could make read repair asynchronous or forgo read
> repair entirely. This would improve the throughput and latency of reads.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14635) Support table level configuration of monotonic reads
[ https://issues.apache.org/jira/browse/CASSANDRA-14635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14635: -- Status: Ready to Commit (was: Patch Available) > Support table level configuration of monotonic reads > > > Key: CASSANDRA-14635 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14635 > Project: Cassandra > Issue Type: Improvement > Components: Coordination >Reporter: Ariel Weisberg >Assignee: Blake Eggleston >Priority: Major > Fix For: 4.0 > > > In CASSANDRA-10726 it was discussed that allowing expert users to forgo > monotonic reads might be desirable. It is practical to control monotonicity > of reads at a fine grained level because it involves changing the behavior of > read repair on a per read basis. > Per CASSANDRA-14593 we already don't preserve update atomicity down to the > column level. You could read the key out of a row and read repair the key, > pass the key to another process which attempts to read the value, but finds > the value is null because read repair only repairs the data (including > columns) that is part of the read. IMO it's a stretch to say that reads are > monotonic. It is technically correct, the best kind of correct, but far from > as useful as it should be. > An initial implementation could make read repair asynchronous or forgo read > repair entirely. This would improve the throughput and latency of reads. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14628:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16590174#comment-16590174
]
Aleksey Yeschenko commented on CASSANDRA-14628:
---
Thanks, committed to 4.0 as
[6a473c4dbe31a64e6fcccdb6760347949d1d321a|https://github.com/apache/cassandra/commit/6a473c4dbe31a64e6fcccdb6760347949d1d321a].
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14628:
--
Reviewers: Benedict, Marcus Eriksson
Reviewer: (was: Benedict)
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14628:
--
Status: Patch Available (was: Reopened)
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Reopened] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko reopened CASSANDRA-14628:
---
As spotted by [~krummas], the final version broke a couple nodetool dtests. I
ran full CI suite on the initial patch, but only unit tests for the revised
version, and did not notice that {{CassandraMetricsRegistry.registerMBean()}}
needed a minor change for {{Metered}} to be properly accepted.
The fix is
[here|https://github.com/iamaleksey/cassandra/commits/14628-fix-jmx], its CI
[here|https://circleci.com/workflow-run/2846f7e8-10a9-4499-bec6-327cc05e2a04] -
there is one unrelated nodetool failure, but the issue with this patch is gone.
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14635) Support table level configuration of monotonic reads
[
https://issues.apache.org/jira/browse/CASSANDRA-14635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16589392#comment-16589392
]
Aleksey Yeschenko commented on CASSANDRA-14635:
---
Looks like a relatively straightforward small, correct change, so I don't have
a lot of feedback here.
1. I'd like the get-repair-strategy-from-row logic to be moved inline in
{{SchemaKeyspace}} - with a comment. Such logic historically belongs to
{{SchemaKeyspace}} or its old equivalent.
2. {{cqlsh}} should be altered to add the new param for completion - it's
relatively straightforward
3. There is a single-iteration (at most) loop in
{{AbstractReadRepair.maybeSendAdditionalReads()}} that should probably be
rewritten using {{Iterables.find()}} or {{Iterables.tryFind()}}
4. {{ReconcileRead}} as the metric name for what it is; we should come up with
something better, though I myself am out of ideas right now
Outside of those three points, some general thoughts that don't need to be
addressed:
- I'm not convinced that {{ASYNC}} is an option that will be used by folks, and
also am concerned with complete lack of backpressure there. With {{BLOCKING}}
we at least have effective pressure via reads being blocked, here it might be
dangerous, like the old replicate on write stage for counters
- Not a fan of {{ReadOnlyReadRepair}} name. I know you aren't either, and I
don't have better ideas
> Support table level configuration of monotonic reads
>
>
> Key: CASSANDRA-14635
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14635
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Ariel Weisberg
>Assignee: Blake Eggleston
>Priority: Major
> Fix For: 4.0
>
>
> In CASSANDRA-10726 it was discussed that allowing expert users to forgo
> monotonic reads might be desirable. It is practical to control monotonicity
> of reads at a fine grained level because it involves changing the behavior of
> read repair on a per read basis.
> Per CASSANDRA-14593 we already don't preserve update atomicity down to the
> column level. You could read the key out of a row and read repair the key,
> pass the key to another process which attempts to read the value, but finds
> the value is null because read repair only repairs the data (including
> columns) that is part of the read. IMO it's a stretch to say that reads are
> monotonic. It is technically correct, the best kind of correct, but far from
> as useful as it should be.
> An initial implementation could make read repair asynchronous or forgo read
> repair entirely. This would improve the throughput and latency of reads.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14592) Reconcile should not be dependent on nowInSec
[ https://issues.apache.org/jira/browse/CASSANDRA-14592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14592: -- Status: Ready to Commit (was: Patch Available) > Reconcile should not be dependent on nowInSec > - > > Key: CASSANDRA-14592 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14592 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Major > Fix For: 4.0 > > > To have the arrival time of a mutation on a replica determine the > reconciliation priority seems to provide for unintuitive database behaviour. > It seems we should formalise our reconciliation logic in a manner that does > not depend on this, and modify our internal APIs to prevent this dependency. > > Take the following example, where both writes have the same timestamp: > > Write X with a value A, TTL of 1s > Write Y with a value B, no TTL > > If X and Y arrive on replicas in < 1s, X and Y are both live, so record Y > wins the reconciliation. The value B appears in the database. > However, if X and Y arrive on replicas in > 1s, X is now (effectively) a > tombstone. This wins the reconciliation race, and NO value is the result. > > Note that the weirdness of this is more pronounced than it might first > appear. If write X gets stuck in hints for a period on the coordinator to > one replica, the value B appears in the database until the hint is replayed. > So now we’re in a very uncertain state - will hints get replayed or not? If > they do, the value B will disappear; if they don’t it won’t. This is despite > a QUORUM of replicas ACKing both writes, and a QUORUM of readers being > engaged on read; the database still changes state to the user suddenly at > some arbitrary future point in time. > > It seems to me that a simple solution to this, is to permit TTL’d data to > always win a reconciliation against non-TTL’d data (of same timestamp), so > that we are consistent across TTLs being transformed into tombstones. > > 4.0 seems like a good opportunity to fix this behaviour, and mention in > CHANGES.txt. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10726) Read repair inserts should not be blocking
[
https://issues.apache.org/jira/browse/CASSANDRA-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-10726:
--
Fix Version/s: (was: 4.x)
4.0
> Read repair inserts should not be blocking
> --
>
> Key: CASSANDRA-10726
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10726
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Richard Low
>Assignee: Blake Eggleston
>Priority: Major
> Fix For: 4.0
>
>
> Today, if there’s a digest mismatch in a foreground read repair, the insert
> to update out of date replicas is blocking. This means, if it fails, the read
> fails with a timeout. If a node is dropping writes (maybe it is overloaded or
> the mutation stage is backed up for some other reason), all reads to a
> replica set could fail. Further, replicas dropping writes get more out of
> sync so will require more read repair.
> The comment on the code for why the writes are blocking is:
> {code}
> // wait for the repair writes to be acknowledged, to minimize impact on any
> replica that's
> // behind on writes in case the out-of-sync row is read multiple times in
> quick succession
> {code}
> but the bad side effect is that reads timeout. Either the writes should not
> be blocking or we should return success for the read even if the write times
> out.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14592) Reconcile should not be dependent on nowInSec
[ https://issues.apache.org/jira/browse/CASSANDRA-14592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587746#comment-16587746 ] Aleksey Yeschenko commented on CASSANDRA-14592: --- +1 > Reconcile should not be dependent on nowInSec > - > > Key: CASSANDRA-14592 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14592 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Major > Fix For: 4.0 > > > To have the arrival time of a mutation on a replica determine the > reconciliation priority seems to provide for unintuitive database behaviour. > It seems we should formalise our reconciliation logic in a manner that does > not depend on this, and modify our internal APIs to prevent this dependency. > > Take the following example, where both writes have the same timestamp: > > Write X with a value A, TTL of 1s > Write Y with a value B, no TTL > > If X and Y arrive on replicas in < 1s, X and Y are both live, so record Y > wins the reconciliation. The value B appears in the database. > However, if X and Y arrive on replicas in > 1s, X is now (effectively) a > tombstone. This wins the reconciliation race, and NO value is the result. > > Note that the weirdness of this is more pronounced than it might first > appear. If write X gets stuck in hints for a period on the coordinator to > one replica, the value B appears in the database until the hint is replayed. > So now we’re in a very uncertain state - will hints get replayed or not? If > they do, the value B will disappear; if they don’t it won’t. This is despite > a QUORUM of replicas ACKing both writes, and a QUORUM of readers being > engaged on read; the database still changes state to the user suddenly at > some arbitrary future point in time. > > It seems to me that a simple solution to this, is to permit TTL’d data to > always win a reconciliation against non-TTL’d data (of same timestamp), so > that we are consistent across TTLs being transformed into tombstones. > > 4.0 seems like a good opportunity to fix this behaviour, and mention in > CHANGES.txt. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14592) Reconcile should not be dependent on nowInSec
[ https://issues.apache.org/jira/browse/CASSANDRA-14592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14592: -- Reviewer: Aleksey Yeschenko > Reconcile should not be dependent on nowInSec > - > > Key: CASSANDRA-14592 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14592 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Assignee: Benedict >Priority: Major > Fix For: 4.0 > > > To have the arrival time of a mutation on a replica determine the > reconciliation priority seems to provide for unintuitive database behaviour. > It seems we should formalise our reconciliation logic in a manner that does > not depend on this, and modify our internal APIs to prevent this dependency. > > Take the following example, where both writes have the same timestamp: > > Write X with a value A, TTL of 1s > Write Y with a value B, no TTL > > If X and Y arrive on replicas in < 1s, X and Y are both live, so record Y > wins the reconciliation. The value B appears in the database. > However, if X and Y arrive on replicas in > 1s, X is now (effectively) a > tombstone. This wins the reconciliation race, and NO value is the result. > > Note that the weirdness of this is more pronounced than it might first > appear. If write X gets stuck in hints for a period on the coordinator to > one replica, the value B appears in the database until the hint is replayed. > So now we’re in a very uncertain state - will hints get replayed or not? If > they do, the value B will disappear; if they don’t it won’t. This is despite > a QUORUM of replicas ACKing both writes, and a QUORUM of readers being > engaged on read; the database still changes state to the user suddenly at > some arbitrary future point in time. > > It seems to me that a simple solution to this, is to permit TTL’d data to > always win a reconciliation against non-TTL’d data (of same timestamp), so > that we are consistent across TTLs being transformed into tombstones. > > 4.0 seems like a good opportunity to fix this behaviour, and mention in > CHANGES.txt. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14573) Expose settings in virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587566#comment-16587566
]
Aleksey Yeschenko commented on CASSANDRA-14573:
---
Committed the end version with some very minor tweaks on top as
[994da255cec95982f52d20c91cb18eb7f9e45fc3|https://github.com/apache/cassandra/commit/994da255cec95982f52d20c91cb18eb7f9e45fc3]
to 4.0.
Cheers.
> Expose settings in virtual table
>
>
> Key: CASSANDRA-14573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14573
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: pull-request-available, virtual-tables
> Fix For: 4.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Allow both viewing what the settings are (currently impossible for some) and
> allow changing some settings.
> Example:
> {code:java}
> UPDATE system_info.settings SET value = 'false' WHERE setting =
> 'hinted_handoff_enabled';
> SELECT * FROM system_info.settings WHERE writable = True;
> setting | value | writable
> --++--
> batch_size_fail_threshold_in_kb | 50 | True
> batch_size_warn_threshold_in_kb | 5 | True
> cas_contention_timeout_in_ms | 1000 | True
> compaction_throughput_mb_per_sec | 16 | True
> concurrent_compactors | 2 | True
>concurrent_validations | 2147483647 | True
> counter_write_request_timeout_in_ms | 5000 | True
>hinted_handoff_enabled | false | True
> hinted_handoff_throttle_in_kb | 1024 | True
> incremental_backups | false | True
> inter_dc_stream_throughput_outbound_megabits_per_sec |200 | True
> phi_convict_threshold |8.0 | True
> range_request_timeout_in_ms | 1 | True
>read_request_timeout_in_ms | 5000 | True
> request_timeout_in_ms | 1 | True
> stream_throughput_outbound_megabits_per_sec |200 | True
> tombstone_failure_threshold | 10 | True
> tombstone_warn_threshold | 1000 | True
>truncate_request_timeout_in_ms | 6 | True
> write_request_timeout_in_ms | 2000 |
> True{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14573) Expose settings in virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14573:
--
Resolution: Fixed
Fix Version/s: (was: 4.x)
4.0
Status: Resolved (was: Patch Available)
> Expose settings in virtual table
>
>
> Key: CASSANDRA-14573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14573
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: pull-request-available, virtual-tables
> Fix For: 4.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Allow both viewing what the settings are (currently impossible for some) and
> allow changing some settings.
> Example:
> {code:java}
> UPDATE system_info.settings SET value = 'false' WHERE setting =
> 'hinted_handoff_enabled';
> SELECT * FROM system_info.settings WHERE writable = True;
> setting | value | writable
> --++--
> batch_size_fail_threshold_in_kb | 50 | True
> batch_size_warn_threshold_in_kb | 5 | True
> cas_contention_timeout_in_ms | 1000 | True
> compaction_throughput_mb_per_sec | 16 | True
> concurrent_compactors | 2 | True
>concurrent_validations | 2147483647 | True
> counter_write_request_timeout_in_ms | 5000 | True
>hinted_handoff_enabled | false | True
> hinted_handoff_throttle_in_kb | 1024 | True
> incremental_backups | false | True
> inter_dc_stream_throughput_outbound_megabits_per_sec |200 | True
> phi_convict_threshold |8.0 | True
> range_request_timeout_in_ms | 1 | True
>read_request_timeout_in_ms | 5000 | True
> request_timeout_in_ms | 1 | True
> stream_throughput_outbound_megabits_per_sec |200 | True
> tombstone_failure_threshold | 10 | True
> tombstone_warn_threshold | 1000 | True
>truncate_request_timeout_in_ms | 6 | True
> write_request_timeout_in_ms | 2000 |
> True{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14626) Expose buffer cache metrics in caches virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-14626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14626: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Expose buffer cache metrics in caches virtual table > --- > > Key: CASSANDRA-14626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14626 > Project: Cassandra > Issue Type: New Feature >Reporter: Benjamin Lerer >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: virtual-tables > Fix For: 4.0 > > > As noted by [~blerer] in CASSANDRA-14538, we should expose buffer cache > metrics in the caches virtual table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14626) Expose buffer cache metrics in caches virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-14626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587360#comment-16587360 ] Aleksey Yeschenko commented on CASSANDRA-14626: --- Committed as [6d1446ff062ac322b203e16ea0bf0ed8fd1fa5ca|https://github.com/apache/cassandra/commit/6d1446ff062ac322b203e16ea0bf0ed8fd1fa5ca] to 4.0, thanks. > Expose buffer cache metrics in caches virtual table > --- > > Key: CASSANDRA-14626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14626 > Project: Cassandra > Issue Type: New Feature >Reporter: Benjamin Lerer >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: virtual-tables > Fix For: 4.0 > > > As noted by [~blerer] in CASSANDRA-14538, we should expose buffer cache > metrics in the caches virtual table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587316#comment-16587316
]
Aleksey Yeschenko commented on CASSANDRA-14628:
---
Cheers, committed as
[e6b8e7a72f783ed0e1b5a2c04381f89b533229a4|https://github.com/apache/cassandra/commit/e6b8e7a72f783ed0e1b5a2c04381f89b533229a4]
to 4.0.
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14628:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587281#comment-16587281
]
Aleksey Yeschenko edited comment on CASSANDRA-14628 at 8/21/18 10:57 AM:
-
bq. Modernise all of the RatioGauge construction as well, via a static method
that accepts a Supplier
Looks good. Can look better though. Might as well go all the way and make a
static method that accepts two {{DoubleSupplier}} s
bq. Replace requests with a Metered that proxies its calls to getX onto
hits.getX() + misses.getX()
Good call.
Cherry-picked, and pushed an updated commit on top that reformats things, fixes
import order, and makes the {{DoubleSupplier}} change.
was (Author: iamaleksey):
bq. Modernise all of the RatioGauge construction as well, via a static method
that accepts a Supplier
Looks good. Can look better though. Might as well go all the way and make a
static method that accepts two {{DoubleSupplier}}s
bq. Replace requests with a Metered that proxies its calls to getX onto
hits.getX() + misses.getX()
Good call.
Cherry-picked, and pushed an updated commit on top that reformats things, fixes
import order, and makes the {{DoubleSupplier}} change.
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16587281#comment-16587281
]
Aleksey Yeschenko commented on CASSANDRA-14628:
---
bq. Modernise all of the RatioGauge construction as well, via a static method
that accepts a Supplier
Looks good. Can look better though. Might as well go all the way and make a
static method that accepts two {{DoubleSupplier}}s
bq. Replace requests with a Metered that proxies its calls to getX onto
hits.getX() + misses.getX()
Good call.
Cherry-picked, and pushed an updated commit on top that reformats things, fixes
import order, and makes the {{DoubleSupplier}} change.
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14635) Support table level configuration of monotonic reads
[ https://issues.apache.org/jira/browse/CASSANDRA-14635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14635: -- Reviewer: Aleksey Yeschenko > Support table level configuration of monotonic reads > > > Key: CASSANDRA-14635 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14635 > Project: Cassandra > Issue Type: Improvement > Components: Coordination >Reporter: Ariel Weisberg >Assignee: Blake Eggleston >Priority: Major > > In CASSANDRA-10726 it was discussed that allowing expert users to forgo > monotonic reads might be desirable. It is practical to control monotonicity > of reads at a fine grained level because it involves changing the behavior of > read repair on a per read basis. > Per CASSANDRA-14593 we already don't preserve update atomicity down to the > column level. You could read the key out of a row and read repair the key, > pass the key to another process which attempts to read the value, but finds > the value is null because read repair only repairs the data (including > columns) that is part of the read. IMO it's a stretch to say that reads are > monotonic. It is technically correct, the best kind of correct, but far from > as useful as it should be. > An initial implementation could make read repair asynchronous or forgo read > repair entirely. This would improve the throughput and latency of reads. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static col
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14638:
--
Resolution: Fixed
Fix Version/s: (was: 4.0.x)
(was: 3.11.x)
(was: 3.0.x)
4.0
3.11.4
3.0.18
Status: Resolved (was: Patch Available)
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.18, 3.11.4, 4.0
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
> t.printStackTrace();
[jira] [Commented] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static c
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582623#comment-16582623
]
Aleksey Yeschenko commented on CASSANDRA-14638:
---
Cheers, committed as
[236c47e65ce95dcc7c8c75706715b5a2a88fd237|https://github.com/apache/cassandra/commit/236c47e65ce95dcc7c8c75706715b5a2a88fd237]
to 3.0 and merged up to 3.11 and trunk.
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
> t.printStackTrace();
> } finally {
>
[jira] [Commented] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static c
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582451#comment-16582451
]
Aleksey Yeschenko commented on CASSANDRA-14638:
---
[~benedict] That's indeed better and less invasive. Force-pushed the new
version, updated CI links in-place in the previous comment.
[~slebresne] Agreed on all points. Updated documentation in the v2 of the patch.
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
> t.printStackTrace();
>
[jira] [Comment Edited] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when sta
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581500#comment-16581500
]
Aleksey Yeschenko edited comment on CASSANDRA-14638 at 8/16/18 12:36 PM:
-
Branches with the fix for
[3.0|https://github.com/iamaleksey/cassandra/commits/14638-3.0],
[3.11|https://github.com/iamaleksey/cassandra/commits/14638-3.11], and
[4.0|https://github.com/iamaleksey/cassandra/commits/14638-4.0]. CI,
respectively:
[3.0|https://circleci.com/workflow-run/6be0f2b2-292c-4c9e-81fe-a1b90a86421e],
[3.11|https://circleci.com/workflow-run/a771039b-aa8f-4eea-a268-822c759a1c8e],
and
[4.0|https://circleci.com/workflow-run/ce2620d7-2317-43ef-8f96-90d4f53ee175].
was (Author: iamaleksey):
Branches with the fix for
[3.0|https://github.com/iamaleksey/cassandra/commits/14638-3.0],
[3.11|https://github.com/iamaleksey/cassandra/commits/14638-3.11], and
[4.0|https://github.com/iamaleksey/cassandra/commits/14638-4.0]. CI,
respectively:
[3.0|https://circleci.com/workflow-run/86ae48d8-ef0c-45ee-a978-7afd99b7b1ad],
[3.11|https://circleci.com/workflow-run/a048437a-beaf-4ba8-9482-635ed3412737],
and
[4.0|https://circleci.com/workflow-run/9eb91625-60ff-46e6-9a8d-289b48314f77].
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p =
[jira] [Updated] (CASSANDRA-14645) Clean up some dead code post CASSANDRA-13910
[ https://issues.apache.org/jira/browse/CASSANDRA-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14645: -- Status: Patch Available (was: Open) > Clean up some dead code post CASSANDRA-13910 > > > Key: CASSANDRA-14645 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14645 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 4.0.x > > > Wasn't quite surgical *enough* and missed one unused field and one dead > branch of code. The former mentioned to me by [~ifesdjeen], the latter by > [~benedict]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14645) Clean up some dead code post CASSANDRA-13910
[ https://issues.apache.org/jira/browse/CASSANDRA-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14645: -- Resolution: Fixed Fix Version/s: (was: 4.0.x) 4.0 Status: Resolved (was: Patch Available) > Clean up some dead code post CASSANDRA-13910 > > > Key: CASSANDRA-14645 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14645 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 4.0 > > > Wasn't quite surgical *enough* and missed one unused field and one dead > branch of code. The former mentioned to me by [~ifesdjeen], the latter by > [~benedict]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14645) Clean up some dead code post CASSANDRA-13910
[ https://issues.apache.org/jira/browse/CASSANDRA-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16582376#comment-16582376 ] Aleksey Yeschenko commented on CASSANDRA-14645: --- Committed as [07b0aca6d3b423f9af405b2d64cd285c597023c4|https://github.com/apache/cassandra/commit/07b0aca6d3b423f9af405b2d64cd285c597023c4] to trunk, without a CHANGES.txt entry. Thanks. > Clean up some dead code post CASSANDRA-13910 > > > Key: CASSANDRA-14645 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14645 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 4.0 > > > Wasn't quite surgical *enough* and missed one unused field and one dead > branch of code. The former mentioned to me by [~ifesdjeen], the latter by > [~benedict]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14645) Clean up some dead code post CASSANDRA-13910
[ https://issues.apache.org/jira/browse/CASSANDRA-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581798#comment-16581798 ] Aleksey Yeschenko edited comment on CASSANDRA-14645 at 8/16/18 10:49 AM: - Code [here|https://github.com/iamaleksey/cassandra/commits/14645-4.0], CI, just in case, [here|https://circleci.com/workflow-run/d0acf81b-bab2-4575-98e3-ba9716b1dcb2] was (Author: iamaleksey): Code [here|https://github.com/iamaleksey/cassandra/commits/14645-4.0], CI, just in case, [here|https://circleci.com/workflow-run/24504926-80a3-4c79-9c80-95fe54a5de53] > Clean up some dead code post CASSANDRA-13910 > > > Key: CASSANDRA-14645 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14645 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 4.0.x > > > Wasn't quite surgical *enough* and missed one unused field and one dead > branch of code. The former mentioned to me by [~ifesdjeen], the latter by > [~benedict]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14645) Clean up some dead code post CASSANDRA-13910
[ https://issues.apache.org/jira/browse/CASSANDRA-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14645: -- Status: Open (was: Patch Available) > Clean up some dead code post CASSANDRA-13910 > > > Key: CASSANDRA-14645 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14645 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 4.0.x > > > Wasn't quite surgical *enough* and missed one unused field and one dead > branch of code. The former mentioned to me by [~ifesdjeen], the latter by > [~benedict]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14645) Clean up some dead code post CASSANDRA-13910
[ https://issues.apache.org/jira/browse/CASSANDRA-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581798#comment-16581798 ] Aleksey Yeschenko edited comment on CASSANDRA-14645 at 8/16/18 12:49 AM: - Code [here|https://github.com/iamaleksey/cassandra/commits/14645-4.0], CI, just in case, [here|https://circleci.com/workflow-run/24504926-80a3-4c79-9c80-95fe54a5de53] was (Author: iamaleksey): Code [here|https://github.com/iamaleksey/cassandra/commits/14645-4.0], CI, just in case, [here|https://circleci.com/workflow-run/2da246dd-7562-47c6-9d68-9a83d92376a3] > Clean up some dead code post CASSANDRA-13910 > > > Key: CASSANDRA-14645 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14645 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 4.0.x > > > Wasn't quite surgical *enough* and missed one unused field and one dead > branch of code. The former mentioned to me by [~ifesdjeen], the latter by > [~benedict]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14645) Clean up some dead code post CASSANDRA-13910
[ https://issues.apache.org/jira/browse/CASSANDRA-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581798#comment-16581798 ] Aleksey Yeschenko commented on CASSANDRA-14645: --- Code [here|https://github.com/iamaleksey/cassandra/commits/14645-4.0], CI, just in case, [here|https://circleci.com/workflow-run/2da246dd-7562-47c6-9d68-9a83d92376a3] > Clean up some dead code post CASSANDRA-13910 > > > Key: CASSANDRA-14645 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14645 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 4.0.x > > > Wasn't quite surgical *enough* and missed one unused field and one dead > branch of code. The former mentioned to me by [~ifesdjeen], the latter by > [~benedict]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14645) Clean up some dead code post CASSANDRA-13910
[ https://issues.apache.org/jira/browse/CASSANDRA-14645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14645: -- Status: Patch Available (was: In Progress) > Clean up some dead code post CASSANDRA-13910 > > > Key: CASSANDRA-14645 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14645 > Project: Cassandra > Issue Type: Improvement >Reporter: Aleksey Yeschenko >Assignee: Aleksey Yeschenko >Priority: Trivial > Fix For: 4.0.x > > > Wasn't quite surgical *enough* and missed one unused field and one dead > branch of code. The former mentioned to me by [~ifesdjeen], the latter by > [~benedict]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14645) Clean up some dead code post CASSANDRA-13910
Aleksey Yeschenko created CASSANDRA-14645: - Summary: Clean up some dead code post CASSANDRA-13910 Key: CASSANDRA-14645 URL: https://issues.apache.org/jira/browse/CASSANDRA-14645 Project: Cassandra Issue Type: Improvement Reporter: Aleksey Yeschenko Assignee: Aleksey Yeschenko Fix For: 4.0.x Wasn't quite surgical *enough* and missed one unused field and one dead branch of code. The former mentioned to me by [~ifesdjeen], the latter by [~benedict]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static col
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14638:
--
Fix Version/s: 4.0.x
Status: Patch Available (was: In Progress)
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
> t.printStackTrace();
> } finally {
> Uninterruptibles.sleepUninterruptibly(30, TimeUnit.SECONDS);
> }
> }
> } finally {
> cluster.close();
> }
> }
> }
>
[jira] [Commented] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static c
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581500#comment-16581500
]
Aleksey Yeschenko commented on CASSANDRA-14638:
---
Branches with the fix for
[3.0|https://github.com/iamaleksey/cassandra/commits/14638-3.0],
[3.11|https://github.com/iamaleksey/cassandra/commits/14638-3.11], and
[4.0|https://github.com/iamaleksey/cassandra/commits/14638-4.0]. CI,
respectively:
[3.0|https://circleci.com/workflow-run/86ae48d8-ef0c-45ee-a978-7afd99b7b1ad],
[3.11|https://circleci.com/workflow-run/a048437a-beaf-4ba8-9482-635ed3412737],
and
[4.0|https://circleci.com/workflow-run/9eb91625-60ff-46e6-9a8d-289b48314f77].
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
>
[jira] [Commented] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static c
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581459#comment-16581459
]
Aleksey Yeschenko commented on CASSANDRA-14638:
---
bq. I was just thinking for the internode part.
Luckily internode messaging is not affected by this bug.
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
> t.printStackTrace();
> } finally {
> Uninterruptibles.sleepUninterruptibly(30, TimeUnit.SECONDS);
> }
> }
>
[jira] [Commented] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static c
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581341#comment-16581341
]
Aleksey Yeschenko commented on CASSANDRA-14638:
---
[~slebresne] I assume you may have an opinion on this. If so, please share.
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
> t.printStackTrace();
> } finally {
> Uninterruptibles.sleepUninterruptibly(30, TimeUnit.SECONDS);
> }
> }
> } finally {
>
[jira] [Commented] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static c
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581304#comment-16581304
]
Aleksey Yeschenko commented on CASSANDRA-14638:
---
bq. Unfortunately I don't think there is an easy fix for this. If the order
was changed back to the previous format, you risk issues for users upgrading
from older 3.0 version.
This is true. That said, there are huge fleets still on 2.1 and 2.2 in the
wild, so leaving this unfixed is also a choice that puts some users in
potential danger.
Our internal contract is quite clear, and is quite clearly broken. I would
prefer to address it here and now, rather than maintaining broken behaviour
just for the sake of it. For the record, here are the relevant bits:
{code}
/**
* An iterator that returns the columns of this object in "select" order
(that
* is in global alphabetical order, where the "normal" iterator returns
simple
* columns first and the complex second).
*
* @return an iterator returning columns in alphabetical order.
*/
public Iterator selectOrderIterator()
{
// In wildcard selection, we want to return all columns in alphabetical
order,
// irregarding of whether they are complex or not
return Iterators.mergeSorted(ImmutableList.of(complexColumns(),
simpleColumns()),
(s, c) ->
{
assert !s.kind.isPrimaryKeyKind();
return
s.name.bytes.compareTo(c.name.bytes);
});
}
{code}
The contact has been defined this way since 2.0, when static columns where
introduced, and was broken relatively recently when 3.0 came out. I would argue
that the correct thing to do is to fix it in the next 3.0 and 3.11 minors, with
a loud NEWS.txt entry and an email to dev@ and users@ warning users who are
both on 2.1/2.2 and 3.0+ and up about the current breakage and the upcoming fix.
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m
[jira] [Updated] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static col
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14638:
--
Fix Version/s: 3.11.x
3.0.x
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
> t.printStackTrace();
> } finally {
> Uninterruptibles.sleepUninterruptibly(30, TimeUnit.SECONDS);
> }
> }
> } finally {
> cluster.close();
> }
> }
> }
> {code}
> To reproduce, set up a
[jira] [Updated] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static col
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14638:
--
Reviewer: Benedict
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
> t.printStackTrace();
> } finally {
> Uninterruptibles.sleepUninterruptibly(30, TimeUnit.SECONDS);
> }
> }
> } finally {
> cluster.close();
> }
> }
> }
> {code}
> To reproduce, set up a cluster, the schema, and run
[jira] [Commented] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static c
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581299#comment-16581299
]
Aleksey Yeschenko commented on CASSANDRA-14638:
---
Proposed work-in-progress fix (test coverage pending) that addresses the
underlying issue can be found
[here|https://github.com/iamaleksey/cassandra/commits/14638-3.0], with CI
upcoming
[here|https://circleci.com/workflow-run/55195e85-6d1a-4864-8c0f-ca656b843f56].
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = session.execute(bs);
> Row row = r.one();
> int i = 0;
> // iterate over the result metadata in order printing the
> // index, name, type, and length of the first row of data.
> for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
> System.out.println(
> i++
> + ": "
> + d.getName()
> + " -> "
> + d.getType()
> + " -> val = "
> + row.getBytesUnsafe(d.getName()).array().length);
> }
> } catch (Throwable t) {
>
[jira] [Commented] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static c
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16581293#comment-16581293
]
Aleksey Yeschenko commented on CASSANDRA-14638:
---
Unfortunately this is an accidental, lower level bug in 3.0, that I believe
should be fixed in the next minor.
The internal contract for wildcard order in 2.1, and, really, everywhere since
introduction of static columns is the following:
1. Partition key columns in their positional order
2. Clustering columns in their positional order
3. All static columns, ordered alphabetically by name, irregardless of whether
they are simple or complex
4. All regular columns, ordered alphabetically by name, irregardless of whether
they are simple or complex
To illustrate, let's say we have a table defined as
{code}
CREATE TABLE ks.tbl (p0 text, p1 text, a set, b set static, c text,
d text static, e set, f set static, g text, h text static, i
set, j set static, PRIMARY KEY (p0, p1));
{code}
And one inserted row:
{code}
INSERT INTO ks.tbl (p0, p1, a, b, c, d, e, f, g, h, i, j) VALUES ('p0', 'p1',
{'a'}, {'b'}, 'c', 'd', {'e'}, {'f'}, 'g', 'h', {'i'}, {'j'});
{code}
In 2.1, performing a {{SELECT * FROM ks.tbl;}} query would return the following
result set:
{code}
p0 | p1 | b | d | f | h | j | a | c | e | g | i
++---+---+---+---+---+---+---+---+---+---
p0 | p1 | {'b'} | d | {'f'} | h | {'j'} | {'a'} | c | {'e'} | g | {'i'}
{code}
3.0, in contrast, would return the following result set:
{code}
p0 | p1 | d | h | b | f | j | a | c | e | g | i
++---+---+---+---+---+---+---+---+---+---
p0 | p1 | d | h | {'b'} | {'f'} | {'j'} | {'a'} | c | {'e'} | g | {'i'}
{code}
Both correctly return {{PRIMARY KEY}} columns in their defined order first,
then all the static columns (although 3.0 order for them is all wrong),
followed by all regular columns in alphabetic order, correct in both 2.1 and
3.0.
What 3.0 does incorrectly is ordering the static columns it returns. Instead of
merging simple and complex static columns, we are getting all simple static
columns first, followed by all complex static columns.
The bug is deep in {{Columns}} class, more specifically in
{{findFirstComplexIdx()}} method. To find the first complex column index we are
using a surrogate {{ColumnDefinition}} value {{FIRST_COMPLEX}}, which is of
kind {{REGULAR}} for {{Columns}} of both {{REGULAR}} and {{STATIC}} kinds. This
quite clearly breaks {{Columns}} of kind {{STATIC}}, making the container
believe that all of its columns are {{COMPLEX}}.
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
>
[jira] [Comment Edited] (CASSANDRA-14640) Set local partitioner for existing virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579791#comment-16579791 ] Aleksey Yeschenko edited comment on CASSANDRA-14640 at 8/14/18 2:11 PM: Committed to trunk as [35750e80589e61dbef0c85f20691764a6c7c3f81|https://github.com/apache/cassandra/commit/35750e80589e61dbef0c85f20691764a6c7c3f81], thanks. EDIT: for posterity, CI results [here|https://circleci.com/workflow-run/8c55d0f9-89fb-4d1d-a05a-80c34b5a92e6]. was (Author: iamaleksey): Committed to trunk as [35750e80589e61dbef0c85f20691764a6c7c3f81|https://github.com/apache/cassandra/commit/35750e80589e61dbef0c85f20691764a6c7c3f81], thanks. > Set local partitioner for existing virtual tables > - > > Key: CASSANDRA-14640 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14640 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Minor > Labels: pull-request-available, virtual-tables > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14640) Set local partitioner for existing virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14640: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Set local partitioner for existing virtual tables > - > > Key: CASSANDRA-14640 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14640 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Minor > Labels: pull-request-available, virtual-tables > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14640) Set local partitioner for existing virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579791#comment-16579791 ] Aleksey Yeschenko commented on CASSANDRA-14640: --- Committed to trunk as [35750e80589e61dbef0c85f20691764a6c7c3f81|https://github.com/apache/cassandra/commit/35750e80589e61dbef0c85f20691764a6c7c3f81], thanks. > Set local partitioner for existing virtual tables > - > > Key: CASSANDRA-14640 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14640 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Minor > Labels: pull-request-available, virtual-tables > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14640) Set local partitioner for existing virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-14640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14640: -- Reviewer: Aleksey Yeschenko Fix Version/s: 4.0 > Set local partitioner for existing virtual tables > - > > Key: CASSANDRA-14640 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14640 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Lohfink >Assignee: Chris Lohfink >Priority: Minor > Labels: pull-request-available, virtual-tables > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14573) Expose settings in virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14573:
--
Fix Version/s: 4.x
> Expose settings in virtual table
>
>
> Key: CASSANDRA-14573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14573
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.x
>
>
> Allow both viewing what the settings are (currently impossible for some) and
> allow changing some settings.
> Example:
> {code:java}
> UPDATE system_info.settings SET value = 'false' WHERE setting =
> 'hinted_handoff_enabled';
> SELECT * FROM system_info.settings WHERE writable = True;
> setting | value | writable
> --++--
> batch_size_fail_threshold_in_kb | 50 | True
> batch_size_warn_threshold_in_kb | 5 | True
> cas_contention_timeout_in_ms | 1000 | True
> compaction_throughput_mb_per_sec | 16 | True
> concurrent_compactors | 2 | True
>concurrent_validations | 2147483647 | True
> counter_write_request_timeout_in_ms | 5000 | True
>hinted_handoff_enabled | false | True
> hinted_handoff_throttle_in_kb | 1024 | True
> incremental_backups | false | True
> inter_dc_stream_throughput_outbound_megabits_per_sec |200 | True
> phi_convict_threshold |8.0 | True
> range_request_timeout_in_ms | 1 | True
>read_request_timeout_in_ms | 5000 | True
> request_timeout_in_ms | 1 | True
> stream_throughput_outbound_megabits_per_sec |200 | True
> tombstone_failure_threshold | 10 | True
> tombstone_warn_threshold | 1000 | True
>truncate_request_timeout_in_ms | 6 | True
> write_request_timeout_in_ms | 2000 |
> True{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13994) Remove COMPACT STORAGE internals before 4.0 release
[
https://issues.apache.org/jira/browse/CASSANDRA-13994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-13994:
--
Priority: Minor (was: Blocker)
> Remove COMPACT STORAGE internals before 4.0 release
> ---
>
> Key: CASSANDRA-13994
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13994
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Minor
> Fix For: 4.0
>
>
> 4.0 comes without thrift (after [CASSANDRA-5]) and COMPACT STORAGE (after
> [CASSANDRA-10857]), and since Compact Storage flags are now disabled, all of
> the related functionality is useless.
> There are still some things to consider:
> 1. One of the system tables (built indexes) was compact. For now, we just
> added {{value}} column to it to make sure it's backwards-compatible, but we
> might want to make sure it's just a "normal" table and doesn't have redundant
> columns.
> 2. Compact Tables were building indexes in {{KEYS}} mode. Removing it is
> trivial, but this would mean that all built indexes will be defunct. We could
> log a warning for now and ask users to migrate off those for now and
> completely remove it from future releases. It's just a couple of classes
> though.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14634) Review Handling Crypto Rules and update ECCN page if needed
[
https://issues.apache.org/jira/browse/CASSANDRA-14634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16575285#comment-16575285
]
Aleksey Yeschenko commented on CASSANDRA-14634:
---
bq. We rely solely on the JDK functionality for all encryption.
This is not quite true anymore with netty-tcnative FWIW.
> Review Handling Crypto Rules and update ECCN page if needed
> ---
>
> Key: CASSANDRA-14634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14634
> Project: Cassandra
> Issue Type: Task
>Reporter: Henri Yandell
>Priority: Blocker
> Labels: security
>
> It is suggested in LEGAL-358 that Cassandra is containing/using cryptographic
> functions and does not have an entry on the ECCN page (
> [http://www.apache.org/licenses/exports/] ).
> See [http://www.apache.org/dev/crypto.html] to review and confirm whether you
> should add something to the ECCN page, and if needed, please do so.
> The text in LEGAL-358 was:
>
> [~zznate] added a comment - 26/Dec/17 14:58
> Ok, I think I have this sorted. Our entry on that page will need to look like
> this:
> {noformat}
> Product Name VersionsECCNControlled Source
> Apache Cassandra development 5D002 ASF
> 0.8 and later 5D002 ASF
> {noformat}
> We first added SSL support in 0.8 via CASSANDRA-1567
> We rely solely on the JDK functionality for all encryption.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10726) Read repair inserts should not be blocking
[
https://issues.apache.org/jira/browse/CASSANDRA-10726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-10726:
--
Reviewers: Alex Petrov, Marcus Eriksson (was: Aleksey Yeschenko, Marcus
Eriksson)
> Read repair inserts should not be blocking
> --
>
> Key: CASSANDRA-10726
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10726
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Richard Low
>Assignee: Blake Eggleston
>Priority: Major
> Fix For: 4.x
>
>
> Today, if there’s a digest mismatch in a foreground read repair, the insert
> to update out of date replicas is blocking. This means, if it fails, the read
> fails with a timeout. If a node is dropping writes (maybe it is overloaded or
> the mutation stage is backed up for some other reason), all reads to a
> replica set could fail. Further, replicas dropping writes get more out of
> sync so will require more read repair.
> The comment on the code for why the writes are blocking is:
> {code}
> // wait for the repair writes to be acknowledged, to minimize impact on any
> replica that's
> // behind on writes in case the out-of-sync row is read multiple times in
> quick succession
> {code}
> but the bad side effect is that reads timeout. Either the writes should not
> be blocking or we should return success for the read even if the write times
> out.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13304) Add checksumming to the native protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573499#comment-16573499
]
Aleksey Yeschenko commented on CASSANDRA-13304:
---
[~mkjellman] zstd supports native checksumming too though. The way I see it
it's not about doubling down on something that's lz4 specific, but about using
compressors' native checksumming support rather than rolling our own.
> Add checksumming to the native protocol
> ---
>
> Key: CASSANDRA-13304
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13304
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Michael Kjellman
>Assignee: Sam Tunnicliffe
>Priority: Blocker
> Labels: client-impacting
> Fix For: 4.x
>
> Attachments: 13304_v1.diff, boxplot-read-throughput.png,
> boxplot-write-throughput.png
>
>
> The native binary transport implementation doesn't include checksums. This
> makes it highly susceptible to silently inserting corrupted data either due
> to hardware issues causing bit flips on the sender/client side, C*/receiver
> side, or network in between.
> Attaching an implementation that makes checksum'ing mandatory (assuming both
> client and server know about a protocol version that supports checksums) --
> and also adds checksumming to clients that request compression.
> The serialized format looks something like this:
> {noformat}
> * 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3
> * 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Number of Compressed Chunks | Compressed Length (e1)/
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * / Compressed Length cont. (e1) |Uncompressed Length (e1) /
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Uncompressed Length cont. (e1)| CRC32 Checksum of Lengths (e1)|
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Checksum of Lengths cont. (e1)|Compressed Bytes (e1)+//
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | CRC32 Checksum (e1) ||
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * |Compressed Length (e2) |
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Uncompressed Length (e2)|
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * |CRC32 Checksum of Lengths (e2) |
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Compressed Bytes (e2) +//
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | CRC32 Checksum (e2) ||
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * |Compressed Length (en) |
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Uncompressed Length (en)|
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * |CRC32 Checksum of Lengths (en) |
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | Compressed Bytes (en) +//
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> * | CRC32 Checksum (en) ||
> * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> {noformat}
> The first pass here adds checksums only to the actual contents of the frame
> body itself (and doesn't actually checksum lengths and headers). While it
> would be great to fully add checksuming across the entire protocol, the
> proposed implementation will ensure we at least catch corrupted data and
> likely protect ourselves pretty well anyways.
> I didn't go to the trouble of implementing a Snappy Checksum'ed Compressor
> implementation as it's been deprecated for a while -- is really slow and
> crappy compared to LZ4 -- and we should do everything in our power to make
> sure no one in the community is still using it. I left it in (for obvious
> backwards compatibility aspects) old for clients that don't know about the
> new protocol.
> The current protocol has a 256MB (max) frame body -- where the serialized
> contents are simply written in to the frame body.
> If the client sends a compression option in the startup, we will install a
> FrameCompressor inline. Unfortunately, we went with
[jira] [Commented] (CASSANDRA-14626) Expose buffer cache metrics in caches virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-14626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573421#comment-16573421 ] Aleksey Yeschenko commented on CASSANDRA-14626: --- Branch [here|https://github.com/iamaleksey/cassandra/commits/14626-4.0], CI [here|https://circleci.com/workflow-run/c1cb3706-6786-456c-ab74-8c79887b8047]. > Expose buffer cache metrics in caches virtual table > --- > > Key: CASSANDRA-14626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14626 > Project: Cassandra > Issue Type: New Feature >Reporter: Benjamin Lerer >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: virtual-tables > Fix For: 4.0 > > > As noted by [~blerer] in CASSANDRA-14538, we should expose buffer cache > metrics in the caches virtual table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14626) Expose buffer cache metrics in caches virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-14626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-14626: -- Reviewer: Benedict Status: Patch Available (was: Open) > Expose buffer cache metrics in caches virtual table > --- > > Key: CASSANDRA-14626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14626 > Project: Cassandra > Issue Type: New Feature >Reporter: Benjamin Lerer >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: virtual-tables > Fix For: 4.0 > > > As noted by [~blerer] in CASSANDRA-14538, we should expose buffer cache > metrics in the caches virtual table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14628:
--
Reviewer: Benedict
Status: Patch Available (was: Open)
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14628) Clean up cache-related metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-14628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573402#comment-16573402
]
Aleksey Yeschenko commented on CASSANDRA-14628:
---
Branch [here|https://github.com/iamaleksey/cassandra/commits/14628-4.0], CI
pending
[here|https://circleci.com/workflow-run/4d4f4fd1-28f4-47e0-9470-2730496c71b1].
> Clean up cache-related metrics
> --
>
> Key: CASSANDRA-14628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> {{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate
> of pre-existing {{CacheMetrics}}. I believe it was done initially because the
> authors thought there was no way to register hits with {{Caffeine}}, only
> misses, but that's not quite true. All we need is to provide a
> {{StatsCounter}} object when building the cache and update our metrics from
> there.
> The patch removes the redundant code and streamlines chunk cache metrics to
> use more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14626) Expose buffer cache metrics in caches virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-14626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573395#comment-16573395 ] Aleksey Yeschenko commented on CASSANDRA-14626: --- Going to address the issue with cache metrics in general in CASSANDRA-14628 then build on it here. > Expose buffer cache metrics in caches virtual table > --- > > Key: CASSANDRA-14626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14626 > Project: Cassandra > Issue Type: New Feature >Reporter: Benjamin Lerer >Assignee: Aleksey Yeschenko >Priority: Minor > Labels: virtual-tables > Fix For: 4.0 > > > As noted by [~blerer] in CASSANDRA-14538, we should expose buffer cache > metrics in the caches virtual table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14628) Clean up cache-related metrics
Aleksey Yeschenko created CASSANDRA-14628:
-
Summary: Clean up cache-related metrics
Key: CASSANDRA-14628
URL: https://issues.apache.org/jira/browse/CASSANDRA-14628
Project: Cassandra
Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Fix For: 4.0
{{ChunkCache}} added {{CacheMissMetrics}} which is an almost exact duplicate of
pre-existing {{CacheMetrics}}. I believe it was done initially because the
authors thought there was no way to register hits with {{Caffeine}}, only
misses, but that's not quite true. All we need is to provide a {{StatsCounter}}
object when building the cache and update our metrics from there.
The patch removes the redundant code and streamlines chunk cache metrics to use
more idiomatic tracking.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14624) Website: Add blog post for Faster Streaming
[
https://issues.apache.org/jira/browse/CASSANDRA-14624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16573181#comment-16573181
]
Aleksey Yeschenko commented on CASSANDRA-14624:
---
The name of the setting is not quite correct - should be
{{stream_entire_sstables}}, not {{streaming_zerocopy_sstables_enabled}}. Some
other issues with the blog post here, but should at the very least fix this.
> Website: Add blog post for Faster Streaming
> ---
>
> Key: CASSANDRA-14624
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14624
> Project: Cassandra
> Issue Type: Task
> Components: Documentation and Website
>Reporter: Dinesh Joshi
>Assignee: Dinesh Joshi
>Priority: Major
> Attachments: faster-streaming-blog.patch
>
>
> Please add a new blog post entry on the Cassandra website. It describes the
> recent work on performance optimizations in Cassandra related to streaming
> enhancements.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14627) CASSANDRA-9608 broke running Cassandra and tests in IntelliJ under Java 8
[
https://issues.apache.org/jira/browse/CASSANDRA-14627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16572981#comment-16572981
]
Aleksey Yeschenko commented on CASSANDRA-14627:
---
See CASSANDRA-14613 too.
> CASSANDRA-9608 broke running Cassandra and tests in IntelliJ under Java 8
> -
>
> Key: CASSANDRA-14627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14627
> Project: Cassandra
> Issue Type: Bug
> Components: Testing
>Reporter: Jordan West
>Priority: Critical
>
> CASSANDRA-9608 added a couple hard-coded options to workspace.xml that are
> not supported in Java 8:
> https://github.com/apache/cassandra/commit/6ba2fb9395226491872b41312d978a169f36fcdb#diff-59e65c5abf01f83a11989765ada76841.
>
> {code}
> Unrecognized option: --add-exports
> Error: Could not create the Java Virtual Machine.
> Error: A fatal exception has occurred. Program will exit.
> {code}
> To reproduce:
> 1. Update to the most recent trunk
> 2. rm -rf .idea && ant generate-idea-files
> 3. Re-open the project in IntelliJ (using Java 8) and run Cassandra or a
> test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14538) Add virtual table to view cache information
[
https://issues.apache.org/jira/browse/CASSANDRA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16571671#comment-16571671
]
Aleksey Yeschenko commented on CASSANDRA-14538:
---
[~blerer] Good point. Agreed. Created CASSANDRA-14626 to take care of it.
> Add virtual table to view cache information
> ---
>
> Key: CASSANDRA-14538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14538
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> Add a table for displaying the general cache information (like in {{nodetool
> info}})
> {code}
> cqlsh> SELECT * FROM system_views.caches;
> name| entries | hit_ratio | hits | recent_hits_per_sec |
> recent_requests_per_sec | requests | size_max | size_used
> -+-+---+--+-+-+--+--+---
> counter | 0 | NaN |0 | 0 |
> 0 |0 | 12582912 | 0
> key | 19 | 0.922509 | 250 | 11 |
> 13 | 271 | 25165824 | 1728
> row | 0 | NaN |0 | 0 |
> 0 |0 | 16777216 | 0
> (3 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14626) Expose buffer cache metrics in caches virtual table
Aleksey Yeschenko created CASSANDRA-14626: - Summary: Expose buffer cache metrics in caches virtual table Key: CASSANDRA-14626 URL: https://issues.apache.org/jira/browse/CASSANDRA-14626 Project: Cassandra Issue Type: New Feature Reporter: Benjamin Lerer Assignee: Aleksey Yeschenko Fix For: 4.0 As noted by [~blerer] in CASSANDRA-14538, we should expose buffer cache metrics in the caches virtual table. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14523) Thread pool stats virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14523:
--
Resolution: Fixed
Fix Version/s: (was: 4.x)
4.0
Status: Resolved (was: Patch Available)
> Thread pool stats virtual table
> ---
>
> Key: CASSANDRA-14523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14523
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> Expose the thread pools like in status logger/tpstats. Additionally be nice
> to include the scheduled executor pools that are currently unmonitored.
> {code:java}
> cqlsh> select * from system_views.thread_pools;
> thread_pool | active | active_max | completed | pending
> | tasks_blocked | total_blocked
> --+++---+-+---+---
>anti_entropy_stage | 0 | 1 | 0 | 0
> | 0 | 0
>cache_cleanup_executor | 0 | 1 | 0 | 0
> | 0 | 0
> compaction_executor | 0 | 4 |41 | 0
> | 0 | 0
>counter_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> gossip_stage | 0 | 1 | 0 | 0
> | 0 | 0
> hints_dispatcher | 0 | 2 | 0 | 0
> | 0 | 0
> internal_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> memtable_flush_writer | 0 | 2 | 5 | 0
> | 0 | 0
> memtable_post_flush | 0 | 1 |20 | 0
> | 0 | 0
> memtable_reclaim_memory | 0 | 1 | 5 | 0
> | 0 | 0
> migration_stage | 0 | 1 | 0 | 0
> | 0 | 0
>misc_stage | 0 | 1 | 0 | 0
> | 0 | 0
>mutation_stage | 0 | 32 | 247 | 0
> | 0 | 0
> native_transport_requests | 1 |128 |28 | 0
> | 0 | 0
> pending_range_calculator | 0 | 1 | 2 | 0
> | 0 | 0
> per_disk_memtable_flush_writer_0 | 0 | 2 | 5 | 0
> | 0 | 0
> read_repair_stage | 0 | 8 | 0 | 0
> | 0 | 0
>read_stage | 0 | 32 |13 | 0
> | 0 | 0
> repair_task | 0 | 2147483647 | 0 | 0
> | 0 | 0
>request_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> sampler | 0 | 1 | 0 | 0
> | 0 | 0
> scheduled_fast_tasks | 0 | 2147483647 | 1398 | 1
> | 0 | 0
> scheduled_heartbeat | 0 | 2147483647 |14 | 1
> | 0 | 0
> scheduled_hotness_tracker | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_non_periodic_tasks | 0 | 2147483647 |10 | 0
> | 0 | 0
> scheduled_optional_tasks | 0 | 2147483647 | 5 | 8
> | 0 | 0
> scheduled_summary_builder | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_tasks | 0 | 2147483647 | 194 | 74
> | 0 | 0
>secondary_index_management | 0 | 1 | 0 | 0
> | 0 | 0
> validation_executor | 0 | 2147483647 | 0 | 0
> | 0 | 0
> view_build_executor | 0 | 1 | 0 | 0
> | 0 | 0
> view_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (CASSANDRA-14523) Thread pool stats virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570571#comment-16570571
]
Aleksey Yeschenko commented on CASSANDRA-14523:
---
Committed to trunk as
[d5ae2ae481545b1fb2332b46013088f2f8cea636|https://github.com/apache/cassandra/commit/d5ae2ae481545b1fb2332b46013088f2f8cea636],
thanks.
> Thread pool stats virtual table
> ---
>
> Key: CASSANDRA-14523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14523
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.x
>
>
> Expose the thread pools like in status logger/tpstats. Additionally be nice
> to include the scheduled executor pools that are currently unmonitored.
> {code:java}
> cqlsh> select * from system_views.thread_pools;
> thread_pool | active | active_max | completed | pending
> | tasks_blocked | total_blocked
> --+++---+-+---+---
>anti_entropy_stage | 0 | 1 | 0 | 0
> | 0 | 0
>cache_cleanup_executor | 0 | 1 | 0 | 0
> | 0 | 0
> compaction_executor | 0 | 4 |41 | 0
> | 0 | 0
>counter_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> gossip_stage | 0 | 1 | 0 | 0
> | 0 | 0
> hints_dispatcher | 0 | 2 | 0 | 0
> | 0 | 0
> internal_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> memtable_flush_writer | 0 | 2 | 5 | 0
> | 0 | 0
> memtable_post_flush | 0 | 1 |20 | 0
> | 0 | 0
> memtable_reclaim_memory | 0 | 1 | 5 | 0
> | 0 | 0
> migration_stage | 0 | 1 | 0 | 0
> | 0 | 0
>misc_stage | 0 | 1 | 0 | 0
> | 0 | 0
>mutation_stage | 0 | 32 | 247 | 0
> | 0 | 0
> native_transport_requests | 1 |128 |28 | 0
> | 0 | 0
> pending_range_calculator | 0 | 1 | 2 | 0
> | 0 | 0
> per_disk_memtable_flush_writer_0 | 0 | 2 | 5 | 0
> | 0 | 0
> read_repair_stage | 0 | 8 | 0 | 0
> | 0 | 0
>read_stage | 0 | 32 |13 | 0
> | 0 | 0
> repair_task | 0 | 2147483647 | 0 | 0
> | 0 | 0
>request_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> sampler | 0 | 1 | 0 | 0
> | 0 | 0
> scheduled_fast_tasks | 0 | 2147483647 | 1398 | 1
> | 0 | 0
> scheduled_heartbeat | 0 | 2147483647 |14 | 1
> | 0 | 0
> scheduled_hotness_tracker | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_non_periodic_tasks | 0 | 2147483647 |10 | 0
> | 0 | 0
> scheduled_optional_tasks | 0 | 2147483647 | 5 | 8
> | 0 | 0
> scheduled_summary_builder | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_tasks | 0 | 2147483647 | 194 | 74
> | 0 | 0
>secondary_index_management | 0 | 1 | 0 | 0
> | 0 | 0
> validation_executor | 0 | 2147483647 | 0 | 0
> | 0 | 0
> view_build_executor | 0 | 1 | 0 | 0
> | 0 | 0
> view_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> {code}
--
This
[jira] [Comment Edited] (CASSANDRA-14523) Thread pool stats virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570370#comment-16570370
]
Aleksey Yeschenko edited comment on CASSANDRA-14523 at 8/6/18 3:53 PM:
---
[~cnlwsu] made most of the requested changes. I pushed a bit more
[here|https://github.com/iamaleksey/cassandra/commits/14523-4.0]. It cleans up
existing things a bit, makes them more consistent, and implements an override
for single-partition-key reads, among other things.
It's going through CI now,
[here|https://circleci.com/workflow-run/f3956836-29d7-4570-aef2-07fa2b6cae11].
If CI is happy, and you are ok with the latest changes, [~cnlwsu], I'll commit
this version.
was (Author: iamaleksey):
[~cnlwsu] made most of the requested changes. I pushed a bit more
[here|https://github.com/iamaleksey/cassandra/commits/14523-4.0]. It cleans up
existing things a bit, makes them more consistent, and implements an override
for single-partition-key reads, among other things.
It's going through CI now,
[here|https://circleci.com/workflow-run/92851a7d-3846-4233-a8b3-7a06a305ad44].
If CI is happy, and you are ok with the latest changes, [~cnlwsu], I'll commit
this version.
> Thread pool stats virtual table
> ---
>
> Key: CASSANDRA-14523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14523
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.x
>
>
> Expose the thread pools like in status logger/tpstats. Additionally be nice
> to include the scheduled executor pools that are currently unmonitored.
> {code:java}
> cqlsh> select * from system_views.thread_pools;
> thread_pool | active | active_max | completed | pending
> | tasks_blocked | total_blocked
> --+++---+-+---+---
>anti_entropy_stage | 0 | 1 | 0 | 0
> | 0 | 0
>cache_cleanup_executor | 0 | 1 | 0 | 0
> | 0 | 0
> compaction_executor | 0 | 4 |41 | 0
> | 0 | 0
>counter_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> gossip_stage | 0 | 1 | 0 | 0
> | 0 | 0
> hints_dispatcher | 0 | 2 | 0 | 0
> | 0 | 0
> internal_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> memtable_flush_writer | 0 | 2 | 5 | 0
> | 0 | 0
> memtable_post_flush | 0 | 1 |20 | 0
> | 0 | 0
> memtable_reclaim_memory | 0 | 1 | 5 | 0
> | 0 | 0
> migration_stage | 0 | 1 | 0 | 0
> | 0 | 0
>misc_stage | 0 | 1 | 0 | 0
> | 0 | 0
>mutation_stage | 0 | 32 | 247 | 0
> | 0 | 0
> native_transport_requests | 1 |128 |28 | 0
> | 0 | 0
> pending_range_calculator | 0 | 1 | 2 | 0
> | 0 | 0
> per_disk_memtable_flush_writer_0 | 0 | 2 | 5 | 0
> | 0 | 0
> read_repair_stage | 0 | 8 | 0 | 0
> | 0 | 0
>read_stage | 0 | 32 |13 | 0
> | 0 | 0
> repair_task | 0 | 2147483647 | 0 | 0
> | 0 | 0
>request_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> sampler | 0 | 1 | 0 | 0
> | 0 | 0
> scheduled_fast_tasks | 0 | 2147483647 | 1398 | 1
> | 0 | 0
> scheduled_heartbeat | 0 | 2147483647 |14 | 1
> | 0 | 0
> scheduled_hotness_tracker | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_non_periodic_tasks | 0 | 2147483647 |
[jira] [Commented] (CASSANDRA-14523) Thread pool stats virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16570370#comment-16570370
]
Aleksey Yeschenko commented on CASSANDRA-14523:
---
[~cnlwsu] made most of the requested changes. I pushed a bit more
[here|https://github.com/iamaleksey/cassandra/commits/14523-4.0]. It cleans up
existing things a bit, makes them more consistent, and implements an override
for single-partition-key reads, among other things.
It's going through CI now,
[here|https://circleci.com/workflow-run/92851a7d-3846-4233-a8b3-7a06a305ad44].
If CI is happy, and you are ok with the latest changes, [~cnlwsu], I'll commit
this version.
> Thread pool stats virtual table
> ---
>
> Key: CASSANDRA-14523
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14523
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.x
>
>
> Expose the thread pools like in status logger/tpstats. Additionally be nice
> to include the scheduled executor pools that are currently unmonitored.
> {code:java}
> cqlsh> select * from system_views.thread_pools;
> thread_pool | active | active_max | completed | pending
> | tasks_blocked | total_blocked
> --+++---+-+---+---
>anti_entropy_stage | 0 | 1 | 0 | 0
> | 0 | 0
>cache_cleanup_executor | 0 | 1 | 0 | 0
> | 0 | 0
> compaction_executor | 0 | 4 |41 | 0
> | 0 | 0
>counter_mutation_stage | 0 | 32 | 0 | 0
> | 0 | 0
> gossip_stage | 0 | 1 | 0 | 0
> | 0 | 0
> hints_dispatcher | 0 | 2 | 0 | 0
> | 0 | 0
> internal_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> memtable_flush_writer | 0 | 2 | 5 | 0
> | 0 | 0
> memtable_post_flush | 0 | 1 |20 | 0
> | 0 | 0
> memtable_reclaim_memory | 0 | 1 | 5 | 0
> | 0 | 0
> migration_stage | 0 | 1 | 0 | 0
> | 0 | 0
>misc_stage | 0 | 1 | 0 | 0
> | 0 | 0
>mutation_stage | 0 | 32 | 247 | 0
> | 0 | 0
> native_transport_requests | 1 |128 |28 | 0
> | 0 | 0
> pending_range_calculator | 0 | 1 | 2 | 0
> | 0 | 0
> per_disk_memtable_flush_writer_0 | 0 | 2 | 5 | 0
> | 0 | 0
> read_repair_stage | 0 | 8 | 0 | 0
> | 0 | 0
>read_stage | 0 | 32 |13 | 0
> | 0 | 0
> repair_task | 0 | 2147483647 | 0 | 0
> | 0 | 0
>request_response_stage | 0 | 8 | 0 | 0
> | 0 | 0
> sampler | 0 | 1 | 0 | 0
> | 0 | 0
> scheduled_fast_tasks | 0 | 2147483647 | 1398 | 1
> | 0 | 0
> scheduled_heartbeat | 0 | 2147483647 |14 | 1
> | 0 | 0
> scheduled_hotness_tracker | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_non_periodic_tasks | 0 | 2147483647 |10 | 0
> | 0 | 0
> scheduled_optional_tasks | 0 | 2147483647 | 5 | 8
> | 0 | 0
> scheduled_summary_builder | 0 | 2147483647 | 0 | 1
> | 0 | 0
> scheduled_tasks | 0 | 2147483647 | 194 | 74
> | 0 | 0
>secondary_index_management | 0 | 1 | 0 | 0
> | 0 | 0
> validation_executor | 0 |
[jira] [Commented] (CASSANDRA-14538) Add virtual table to view cache information
[
https://issues.apache.org/jira/browse/CASSANDRA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568497#comment-16568497
]
Aleksey Yeschenko commented on CASSANDRA-14538:
---
Thanks. Committed as
[45c3b461ec28abdb24353c7317dfa7544c415f11|https://github.com/apache/cassandra/commit/45c3b461ec28abdb24353c7317dfa7544c415f11]
to trunk. CI results
[here|https://circleci.com/workflow-run/43956bc9-b649-467a-9b74-2ea4d1c7f36f].
> Add virtual table to view cache information
> ---
>
> Key: CASSANDRA-14538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14538
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> Add a table for displaying the general cache information (like in {{nodetool
> info}})
> {code}
> cqlsh> SELECT * FROM system_views.caches;
> name| entries | hit_ratio | hits | recent_hits_per_sec |
> recent_requests_per_sec | requests | size_max | size_used
> -+-+---+--+-+-+--+--+---
> counter | 0 | NaN |0 | 0 |
> 0 |0 | 12582912 | 0
> key | 19 | 0.922509 | 250 | 11 |
> 13 | 271 | 25165824 | 1728
> row | 0 | NaN |0 | 0 |
> 0 |0 | 16777216 | 0
> (3 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14538) Add virtual table to view cache information
[
https://issues.apache.org/jira/browse/CASSANDRA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14538:
--
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Add virtual table to view cache information
> ---
>
> Key: CASSANDRA-14538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14538
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> Add a table for displaying the general cache information (like in {{nodetool
> info}})
> {code}
> cqlsh> SELECT * FROM system_views.caches;
> name| entries | hit_ratio | hits | recent_hits_per_sec |
> recent_requests_per_sec | requests | size_max | size_used
> -+-+---+--+-+-+--+--+---
> counter | 0 | NaN |0 | 0 |
> 0 |0 | 12582912 | 0
> key | 19 | 0.922509 | 250 | 11 |
> 13 | 271 | 25165824 | 1728
> row | 0 | NaN |0 | 0 |
> 0 |0 | 16777216 | 0
> (3 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14538) Add virtual table to view cache information
[
https://issues.apache.org/jira/browse/CASSANDRA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-14538:
--
Status: Ready to Commit (was: Patch Available)
> Add virtual table to view cache information
> ---
>
> Key: CASSANDRA-14538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14538
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> Add a table for displaying the general cache information (like in {{nodetool
> info}})
> {code}
> cqlsh> SELECT * FROM system_views.caches;
> name| entries | hit_ratio | hits | recent_hits_per_sec |
> recent_requests_per_sec | requests | size_max | size_used
> -+-+---+--+-+-+--+--+---
> counter | 0 | NaN |0 | 0 |
> 0 |0 | 12582912 | 0
> key | 19 | 0.922509 | 250 | 11 |
> 13 | 271 | 25165824 | 1728
> row | 0 | NaN |0 | 0 |
> 0 |0 | 16777216 | 0
> (3 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14538) Add virtual table to view cache information
[
https://issues.apache.org/jira/browse/CASSANDRA-14538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16568332#comment-16568332
]
Aleksey Yeschenko commented on CASSANDRA-14538:
---
Code looks good. Fixed some nits here:
https://github.com/iamaleksey/cassandra/commits/caches_vtable_review
Missing license, inconsistent modifier order, stuff like that. Also gave things
(arguably) more descriptive names. If you are fine with those changes, I'll
commit, if not, we'd have to debate a little.
> Add virtual table to view cache information
> ---
>
> Key: CASSANDRA-14538
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14538
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Minor
> Labels: virtual-tables
> Fix For: 4.0
>
>
> Add a table for displaying the general cache information (like in {{nodetool
> info}})
> {code}
> cqlsh> SELECT * FROM system_views.caches;
> name| entries | hit_ratio | hits | recent_hits_per_sec |
> recent_requests_per_sec | requests | size_max | size_used
> -+-+---+--+-+-+--+--+---
> counter | 0 | NaN |0 | 0 |
> 0 |0 | 12582912 | 0
> key | 19 | 0.922509 | 250 | 11 |
> 13 | 271 | 25165824 | 1728
> row | 0 | NaN |0 | 0 |
> 0 |0 | 16777216 | 0
> (3 rows)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
