[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id
[
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337240#comment-15337240
]
Andy Tolbert commented on CASSANDRA-10786:
--
{quote}
What's our normal process for replacing the bundled Java driver when there's a
protocol version change? Andy Tolbert any preferences on how to handle the
review of the driver changes?
{quote}
Good question, I recall with protocol v4 and with C* 3.0 that there was a
period of time where a pre-released driver jar was put in lib which I think is
reasonable, but maybe there is a better way to do this, will discuss with the
team on Monday. There was [a
PR|https://github.com/datastax/java-driver/pull/675] for reviewing this, but we
will need to update it with the commit from
[~ifesdjeen's|https://github.com/ifesdjeen/java-driver/tree/10786-v5] branch.
> Include hash of result set metadata in prepared statement id
>
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
> Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a
> prepared statement when the table is altered, to force clients to update
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host.
> The first client to execute the query after the cache was invalidated will
> receive an UNPREPARED response, re-prepare, and update its local metadata.
> But other clients might miss it entirely (the MD5 hasn't changed), and they
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123,
> clientA and clientB both have a cache of the metadata (columns b and c)
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response,
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been
> populated again, the query succeeds. But clientB still has not updated its
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set
> metadata in the md5. This way the md5 would change at step 3, and any client
> using the old md5 would get an UNPREPARED, regardless of whether another
> client already reprepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id
[
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337240#comment-15337240
]
Andy Tolbert edited comment on CASSANDRA-10786 at 6/17/16 11:54 PM:
{quote}
What's our normal process for replacing the bundled Java driver when there's a
protocol version change? Andy Tolbert any preferences on how to handle the
review of the driver changes?
{quote}
Good question, I recall with protocol v4 and with C* 3.0 that there was a
period of time where a pre-released driver jar was put in lib which I think is
reasonable, but maybe there is a better way to do this, will discuss with the
team on Monday. There was [a
PR|https://github.com/datastax/java-driver/pull/675] for reviewing this, but we
will need to update it with the commit from [~ifesdjeen]'s
[branch|https://github.com/ifesdjeen/java-driver/tree/10786-v5].
was (Author: andrew.tolbert):
{quote}
What's our normal process for replacing the bundled Java driver when there's a
protocol version change? Andy Tolbert any preferences on how to handle the
review of the driver changes?
{quote}
Good question, I recall with protocol v4 and with C* 3.0 that there was a
period of time where a pre-released driver jar was put in lib which I think is
reasonable, but maybe there is a better way to do this, will discuss with the
team on Monday. There was [a
PR|https://github.com/datastax/java-driver/pull/675] for reviewing this, but we
will need to update it with the commit from
[~ifesdjeen's|https://github.com/ifesdjeen/java-driver/tree/10786-v5] branch.
> Include hash of result set metadata in prepared statement id
>
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
> Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a
> prepared statement when the table is altered, to force clients to update
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host.
> The first client to execute the query after the cache was invalidated will
> receive an UNPREPARED response, re-prepare, and update its local metadata.
> But other clients might miss it entirely (the MD5 hasn't changed), and they
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123,
> clientA and clientB both have a cache of the metadata (columns b and c)
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response,
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been
> populated again, the query succeeds. But clientB still has not updated its
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set
> metadata in the md5. This way the md5 would change at step 3, and any client
> using the old md5 would get an UNPREPARED, regardless of whether another
> client already reprepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id
[
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339842#comment-15339842
]
Andy Tolbert commented on CASSANDRA-10786:
--
We (java driver team) discussed this and we agree with taking the approach
we've previously taken, we can do a separate branch with this change, build a
jar and include it with C* (in the lib/ directory as currently done). We'll do
some extra validation to make sure everything is good from the driver side.
After a driver has been formally released with those changes, we would want to
update C* to use that.
A few questions though:
* What is the timeline of this change? Will it be in 3.8 or 3.10?
* Will this be the only change in for protocol v5 or will some of the tickets
in [CASSANDRA-9362] be included as well?
> Include hash of result set metadata in prepared statement id
>
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
> Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a
> prepared statement when the table is altered, to force clients to update
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host.
> The first client to execute the query after the cache was invalidated will
> receive an UNPREPARED response, re-prepare, and update its local metadata.
> But other clients might miss it entirely (the MD5 hasn't changed), and they
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123,
> clientA and clientB both have a cache of the metadata (columns b and c)
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response,
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been
> populated again, the query succeeds. But clientB still has not updated its
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set
> metadata in the md5. This way the md5 would change at step 3, and any client
> using the old md5 would get an UNPREPARED, regardless of whether another
> client already reprepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-11682) 'keyspace' option support on STARTUP message
Andy Tolbert created CASSANDRA-11682:
Summary: 'keyspace' option support on STARTUP message
Key: CASSANDRA-11682
URL: https://issues.apache.org/jira/browse/CASSANDRA-11682
Project: Cassandra
Issue Type: Improvement
Reporter: Andy Tolbert
Most of the drivers provide a way to bind a keyspace to a Client/Session and
subsequently its connections, i.e. in the java-driver:
{code}
Session session = cluster.connect("mykeyspace");
{code}
Currently to bind keyspaces to connections that drivers need to issue a {{USE
}} query and keep track of which keyspace is bound to a connection.
This is non-trivial to manage and this has been a source of quite a few bugs in
the drivers.
It would be really nice if in a future protocol version that the {{STARTUP}}
message supported an option to provide the keyspace to bind a connection to.
This way no separate {{USE }} request needs to be issued.
If the keyspace is provided in this way, it could also be a nice consideration
to make the keyspace bound to the connection unchangeable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11682) 'keyspace' option support on STARTUP message
[
https://issues.apache.org/jira/browse/CASSANDRA-11682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-11682:
-
Priority: Minor (was: Major)
> 'keyspace' option support on STARTUP message
>
>
> Key: CASSANDRA-11682
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11682
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Andy Tolbert
>Priority: Minor
>
> Most of the drivers provide a way to bind a keyspace to a Client/Session and
> subsequently its connections, i.e. in the java-driver:
> {code}
> Session session = cluster.connect("mykeyspace");
> {code}
> Currently to bind keyspaces to connections that drivers need to issue a {{USE
> }} query and keep track of which keyspace is bound to a connection.
> This is non-trivial to manage and this has been a source of quite a few bugs
> in the drivers.
> It would be really nice if in a future protocol version that the {{STARTUP}}
> message supported an option to provide the keyspace to bind a connection to.
> This way no separate {{USE }} request needs to be issued.
> If the keyspace is provided in this way, it could also be a nice
> consideration to make the keyspace bound to the connection unchangeable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11615) cassandra-stress blocks when connecting to a big cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-11615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15267306#comment-15267306
]
Andy Tolbert commented on CASSANDRA-11615:
--
java-driver 3.0.1 has been released with this fix! :)
> cassandra-stress blocks when connecting to a big cluster
>
>
> Key: CASSANDRA-11615
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11615
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
> Fix For: 3.0.x
>
> Attachments: 11615-3.0-2nd.patch, 11615-3.0.patch
>
>
> I had a *100* node cluster and was running
> {code}
> cassandra-stress read n=100 no-warmup cl=LOCAL_QUORUM -rate 'threads=20'
> 'limit=1000/s'
> {code}
> Based on the thread dump it looks like it's been blocked at
> https://github.com/apache/cassandra/blob/cassandra-3.0/tools/stress/src/org/apache/cassandra/stress/util/JavaDriverClient.java#L96
> {code}
> "Thread-20" #245 prio=5 os_prio=0 tid=0x7f3781822000 nid=0x46c4 waiting
> for monitor entry [0x7f36cc788000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.stress.util.JavaDriverClient.prepare(JavaDriverClient.java:96)
> - waiting to lock <0x0005c003d920> (a
> java.util.concurrent.ConcurrentHashMap)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation$JavaDriverWrapper.createPreparedStatement(CqlOperation.java:314)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:77)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> "Thread-19" #244 prio=5 os_prio=0 tid=0x7f378182 nid=0x46c3 waiting
> for monitor entry [0x7f36cc889000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.stress.util.JavaDriverClient.prepare(JavaDriverClient.java:96)
> - waiting to lock <0x0005c003d920> (a
> java.util.concurrent.ConcurrentHashMap)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation$JavaDriverWrapper.createPreparedStatement(CqlOperation.java:314)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:77)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> {code}
> I was trying the same with with a smaller cluster (50 nodes) and it was
> working fine.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-11798) Allow specification of 'time' column value as number in CQL query.
Andy Tolbert created CASSANDRA-11798:
Summary: Allow specification of 'time' column value as number in
CQL query.
Key: CASSANDRA-11798
URL: https://issues.apache.org/jira/browse/CASSANDRA-11798
Project: Cassandra
Issue Type: Improvement
Components: CQL
Reporter: Andy Tolbert
Priority: Minor
The 'time' cql type is internally stored and sent over the protocol as an
8-byte long value representing nanoseconds since midnight.
When specifying a time column value as a number in a simple statement,, C*
currently responds with:
{noformat}
InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant
(420) for "time" of type time"
{noformat}
Instead one must provide this value as a string (i.e. {{'420'}} or use
an {{HH.MM.SS.s}} format, i.e. {{'00:00:42.0'}}). It would be
nice if it supported unquoted numbers as well.
Example:
{code}
cqlsh:simple> CREATE TABLE timeentity (id varchar PRIMARY KEY, time time);
# Doesn't work
cqlsh:simple> INSERT into timeentity (id, time) values ('1', 420);
InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant
(420) for "time" of type time"
# Works
cqlsh:simple> INSERT into timeentity (id, time) values ('1', '420');
{code}
When using prepared statements or simple statements with parameters, one could
provide a long value, depending on the driver implementation. I.E. the java
driver has {{setTime(int index, long v)}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11798) Allow specification of 'time' column value as number in CQL query.
[
https://issues.apache.org/jira/browse/CASSANDRA-11798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15281900#comment-15281900
]
Andy Tolbert commented on CASSANDRA-11798:
--
/cc [~mp911de] (sorry, don't have access to add you as a watcher).
> Allow specification of 'time' column value as number in CQL query.
> --
>
> Key: CASSANDRA-11798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11798
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Andy Tolbert
>Priority: Minor
>
> The 'time' cql type is internally stored and sent over the protocol as an
> 8-byte long value representing nanoseconds since midnight.
> When specifying a time column value as a number in a simple statement,, C*
> currently responds with:
> {noformat}
> InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant
> (420) for "time" of type time"
> {noformat}
> Instead one must provide this value as a string (i.e. {{'420'}} or
> use an {{HH.MM.SS.s}} format, i.e. {{'00:00:42.0'}}). It
> would be nice if it supported unquoted numbers as well.
> Example:
> {code}
> cqlsh:simple> CREATE TABLE timeentity (id varchar PRIMARY KEY, time time);
> # Doesn't work
> cqlsh:simple> INSERT into timeentity (id, time) values ('1', 420);
> InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant
> (420) for "time" of type time"
> # Works
> cqlsh:simple> INSERT into timeentity (id, time) values ('1', '420');
> {code}
> When using prepared statements or simple statements with parameters, one
> could provide a long value, depending on the driver implementation. I.E. the
> java driver has {{setTime(int index, long v)}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11798) Allow specification of 'time' column value as number in CQL query.
[
https://issues.apache.org/jira/browse/CASSANDRA-11798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-11798:
-
Description:
The 'time' cql type is internally stored and sent over the protocol as an
8-byte long value representing nanoseconds since midnight.
When specifying a time column value as a number in a simple statement,, C*
currently responds with:
{noformat}
InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant
(420) for "time" of type time"
{noformat}
Instead one must provide this value as a string (i.e. {{'420'}}) or use
an {{HH.MM.SS.s}} format (i.e. {{'00:00:42.0'}}). It would be
nice if it supported unquoted numbers as well.
Example:
{code}
cqlsh:simple> CREATE TABLE timeentity (id varchar PRIMARY KEY, time time);
# Doesn't work
cqlsh:simple> INSERT into timeentity (id, time) values ('1', 420);
InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant
(420) for "time" of type time"
# Works
cqlsh:simple> INSERT into timeentity (id, time) values ('1', '420');
{code}
When using prepared statements or simple statements with parameters, one could
provide a long value, depending on the driver implementation. I.E. the java
driver has {{setTime(int index, long v)}}.
was:
The 'time' cql type is internally stored and sent over the protocol as an
8-byte long value representing nanoseconds since midnight.
When specifying a time column value as a number in a simple statement,, C*
currently responds with:
{noformat}
InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant
(420) for "time" of type time"
{noformat}
Instead one must provide this value as a string (i.e. {{'420'}} or use
an {{HH.MM.SS.s}} format, i.e. {{'00:00:42.0'}}). It would be
nice if it supported unquoted numbers as well.
Example:
{code}
cqlsh:simple> CREATE TABLE timeentity (id varchar PRIMARY KEY, time time);
# Doesn't work
cqlsh:simple> INSERT into timeentity (id, time) values ('1', 420);
InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant
(420) for "time" of type time"
# Works
cqlsh:simple> INSERT into timeentity (id, time) values ('1', '420');
{code}
When using prepared statements or simple statements with parameters, one could
provide a long value, depending on the driver implementation. I.E. the java
driver has {{setTime(int index, long v)}}.
> Allow specification of 'time' column value as number in CQL query.
> --
>
> Key: CASSANDRA-11798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11798
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Andy Tolbert
>Priority: Minor
>
> The 'time' cql type is internally stored and sent over the protocol as an
> 8-byte long value representing nanoseconds since midnight.
> When specifying a time column value as a number in a simple statement,, C*
> currently responds with:
> {noformat}
> InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant
> (420) for "time" of type time"
> {noformat}
> Instead one must provide this value as a string (i.e. {{'420'}}) or
> use an {{HH.MM.SS.s}} format (i.e. {{'00:00:42.0'}}). It
> would be nice if it supported unquoted numbers as well.
> Example:
> {code}
> cqlsh:simple> CREATE TABLE timeentity (id varchar PRIMARY KEY, time time);
> # Doesn't work
> cqlsh:simple> INSERT into timeentity (id, time) values ('1', 420);
> InvalidRequest: code=2200 [Invalid query] message="Invalid INTEGER constant
> (420) for "time" of type time"
> # Works
> cqlsh:simple> INSERT into timeentity (id, time) values ('1', '420');
> {code}
> When using prepared statements or simple statements with parameters, one
> could provide a long value, depending on the driver implementation. I.E. the
> java driver has {{setTime(int index, long v)}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id
[
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15289228#comment-15289228
]
Andy Tolbert commented on CASSANDRA-10786:
--
With the issue that [~ifesdjeen] described on the driver end, there are some
things to worry about:
# this will be an issue in the case where a user has an older client install
with a newer C* install and they have this case where altering a table causes
the prepared statement hash to change, causing the repeating prepare as
described.
# this could also be a problem for cases of a cluster with mixed versions. If
you have some nodes at a newer version using this calculation, and others using
the previous calculation you will get different prepared ids depending on which
node you hit. Depending on the driver implementation this could be an issue.
What do you think about the possibility of tying prepared statement id
calculation to protocol versions? This way older clients that don't know to
handle such a case where the prepared id can change will continue to work, and
newer clients can be updated to support these scenarios as part of also
supporting a new protocol version (as [~ifesdjeen] is doing for the java
driver). This also would solve #2.The downside of tying this change to a
protocol version bump is that I imagine it defers this change quite a bit, and
this fix would be really nice to have as it will fix a problem a lot of users
have run into.
> Include hash of result set metadata in prepared statement id
>
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
> Labels: client-impacting
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a
> prepared statement when the table is altered, to force clients to update
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host.
> The first client to execute the query after the cache was invalidated will
> receive an UNPREPARED response, re-prepare, and update its local metadata.
> But other clients might miss it entirely (the MD5 hasn't changed), and they
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123,
> clientA and clientB both have a cache of the metadata (columns b and c)
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response,
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been
> populated again, the query succeeds. But clientB still has not updated its
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set
> metadata in the md5. This way the md5 would change at step 3, and any client
> using the old md5 would get an UNPREPARED, regardless of whether another
> client already reprepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11595) Cassandra cannot start because of empty commitlog
[ https://issues.apache.org/jira/browse/CASSANDRA-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15416539#comment-15416539 ] Andy Tolbert commented on CASSANDRA-11595: -- Was talking with a user who also encountered this same problem (also on windows), however in their case their Commitlog was 32MB, but was completely zero'd out. Their system had been power cycled and upon restart they hit this error. I was looking at the code and saw that when reading commitlogs prior to 2.1 a [different code path|https://github.com/apache/cassandra/blob/cassandra-3.7/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L358] would be taken for reading commitlogs which seemed to [gracefully proceed|https://github.com/apache/cassandra/blob/cassandra-3.7/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L464-L468] when reading 0. Although in that case no header would be read from like it is for [newer commitlogs|https://github.com/apache/cassandra/blob/cassandra-3.7/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java#L370]. Should it be considered mandatory that the header content is present in the commitlog as it currently is? In this case it seems like there is no risk in the user simply deleting the file and moving on since it doesn't contain anything anyways. Although maybe there is still a problem somewhere since the file has no header in the first place. > Cassandra cannot start because of empty commitlog > - > > Key: CASSANDRA-11595 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11595 > Project: Cassandra > Issue Type: Bug >Reporter: n0rad >Assignee: Benjamin Lerer > > After the crash of CASSANDRA-11594. > Cassandra try to restart and fail because of commit log replay. > Same on 4 of the crashed nodes out of 6. > ``` > org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: > Could not read commit log descriptor in file > /data/commitlog/CommitLog-6-1460632496764.log > at > org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:644) > [apache-cassandra-3.0.5.jar:3.0.5] > ``` > This file is empty and is not the commitlog with the latest date. > ``` > ... > -rw-r--r-- 1 root root 32M Apr 16 21:46 CommitLog-6-1460632496761.log > -rw-r--r-- 1 root root 32M Apr 16 21:47 CommitLog-6-1460632496762.log > -rw-r--r-- 1 root root 32M Apr 16 21:47 CommitLog-6-1460632496763.log > -rw-r--r-- 1 root root 0 Apr 16 21:47 CommitLog-6-1460632496764.log > -rw-r--r-- 1 root root 32M Apr 16 21:50 CommitLog-6-1460843401097.log > -rw-r--r-- 1 root root 32M Apr 16 21:51 CommitLog-6-1460843513346.log > -rw-r--r-- 1 root root 32M Apr 16 21:53 CommitLog-6-1460843619271.log > -rw-r--r-- 1 root root 32M Apr 16 21:55 CommitLog-6-1460843730533.log > -rw-r--r-- 1 root root 32M Apr 16 21:57 CommitLog-6-1460843834129.log > -rw-r--r-- 1 root root 32M Apr 16 21:58 CommitLog-6-1460843935094.log > -rw-r--r-- 1 root root 32M Apr 16 22:00 CommitLog-6-1460844038543.log > -rw-r--r-- 1 root root 32M Apr 16 22:02 CommitLog-6-1460844141003.log > ... > ``` -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-12434) Improve deletion_info in sstabledump to include calculated 'expires_at' based on local_delete_time + gc_grace_seconds
Andy Tolbert created CASSANDRA-12434:
Summary: Improve deletion_info in sstabledump to include
calculated 'expires_at' based on local_delete_time + gc_grace_seconds
Key: CASSANDRA-12434
URL: https://issues.apache.org/jira/browse/CASSANDRA-12434
Project: Cassandra
Issue Type: Improvement
Reporter: Andy Tolbert
Priority: Minor
It would be helpful if sstabledump included tombstone expiration time in
{{deletion_info}} output by calculating {{local_delete_time +
gc_grace_seconds}}. To do this it would need access to the table schema to get
at {{gc_grace_seconds}} which is currently not possible until [CASSANDRA-9587].
Example output, assuming gc_grace_seconds is 86400 (10 days):
{code:javascript}
{ "name" : "data", "deletion_info" : { "marked_deleted" :
"2016-07-01T18:00:06.800550Z", "local_delete_time" : "2016-07-01T18:00:06Z",
"expires_at": "2016-07-11T18:00:06Z" } }
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12434) Improve deletion_info in sstabledump to include calculated 'expires_at' based on local_delete_time + gc_grace_seconds
[
https://issues.apache.org/jira/browse/CASSANDRA-12434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15416570#comment-15416570
]
Andy Tolbert commented on CASSANDRA-12434:
--
Maybe 'expires_at' isn't the appropriate term since it could be conflated with
TTLs, can probably think of something better.
> Improve deletion_info in sstabledump to include calculated 'expires_at' based
> on local_delete_time + gc_grace_seconds
> -
>
> Key: CASSANDRA-12434
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12434
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Andy Tolbert
>Priority: Minor
>
> It would be helpful if sstabledump included tombstone expiration time in
> {{deletion_info}} output by calculating {{local_delete_time +
> gc_grace_seconds}}. To do this it would need access to the table schema to
> get at {{gc_grace_seconds}} which is currently not possible until
> [CASSANDRA-9587].
> Example output, assuming gc_grace_seconds is 86400 (10 days):
> {code:javascript}
> { "name" : "data", "deletion_info" : { "marked_deleted" :
> "2016-07-01T18:00:06.800550Z", "local_delete_time" : "2016-07-01T18:00:06Z",
> "expires_at": "2016-07-11T18:00:06Z" } }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11595) Cassandra cannot start because of empty commitlog
[
https://issues.apache.org/jira/browse/CASSANDRA-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421501#comment-15421501
]
Andy Tolbert commented on CASSANDRA-11595:
--
{quote}
In general, our philosophy is that if there's a problem with your commitlog we
die and notify you unless you specifically configure otherwise. Also, the
possibility of introducing a bug that leads to data loss with auto-deletion is
sufficient for me to advocate against it.
{quote}
That seems fair to me. Something went wrong, the user should take a step back
and think about it.
Although in this case it's not particularly user friendly, there's nothing in
the file, so there really is no recovery path for that particular file. The
user may not understand that the file is empty - since it is 32MB after all,
albeit zero'd out - and may not understand the only corrective course is to
remove/move away the commitlog.
Overall, I agree the fact that the file is empty in the first place is the main
problem (although caused by some external factor like powercycling the system)
- not that C* refuses to start because of it.
> Cassandra cannot start because of empty commitlog
> -
>
> Key: CASSANDRA-11595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11595
> Project: Cassandra
> Issue Type: Bug
>Reporter: n0rad
>Assignee: Benjamin Lerer
>
> After the crash of CASSANDRA-11594.
> Cassandra try to restart and fail because of commit log replay.
> Same on 4 of the crashed nodes out of 6.
> ```
> org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException:
> Could not read commit log descriptor in file
> /data/commitlog/CommitLog-6-1460632496764.log
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:644)
> [apache-cassandra-3.0.5.jar:3.0.5]
> ```
> This file is empty and is not the commitlog with the latest date.
> ```
> ...
> -rw-r--r-- 1 root root 32M Apr 16 21:46 CommitLog-6-1460632496761.log
> -rw-r--r-- 1 root root 32M Apr 16 21:47 CommitLog-6-1460632496762.log
> -rw-r--r-- 1 root root 32M Apr 16 21:47 CommitLog-6-1460632496763.log
> -rw-r--r-- 1 root root 0 Apr 16 21:47 CommitLog-6-1460632496764.log
> -rw-r--r-- 1 root root 32M Apr 16 21:50 CommitLog-6-1460843401097.log
> -rw-r--r-- 1 root root 32M Apr 16 21:51 CommitLog-6-1460843513346.log
> -rw-r--r-- 1 root root 32M Apr 16 21:53 CommitLog-6-1460843619271.log
> -rw-r--r-- 1 root root 32M Apr 16 21:55 CommitLog-6-1460843730533.log
> -rw-r--r-- 1 root root 32M Apr 16 21:57 CommitLog-6-1460843834129.log
> -rw-r--r-- 1 root root 32M Apr 16 21:58 CommitLog-6-1460843935094.log
> -rw-r--r-- 1 root root 32M Apr 16 22:00 CommitLog-6-1460844038543.log
> -rw-r--r-- 1 root root 32M Apr 16 22:02 CommitLog-6-1460844141003.log
> ...
> ```
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11595) Cassandra cannot start because of empty commitlog
[
https://issues.apache.org/jira/browse/CASSANDRA-11595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421556#comment-15421556
]
Andy Tolbert commented on CASSANDRA-11595:
--
{quote}
I think the safest choice here is to prevent C* from starting and to let the
administrator deal with the problem. As in this case the file is obviously
empty the solution to the problem is pretty clear.
Andy Tolbert you are talking in my opinion of a different problem in the same
area that affect 2.1 where C* has probably crashed after the resize.
I will clause the ticket as Not A Problem as C* behaved as expected.
{quote}
(y), agreed.
> Cassandra cannot start because of empty commitlog
> -
>
> Key: CASSANDRA-11595
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11595
> Project: Cassandra
> Issue Type: Bug
>Reporter: n0rad
>Assignee: Benjamin Lerer
>
> After the crash of CASSANDRA-11594.
> Cassandra try to restart and fail because of commit log replay.
> Same on 4 of the crashed nodes out of 6.
> ```
> org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException:
> Could not read commit log descriptor in file
> /data/commitlog/CommitLog-6-1460632496764.log
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:644)
> [apache-cassandra-3.0.5.jar:3.0.5]
> ```
> This file is empty and is not the commitlog with the latest date.
> ```
> ...
> -rw-r--r-- 1 root root 32M Apr 16 21:46 CommitLog-6-1460632496761.log
> -rw-r--r-- 1 root root 32M Apr 16 21:47 CommitLog-6-1460632496762.log
> -rw-r--r-- 1 root root 32M Apr 16 21:47 CommitLog-6-1460632496763.log
> -rw-r--r-- 1 root root 0 Apr 16 21:47 CommitLog-6-1460632496764.log
> -rw-r--r-- 1 root root 32M Apr 16 21:50 CommitLog-6-1460843401097.log
> -rw-r--r-- 1 root root 32M Apr 16 21:51 CommitLog-6-1460843513346.log
> -rw-r--r-- 1 root root 32M Apr 16 21:53 CommitLog-6-1460843619271.log
> -rw-r--r-- 1 root root 32M Apr 16 21:55 CommitLog-6-1460843730533.log
> -rw-r--r-- 1 root root 32M Apr 16 21:57 CommitLog-6-1460843834129.log
> -rw-r--r-- 1 root root 32M Apr 16 21:58 CommitLog-6-1460843935094.log
> -rw-r--r-- 1 root root 32M Apr 16 22:00 CommitLog-6-1460844038543.log
> -rw-r--r-- 1 root root 32M Apr 16 22:02 CommitLog-6-1460844141003.log
> ...
> ```
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12594) sstabledump fails on frozen collection cells
Andy Tolbert created CASSANDRA-12594:
Summary: sstabledump fails on frozen collection cells
Key: CASSANDRA-12594
URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
Project: Cassandra
Issue Type: Bug
Reporter: Andy Tolbert
Assignee: Andy Tolbert
Priority: Minor
sstabledump throws an exception when attempting to parse a cell that is a
frozen collection, i.e.:
{noformat}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
"cells" : [
{ "name" : "m", "value" }
] }
] }
]Exception in thread "main" java.lang.IllegalArgumentException
at java.nio.Buffer.limit(Buffer.java:275)
at
org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
at
org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
at
org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
at
org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
at
org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
at
org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.Iterator.forEachRemaining(Iterator.java:116)
at
java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at
org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
{noformat}
This is because the code doesn't consider that the cell may be a frozen
collection, and attempts to get the string representation using the value type
which doesn't work.
Example data:
{noformat}
CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy',
'replication_factor': '1'};
CREATE TABLE simple.unfrozen_map (
k int PRIMARY KEY,
m map>>
);
CREATE TABLE simple.frozen_map (
k int PRIMARY KEY,
m frozen>>>
);
insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
'g': ('h', 'i')});
insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
'g': ('h', 'i')});
{noformat}
unfrozen_map will properly dump each cell individually, but frozen_map fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12594) sstabledump fails on frozen collection cells
[
https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456906#comment-15456906
]
Andy Tolbert commented on CASSANDRA-12594:
--
I have a fix for this which may or may not be a good solution. As the
individual {{CollecitonType}} implementation's {{getString()}} method just
returns a raw hexString, we could instead use {{AbstractType#toJSONString}} if
the given cell type {{isFrozenCollection}} evaluates to true.
This provides the following output for the example data above:
unfrozen case (existing that works):
{code}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:34.950608Z" },
"cells" : [
{ "name" : "m", "deletion_info" : { "marked_deleted" :
"2016-09-01T22:06:34.950607Z", "local_delete_time" : "2016-09-01T22:06:34Z" } },
{ "name" : "m", "path" : [ "a" ], "value" : "b:c" },
{ "name" : "m", "path" : [ "d" ], "value" : "e:f" },
{ "name" : "m", "path" : [ "g" ], "value" : "h:i" }
]
}
]
}
]
{code}
frozen case:
{code}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
"cells" : [
{ "name" : "m", "value" : {"a": ["b", "c"], "d": ["e", "f"], "g":
["h", "i"]} }
]
}
]
}
]
{code}
Attaching patch shortly.
> sstabledump fails on frozen collection cells
>
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
>
> sstabledump throws an exception when attempting to parse a cell that is a
> frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at
> org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
> {noformat}
> This is because the code doesn't consider that the cell may be a frozen
> collection, and attempts to get the string representation using the value
> type which doesn't work.
> Example data:
> {noformat}
> CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE simple.unfrozen_map (
> k int PRIMARY KEY,
> m map>>
> );
> CREATE TABLE simple.frozen_map (
> k int PRIMARY KEY,
> m frozen>>>
> );
> insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> {noformat}
> unfrozen_map will properly dump each cell individ
[jira] [Comment Edited] (CASSANDRA-12594) sstabledump fails on frozen collection cells
[
https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456906#comment-15456906
]
Andy Tolbert edited comment on CASSANDRA-12594 at 9/1/16 11:15 PM:
---
I have a fix for this which may or may not be a good solution. As the
individual {{CollectionType}} implementation's {{getString()}} method just
returns a raw hexString, we could instead use {{AbstractType#toJSONString}} if
the given cell type {{isFrozenCollection}} evaluates to true.
This provides the following output for the example data above:
unfrozen case (existing that works):
{code}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:34.950608Z" },
"cells" : [
{ "name" : "m", "deletion_info" : { "marked_deleted" :
"2016-09-01T22:06:34.950607Z", "local_delete_time" : "2016-09-01T22:06:34Z" } },
{ "name" : "m", "path" : [ "a" ], "value" : "b:c" },
{ "name" : "m", "path" : [ "d" ], "value" : "e:f" },
{ "name" : "m", "path" : [ "g" ], "value" : "h:i" }
]
}
]
}
]
{code}
frozen case:
{code}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
"cells" : [
{ "name" : "m", "value" : {"a": ["b", "c"], "d": ["e", "f"], "g":
["h", "i"]} }
]
}
]
}
]
{code}
Attaching patch shortly.
was (Author: andrew.tolbert):
I have a fix for this which may or may not be a good solution. As the
individual {{CollecitonType}} implementation's {{getString()}} method just
returns a raw hexString, we could instead use {{AbstractType#toJSONString}} if
the given cell type {{isFrozenCollection}} evaluates to true.
This provides the following output for the example data above:
unfrozen case (existing that works):
{code}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:34.950608Z" },
"cells" : [
{ "name" : "m", "deletion_info" : { "marked_deleted" :
"2016-09-01T22:06:34.950607Z", "local_delete_time" : "2016-09-01T22:06:34Z" } },
{ "name" : "m", "path" : [ "a" ], "value" : "b:c" },
{ "name" : "m", "path" : [ "d" ], "value" : "e:f" },
{ "name" : "m", "path" : [ "g" ], "value" : "h:i" }
]
}
]
}
]
{code}
frozen case:
{code}
[
{
"partition" : {
"key" : [ "0" ],
"position" : 0
},
"rows" : [
{
"type" : "row",
"position" : 18,
"liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
"cells" : [
{ "name" : "m", "value" : {"a": ["b", "c"], "d": ["e", "f"], "g":
["h", "i"]} }
]
}
]
}
]
{code}
Attaching patch shortly.
> sstabledump fails on frozen collection cells
>
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
>
> sstabledump throws an exception when attempting to parse a cell that is a
> frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at
> org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$It
[jira] [Updated] (CASSANDRA-12594) sstabledump fails on frozen collection cells
[
https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-12594:
-
Fix Version/s: 3.9
3.0.9
Reproduced In: 3.0.8
Status: Patch Available (was: Open)
> sstabledump fails on frozen collection cells
>
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
> Fix For: 3.0.9, 3.9
>
>
> sstabledump throws an exception when attempting to parse a cell that is a
> frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at
> org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
> {noformat}
> This is because the code doesn't consider that the cell may be a frozen
> collection, and attempts to get the string representation using the value
> type which doesn't work.
> Example data:
> {noformat}
> CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE simple.unfrozen_map (
> k int PRIMARY KEY,
> m map>>
> );
> CREATE TABLE simple.frozen_map (
> k int PRIMARY KEY,
> m frozen>>>
> );
> insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> {noformat}
> unfrozen_map will properly dump each cell individually, but frozen_map fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12594) sstabledump fails on frozen collection cells
[
https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-12594:
-
Attachment: CASSANDRA-12594-3.0.txt
> sstabledump fails on frozen collection cells
>
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
> Fix For: 3.0.9, 3.9
>
> Attachments: CASSANDRA-12594-3.0.txt
>
>
> sstabledump throws an exception when attempting to parse a cell that is a
> frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at
> org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
> {noformat}
> This is because the code doesn't consider that the cell may be a frozen
> collection, and attempts to get the string representation using the value
> type which doesn't work.
> Example data:
> {noformat}
> CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE simple.unfrozen_map (
> k int PRIMARY KEY,
> m map>>
> );
> CREATE TABLE simple.frozen_map (
> k int PRIMARY KEY,
> m frozen>>>
> );
> insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> {noformat}
> unfrozen_map will properly dump each cell individually, but frozen_map fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12594) sstabledump fails on frozen collection cells
[
https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-12594:
-
Attachment: CASSANDRA-12594-3.0_2.txt
Fixed patch [^CASSANDRA-12594-3.0_2.txt] which uses current server version
protocol version instead of hardcoding to 4.
> sstabledump fails on frozen collection cells
>
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
> Fix For: 3.0.9, 3.9
>
> Attachments: CASSANDRA-12594-3.0.txt, CASSANDRA-12594-3.0_2.txt
>
>
> sstabledump throws an exception when attempting to parse a cell that is a
> frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at
> org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
> {noformat}
> This is because the code doesn't consider that the cell may be a frozen
> collection, and attempts to get the string representation using the value
> type which doesn't work.
> Example data:
> {noformat}
> CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE simple.unfrozen_map (
> k int PRIMARY KEY,
> m map>>
> );
> CREATE TABLE simple.frozen_map (
> k int PRIMARY KEY,
> m frozen>>>
> );
> insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> {noformat}
> unfrozen_map will properly dump each cell individually, but frozen_map fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12594) sstabledump fails on frozen collection cells
[
https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456961#comment-15456961
]
Andy Tolbert edited comment on CASSANDRA-12594 at 9/1/16 11:44 PM:
---
Fixed patch [^CASSANDRA-12594-3.0_2.txt] which uses current server version
protocol version instead of hardcoding to 4. It seems to apply cleanly on 3.9
and trunk as well.
was (Author: andrew.tolbert):
Fixed patch [^CASSANDRA-12594-3.0_2.txt] which uses current server version
protocol version instead of hardcoding to 4.
> sstabledump fails on frozen collection cells
>
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
> Fix For: 3.0.9, 3.9
>
> Attachments: CASSANDRA-12594-3.0.txt, CASSANDRA-12594-3.0_2.txt
>
>
> sstabledump throws an exception when attempting to parse a cell that is a
> frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at
> org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
> {noformat}
> This is because the code doesn't consider that the cell may be a frozen
> collection, and attempts to get the string representation using the value
> type which doesn't work.
> Example data:
> {noformat}
> CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE simple.unfrozen_map (
> k int PRIMARY KEY,
> m map>>
> );
> CREATE TABLE simple.frozen_map (
> k int PRIMARY KEY,
> m frozen>>>
> );
> insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> {noformat}
> unfrozen_map will properly dump each cell individually, but frozen_map fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12594) sstabledump fails on frozen collection cells
[
https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458966#comment-15458966
]
Andy Tolbert commented on CASSANDRA-12594:
--
That is understandable (y). I was working around
{{ColumnDefinition.cellValueType()}} returning the same thing it returns for a
non-frozen collection. Wouldn't there also be an issue in the tools'
dependence on {{AbstractType#getString}} to get a string representation of
data? Currently collections return hexstring representation for {{getString}}
(via {{CollectionType}}, which doesn't seem useful for the sstabledump output.
Should the various {{CollectionType}} implementations implement their own
{{getString}} as well?
> sstabledump fails on frozen collection cells
>
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
> Fix For: 3.0.9, 3.9
>
> Attachments: CASSANDRA-12594-3.0.txt, CASSANDRA-12594-3.0_2.txt
>
>
> sstabledump throws an exception when attempting to parse a cell that is a
> frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at
> org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
> {noformat}
> This is because the code doesn't consider that the cell may be a frozen
> collection, and attempts to get the string representation using the value
> type which doesn't work.
> Example data:
> {noformat}
> CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE simple.unfrozen_map (
> k int PRIMARY KEY,
> m map>>
> );
> CREATE TABLE simple.frozen_map (
> k int PRIMARY KEY,
> m frozen>>>
> );
> insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> {noformat}
> unfrozen_map will properly dump each cell individually, but frozen_map fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12594) sstabledump fails on frozen collection cells
[
https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15458966#comment-15458966
]
Andy Tolbert edited comment on CASSANDRA-12594 at 9/2/16 4:24 PM:
--
That is understandable (y). I was working around
{{ColumnDefinition.cellValueType()}} returning the same thing it returns for a
non-frozen collection. Wouldn't there also be an issue in the tools'
dependence on {{AbstractType#getString}} to get a string representation of
data? Currently collections return hexstring representation for {{getString}}
(via {{CollectionType}}, which doesn't seem useful for the sstabledump output).
Should the various {{CollectionType}} implementations implement their own
{{getString}} as well?
was (Author: andrew.tolbert):
That is understandable (y). I was working around
{{ColumnDefinition.cellValueType()}} returning the same thing it returns for a
non-frozen collection. Wouldn't there also be an issue in the tools'
dependence on {{AbstractType#getString}} to get a string representation of
data? Currently collections return hexstring representation for {{getString}}
(via {{CollectionType}}, which doesn't seem useful for the sstabledump output.
Should the various {{CollectionType}} implementations implement their own
{{getString}} as well?
> sstabledump fails on frozen collection cells
>
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
> Fix For: 3.0.9, 3.9
>
> Attachments: CASSANDRA-12594-3.0.txt, CASSANDRA-12594-3.0_2.txt
>
>
> sstabledump throws an exception when attempting to parse a cell that is a
> frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at
> org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
> {noformat}
> This is because the code doesn't consider that the cell may be a frozen
> collection, and attempts to get the string representation using the value
> type which doesn't work.
> Example data:
> {noformat}
> CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE simple.unfrozen_map (
> k int PRIMARY KEY,
> m map>>
> );
> CREATE TABLE simple.frozen_map (
> k int PRIMARY KEY,
> m frozen>>>
> );
> insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> {noformat}
> unfrozen_map will properly dump each cell individually, but frozen_map fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12594) sstabledump fails on frozen collection cells
[
https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469464#comment-15469464
]
Andy Tolbert commented on CASSANDRA-12594:
--
{quote}
sstabledump returns JSON and we now have a proper toJSONString() method in
AbstractType that handles that properly so it would make sense to simply use
that for everything in sstabledump in my opinion
{quote}
That makes perfect sense to me, and also likely easy to change. I'll play with
that a bit and follow on sometime this week.
> sstabledump fails on frozen collection cells
>
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
> Fix For: 3.0.9, 3.9
>
> Attachments: CASSANDRA-12594-3.0.txt, CASSANDRA-12594-3.0_2.txt
>
>
> sstabledump throws an exception when attempting to parse a cell that is a
> frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at
> org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
> {noformat}
> This is because the code doesn't consider that the cell may be a frozen
> collection, and attempts to get the string representation using the value
> type which doesn't work.
> Example data:
> {noformat}
> CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE simple.unfrozen_map (
> k int PRIMARY KEY,
> m map>>
> );
> CREATE TABLE simple.frozen_map (
> k int PRIMARY KEY,
> m frozen>>>
> );
> insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> {noformat}
> unfrozen_map will properly dump each cell individually, but frozen_map fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12628) sstabledump should use AbstractType.toJSONString() instead of getString()
Andy Tolbert created CASSANDRA-12628:
Summary: sstabledump should use AbstractType.toJSONString()
instead of getString()
Key: CASSANDRA-12628
URL: https://issues.apache.org/jira/browse/CASSANDRA-12628
Project: Cassandra
Issue Type: Bug
Reporter: Andy Tolbert
Assignee: Andy Tolbert
Priority: Minor
Since sstabledump outputs json, it makes sense for it to use
{{AbstractType.toJSONString()}} instead of {{getString()}} as discussed in
[CASSANDRA-12594]:
{quote}
sstabledump returns JSON and we now have a proper toJSONString() method in
AbstractType that handles that properly so it would make sense to simply use
that for everything in sstabledump in my opinion.
{quote}
Doing this should also fix [CASSANDRA-12594]. There may be a few small
differences in the output format, which I will attempt to document here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12594) sstabledump fails on frozen collection cells
[
https://issues.apache.org/jira/browse/CASSANDRA-12594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-12594:
-
Fix Version/s: (was: 3.0.9)
(was: 3.9)
> sstabledump fails on frozen collection cells
>
>
> Key: CASSANDRA-12594
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12594
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
> Attachments: CASSANDRA-12594-3.0.txt, CASSANDRA-12594-3.0_2.txt
>
>
> sstabledump throws an exception when attempting to parse a cell that is a
> frozen collection, i.e.:
> {noformat}
> [
> {
> "partition" : {
> "key" : [ "0" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 18,
> "liveness_info" : { "tstamp" : "2016-09-01T22:06:45.670810Z" },
> "cells" : [
> { "name" : "m", "value" }
> ] }
> ] }
> ]Exception in thread "main" java.lang.IllegalArgumentException
> at java.nio.Buffer.limit(Buffer.java:275)
> at
> org.apache.cassandra.utils.ByteBufferUtil.readBytes(ByteBufferUtil.java:613)
> at
> org.apache.cassandra.db.marshal.TupleType.getString(TupleType.java:211)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeCell(JsonTransformer.java:441)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeColumnData(JsonTransformer.java:375)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:279)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:214)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:102)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:242)
> {noformat}
> This is because the code doesn't consider that the cell may be a frozen
> collection, and attempts to get the string representation using the value
> type which doesn't work.
> Example data:
> {noformat}
> CREATE KEYSPACE simple WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': '1'};
> CREATE TABLE simple.unfrozen_map (
> k int PRIMARY KEY,
> m map>>
> );
> CREATE TABLE simple.frozen_map (
> k int PRIMARY KEY,
> m frozen>>>
> );
> insert into unfrozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> insert into frozen_map (k, m) values (0, {'a': ('b', 'c'), 'd': ('e', 'f'),
> 'g': ('h', 'i')});
> {noformat}
> unfrozen_map will properly dump each cell individually, but frozen_map fails.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12628) sstabledump should use AbstractType.toJSONString() instead of getString()
[
https://issues.apache.org/jira/browse/CASSANDRA-12628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15482396#comment-15482396
]
Andy Tolbert commented on CASSANDRA-12628:
--
[wip branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-12628], needs
a bit of testing so will hold off on attaching a patch until comfortable. Also
noticed that unfrozen UDTs don't work either, added support for that as well.
> sstabledump should use AbstractType.toJSONString() instead of getString()
> -
>
> Key: CASSANDRA-12628
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12628
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Minor
>
> Since sstabledump outputs json, it makes sense for it to use
> {{AbstractType.toJSONString()}} instead of {{getString()}} as discussed in
> [CASSANDRA-12594]:
> {quote}
> sstabledump returns JSON and we now have a proper toJSONString() method in
> AbstractType that handles that properly so it would make sense to simply use
> that for everything in sstabledump in my opinion.
> {quote}
> Doing this should also fix [CASSANDRA-12594]. There may be a few small
> differences in the output format, which I will attempt to document here.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12630) Ability to restrict size of outbound native protocol frames
Andy Tolbert created CASSANDRA-12630:
Summary: Ability to restrict size of outbound native protocol
frames
Key: CASSANDRA-12630
URL: https://issues.apache.org/jira/browse/CASSANDRA-12630
Project: Cassandra
Issue Type: Improvement
Reporter: Andy Tolbert
{{native_transport_max_frame_size_in_mb}} [is documented
as|http://cassandra.apache.org/doc/latest/configuration/cassandra_config_file.html?highlight=native_transport_max_frame_size_in_mb#native-transport-max-frame-size-in-mb]:
{quote}
The maximum size of allowed frame. Frame (requests) larger than this will be
rejected as invalid. The default is 256MB. If you’re changing this parameter,
you may want to adjust max_value_size_in_mb accordingly.
{quote}
It wasn't immediately clear to me (and others) that this value is only used for
validating inbound frames coming from a client and is not used for frames
generated by the server to be sent outbound. Although the {{Frame (requests)}}
part indicates that this is only for inbound messages.
The java driver will currently fail any frames larger than 256mb and the native
protocol spec claims:
{quote}
2.5. length
A 4 byte integer representing the length of the body of the frame (note:
currently a frame is limited to 256MB in length).
{quote}
But it is currently possible for C* to generate frames larger than this and
send them out. It would to nice if C* could restrict this behavior, either by
native_transport_max_frame_size_in_mb or some other config, and prevent larger
payloads from being sent by the server.
More discussion @
[JAVA-1292|https://datastax-oss.atlassian.net/browse/JAVA-1292]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12686) Communicate timeouts and other driver relevant options in SUPPORTED response or some other mechanism
Andy Tolbert created CASSANDRA-12686:
Summary: Communicate timeouts and other driver relevant options in
SUPPORTED response or some other mechanism
Key: CASSANDRA-12686
URL: https://issues.apache.org/jira/browse/CASSANDRA-12686
Project: Cassandra
Issue Type: Improvement
Reporter: Andy Tolbert
It would be really useful if driver clients had a mechanism to understand what
the configured timeouts on the C* side are.
Ideally a driver should be configured in such a way that it's client timeout is
greater than the C* timeouts ({{write_request_timeout_in_ms}},
{{read_request_timeout_in_ms}}, etc.) so its retry policy may make the
appropriate decision based on the kind of timeout received from cassandra.
This is why most driver clients have a client timeout of 12 seconds. If the
client knew the server timeouts, it could adjust its client timeout accordingly.
At the moment, the only place I think where this could be communicated is
through a {{SUPPORTED}} message when the client sends an {{OPTIONS}} message,
but that could be viewed as awkward. Also consider that some clients use the
{{OPTIONS}} message as a form of heartbeat, so adding more to a {{SUPPORTED}}
message could add some (likely trivial) data on the wire between server and
client.
Alternatively, it could also be interesting if the client could configure the
timeout on the server end (with some ceiling set by C*).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12687) Protocol level heartbeat
Andy Tolbert created CASSANDRA-12687:
Summary: Protocol level heartbeat
Key: CASSANDRA-12687
URL: https://issues.apache.org/jira/browse/CASSANDRA-12687
Project: Cassandra
Issue Type: Improvement
Reporter: Andy Tolbert
Most of the datastax drivers use the {{OPTIONS}} message as a means of doing a
protocol level heartbeat, to which the server responds with a {{SUPPORTED}}
message (see: http://datastax.github.io/java-driver/manual/pooling/#heartbeat).
It would be great if there was a simple {{HEARTBEAT}} message type that could
be sent in either direction to indicate that the other side of a connection is
still responding at a protocol level.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12686) Communicate timeouts and other driver relevant options in SUPPORTED response or some other mechanism
[
https://issues.apache.org/jira/browse/CASSANDRA-12686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-12686:
-
Labels: client-impacting (was: )
> Communicate timeouts and other driver relevant options in SUPPORTED response
> or some other mechanism
>
>
> Key: CASSANDRA-12686
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12686
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Andy Tolbert
> Labels: client-impacting
>
> It would be really useful if driver clients had a mechanism to understand
> what the configured timeouts on the C* side are.
> Ideally a driver should be configured in such a way that it's client timeout
> is greater than the C* timeouts ({{write_request_timeout_in_ms}},
> {{read_request_timeout_in_ms}}, etc.) so its retry policy may make the
> appropriate decision based on the kind of timeout received from cassandra.
> This is why most driver clients have a client timeout of 12 seconds. If the
> client knew the server timeouts, it could adjust its client timeout
> accordingly.
> At the moment, the only place I think where this could be communicated is
> through a {{SUPPORTED}} message when the client sends an {{OPTIONS}} message,
> but that could be viewed as awkward. Also consider that some clients use the
> {{OPTIONS}} message as a form of heartbeat, so adding more to a {{SUPPORTED}}
> message could add some (likely trivial) data on the wire between server and
> client.
> Alternatively, it could also be interesting if the client could configure the
> timeout on the server end (with some ceiling set by C*).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12754) Change cassandra.wait_for_tracing_events_timeout_secs default to -1 so C* doesn't wait on trace events to be written before responding to request by default
Andy Tolbert created CASSANDRA-12754:
Summary: Change cassandra.wait_for_tracing_events_timeout_secs
default to -1 so C* doesn't wait on trace events to be written before
responding to request by default
Key: CASSANDRA-12754
URL: https://issues.apache.org/jira/browse/CASSANDRA-12754
Project: Cassandra
Issue Type: Bug
Reporter: Andy Tolbert
[CASSANDRA-11465] introduces a new system property
{{cassandra.wait_for_tracing_events_timeout_secs}} that controls whether or not
C* waits for events to be written before responding to client. The current
default behavior is to wait up to 1 second and then respond and timeout.
If using probabilistic tracing this can cause queries to be randomly delayed up
to 1 second.
Changing the default to -1 (disabled and enabling it explicitly in
{{cql_tracing_test.TestCqlTracing.tracing_unknown_impl_test}}.
Ideally it would be nice to be able to control this behavior on a per request
basis (which would require native protocol changes).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12755) Ability to control whether or not to wait on trace events being written before responding to client
Andy Tolbert created CASSANDRA-12755:
Summary: Ability to control whether or not to wait on trace events
being written before responding to client
Key: CASSANDRA-12755
URL: https://issues.apache.org/jira/browse/CASSANDRA-12755
Project: Cassandra
Issue Type: Improvement
Reporter: Andy Tolbert
Priority: Minor
[CASSANDRA-11465] introduces a new system property
{{cassandra.wait_for_tracing_events_timeout_secs}} that controls whether or not
C* waits for events to be written before responding to client.
Instead it would be great if this could be configured on a per request basis
(which would require a new flag at frame level or something else in protocol)
and also as an option on the {{settraceprobability}} command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12755) Ability to control whether or not to wait on trace events being written before responding to client at request level
[
https://issues.apache.org/jira/browse/CASSANDRA-12755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-12755:
-
Summary: Ability to control whether or not to wait on trace events being
written before responding to client at request level (was: Ability to control
whether or not to wait on trace events being written before responding to
client)
> Ability to control whether or not to wait on trace events being written
> before responding to client at request level
>
>
> Key: CASSANDRA-12755
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12755
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Andy Tolbert
>Priority: Minor
>
> [CASSANDRA-11465] introduces a new system property
> {{cassandra.wait_for_tracing_events_timeout_secs}} that controls whether or
> not C* waits for events to be written before responding to client.
> Instead it would be great if this could be configured on a per request basis
> (which would require a new flag at frame level or something else in protocol)
> and also as an option on the {{settraceprobability}} command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-12774) Expose dc in Unavailable exception errors
Andy Tolbert created CASSANDRA-12774:
Summary: Expose dc in Unavailable exception errors
Key: CASSANDRA-12774
URL: https://issues.apache.org/jira/browse/CASSANDRA-12774
Project: Cassandra
Issue Type: Bug
Reporter: Andy Tolbert
Priority: Minor
For protocol v5 or later, it could be useful if Unavailable optionally included
the DC that could not meet the CL.
For example. if a user has a keyspace with RF of { dc1: 3, and dc2: 3 } and
they make a query at {{EACH_QUORUM}} and not enough replicas are available in
dc2, an {{UnavailableException}} will be sent to the client with X available
and 2 required, but we don't know which DC failed. It looks like
{{UnavailableException}} already has a constructor that takes in the DC (see
[here|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/exceptions/UnavailableException.java#L33])
so this could be feasible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12774) Expose dc in Unavailable exception errors
[
https://issues.apache.org/jira/browse/CASSANDRA-12774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-12774:
-
Description:
For protocol v5 or later, it could be useful if Unavailable optionally included
the DC that could not meet the CL.
For example. if a user has a keyspace with RF of { dc1: 3, dc2: 3 } and they
make a query at {{EACH_QUORUM}} and not enough replicas are available in dc2,
an {{UnavailableException}} will be sent to the client with X available and 2
required, but we don't know which DC failed. It looks like
{{UnavailableException}} already has a constructor that takes in the DC (see
[here|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/exceptions/UnavailableException.java#L33])
so this could be feasible.
was:
For protocol v5 or later, it could be useful if Unavailable optionally included
the DC that could not meet the CL.
For example. if a user has a keyspace with RF of { dc1: 3, and dc2: 3 } and
they make a query at {{EACH_QUORUM}} and not enough replicas are available in
dc2, an {{UnavailableException}} will be sent to the client with X available
and 2 required, but we don't know which DC failed. It looks like
{{UnavailableException}} already has a constructor that takes in the DC (see
[here|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/exceptions/UnavailableException.java#L33])
so this could be feasible.
> Expose dc in Unavailable exception errors
> -
>
> Key: CASSANDRA-12774
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12774
> Project: Cassandra
> Issue Type: Bug
>Reporter: Andy Tolbert
>Priority: Minor
>
> For protocol v5 or later, it could be useful if Unavailable optionally
> included the DC that could not meet the CL.
> For example. if a user has a keyspace with RF of { dc1: 3, dc2: 3 } and they
> make a query at {{EACH_QUORUM}} and not enough replicas are available in dc2,
> an {{UnavailableException}} will be sent to the client with X available and 2
> required, but we don't know which DC failed. It looks like
> {{UnavailableException}} already has a constructor that takes in the DC (see
> [here|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/exceptions/UnavailableException.java#L33])
> so this could be feasible.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8871) Non-null paging state returned if last page
[
https://issues.apache.org/jira/browse/CASSANDRA-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-8871:
Description:
tWhen retrieving the next page from the result of a simple statement, the
result will return a non-null paging state even if it's the last page of the
query. This only happens if it's the last page, and the results of the last
page exactly matches the paging size.
Schema:
{noformat}
CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
USE simplex;
CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
INSERT INTO test (k, v) VALUES ('a', 0);
INSERT INTO test (k, v) VALUES ('b', 1);
INSERT INTO test (k, v) VALUES ('c', 2);
INSERT INTO test (k, v) VALUES ('d', 3);
INSERT INTO test (k, v) VALUES ('e', 4);
{noformat}
Query:
{noformat}
result = session.execute("SELECT * FROM test", page_size: 5)
loop do
puts "last page? #{result.last_page?}"
puts "page size: #{result.size}"
result.each do |row|
puts row
end
puts ""
break if result.last_page?
result = result.next_page
end
{noformat}
Result:
{noformat}
last page? false
page size: 5
{"k"=>"a", "v"=>0}
{"k"=>"c", "v"=>2}
{"k"=>"m", "v"=>12}
{"k"=>"f", "v"=>5}
{"k"=>"o", "v"=>14}
last page? true
page size: 0
{noformat}
was:
When retrieving the next page from the result of a simple statement, the result
will return a non-null paging state even if it's the last page of the query.
This only happens if it's the last page, and the results of the last page
exactly matches the paging size.
Schema:
{noformat}
CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
USE simplex;
CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
INSERT INTO test (k, v) VALUES ('a', 0);
INSERT INTO test (k, v) VALUES ('b', 1);
INSERT INTO test (k, v) VALUES ('c', 2);
INSERT INTO test (k, v) VALUES ('d', 3);
INSERT INTO test (k, v) VALUES ('e', 4);
{noformat}
Query:
{noformat}
result = session.execute("SELECT * FROM test", page_size: 5)
loop do
puts "last page? #{result.last_page?}"
puts "page size: #{result.size}"
result.each do |row|
puts row
end
puts ""
break if result.last_page?
result = result.next_page
end
{noformat}
Result:
{noformat}
last page? false
page size: 5
{"k"=>"a", "v"=>0}
{"k"=>"c", "v"=>2}
{"k"=>"m", "v"=>12}
{"k"=>"f", "v"=>5}
{"k"=>"o", "v"=>14}
last page? true
page size: 0
{noformat}
> Non-null paging state returned if last page
> ---
>
> Key: CASSANDRA-8871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8871
> Project: Cassandra
> Issue Type: Bug
> Environment: ruby-driver 2.1.0 | C* 2.0.12 | C* 2.1.3
>Reporter: Kishan Karunaratne
>Assignee: Tyler Hobbs
>
> tWhen retrieving the next page from the result of a simple statement, the
> result will return a non-null paging state even if it's the last page of the
> query. This only happens if it's the last page, and the results of the last
> page exactly matches the paging size.
> Schema:
> {noformat}
> CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> USE simplex;
> CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
> INSERT INTO test (k, v) VALUES ('a', 0);
> INSERT INTO test (k, v) VALUES ('b', 1);
> INSERT INTO test (k, v) VALUES ('c', 2);
> INSERT INTO test (k, v) VALUES ('d', 3);
> INSERT INTO test (k, v) VALUES ('e', 4);
> {noformat}
> Query:
> {noformat}
> result = session.execute("SELECT * FROM test", page_size: 5)
> loop do
> puts "last page? #{result.last_page?}"
> puts "page size: #{result.size}"
> result.each do |row|
> puts row
> end
> puts ""
> break if result.last_page?
> result = result.next_page
> end
> {noformat}
> Result:
> {noformat}
> last page? false
> page size: 5
> {"k"=>"a", "v"=>0}
> {"k"=>"c", "v"=>2}
> {"k"=>"m", "v"=>12}
> {"k"=>"f", "v"=>5}
> {"k"=>"o", "v"=>14}
>
> last page? true
> page size: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-8871) Non-null paging state returned if last page
[
https://issues.apache.org/jira/browse/CASSANDRA-8871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-8871:
Description:
When retrieving the next page from the result of a simple statement, the result
will return a non-null paging state even if it's the last page of the query.
This only happens if it's the last page, and the results of the last page
exactly matches the paging size.
Schema:
{noformat}
CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
USE simplex;
CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
INSERT INTO test (k, v) VALUES ('a', 0);
INSERT INTO test (k, v) VALUES ('b', 1);
INSERT INTO test (k, v) VALUES ('c', 2);
INSERT INTO test (k, v) VALUES ('d', 3);
INSERT INTO test (k, v) VALUES ('e', 4);
{noformat}
Query:
{noformat}
result = session.execute("SELECT * FROM test", page_size: 5)
loop do
puts "last page? #{result.last_page?}"
puts "page size: #{result.size}"
result.each do |row|
puts row
end
puts ""
break if result.last_page?
result = result.next_page
end
{noformat}
Result:
{noformat}
last page? false
page size: 5
{"k"=>"a", "v"=>0}
{"k"=>"c", "v"=>2}
{"k"=>"m", "v"=>12}
{"k"=>"f", "v"=>5}
{"k"=>"o", "v"=>14}
last page? true
page size: 0
{noformat}
was:
tWhen retrieving the next page from the result of a simple statement, the
result will return a non-null paging state even if it's the last page of the
query. This only happens if it's the last page, and the results of the last
page exactly matches the paging size.
Schema:
{noformat}
CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 3};
USE simplex;
CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
INSERT INTO test (k, v) VALUES ('a', 0);
INSERT INTO test (k, v) VALUES ('b', 1);
INSERT INTO test (k, v) VALUES ('c', 2);
INSERT INTO test (k, v) VALUES ('d', 3);
INSERT INTO test (k, v) VALUES ('e', 4);
{noformat}
Query:
{noformat}
result = session.execute("SELECT * FROM test", page_size: 5)
loop do
puts "last page? #{result.last_page?}"
puts "page size: #{result.size}"
result.each do |row|
puts row
end
puts ""
break if result.last_page?
result = result.next_page
end
{noformat}
Result:
{noformat}
last page? false
page size: 5
{"k"=>"a", "v"=>0}
{"k"=>"c", "v"=>2}
{"k"=>"m", "v"=>12}
{"k"=>"f", "v"=>5}
{"k"=>"o", "v"=>14}
last page? true
page size: 0
{noformat}
> Non-null paging state returned if last page
> ---
>
> Key: CASSANDRA-8871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8871
> Project: Cassandra
> Issue Type: Bug
> Environment: ruby-driver 2.1.0 | C* 2.0.12 | C* 2.1.3
>Reporter: Kishan Karunaratne
>Assignee: Tyler Hobbs
>
> When retrieving the next page from the result of a simple statement, the
> result will return a non-null paging state even if it's the last page of the
> query. This only happens if it's the last page, and the results of the last
> page exactly matches the paging size.
> Schema:
> {noformat}
> CREATE KEYSPACE simplex WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 3};
> USE simplex;
> CREATE TABLE test (k text, v int, PRIMARY KEY (k, v));
> INSERT INTO test (k, v) VALUES ('a', 0);
> INSERT INTO test (k, v) VALUES ('b', 1);
> INSERT INTO test (k, v) VALUES ('c', 2);
> INSERT INTO test (k, v) VALUES ('d', 3);
> INSERT INTO test (k, v) VALUES ('e', 4);
> {noformat}
> Query:
> {noformat}
> result = session.execute("SELECT * FROM test", page_size: 5)
> loop do
> puts "last page? #{result.last_page?}"
> puts "page size: #{result.size}"
> result.each do |row|
> puts row
> end
> puts ""
> break if result.last_page?
> result = result.next_page
> end
> {noformat}
> Result:
> {noformat}
> last page? false
> page size: 5
> {"k"=>"a", "v"=>0}
> {"k"=>"c", "v"=>2}
> {"k"=>"m", "v"=>12}
> {"k"=>"f", "v"=>5}
> {"k"=>"o", "v"=>14}
>
> last page? true
> page size: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-11181) Add broadcast_rpc_address to system.local
[
https://issues.apache.org/jira/browse/CASSANDRA-11181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115003#comment-16115003
]
Andy Tolbert commented on CASSANDRA-11181:
--
{quote}
I think we should simply be storing the broadcast_rpc_address in the
rpc_address column, since that's what we do for system.peers.
{quote}
+100, if {{rpc_address}} is set to 0.0.0.0, and {{broadcast_rpc_address}} is
set to something, you can't really use the {{rpc_address}} column in
{{system.local}}. I would consider that a bug.
> Add broadcast_rpc_address to system.local
> -
>
> Key: CASSANDRA-11181
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11181
> Project: Cassandra
> Issue Type: Improvement
> Components: Distributed Metadata
>Reporter: Nick Bailey
> Labels: client-impacting
> Fix For: 4.x
>
>
> Right now it's impossible to get the broadcast_rpc_address of the node you
> are connected to via the drivers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-13302) last row of previous page == first row of next page while querying data using SASI index
Andy Tolbert created CASSANDRA-13302:
Summary: last row of previous page == first row of next page while
querying data using SASI index
Key: CASSANDRA-13302
URL: https://issues.apache.org/jira/browse/CASSANDRA-13302
Project: Cassandra
Issue Type: Bug
Environment: Tested with C* 3.9 and 3.10.
Reporter: Andy Tolbert
Apologies if this is a duplicate (couldn't track down an existing bug).
Similarly to [CASSANDRA-11208], it appears it is possible to retrieve duplicate
rows when paging using a SASI index as documented in
[JAVA-1362|https://datastax-oss.atlassian.net/browse/JAVA-1362], the following
test demonstrates that data is repeated while querying using a SASI index:
{code:java}
public class TestPagingBug
{
public static void main(String[] args)
{
Cluster.Builder builder = Cluster.builder();
Cluster c = builder.addContactPoints("192.168.98.190").build();
Session s = c.connect();
s.execute("CREATE KEYSPACE IF NOT EXISTS test WITH replication
= { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }");
s.execute("CREATE TABLE IF NOT EXISTS test.test_table_sec(sec
BIGINT PRIMARY KEY, id INT)");
//create secondary index on ID column, used for select statement
String index = "CREATE CUSTOM INDEX test_table_sec_idx ON
test.test_table_sec (id) USING 'org.apache.cassandra.index.sasi.SASIIndex' "
+ "WITH OPTIONS = { 'mode': 'PREFIX' }";
s.execute(index);
PreparedStatement insert = s.prepare("INSERT INTO
test.test_table_sec (id, sec) VALUES (1, ?)");
for (int i = 0; i < 1000; i++)
s.execute(insert.bind((long) i));
PreparedStatement select = s.prepare("SELECT sec FROM
test.test_table_sec WHERE id = 1");
long lastSec = -1;
for (Row row : s.execute(select.bind().setFetchSize(300)))
{
long sec = row.getLong("sec");
if (sec == lastSec)
System.out.println(String.format("Duplicated id
%d", sec));
lastSec = sec;
}
System.exit(0);
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13302) last row of previous page == first row of next page while querying data using SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-13302:
-
Description:
Apologies if this is a duplicate (couldn't track down an existing bug).
Similarly to [CASSANDRA-11208], it appears it is possible to retrieve duplicate
rows when paging using a SASI index as documented in
[JAVA-1413|https://datastax-oss.atlassian.net/browse/JAVA-1413], the following
test demonstrates that data is repeated while querying using a SASI index:
{code:java}
public class TestPagingBug
{
public static void main(String[] args)
{
Cluster.Builder builder = Cluster.builder();
Cluster c = builder.addContactPoints("192.168.98.190").build();
Session s = c.connect();
s.execute("CREATE KEYSPACE IF NOT EXISTS test WITH replication
= { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }");
s.execute("CREATE TABLE IF NOT EXISTS test.test_table_sec(sec
BIGINT PRIMARY KEY, id INT)");
//create secondary index on ID column, used for select statement
String index = "CREATE CUSTOM INDEX test_table_sec_idx ON
test.test_table_sec (id) USING 'org.apache.cassandra.index.sasi.SASIIndex' "
+ "WITH OPTIONS = { 'mode': 'PREFIX' }";
s.execute(index);
PreparedStatement insert = s.prepare("INSERT INTO
test.test_table_sec (id, sec) VALUES (1, ?)");
for (int i = 0; i < 1000; i++)
s.execute(insert.bind((long) i));
PreparedStatement select = s.prepare("SELECT sec FROM
test.test_table_sec WHERE id = 1");
long lastSec = -1;
for (Row row : s.execute(select.bind().setFetchSize(300)))
{
long sec = row.getLong("sec");
if (sec == lastSec)
System.out.println(String.format("Duplicated id
%d", sec));
lastSec = sec;
}
System.exit(0);
}
}
{code}
was:
Apologies if this is a duplicate (couldn't track down an existing bug).
Similarly to [CASSANDRA-11208], it appears it is possible to retrieve duplicate
rows when paging using a SASI index as documented in
[JAVA-1362|https://datastax-oss.atlassian.net/browse/JAVA-1362], the following
test demonstrates that data is repeated while querying using a SASI index:
{code:java}
public class TestPagingBug
{
public static void main(String[] args)
{
Cluster.Builder builder = Cluster.builder();
Cluster c = builder.addContactPoints("192.168.98.190").build();
Session s = c.connect();
s.execute("CREATE KEYSPACE IF NOT EXISTS test WITH replication
= { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }");
s.execute("CREATE TABLE IF NOT EXISTS test.test_table_sec(sec
BIGINT PRIMARY KEY, id INT)");
//create secondary index on ID column, used for select statement
String index = "CREATE CUSTOM INDEX test_table_sec_idx ON
test.test_table_sec (id) USING 'org.apache.cassandra.index.sasi.SASIIndex' "
+ "WITH OPTIONS = { 'mode': 'PREFIX' }";
s.execute(index);
PreparedStatement insert = s.prepare("INSERT INTO
test.test_table_sec (id, sec) VALUES (1, ?)");
for (int i = 0; i < 1000; i++)
s.execute(insert.bind((long) i));
PreparedStatement select = s.prepare("SELECT sec FROM
test.test_table_sec WHERE id = 1");
long lastSec = -1;
for (Row row : s.execute(select.bind().setFetchSize(300)))
{
long sec = row.getLong("sec");
if (sec == lastSec)
System.out.println(String.format("Duplicated id
%d", sec));
lastSec = sec;
}
System.exit(0);
}
}
{code}
> last row of previous page == first row of next page while querying data using
> SASI index
>
>
> Key: CASSANDRA-13302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13302
> Project: Cassandra
> Issue Type: Bug
> Environment: Tested with C* 3.9 and 3.10.
>Reporter: Andy Tolbert
>
> Apologies if this is a duplicate (couldn't track down an existing bug).
> Similarly to [CASSA
[jira] [Updated] (CASSANDRA-13302) last row of previous page == first row of next page while querying data using SASI index
[
https://issues.apache.org/jira/browse/CASSANDRA-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-13302:
-
Description:
Apologies if this is a duplicate (couldn't track down an existing bug).
Similarly to [CASSANDRA-11208], it appears it is possible to retrieve duplicate
rows when paging using a SASI index as documented in
[JAVA-1413|https://datastax-oss.atlassian.net/browse/JAVA-1413], the following
test demonstrates that data is repeated while querying using a SASI index:
{code:java}
public class TestPagingBug
{
public static void main(String[] args)
{
Cluster.Builder builder = Cluster.builder();
Cluster c = builder.addContactPoints("192.168.98.190").build();
Session s = c.connect();
s.execute("CREATE KEYSPACE IF NOT EXISTS test WITH replication
= { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }");
s.execute("CREATE TABLE IF NOT EXISTS test.test_table_sec(sec
BIGINT PRIMARY KEY, id INT)");
//create secondary index on ID column, used for select statement
String index = "CREATE CUSTOM INDEX test_table_sec_idx ON
test.test_table_sec (id) USING 'org.apache.cassandra.index.sasi.SASIIndex' "
+ "WITH OPTIONS = { 'mode': 'PREFIX' }";
s.execute(index);
PreparedStatement insert = s.prepare("INSERT INTO
test.test_table_sec (id, sec) VALUES (1, ?)");
for (int i = 0; i < 1000; i++)
s.execute(insert.bind((long) i));
PreparedStatement select = s.prepare("SELECT sec FROM
test.test_table_sec WHERE id = 1");
long lastSec = -1;
for (Row row : s.execute(select.bind().setFetchSize(300)))
{
long sec = row.getLong("sec");
if (sec == lastSec)
System.out.println(String.format("Duplicated id
%d", sec));
lastSec = sec;
}
System.exit(0);
}
}
{code}
The program outputs the following:
{noformat}
Duplicated id 23
Duplicated id 192
Duplicated id 684
{noformat}
Note that the simple primary key is required to reproduce this.
was:
Apologies if this is a duplicate (couldn't track down an existing bug).
Similarly to [CASSANDRA-11208], it appears it is possible to retrieve duplicate
rows when paging using a SASI index as documented in
[JAVA-1413|https://datastax-oss.atlassian.net/browse/JAVA-1413], the following
test demonstrates that data is repeated while querying using a SASI index:
{code:java}
public class TestPagingBug
{
public static void main(String[] args)
{
Cluster.Builder builder = Cluster.builder();
Cluster c = builder.addContactPoints("192.168.98.190").build();
Session s = c.connect();
s.execute("CREATE KEYSPACE IF NOT EXISTS test WITH replication
= { 'class' : 'SimpleStrategy', 'replication_factor' : 3 }");
s.execute("CREATE TABLE IF NOT EXISTS test.test_table_sec(sec
BIGINT PRIMARY KEY, id INT)");
//create secondary index on ID column, used for select statement
String index = "CREATE CUSTOM INDEX test_table_sec_idx ON
test.test_table_sec (id) USING 'org.apache.cassandra.index.sasi.SASIIndex' "
+ "WITH OPTIONS = { 'mode': 'PREFIX' }";
s.execute(index);
PreparedStatement insert = s.prepare("INSERT INTO
test.test_table_sec (id, sec) VALUES (1, ?)");
for (int i = 0; i < 1000; i++)
s.execute(insert.bind((long) i));
PreparedStatement select = s.prepare("SELECT sec FROM
test.test_table_sec WHERE id = 1");
long lastSec = -1;
for (Row row : s.execute(select.bind().setFetchSize(300)))
{
long sec = row.getLong("sec");
if (sec == lastSec)
System.out.println(String.format("Duplicated id
%d", sec));
lastSec = sec;
}
System.exit(0);
}
}
{code}
> last row of previous page == first row of next page while querying data using
> SASI index
>
>
> Key: CASSANDRA-13302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13302
> Project: Cassandra
> Issue Type: Bug
>
[jira] [Created] (CASSANDRA-13697) CDC and VIEW writeType missing from spec for write_timeout / write_failure
Andy Tolbert created CASSANDRA-13697:
Summary: CDC and VIEW writeType missing from spec for
write_timeout / write_failure
Key: CASSANDRA-13697
URL: https://issues.apache.org/jira/browse/CASSANDRA-13697
Project: Cassandra
Issue Type: Improvement
Reporter: Andy Tolbert
Priority: Minor
In cassandra 3.0 a new {{WriteType}} {{VIEW}} was added which appears to be
used when raising a {{WriteTimeoutException}} when the local view lock for a
key cannot be acquired within timeout.
In cassandra 3.8 {{CDC}} {{WriteType}} was added for when
{{cdc_total_space_in_mb}} is exceeded when doing a write to data tracked by cdc.
The [v4
spec|https://github.com/apache/cassandra/blob/cassandra-3.11.0/doc/native_protocol_v4.spec#L1051-L1066]
currently doesn't cover these two write types. While the protocol allows for
a free form string for write type, it would be nice to document that types are
available since some drivers (java, cpp, python) attempt to deserialize write
type into an enum and may not handle it well.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-11936) Add support for + and - operations on dates
[
https://issues.apache.org/jira/browse/CASSANDRA-11936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822484#comment-15822484
]
Andy Tolbert commented on CASSANDRA-11936:
--
A couple questions about this from experimenting with the branch:
1. Should it also work for timeuuid?
2. Does {{WHERE reading_time < now() - 2h}} work yet or will that be in another
ticket? The now() function is documented as generating a timeuuid, although
maybe it's contextual by type? When I try it I get an error {{the '-'
operation is not supported between now() and 2h}}.
> Add support for + and - operations on dates
> ---
>
> Key: CASSANDRA-11936
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11936
> Project: Cassandra
> Issue Type: Sub-task
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Fix For: 3.x
>
>
> For time series it can be interesting to allow queries with {{WHERE}} clause
> like: {{... WHERE reading_time < now() - 2h}}
> In order to do that we need to add support for: {{+}} and {{-}} operation
> with date.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-13952) C* still sending WriteFailure instead of CDCWriteFailure
[
https://issues.apache.org/jira/browse/CASSANDRA-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16206283#comment-16206283
]
Andy Tolbert commented on CASSANDRA-13952:
--
To clarify the intent of this error, should it be its own protocol error
({{CDC_WRITE_FAILURE}}) or should it be a new reason code on {{WRITE_FAILURE}}.
I think if the error only happens on the coordinator node a separate error
may make more sense, but if it can happen on any replica {{WRITE_FAILURE}}
would make more sense as with {{WRITE_FAILURE}} you get a mapping of failed
replicas with their failure codes.
> C* still sending WriteFailure instead of CDCWriteFailure
>
>
> Key: CASSANDRA-13952
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13952
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jaume M
>Assignee: Jaume M
> Fix For: 4.0
>
>
> I've tested this setting setting cdc_total_space_in_mb: 1 and with the
> current python driver master
> {code}
> from cassandra.cluster import Cluster
> cluster = Cluster(protocol_version=5, allow_beta_protocol_version=True)
> session = cluster.connect()
> session.execute("""
> CREATE KEYSPACE IF NOT EXISTS %s
> WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '2'
> }
> """ % "query_per_range")
> session.execute("""
> CREATE TABLE IF NOT EXISTS query_per_range.mytable_int (
> key_one int,
> key_two int,
> col1 text,
> col2 text,
> PRIMARY KEY ((key_one, key_two), col1)
> ) WITH cdc=true;
> """)
> for i in range(1000):
> session.execute("INSERT INTO query_per_range.mytable_int(key_one,
> key_two, col1, col2) VALUES (%s, %s, %s, %s)",
> (i, i, str(i), str(i)))
> {code}
> I'm receiving a {{WriteFailure}} every time but I was expecting a
> {{CDCWriteFailure}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13952) C* still sending WriteFailure instead of CDCWriteFailure
[
https://issues.apache.org/jira/browse/CASSANDRA-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16206283#comment-16206283
]
Andy Tolbert edited comment on CASSANDRA-13952 at 10/16/17 5:48 PM:
To clarify the intent of this error, should it be its own protocol error
({{CDC_WRITE_FAILURE}}) or should it be a new reason code on {{WRITE_FAILURE}}?
I think if the error only happens on the coordinator node a separate error
may make more sense, but if it can happen on any replica {{WRITE_FAILURE}}
would make more sense as with {{WRITE_FAILURE}} you get a mapping of failed
replicas with their failure codes.
was (Author: andrew.tolbert):
To clarify the intent of this error, should it be its own protocol error
({{CDC_WRITE_FAILURE}}) or should it be a new reason code on {{WRITE_FAILURE}}.
I think if the error only happens on the coordinator node a separate error
may make more sense, but if it can happen on any replica {{WRITE_FAILURE}}
would make more sense as with {{WRITE_FAILURE}} you get a mapping of failed
replicas with their failure codes.
> C* still sending WriteFailure instead of CDCWriteFailure
>
>
> Key: CASSANDRA-13952
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13952
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jaume M
>Assignee: Jaume M
> Fix For: 4.0
>
>
> I've tested this setting setting cdc_total_space_in_mb: 1 and with the
> current python driver master
> {code}
> from cassandra.cluster import Cluster
> cluster = Cluster(protocol_version=5, allow_beta_protocol_version=True)
> session = cluster.connect()
> session.execute("""
> CREATE KEYSPACE IF NOT EXISTS %s
> WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '2'
> }
> """ % "query_per_range")
> session.execute("""
> CREATE TABLE IF NOT EXISTS query_per_range.mytable_int (
> key_one int,
> key_two int,
> col1 text,
> col2 text,
> PRIMARY KEY ((key_one, key_two), col1)
> ) WITH cdc=true;
> """)
> for i in range(1000):
> session.execute("INSERT INTO query_per_range.mytable_int(key_one,
> key_two, col1, col2) VALUES (%s, %s, %s, %s)",
> (i, i, str(i), str(i)))
> {code}
> I'm receiving a {{WriteFailure}} every time but I was expecting a
> {{CDCWriteFailure}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13952) C* still sending WriteFailure instead of CDCWriteFailure
[
https://issues.apache.org/jira/browse/CASSANDRA-13952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16215412#comment-16215412
]
Andy Tolbert commented on CASSANDRA-13952:
--
Just to clarify your last comment, are you ok with this returning
{{WriteFailureException}} with a reason code (the way it currently works), or
do you want there to be a new error type?
I prefer the former since it seems to be a good fit and doesn't require any
client changes.
> C* still sending WriteFailure instead of CDCWriteFailure
>
>
> Key: CASSANDRA-13952
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13952
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jaume M
>Assignee: Jaume M
> Fix For: 4.0
>
>
> I've tested this setting setting cdc_total_space_in_mb: 1 and with the
> current python driver master
> {code}
> from cassandra.cluster import Cluster
> cluster = Cluster(protocol_version=5, allow_beta_protocol_version=True)
> session = cluster.connect()
> session.execute("""
> CREATE KEYSPACE IF NOT EXISTS %s
> WITH replication = { 'class': 'SimpleStrategy', 'replication_factor': '2'
> }
> """ % "query_per_range")
> session.execute("""
> CREATE TABLE IF NOT EXISTS query_per_range.mytable_int (
> key_one int,
> key_two int,
> col1 text,
> col2 text,
> PRIMARY KEY ((key_one, key_two), col1)
> ) WITH cdc=true;
> """)
> for i in range(1000):
> session.execute("INSERT INTO query_per_range.mytable_int(key_one,
> key_two, col1, col2) VALUES (%s, %s, %s, %s)",
> (i, i, str(i), str(i)))
> {code}
> I'm receiving a {{WriteFailure}} every time but I was expecting a
> {{CDCWriteFailure}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10751) "Pool is shutdown" error when running Hadoop jobs on Yarn
[
https://issues.apache.org/jira/browse/CASSANDRA-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511528#comment-16511528
]
Andy Tolbert commented on CASSANDRA-10751:
--
(y), to further expound on [~jjordan]'s comment, it's misleading but the
[original
code|https://github.com/datastax/java-driver/blob/2.1.8/driver-core/src/main/java/com/datastax/driver/core/Connection.java#L477]
referenced is quoting the value returned from a {{set_keyspace}} response sent
by C*. In that case, C* sends the internal form of the keyspace, which is
never quoted, so we have to quote it there.
> "Pool is shutdown" error when running Hadoop jobs on Yarn
> -
>
> Key: CASSANDRA-10751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10751
> Project: Cassandra
> Issue Type: Bug
> Environment: Hadoop 2.7.1 (HDP 2.3.2)
> Cassandra 2.1.11
>Reporter: Cyril Scetbon
>Assignee: Cyril Scetbon
>Priority: Major
> Fix For: 4.0, 2.2.13, 3.0.17, 3.11.3
>
> Attachments: CASSANDRA-10751-2.2.patch, CASSANDRA-10751-3.0.patch,
> output.log
>
>
> Trying to execute an Hadoop job on Yarn, I get errors from Cassandra's
> internal code. It seems that connections are shutdown but we can't understand
> why ...
> Here is a subtract of the errors. I also add a file with the complete debug
> logs.
> {code}
> 15/11/22 20:05:54 [main]: DEBUG core.RequestHandler: Error querying
> node006.internal.net/192.168.12.22:9042, trying next host (error is:
> com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown)
> Failed with exception java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> 15/11/22 20:05:54 [main]: ERROR CliDriver: Failed with exception
> java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> java.io.IOException: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1672)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> org.apache.hadoop.hive.cassandra.input.cql.HiveCqlInputFormat.getRecordReader(HiveCqlInputFormat.java:132)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:674)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:324)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:446)
> ... 15 more
> Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All
> host(s) tried for query failed (tried:
> node006.internal.net/192.168.12
[jira] [Comment Edited] (CASSANDRA-10751) "Pool is shutdown" error when running Hadoop jobs on Yarn
[
https://issues.apache.org/jira/browse/CASSANDRA-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511528#comment-16511528
]
Andy Tolbert edited comment on CASSANDRA-10751 at 6/13/18 6:37 PM:
---
(y) the driver does not quote the keyspace value given to {{Cluster.connect
}}so this change will cause problems for mixed-case keyspaces. To further
expound on [~jjordan]'s comment, it's misleading but the [original
code|https://github.com/datastax/java-driver/blob/2.1.8/driver-core/src/main/java/com/datastax/driver/core/Connection.java#L477]
referenced is quoting the value returned from a {{set_keyspace}} response sent
by C*. In that case, C* sends the internal form of the keyspace, which is
never quoted, so we have to quote it there.
was (Author: andrew.tolbert):
(y), to further expound on [~jjordan]'s comment, it's misleading but the
[original
code|https://github.com/datastax/java-driver/blob/2.1.8/driver-core/src/main/java/com/datastax/driver/core/Connection.java#L477]
referenced is quoting the value returned from a {{set_keyspace}} response sent
by C*. In that case, C* sends the internal form of the keyspace, which is
never quoted, so we have to quote it there.
> "Pool is shutdown" error when running Hadoop jobs on Yarn
> -
>
> Key: CASSANDRA-10751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10751
> Project: Cassandra
> Issue Type: Bug
> Environment: Hadoop 2.7.1 (HDP 2.3.2)
> Cassandra 2.1.11
>Reporter: Cyril Scetbon
>Assignee: Cyril Scetbon
>Priority: Major
> Fix For: 4.0, 2.2.13, 3.0.17, 3.11.3
>
> Attachments: CASSANDRA-10751-2.2.patch, CASSANDRA-10751-3.0.patch,
> output.log
>
>
> Trying to execute an Hadoop job on Yarn, I get errors from Cassandra's
> internal code. It seems that connections are shutdown but we can't understand
> why ...
> Here is a subtract of the errors. I also add a file with the complete debug
> logs.
> {code}
> 15/11/22 20:05:54 [main]: DEBUG core.RequestHandler: Error querying
> node006.internal.net/192.168.12.22:9042, trying next host (error is:
> com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown)
> Failed with exception java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> 15/11/22 20:05:54 [main]: ERROR CliDriver: Failed with exception
> java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> java.io.IOException: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1672)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:904
[jira] [Comment Edited] (CASSANDRA-10751) "Pool is shutdown" error when running Hadoop jobs on Yarn
[
https://issues.apache.org/jira/browse/CASSANDRA-10751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16511528#comment-16511528
]
Andy Tolbert edited comment on CASSANDRA-10751 at 6/13/18 6:51 PM:
---
(y) the driver does not quote the keyspace value given to {{Cluster.connect}}
so this change will cause problems for mixed-case keyspaces. To further
expound on [~jjordan]'s comment, it's misleading but the [original
code|https://github.com/datastax/java-driver/blob/2.1.8/driver-core/src/main/java/com/datastax/driver/core/Connection.java#L477]
referenced is quoting the value returned from a {{set_keyspace}} response sent
by C*. In that case, C* sends the internal form of the keyspace, which is
never quoted, so we have to quote it there.
was (Author: andrew.tolbert):
(y) the driver does not quote the keyspace value given to {{Cluster.connect
}}so this change will cause problems for mixed-case keyspaces. To further
expound on [~jjordan]'s comment, it's misleading but the [original
code|https://github.com/datastax/java-driver/blob/2.1.8/driver-core/src/main/java/com/datastax/driver/core/Connection.java#L477]
referenced is quoting the value returned from a {{set_keyspace}} response sent
by C*. In that case, C* sends the internal form of the keyspace, which is
never quoted, so we have to quote it there.
> "Pool is shutdown" error when running Hadoop jobs on Yarn
> -
>
> Key: CASSANDRA-10751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10751
> Project: Cassandra
> Issue Type: Bug
> Environment: Hadoop 2.7.1 (HDP 2.3.2)
> Cassandra 2.1.11
>Reporter: Cyril Scetbon
>Assignee: Cyril Scetbon
>Priority: Major
> Fix For: 4.0, 2.2.13, 3.0.17, 3.11.3
>
> Attachments: CASSANDRA-10751-2.2.patch, CASSANDRA-10751-3.0.patch,
> output.log
>
>
> Trying to execute an Hadoop job on Yarn, I get errors from Cassandra's
> internal code. It seems that connections are shutdown but we can't understand
> why ...
> Here is a subtract of the errors. I also add a file with the complete debug
> logs.
> {code}
> 15/11/22 20:05:54 [main]: DEBUG core.RequestHandler: Error querying
> node006.internal.net/192.168.12.22:9042, trying next host (error is:
> com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown)
> Failed with exception java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> 15/11/22 20:05:54 [main]: ERROR CliDriver: Failed with exception
> java.io.IOException:java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> java.io.IOException: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tried: node006.internal.net/192.168.12.22:9042
> (com.datastax.driver.core.ConnectionException:
> [node006.internal.net/192.168.12.22:9042] Pool is shutdown))
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:508)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:415)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1672)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.io.IOException:
> com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s)
> tried for query failed (tri
[jira] [Commented] (CASSANDRA-15472) Read failure due to exception from metrics-core dependency
[
https://issues.apache.org/jira/browse/CASSANDRA-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089193#comment-17089193
]
Andy Tolbert commented on CASSANDRA-15472:
--
Sorry for the delay in response [~sumanth.pasupuleti].
I think its probably reasonable to add {{metrics-jmx}} as a dependency while
upgrading to metrics 4, since metrics 3 included the JMX bits anyways so this
isn't really adding any additional baggage. The driver does work with metrics
4 (just like it works with netty 4.1 but depends on 4.0), we just didn't want
to bump the dependency as if I recall right we wanted to avoid making major
version upgrades of dependencies in minor releases.
Alternatively we could use {{withoutJMXReporting}} in all the cluster builders,
but not sure if anyone is dependent on the driver emitting metrics via JMX, so
i'd rather err on the side of adding {{metrics-jmx}}.
> Read failure due to exception from metrics-core dependency
> --
>
> Key: CASSANDRA-15472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15472
> Project: Cassandra
> Issue Type: Bug
> Components: Dependencies
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> Stacktrace
> {code:java}
> Uncaught exception on thread Thread[SharedPool-Worker-27,5,main]: {}
> java.util.NoSuchElementException: null
> at
> java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2053)
> ~[na:1.8.0_222]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:102)
> ~[metrics-core-2.2.0.jar:na]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:81)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Histogram.update(Histogram.java:110)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:198)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:76)
> ~[metrics-core-2.2.0.jar:na]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:108)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:114)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1897)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_222]
> at
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]
> {code}
> This [issue|https://github.com/dropwizard/metrics/issues/1278] has been
> [fixed|https://github.com/dropwizard/metrics/pull/1436] in
> [v4.0.6|https://github.com/dropwizard/metrics/releases/tag/v4.0.6].
> This is observed on a 2.1.19 cluster, but this would impact pretty much any
> version of C* since we depend on lower versions of metrics-core that do not
> have the fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15472) Read failure due to exception from metrics-core dependency
[
https://issues.apache.org/jira/browse/CASSANDRA-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090283#comment-17090283
]
Andy Tolbert commented on CASSANDRA-15472:
--
You are right, I should have been more clear there. Adding the dependency
alone is not enough, wherever a {{Cluster.builder}} is registered, we would
also need to do the following, per the java driver docs you linked:
{code}
Cluster cluster = Cluster.builder()
.withoutJMXReporting()
.build();
JmxReporter reporter =
JmxReporter.forRegistry(cluster.getMetrics().getRegistry())
.inDomain(cluster.getClusterName() + "-metrics")
.build();
reporter.start();
{code}
I think this is a reasonable compromise, it'll take some extra work but the end
result is we use metrics 4, and the behavior of our tooling that use the driver
should be the same.
> Read failure due to exception from metrics-core dependency
> --
>
> Key: CASSANDRA-15472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15472
> Project: Cassandra
> Issue Type: Bug
> Components: Dependencies
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> Stacktrace
> {code:java}
> Uncaught exception on thread Thread[SharedPool-Worker-27,5,main]: {}
> java.util.NoSuchElementException: null
> at
> java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2053)
> ~[na:1.8.0_222]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:102)
> ~[metrics-core-2.2.0.jar:na]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:81)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Histogram.update(Histogram.java:110)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:198)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:76)
> ~[metrics-core-2.2.0.jar:na]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:108)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:114)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1897)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_222]
> at
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]
> {code}
> This [issue|https://github.com/dropwizard/metrics/issues/1278] has been
> [fixed|https://github.com/dropwizard/metrics/pull/1436] in
> [v4.0.6|https://github.com/dropwizard/metrics/releases/tag/v4.0.6].
> This is observed on a 2.1.19 cluster, but this would impact pretty much any
> version of C* since we depend on lower versions of metrics-core that do not
> have the fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15472) Read failure due to exception from metrics-core dependency
[
https://issues.apache.org/jira/browse/CASSANDRA-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17090283#comment-17090283
]
Andy Tolbert edited comment on CASSANDRA-15472 at 4/23/20, 5:52 AM:
You are right, I should have been more clear there. Adding the dependency
alone is not enough, wherever a {{Cluster.builder}} is registered, we would
also need to do the following, per the java driver docs you linked:
{code}
Cluster cluster = Cluster.builder()
.withoutJMXReporting()
.build();
JmxReporter reporter =
JmxReporter.forRegistry(cluster.getMetrics().getRegistry())
.inDomain(cluster.getClusterName() + "-metrics")
.build();
reporter.start();
{code}
I think this is a reasonable compromise, it'll take some extra work but the end
result is we use metrics 4, and the behavior of our tooling that use the driver
should be the same as it pertains to emitting JMX metrics.
was (Author: andrew.tolbert):
You are right, I should have been more clear there. Adding the dependency
alone is not enough, wherever a {{Cluster.builder}} is registered, we would
also need to do the following, per the java driver docs you linked:
{code}
Cluster cluster = Cluster.builder()
.withoutJMXReporting()
.build();
JmxReporter reporter =
JmxReporter.forRegistry(cluster.getMetrics().getRegistry())
.inDomain(cluster.getClusterName() + "-metrics")
.build();
reporter.start();
{code}
I think this is a reasonable compromise, it'll take some extra work but the end
result is we use metrics 4, and the behavior of our tooling that use the driver
should be the same.
> Read failure due to exception from metrics-core dependency
> --
>
> Key: CASSANDRA-15472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15472
> Project: Cassandra
> Issue Type: Bug
> Components: Dependencies
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> Stacktrace
> {code:java}
> Uncaught exception on thread Thread[SharedPool-Worker-27,5,main]: {}
> java.util.NoSuchElementException: null
> at
> java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2053)
> ~[na:1.8.0_222]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:102)
> ~[metrics-core-2.2.0.jar:na]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:81)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Histogram.update(Histogram.java:110)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:198)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:76)
> ~[metrics-core-2.2.0.jar:na]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:108)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:114)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1897)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_222]
> at
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]
> {code}
> This [issue|https://github.com/dropwizard/metrics/issues/1278] has been
> [fixed|https://github.com/dropwizard/metrics/pull/1436] in
> [v4.0.6|https://github.com/dropwizard/metrics/releases/tag/v4.0.6].
> This is observed on a 2.1.19 cluster, but this would impact pretty much any
> version of C* since we depend on lower versions of metrics-core that do not
> have the fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
--
[jira] [Created] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static col
Andy Tolbert created CASSANDRA-14638:
Summary: Column result order can change in 'SELECT *' results when
upgrading from 2.1 to 3.0 causing response corruption for queries using
prepared statements when static columns are used
Key: CASSANDRA-14638
URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
Project: Cassandra
Issue Type: Bug
Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
3.0.17
Reporter: Andy Tolbert
When performing an upgrade from C* 2.1.20 to 3.1.17 I observed that the order
of columns returned from a 'SELECT *' query changes, particularly when static
columns are involved.
This may not seem like that much of a problem, however if using Prepared
Statements, any clients that remain connected during the upgrade may encounter
issues consuming results from these queries, as data is reordered and the
client not aware of it. The result definition is sent in the original prepared
statement response, so if order changes the client has no way of knowing (until
C* 4.0 via CASSANDRA-10786) without re-preparing, which is non-trivial as most
client drivers cache prepared statements.
This could lead to reading the wrong values for columns, which could result in
some kind of deserialization exception or if the data types of the switched
columns are compatible, the wrong values. This happens even if the client
attempts to retrieve a column value by name (i.e. row.getInt("colx")).
Unfortunately I don't think there is an easy fix for this. If the order was
changed back to the previous format, you risk issues for users upgrading from
older 3.0 version. I think it would be nice to add a note in the NEWS file in
the 3.0 upgrade section that describes this issue, and how to work around it
(specify all column names of interest explicitly in query).
Example schema and code to reproduce:
{noformat}
create keyspace ks with replication = {'class': 'SimpleStrategy',
'replication_factor': 1};
create table ks.tbl (p0 text,
p1 text,
m map static,
t text,
u text static,
primary key (p0, p1)
);
insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' }, 't',
'u');{noformat}
When querying with 2.1 you'll observe the following order via cqlsh:
{noformat}
p0 | p1 | m| u | t
++--+---+---
p0 | p1 | {'m0': 'm1'} | u | t{noformat}
With 3.0, observe that u and m are transposed:
{noformat}
p0 | p1 | u | m | t
++---+--+---
p0 | p1 | u | {'m0': 'm1'} | t{noformat}
{code:java}
import com.datastax.driver.core.BoundStatement;
import com.datastax.driver.core.Cluster;
import com.datastax.driver.core.ColumnDefinitions;
import com.datastax.driver.core.PreparedStatement;
import com.datastax.driver.core.ResultSet;
import com.datastax.driver.core.Row;
import com.datastax.driver.core.Session;
import com.google.common.util.concurrent.Uninterruptibles;
import java.util.concurrent.TimeUnit;
public class LiveUpgradeTest {
public static void main(String args[]) {
Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
try {
Session session = cluster.connect();
PreparedStatement p = session.prepare("SELECT * from ks.tbl");
BoundStatement bs = p.bind();
// continually query every 30 seconds
while (true) {
try {
ResultSet r = session.execute(bs);
Row row = r.one();
int i = 0;
// iterate over the result metadata in order printing the
// index, name, type, and length of the first row of data.
for (ColumnDefinitions.Definition d : r.getColumnDefinitions()) {
System.out.println(
i++
+ ": "
+ d.getName()
+ " -> "
+ d.getType()
+ " -> val = "
+ row.getBytesUnsafe(d.getName()).array().length);
}
} catch (Throwable t) {
t.printStackTrace();
} finally {
Uninterruptibles.sleepUninterruptibly(30, TimeUnit.SECONDS);
}
}
} finally {
cluster.close();
}
}
}
{code}
To reproduce, set up a cluster, the schema, and run this script. Then upgrade
the cluster to 3.0.17 (with ccm, ccm stop; ccm node1 setdir -v 3.0.17; ccm
start works) and observe after the client is able to reconnect that the results
are in a different order. i.e.:
With 2.x:
{noformat}
0: p0 -> varchar -> val = 2
1: p1 -> varchar -> val = 2
2: m -> map -> val = 16
3: u -> varchar -> val = 1
4: t -> varchar -> val = 1{noformat}
With 3.x:
{noformat}
0: p0 -> varchar -> val = 2
1: p1 -> varchar -> val = 2
2: m -> map -> val = 1
3: u -> varchar -> val = 16 (<-- the data for 'm' is now at index 3)
4: t -> varchar -> val = 1{noformat}
--
This messa
[jira] [Commented] (CASSANDRA-14638) Column result order can change in 'SELECT *' results when upgrading from 2.1 to 3.0 causing response corruption for queries using prepared statements when static c
[
https://issues.apache.org/jira/browse/CASSANDRA-14638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16581355#comment-16581355
]
Andy Tolbert commented on CASSANDRA-14638:
--
{quote}
What 3.0 does incorrectly is ordering the static columns it returns. Instead of
merging simple and complex static columns, we are getting all simple static
columns first, followed by all complex static columns.
{quote}
Ah, that explains it! Thanks for looking into this. I was trying to
understand why it was ordering the static columns the way it was, that makes
sense.
{quote}
This is true. That said, there are huge fleets still on 2.1 and 2.2 in the
wild, so leaving this unfixed is also a choice that puts some users in
potential danger.
{quote}
That's a great point. We need to balance the risk between users upgrading from
2.x to those upgrading from previous 3.x versions. I may not have the best
sense of which should win out, but I think I agree that if we can make it
consistent and work the way it did pre-3.0, that is probably for the best long
term.
> Column result order can change in 'SELECT *' results when upgrading from 2.1
> to 3.0 causing response corruption for queries using prepared statements when
> static columns are used
> --
>
> Key: CASSANDRA-14638
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14638
> Project: Cassandra
> Issue Type: Bug
> Environment: Single C* node ccm cluster upgraded from C* 2.1.20 to
> 3.0.17
>Reporter: Andy Tolbert
>Assignee: Aleksey Yeschenko
>Priority: Major
> Fix For: 3.0.x, 3.11.x
>
>
> When performing an upgrade from C* 2.1.20 to 3.0.17 I observed that the order
> of columns returned from a 'SELECT *' query changes, particularly when static
> columns are involved.
> This may not seem like that much of a problem, however if using Prepared
> Statements, any clients that remain connected during the upgrade may
> encounter issues consuming results from these queries, as data is reordered
> and the client not aware of it. The result definition is sent in the
> original prepared statement response, so if order changes the client has no
> way of knowing (until C* 4.0 via CASSANDRA-10786) without re-preparing, which
> is non-trivial as most client drivers cache prepared statements.
> This could lead to reading the wrong values for columns, which could result
> in some kind of deserialization exception or if the data types of the
> switched columns are compatible, the wrong values. This happens even if the
> client attempts to retrieve a column value by name (i.e. row.getInt("colx")).
> Unfortunately I don't think there is an easy fix for this. If the order was
> changed back to the previous format, you risk issues for users upgrading from
> older 3.0 version. I think it would be nice to add a note in the NEWS file
> in the 3.0 upgrade section that describes this issue, and how to work around
> it (specify all column names of interest explicitly in query).
> Example schema and code to reproduce:
>
> {noformat}
> create keyspace ks with replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> create table ks.tbl (p0 text,
> p1 text,
> m map static,
> t text,
> u text static,
> primary key (p0, p1)
> );
> insert into ks.tbl (p0, p1, m, t, u) values ('p0', 'p1', { 'm0' : 'm1' },
> 't', 'u');{noformat}
>
> When querying with 2.1 you'll observe the following order via cqlsh:
> {noformat}
> p0 | p1 | m| u | t
> ++--+---+---
> p0 | p1 | {'m0': 'm1'} | u | t{noformat}
>
> With 3.0, observe that u and m are transposed:
>
> {noformat}
> p0 | p1 | u | m | t
> ++---+--+---
> p0 | p1 | u | {'m0': 'm1'} | t{noformat}
>
>
> {code:java}
> import com.datastax.driver.core.BoundStatement;
> import com.datastax.driver.core.Cluster;
> import com.datastax.driver.core.ColumnDefinitions;
> import com.datastax.driver.core.PreparedStatement;
> import com.datastax.driver.core.ResultSet;
> import com.datastax.driver.core.Row;
> import com.datastax.driver.core.Session;
> import com.google.common.util.concurrent.Uninterruptibles;
> import java.util.concurrent.TimeUnit;
> public class LiveUpgradeTest {
> public static void main(String args[]) {
> Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").build();
> try {
> Session session = cluster.connect();
> PreparedStatement p = session.prepare("SELECT * from ks.tbl");
> BoundStatement bs = p.bind();
> // continually query every 30 seconds
> while (true) {
> try {
> ResultSet r = sess
[jira] [Commented] (CASSANDRA-14655) Upgrade C* to use latest guava (26.0)
[ https://issues.apache.org/jira/browse/CASSANDRA-14655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586142#comment-16586142 ] Andy Tolbert commented on CASSANDRA-14655: -- We are tentatively planning on releasing version 3.6.0 of the driver which will include guava 26 compatibility sometime in the latter part of next week. I'll add a comment to this ticket when that's released (y). > Upgrade C* to use latest guava (26.0) > - > > Key: CASSANDRA-14655 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14655 > Project: Cassandra > Issue Type: Improvement > Components: Libraries >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Minor > Fix For: 4.x > > > C* currently uses guava 23.3. This JIRA is about changing C* to use latest > guava (26.0). Originated from a discussion in the mailing list. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14655) Upgrade C* to use latest guava (26.0)
[ https://issues.apache.org/jira/browse/CASSANDRA-14655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596731#comment-16596731 ] Andy Tolbert commented on CASSANDRA-14655: -- driver 3.6.0 has been released! This version supports guava versions 16.0.1 through 26.0-jre. > Upgrade C* to use latest guava (26.0) > - > > Key: CASSANDRA-14655 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14655 > Project: Cassandra > Issue Type: Improvement > Components: Libraries >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Minor > Fix For: 4.x > > > C* currently uses guava 23.3. This JIRA is about changing C* to use latest > guava (26.0). Originated from a discussion in the mailing list. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14655) Upgrade C* to use latest guava (26.0)
[
https://issues.apache.org/jira/browse/CASSANDRA-14655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596903#comment-16596903
]
Andy Tolbert commented on CASSANDRA-14655:
--
[~sumanth.pasupuleti], hrmm I'm betting that current trunk depends on a Driver
snapshot based on [this PR|https://github.com/datastax/java-driver/pull/931]
for CASSANDRA-7544. The way we ended up implementing this feature in the
driver, {{allowServerPortDiscovery}} is no longer needed as the driver will
always query the {{system.peers_v2}} table and use it if exists, so I think you
can safely remove that option from NativeSSTableLoaderClient.
> Upgrade C* to use latest guava (26.0)
> -
>
> Key: CASSANDRA-14655
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14655
> Project: Cassandra
> Issue Type: Improvement
> Components: Libraries
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Minor
> Fix For: 4.x
>
>
> C* currently uses guava 23.3. This JIRA is about changing C* to use latest
> guava (26.0). Originated from a discussion in the mailing list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14655) Upgrade C* to use latest guava (26.0)
[
https://issues.apache.org/jira/browse/CASSANDRA-14655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596906#comment-16596906
]
Andy Tolbert commented on CASSANDRA-14655:
--
Ah, I also forgot to mention, you can use {{getBroadcastSocketAddress}} in
favor of {{getBroadcastAddressOptPort}}.
The code change should look something like this:
{code:java}
int broadcastPort = endpoint.getBroadcastSocketAddress().getPort();
// use port from broadcast address if set.
int portToUse = broadcastPort != 0 ? broadcastPort : storagePort;
{code}
> Upgrade C* to use latest guava (26.0)
> -
>
> Key: CASSANDRA-14655
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14655
> Project: Cassandra
> Issue Type: Improvement
> Components: Libraries
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Minor
> Fix For: 4.x
>
>
> C* currently uses guava 23.3. This JIRA is about changing C* to use latest
> guava (26.0). Originated from a discussion in the mailing list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14655) Upgrade C* to use latest guava (26.0)
[
https://issues.apache.org/jira/browse/CASSANDRA-14655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598987#comment-16598987
]
Andy Tolbert commented on CASSANDRA-14655:
--
Ah - curious about the missing {{system_virtual_schema}} errors. The way our
schema resolution logic works for the driver is if any of tables we query for
schema don't exist, we bail out (unless you use
{{setSchemaMetadataEnabled(false)}} in your {{Cluster.builder}}). We had
assumed that {{system_virtual_schema}} would be present for C* 4.0 always,
which I believe should be the case, but maybe in the context of the tests, the
Cassandra server code is not completely loaded? We could consider silenty
skipping parsing of virtual tables in this case in the driver, although I think
it's reasonable to expect that they should always be there (just like we assume
{{system_schema}} is).
With regards to the NPE, is there an associated stack trace? Curious to know
if the {{NPE}} is raised by the driver or C* code.
> Upgrade C* to use latest guava (26.0)
> -
>
> Key: CASSANDRA-14655
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14655
> Project: Cassandra
> Issue Type: Improvement
> Components: Libraries
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Minor
> Fix For: 4.x
>
>
> C* currently uses guava 23.3. This JIRA is about changing C* to use latest
> guava (26.0). Originated from a discussion in the mailing list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14655) Upgrade C* to use latest guava (26.0)
[
https://issues.apache.org/jira/browse/CASSANDRA-14655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599028#comment-16599028
]
Andy Tolbert commented on CASSANDRA-14655:
--
Thanks for providing the stack traces.
Ahh, I suppose that the cause of the NPE could just be that no {{Cluster}}
instance is provided in {{executeNet}} because it can't properly initialize
because of the {{system_virtual_schema}} issue.
In terms of {{system_virtual_schema}} being missing, I think the code that
loads/creates that keyspace is
[here|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CassandraDaemon.java#L255-L256]
during the setup of {{CassandraDaemon}}. My guess is that if you add the
relevant code in CQLTester.requireNetwork, like [is
done|https://github.com/apache/cassandra/blob/207c80c1fd63dfbd8ca7e615ec8002ee8983c5d6/test/unit/org/apache/cassandra/cql3/CQLTester.java#L381]
for the system keyspace via {{SystemKeyspace.finishStartup()}} that will fix
this issue.
> Upgrade C* to use latest guava (26.0)
> -
>
> Key: CASSANDRA-14655
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14655
> Project: Cassandra
> Issue Type: Improvement
> Components: Libraries
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Minor
> Fix For: 4.x
>
>
> C* currently uses guava 23.3. This JIRA is about changing C* to use latest
> guava (26.0). Originated from a discussion in the mailing list.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13871) cassandra-stress user command misbehaves when retrying operations
[
https://issues.apache.org/jira/browse/CASSANDRA-13871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16640549#comment-16640549
]
Andy Tolbert commented on CASSANDRA-13871:
--
I recently ran into this and was confounded why stress started misbehaving very
oddly. I had {{-errors ignore}} specified, so the exception wasn't surfaced at
all, but after removing that option and updating code to include the full cause
trace, I realized I was encountering this issue. It looks like the patch is no
longer valid because of recent changes, I will attach a follow on patch
sometime this weekend.
> cassandra-stress user command misbehaves when retrying operations
> -
>
> Key: CASSANDRA-13871
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13871
> Project: Cassandra
> Issue Type: Bug
> Components: Stress
>Reporter: Daniel Cranford
>Priority: Minor
> Attachments: 0001-Fixing-cassandra-stress-user-operations-retry.patch
>
>
> o.a.c.stress.Operation will retry queries a configurable number of times.
> When the "user" command is invoked the o.a.c.stress.operations.userdefined
> SchemaInsert and SchemaQuery operations are used.
> When SchemaInsert and SchemaQuery are retried (eg after a Read/WriteTimeout
> exception), they advance the PartitionIterator used to generate the keys to
> insert/query (SchemaInsert.java:85 SchemaQuery.java:129) This means each
> retry will use a different set of keys.
> The predefined set of operations avoid this problem by packaging up the
> arguments to bind to the query into the RunOp object so that retrying the
> operation results in exactly the same query with the same arguments being run.
> This problem was introduced by CASSANDRA-7964. Prior to CASSANDRA-7964 the
> PartitionIterator (Partition.RowIterator before the change) was reinitialized
> prior to each query retry, thus generating the same set of keys each time.
> This problem is reported rather confusingly. The only error that shows up in
> a log file (specified with -log file=foo.log) is the unhelpful
> {noformat}
> java.io.IOException Operation x10 on key(s) [foobarkey]: Error executing:
> (NoSuchElementException)
> at org.apache.cassandra.stress.Operation.error(Operation.java:136)
> at org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:114)
> at
> org.apache.cassandra.stress.userdefined.SchemaQuery.run(SchemaQuery.java:158)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:459)
> {noformat}
> Standard error is only slightly more helpful, displaying the ignorable
> initial read/write error, and confusing java.util.NoSuchElementException
> lines (caused by PartitionIterator exhaustion) followed by the above
> IOException with stack trace, eg
> {noformat}
> com.datastax.drive.core.exceptions.ReadTimeoutException: Cassandra timeout
> during read query
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.util.NoSuchElementException
> java.io.IOException Operation x10 on key(s) [foobarkey]: Error executing:
> (NoSuchElementException)
> at org.apache.cassandra.stress.Operation.error(Operation.java:136)
> at org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:114)
> at
> org.apache.cassandra.stress.userdefined.SchemaQuery.run(SchemaQuery.java:158)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:459)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15349) Add “Going away” message to the client protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-15349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968498#comment-16968498
]
Andy Tolbert commented on CASSANDRA-15349:
--
I think a new event for {{REGISTER}} seems adequate if the only information we
are communicating is that the node is going away. It seems to fit in that it's
the only current type that causes messages that aren't bound to a request. One
benefit of this is that sending this as an event shouldn't break any existing
clients that don't support it since they would have to know to {{REGISTER}} for
it.
Some thoughts about how I think it should work on the server side when it sends
{{GOING_AWAY}}:
* The node stops accepting new connections immediately.
* The node still accepts requests for some time, like 2 seconds. This gives
client time to react to the message. Another alternative is to make this a
quiet period, similar to Netty's
[shutdownGracefully|https://netty.io/4.0/api/io/netty/util/concurrent/GlobalEventExecutor.html#shutdownGracefully-long-long-java.util.concurrent.TimeUnit-]
where it will continue accepting requests until some quiet period or timeout
has elapsed.
* After that time, it sends some kind of error, something like
{{Overloaded}}), which would cause most clients to retry on next host
* Then after some other configurable period of time (10 seconds?), it closes
the connections.
On driver side, some ideas, but depends on driver's implementation:
1. If we go register route, each connection would need to register on this when
its initialized. This is probably easier to implement than doing it for one
connection per node I think.
2. When receiving {{GOING_AWAY}}, stop sending new requests to that node, but
don't consider it DOWN to allow pending requests to complete. Most drivers
will mark the host DOWN once all connections to it are closed, and will try
reconnecting per usual.
3. Don't close connections explicitly, let the server do it for you.
> Add “Going away” message to the client protocol
> ---
>
> Key: CASSANDRA-15349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15349
> Project: Cassandra
> Issue Type: New Feature
> Components: Messaging/Client
>Reporter: Alex Petrov
>Priority: Normal
> Labels: client-impacting
>
> Add “Going away” message that allows node to announce its shutdown and let
> clients gracefully shutdown and not attempt further requests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15349) Add “Going away” message to the client protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-15349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968498#comment-16968498
]
Andy Tolbert edited comment on CASSANDRA-15349 at 11/6/19 4:48 PM:
---
I think a new event for {{REGISTER}} seems adequate if the only information we
are communicating is that the node is going away. It seems to fit in that it's
the only current type that causes messages that aren't bound to a request. One
benefit of this is that sending this as an event shouldn't break any existing
clients that don't support it since they would have to know to {{REGISTER}} for
it.
Some thoughts about how I think it should work on the server side when it sends
{{GOING_AWAY}}:
* The node stops accepting new connections immediately.
* The node still accepts requests for some time, like 2 seconds. This gives
client time to react to the message. Another alternative is to make this a
quiet period, similar to Netty's
[shutdownGracefully|https://netty.io/4.0/api/io/netty/util/concurrent/GlobalEventExecutor.html#shutdownGracefully-long-long-java.util.concurrent.TimeUnit-]
where it will continue accepting requests until some quiet period or timeout
has elapsed.
* After that time, it sends some kind of error, something like
{{Overloaded}}, which would cause most clients to retry on next host
* Then after some other configurable period of time (10 seconds?), it closes
the connections.
On driver side, some ideas, but depends on driver's implementation:
1. If we go register route, each connection would need to register on this when
its initialized. This is probably easier to implement than doing it for one
connection per node I think.
2. When receiving {{GOING_AWAY}}, stop sending new requests to that node, but
don't consider it DOWN to allow pending requests to complete. Most drivers
will mark the host DOWN once all connections to it are closed, and will try
reconnecting per usual.
3. Don't close connections explicitly, let the server do it for you.
was (Author: andrew.tolbert):
I think a new event for {{REGISTER}} seems adequate if the only information we
are communicating is that the node is going away. It seems to fit in that it's
the only current type that causes messages that aren't bound to a request. One
benefit of this is that sending this as an event shouldn't break any existing
clients that don't support it since they would have to know to {{REGISTER}} for
it.
Some thoughts about how I think it should work on the server side when it sends
{{GOING_AWAY}}:
* The node stops accepting new connections immediately.
* The node still accepts requests for some time, like 2 seconds. This gives
client time to react to the message. Another alternative is to make this a
quiet period, similar to Netty's
[shutdownGracefully|https://netty.io/4.0/api/io/netty/util/concurrent/GlobalEventExecutor.html#shutdownGracefully-long-long-java.util.concurrent.TimeUnit-]
where it will continue accepting requests until some quiet period or timeout
has elapsed.
* After that time, it sends some kind of error, something like
{{Overloaded}}), which would cause most clients to retry on next host
* Then after some other configurable period of time (10 seconds?), it closes
the connections.
On driver side, some ideas, but depends on driver's implementation:
1. If we go register route, each connection would need to register on this when
its initialized. This is probably easier to implement than doing it for one
connection per node I think.
2. When receiving {{GOING_AWAY}}, stop sending new requests to that node, but
don't consider it DOWN to allow pending requests to complete. Most drivers
will mark the host DOWN once all connections to it are closed, and will try
reconnecting per usual.
3. Don't close connections explicitly, let the server do it for you.
> Add “Going away” message to the client protocol
> ---
>
> Key: CASSANDRA-15349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15349
> Project: Cassandra
> Issue Type: New Feature
> Components: Messaging/Client
>Reporter: Alex Petrov
>Priority: Normal
> Labels: client-impacting
>
> Add “Going away” message that allows node to announce its shutdown and let
> clients gracefully shutdown and not attempt further requests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15349) Add “Going away” message to the client protocol
[
https://issues.apache.org/jira/browse/CASSANDRA-15349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968498#comment-16968498
]
Andy Tolbert edited comment on CASSANDRA-15349 at 11/6/19 4:48 PM:
---
I think a new event for {{REGISTER}} seems adequate if the only information we
are communicating is that the node is going away. It seems to fit in that it's
the only current type that causes messages that aren't bound to a request. One
benefit of this is that sending this as an event shouldn't break any existing
clients that don't support it since they would have to know to {{REGISTER}} for
it.
Some thoughts about how I think it should work on the server side when it sends
{{GOING_AWAY}}:
* The node stops accepting new connections immediately.
* The node still accepts requests for some time, like 2 seconds. This gives
client time to react to the message. Another alternative is to make this a
quiet period, similar to Netty's
[shutdownGracefully|https://netty.io/4.0/api/io/netty/util/concurrent/GlobalEventExecutor.html#shutdownGracefully-long-long-java.util.concurrent.TimeUnit-]
where it will continue accepting requests until some quiet period or timeout
has elapsed.
* After that time, it sends some kind of error, something like
{{Overloaded}}, which would cause most clients to retry on next host
* Then after some other configurable period of time (10 seconds?), it closes
the connections. Alternatively, connections could be closed when there are no
more inflight requests on them (after a quiet period).
On driver side, some ideas, but depends on driver's implementation:
1. If we go register route, each connection would need to register on this when
its initialized. This is probably easier to implement than doing it for one
connection per node I think.
2. When receiving {{GOING_AWAY}}, stop sending new requests to that node, but
don't consider it DOWN to allow pending requests to complete. Most drivers
will mark the host DOWN once all connections to it are closed, and will try
reconnecting per usual.
3. Don't close connections explicitly, let the server do it for you.
was (Author: andrew.tolbert):
I think a new event for {{REGISTER}} seems adequate if the only information we
are communicating is that the node is going away. It seems to fit in that it's
the only current type that causes messages that aren't bound to a request. One
benefit of this is that sending this as an event shouldn't break any existing
clients that don't support it since they would have to know to {{REGISTER}} for
it.
Some thoughts about how I think it should work on the server side when it sends
{{GOING_AWAY}}:
* The node stops accepting new connections immediately.
* The node still accepts requests for some time, like 2 seconds. This gives
client time to react to the message. Another alternative is to make this a
quiet period, similar to Netty's
[shutdownGracefully|https://netty.io/4.0/api/io/netty/util/concurrent/GlobalEventExecutor.html#shutdownGracefully-long-long-java.util.concurrent.TimeUnit-]
where it will continue accepting requests until some quiet period or timeout
has elapsed.
* After that time, it sends some kind of error, something like
{{Overloaded}}, which would cause most clients to retry on next host
* Then after some other configurable period of time (10 seconds?), it closes
the connections.
On driver side, some ideas, but depends on driver's implementation:
1. If we go register route, each connection would need to register on this when
its initialized. This is probably easier to implement than doing it for one
connection per node I think.
2. When receiving {{GOING_AWAY}}, stop sending new requests to that node, but
don't consider it DOWN to allow pending requests to complete. Most drivers
will mark the host DOWN once all connections to it are closed, and will try
reconnecting per usual.
3. Don't close connections explicitly, let the server do it for you.
> Add “Going away” message to the client protocol
> ---
>
> Key: CASSANDRA-15349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15349
> Project: Cassandra
> Issue Type: New Feature
> Components: Messaging/Client
>Reporter: Alex Petrov
>Priority: Normal
> Labels: client-impacting
>
> Add “Going away” message that allows node to announce its shutdown and let
> clients gracefully shutdown and not attempt further requests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15349) Add “Going away” message to the client protocol
[ https://issues.apache.org/jira/browse/CASSANDRA-15349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16968651#comment-16968651 ] Andy Tolbert commented on CASSANDRA-15349: -- With regards to #3, I was only thinking about limiting the complexity on the client side. If the server could close the connection when there are no more inflight requests (or after some timeout if its taking too long) the client wouldn't need to worry about checking for it as it gets responses. The timeout could be more of a ceiling in cases where requests aren't being processed for whatever reason, or if the client won't stop sending requests. > Add “Going away” message to the client protocol > --- > > Key: CASSANDRA-15349 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15349 > Project: Cassandra > Issue Type: New Feature > Components: Messaging/Client >Reporter: Alex Petrov >Priority: Normal > Labels: client-impacting > > Add “Going away” message that allows node to announce its shutdown and let > clients gracefully shutdown and not attempt further requests. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14872) Update to version of python driver and update cqlsh to use driver metadata for virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040686#comment-17040686
]
Andy Tolbert commented on CASSANDRA-14872:
--
I remember trying transient replication with the java driver and it not working
([JAVA-2105|https://datastax-oss.atlassian.net/browse/JAVA-2105]), I suspect
most/all client libraries that read schema need updates to parse it.
> Update to version of python driver and update cqlsh to use driver metadata
> for virtual tables
> -
>
> Key: CASSANDRA-14872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14872
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Andy Tolbert
>Assignee: Dinesh Joshi
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> When virtual tables were implemented ([CASSANDRA-7622]), cqlsh.py was updated
> to parse virtual keyspace metadata by making queries to the
> {{system_virtual_schema}} table and included a TODO:
> {code:python}
> # TODO remove after virtual tables are added to connection metadata
> {code}
> Since python driver 3.15.0 (released in August), the driver now parses
> virtual keyspace metadata. It would be good to update the bundled python
> driver and simplify cqlsh code to utilize its capability to parse virtual
> tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14872) Update to version of python driver and update cqlsh to use driver metadata for virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040697#comment-17040697
]
Andy Tolbert commented on CASSANDRA-14872:
--
Changes look great! Played around with it a little bit and it's working well.
I like that autocompletion works flawlessly for vtables :)
One thing that is interesting is that geomet is a required dependency. Since
it's apache licensed and seems pretty small that doesn't seem like much of a
deal. I think it could be made optional in the python driver (it's used for
parsing DSE-specific geospatial types) so might be something worth pursuing on
that end.
> Update to version of python driver and update cqlsh to use driver metadata
> for virtual tables
> -
>
> Key: CASSANDRA-14872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14872
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Andy Tolbert
>Assignee: Dinesh Joshi
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> When virtual tables were implemented ([CASSANDRA-7622]), cqlsh.py was updated
> to parse virtual keyspace metadata by making queries to the
> {{system_virtual_schema}} table and included a TODO:
> {code:python}
> # TODO remove after virtual tables are added to connection metadata
> {code}
> Since python driver 3.15.0 (released in August), the driver now parses
> virtual keyspace metadata. It would be good to update the bundled python
> driver and simplify cqlsh code to utilize its capability to parse virtual
> tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14872) Update to version of python driver and update cqlsh to use driver metadata for virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040697#comment-17040697
]
Andy Tolbert edited comment on CASSANDRA-14872 at 2/20/20 7:37 AM:
---
Changes look great! Played around with it a little bit and it's working well.
I like that autocompletion works flawlessly for vtables :)
One thing that is interesting is that geomet is a required dependency. Since
it's apache licensed and seems pretty small that doesn't seem like much of a
deal. I think it could be made optional in the python driver (it's used for
parsing DSE-specific geospatial types) so might be something worth pursuing on
that end (but don't think it's needed as a prereq to this).
was (Author: andrew.tolbert):
Changes look great! Played around with it a little bit and it's working well.
I like that autocompletion works flawlessly for vtables :)
One thing that is interesting is that geomet is a required dependency. Since
it's apache licensed and seems pretty small that doesn't seem like much of a
deal. I think it could be made optional in the python driver (it's used for
parsing DSE-specific geospatial types) so might be something worth pursuing on
that end.
> Update to version of python driver and update cqlsh to use driver metadata
> for virtual tables
> -
>
> Key: CASSANDRA-14872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14872
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Andy Tolbert
>Assignee: Dinesh Joshi
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> When virtual tables were implemented ([CASSANDRA-7622]), cqlsh.py was updated
> to parse virtual keyspace metadata by making queries to the
> {{system_virtual_schema}} table and included a TODO:
> {code:python}
> # TODO remove after virtual tables are added to connection metadata
> {code}
> Since python driver 3.15.0 (released in August), the driver now parses
> virtual keyspace metadata. It would be good to update the bundled python
> driver and simplify cqlsh code to utilize its capability to parse virtual
> tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14872) Update to version of python driver and update cqlsh to use driver metadata for virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17040727#comment-17040727
]
Andy Tolbert commented on CASSANDRA-14872:
--
{quote}
It will need some driver changes and I am reluctant to check in a custom build
of the driver. So this is an ok compromise for the time being
{quote}
+1 agree we should proceed with it included as it seems like a small
self-contained dependency, and it is really nice to not use a custom build.
> Update to version of python driver and update cqlsh to use driver metadata
> for virtual tables
> -
>
> Key: CASSANDRA-14872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14872
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Andy Tolbert
>Assignee: Dinesh Joshi
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> When virtual tables were implemented ([CASSANDRA-7622]), cqlsh.py was updated
> to parse virtual keyspace metadata by making queries to the
> {{system_virtual_schema}} table and included a TODO:
> {code:python}
> # TODO remove after virtual tables are added to connection metadata
> {code}
> Since python driver 3.15.0 (released in August), the driver now parses
> virtual keyspace metadata. It would be good to update the bundled python
> driver and simplify cqlsh code to utilize its capability to parse virtual
> tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15032) Adjust transient replication keyspace replication options to be more friendly to naive clients
Andy Tolbert created CASSANDRA-15032:
Summary: Adjust transient replication keyspace replication options
to be more friendly to naive clients
Key: CASSANDRA-15032
URL: https://issues.apache.org/jira/browse/CASSANDRA-15032
Project: Cassandra
Issue Type: Improvement
Components: Feature/Transient Replication
Reporter: Andy Tolbert
To specify the number of transient replicas, replication options are specified
like:
{code}
ALTER KEYSPACE foo WITH REPLICATION = {'class' : 'NetworkTopologyStrategy',
'DC1' : '3/1'}; ALTER KEYSPACE foo WITH REPLICATION = {'class' :
'SimpleStrategy', 'replication_factor' : '3/1'}
{code}
It occurred to me that existing client drivers that parse keyspace options may
not handle this gracefully.
For example, the datastax java driver tries to parse {{3/1}} as a number and
fails. In this case, the parsing error is not fatal, its just that the
metadata for that keyspace in the driver is incomplete, and things like token
aware routing can't be utilized.
It is possible that other libraries may not handle this as well.
As an alternative, I propose adding a separate option like:
{{'transient_replicas': 1}}. {{replication_factor}} would represent the total
number of replicas (full and transient) in this case.
This would allow existing client libraries to continue working, and while
things like routing may be suboptimal (i.e. driver won't know to differentiate
between replicas and transient replicas), at least parsing won't fail in
possibly fatal ways.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15032) Adjust transient replication keyspace replication options to be more friendly to naive clients
[
https://issues.apache.org/jira/browse/CASSANDRA-15032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-15032:
-
Description:
To specify the number of transient replicas, replication options are specified
like:
{code}
ALTER KEYSPACE foo WITH REPLICATION = {'class' : 'NetworkTopologyStrategy',
'DC1' : '3/1'};
ALTER KEYSPACE foo WITH REPLICATION = {'class' : 'SimpleStrategy',
'replication_factor' : '3/1'}
{code}
It occurred to me that existing client drivers that parse keyspace options may
not handle this gracefully.
For example, the datastax java driver tries to parse {{3/1}} as a number and
fails. In this case, the parsing error is not fatal, its just that the
metadata for that keyspace in the driver is incomplete, and things like token
aware routing can't be utilized.
It is possible that other libraries may not handle this as well.
As an alternative, I propose adding a separate option like:
{{'transient_replicas': 1}}. {{replication_factor}} would represent the total
number of replicas (full and transient) in this case. Something similar could
be done for the NTS case, but might be slightly clumsy to express.
This would allow existing client libraries to continue working, and while
things like routing may be suboptimal (i.e. driver won't know to differentiate
between replicas and transient replicas), at least parsing won't fail in
possibly fatal ways.
was:
To specify the number of transient replicas, replication options are specified
like:
{code}
ALTER KEYSPACE foo WITH REPLICATION = {'class' : 'NetworkTopologyStrategy',
'DC1' : '3/1'}; ALTER KEYSPACE foo WITH REPLICATION = {'class' :
'SimpleStrategy', 'replication_factor' : '3/1'}
{code}
It occurred to me that existing client drivers that parse keyspace options may
not handle this gracefully.
For example, the datastax java driver tries to parse {{3/1}} as a number and
fails. In this case, the parsing error is not fatal, its just that the
metadata for that keyspace in the driver is incomplete, and things like token
aware routing can't be utilized.
It is possible that other libraries may not handle this as well.
As an alternative, I propose adding a separate option like:
{{'transient_replicas': 1}}. {{replication_factor}} would represent the total
number of replicas (full and transient) in this case.
This would allow existing client libraries to continue working, and while
things like routing may be suboptimal (i.e. driver won't know to differentiate
between replicas and transient replicas), at least parsing won't fail in
possibly fatal ways.
> Adjust transient replication keyspace replication options to be more friendly
> to naive clients
> --
>
> Key: CASSANDRA-15032
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15032
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Transient Replication
>Reporter: Andy Tolbert
>Priority: Major
>
> To specify the number of transient replicas, replication options are
> specified like:
> {code}
> ALTER KEYSPACE foo WITH REPLICATION = {'class' : 'NetworkTopologyStrategy',
> 'DC1' : '3/1'};
> ALTER KEYSPACE foo WITH REPLICATION = {'class' : 'SimpleStrategy',
> 'replication_factor' : '3/1'}
> {code}
> It occurred to me that existing client drivers that parse keyspace options
> may not handle this gracefully.
> For example, the datastax java driver tries to parse {{3/1}} as a number and
> fails. In this case, the parsing error is not fatal, its just that the
> metadata for that keyspace in the driver is incomplete, and things like token
> aware routing can't be utilized.
> It is possible that other libraries may not handle this as well.
> As an alternative, I propose adding a separate option like:
> {{'transient_replicas': 1}}. {{replication_factor}} would represent the
> total number of replicas (full and transient) in this case. Something similar
> could be done for the NTS case, but might be slightly clumsy to express.
> This would allow existing client libraries to continue working, and while
> things like routing may be suboptimal (i.e. driver won't know to
> differentiate between replicas and transient replicas), at least parsing
> won't fail in possibly fatal ways.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-11097) Idle session timeout for secure environments
[
https://issues.apache.org/jira/browse/CASSANDRA-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876952#comment-16876952
]
Andy Tolbert commented on CASSANDRA-11097:
--
Might already be aware of this but some of the datastax drivers send a
'heartbeat' {{OPTIONS}} message every 30 seconds to keep connections active as
we've seen some environments disconnect idle connections as soon as 5 minutes
([doc|https://docs.datastax.com/en/developer/java-driver/3.7/manual/pooling/#heartbeat]).
Is the expectation that to make use of this feature that client side
heartbeats should be disabled?
> Idle session timeout for secure environments
>
>
> Key: CASSANDRA-11097
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11097
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Jeff Jirsa
>Assignee: Alex Petrov
>Priority: Low
> Labels: lhf, ponies
>
> A thread on the user list pointed out that some use cases may prefer to have
> a database disconnect sessions after some idle timeout. An example would be
> an administrator who connected via ssh+cqlsh and then walked away.
> Disconnecting that user and forcing it to re-authenticate could protect
> against unauthorized access.
> It seems like it may be possible to do this using a netty
> {{IdleStateHandler}} in a way that's low risk and perhaps off by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-11097) Idle session timeout for secure environments
[
https://issues.apache.org/jira/browse/CASSANDRA-11097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16878821#comment-16878821
]
Andy Tolbert commented on CASSANDRA-11097:
--
Thanks for the background [~ifesdjeen], that makes sense to me (y). I can see
how being able to clean up defunct connections that aren't doing anything would
be nice, especially since TCP keepalive defaults are usually large (2 hours).
With regards to this case from description:
{quote}
An example would be an administrator who connected via ssh+cqlsh and then
walked away. Disconnecting that user and forcing it to re-authenticate could
protect against unauthorized access.
{quote}
If the underlying python-driver was using the default heartbeat configuration,
wouldn't the connections stay alive then even with this new configuration? Even
if it did close the connections, the driver would likely successfully reconnect
since it looks like the configured auth provider for cqlsh will retain
credentials between reconnections.
That being said I still like the change as it enables the capability of timing
out idle connections, but may require some configuration on the client side to
achieve the "walk way from my cqlsh session" case.
> Idle session timeout for secure environments
>
>
> Key: CASSANDRA-11097
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11097
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Jeff Jirsa
>Assignee: Alex Petrov
>Priority: Low
> Labels: lhf, ponies
> Fix For: 4.0
>
>
> A thread on the user list pointed out that some use cases may prefer to have
> a database disconnect sessions after some idle timeout. An example would be
> an administrator who connected via ssh+cqlsh and then walked away.
> Disconnecting that user and forcing it to re-authenticate could protect
> against unauthorized access.
> It seems like it may be possible to do this using a netty
> {{IdleStateHandler}} in a way that's low risk and perhaps off by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15206) cassandra 4.0 cqlsh not working with jdk 11
[ https://issues.apache.org/jira/browse/CASSANDRA-15206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883754#comment-16883754 ] Andy Tolbert commented on CASSANDRA-15206: -- I'm somewhat confident this isn't related to building with JDK11. Just to verify I built latest trunk (149caf01) and everything seems in working order. Rather I suspect a compatibility issue with cqlsh and the python-driver being used. cqlsh scans your lib directory for a file name starting with 'cassandra-driver-internal-only-' and adds it to the beginning of the python lib path. In the case of trunk, there should be a file named 'cassandra-driver-internal-only-3.12.0.post0-5838e2fd.zip'. Can you verify whether or not that is the case? What could be happening is the file is not found, and python is falling back on your installed libraries, and maybe you have a different version of the python driver installed than expected. Or maybe there are multiple files prefixed with 'cassandra-driver-internal-only-' in your path and an older one is being used. Another possibility is that you may have moved cqlsh/cqlsh.py out of the bin directory and therefore the zip file can't be found in a relative way, and cqlsh is falling back on your installed libraries, which may contain a different version of the python driver. Although I see in your stack trace that the script is running out of cassandra/bin, so maybe that isn't the case. > cassandra 4.0 cqlsh not working with jdk 11 > --- > > Key: CASSANDRA-15206 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15206 > Project: Cassandra > Issue Type: Bug > Components: Tool/cqlsh >Reporter: RamyaK >Priority: Urgent > > Im able to start cassandra by compiling the latest git code with OpenJDK11, > but facing below error with cqlsh. please suggest. > > Traceback (most recent call last): > File "/home/id/cassandra/bin/cqlsh.py", line 2520, in > main(*read_options(sys.argv[1:], os.environ)) > File "/home/id/cassandra/bin/cqlsh.py", line 2498, in main > allow_server_port_discovery=options.allow_server_port_discovery) > File "/home/id/cassandra/bin/cqlsh.py", line 491, in __init__ > **kwargs) > File "cassandra/cluster.py", line 802, in cassandra.cluster.Cluster.__init__ > TypeError: __init__() got an unexpected keyword argument > 'allow_server_port_discovery' -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15248) Upgrade Guava to latest on master branch
[ https://issues.apache.org/jira/browse/CASSANDRA-15248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892972#comment-16892972 ] Andy Tolbert commented on CASSANDRA-15248: -- [~sumanth.pasupuleti] had provided a patch for upgrading to Guava 27.0 in [CASSANDRA-14655]. There may be some follow on work to do with Guava 28 out and other code changing since then but that may be a good starting point. > Upgrade Guava to latest on master branch > > > Key: CASSANDRA-15248 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15248 > Project: Cassandra > Issue Type: Task > Components: Build, Dependencies, Packaging >Reporter: Abhijit Sarkar >Priority: Normal > > Upgrade Guava to latest on master branch. See > https://issues.apache.org/jira/browse/CASSANDRA-15245. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15248) Upgrade Guava to latest on master branch
[ https://issues.apache.org/jira/browse/CASSANDRA-15248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16895307#comment-16895307 ] Andy Tolbert commented on CASSANDRA-15248: -- 4.0 isn’t too far away so we should be carefully considering any changes at this point. Since guava is used in a number of places it makes sense to be deliberate about whether or not to upgrade it. > Upgrade Guava to latest on master branch > > > Key: CASSANDRA-15248 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15248 > Project: Cassandra > Issue Type: Task > Components: Build, Dependencies, Packaging >Reporter: Abhijit Sarkar >Priority: Normal > > Upgrade Guava to latest on master branch. See > https://issues.apache.org/jira/browse/CASSANDRA-15245. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15269) Cassandra fails to process OperationExecutionException which causes ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16911757#comment-16911757
]
Andy Tolbert commented on CASSANDRA-15269:
--
I made a small comment that this should be part of the protocol v5 spec instead
of v4, but other than that I think this looks good. I went ahead and created a
java driver ticket for awareness:
[JAVA-2397|https://datastax-oss.atlassian.net/browse/JAVA-2397]. The native
protocol change will be great for the 4.0 java driver. We should also consider
making the change in the 3.0 driver since that intends on supporting C* 4.0 as
well.
> Cassandra fails to process OperationExecutionException which causes
> ClassCastException
> --
>
> Key: CASSANDRA-15269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15269
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Liudmila Kornilova
>Assignee: Liudmila Kornilova
>Priority: Normal
>
> While working on CASSANDRA-15232 I noticed that OperationExecutionException
> is not processed correctly.
> How to reproduce the issue:
> 1. {{create table d (numerator decimal primary key, denominator decimal);}}
> 2. {{insert into d (numerator, denominator) values
> (123456789112345678921234567893123456, 2);}}
> 3. {{select numerator % denominator from d;}}
> What happens:
> 1. remainder operation throws ArithmeticException (BigDecimal:1854)
> 2. The exception is wrapped in OperationExecutionException
> 3. ClassCastException appears (OperationExecutionException cannot be cast to
> FunctionExecutionException at ErrorMessage.java:280)
> What should happen:
> OperationExecutionException with message "the operation 'decimal % decimal'
> failed: Division impossible" should be delivered to user
> Note that after fixing CASSANDRA-15232 {{select numerator % denominator from
> d;}} will produce correct result of remainder operation.
> Currently I am not aware of other cases when OperationExecutionException may
> be treated as FunctionExecutionException
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15288) Read sidecar.yaml from sidecar.config System Property instead of classpath
[
https://issues.apache.org/jira/browse/CASSANDRA-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-15288:
-
Labels: pull-request-available (was: )
> Read sidecar.yaml from sidecar.config System Property instead of classpath
> --
>
> Key: CASSANDRA-15288
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15288
> Project: Cassandra
> Issue Type: Improvement
> Components: Sidecar
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Normal
> Labels: pull-request-available
>
> I was experimenting earlier with running multiple sidecar instances against
> multiple local C* nodes (using a ccm cluster) and noticed that the
> distribution of sidecar did not include the conf directory and instead was
> loading sidecar.yaml from the classpath. I noticed the lines in build.gradle:
> {code:java}
> // Config file location should be in file:/// format for local files,
> // when we have the fix for adding /conf directory to classpaht, we can get
> away with below default JvmArg
> {code}
> From this statement I assume the expectation was that eventually the
> sidecar.config system property would be used to locate the sidecar.yaml file.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15288) Read sidecar.yaml from sidecar.config System Property instead of classpath
Andy Tolbert created CASSANDRA-15288:
Summary: Read sidecar.yaml from sidecar.config System Property
instead of classpath
Key: CASSANDRA-15288
URL: https://issues.apache.org/jira/browse/CASSANDRA-15288
Project: Cassandra
Issue Type: Improvement
Components: Sidecar
Reporter: Andy Tolbert
Assignee: Andy Tolbert
I was experimenting earlier with running multiple sidecar instances against
multiple local C* nodes (using a ccm cluster) and noticed that the distribution
of sidecar did not include the conf directory and instead was loading
sidecar.yaml from the classpath. I noticed the lines in build.gradle:
{code:java}
// Config file location should be in file:/// format for local files,
// when we have the fix for adding /conf directory to classpaht, we can get
away with below default JvmArg
{code}
>From this statement I assume the expectation was that eventually the
>sidecar.config system property would be used to locate the sidecar.yaml file.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15288) Read sidecar.yaml from sidecar.config System Property instead of classpath
[
https://issues.apache.org/jira/browse/CASSANDRA-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-15288:
-
Priority: Low (was: Normal)
> Read sidecar.yaml from sidecar.config System Property instead of classpath
> --
>
> Key: CASSANDRA-15288
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15288
> Project: Cassandra
> Issue Type: Improvement
> Components: Sidecar
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Low
> Labels: pull-request-available
>
> I was experimenting earlier with running multiple sidecar instances against
> multiple local C* nodes (using a ccm cluster) and noticed that the
> distribution of sidecar did not include the conf directory and instead was
> loading sidecar.yaml from the classpath. I noticed the lines in build.gradle:
> {code:java}
> // Config file location should be in file:/// format for local files,
> // when we have the fix for adding /conf directory to classpaht, we can get
> away with below default JvmArg
> {code}
> From this statement I assume the expectation was that eventually the
> sidecar.config system property would be used to locate the sidecar.yaml file.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15288) Read sidecar.yaml from sidecar.config System Property instead of classpath
[
https://issues.apache.org/jira/browse/CASSANDRA-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-15288:
-
Reviewers: Dinesh Joshi, Vinay Chella
> Read sidecar.yaml from sidecar.config System Property instead of classpath
> --
>
> Key: CASSANDRA-15288
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15288
> Project: Cassandra
> Issue Type: Improvement
> Components: Sidecar
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Low
> Labels: pull-request-available
>
> I was experimenting earlier with running multiple sidecar instances against
> multiple local C* nodes (using a ccm cluster) and noticed that the
> distribution of sidecar did not include the conf directory and instead was
> loading sidecar.yaml from the classpath. I noticed the lines in build.gradle:
> {code:java}
> // Config file location should be in file:/// format for local files,
> // when we have the fix for adding /conf directory to classpaht, we can get
> away with below default JvmArg
> {code}
> From this statement I assume the expectation was that eventually the
> sidecar.config system property would be used to locate the sidecar.yaml file.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15288) Read sidecar.yaml from sidecar.config System Property instead of classpath
[
https://issues.apache.org/jira/browse/CASSANDRA-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-15288:
-
Change Category: Semantic
Complexity: Low Hanging Fruit
Status: Open (was: Triage Needed)
> Read sidecar.yaml from sidecar.config System Property instead of classpath
> --
>
> Key: CASSANDRA-15288
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15288
> Project: Cassandra
> Issue Type: Improvement
> Components: Sidecar
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Low
> Labels: pull-request-available
>
> I was experimenting earlier with running multiple sidecar instances against
> multiple local C* nodes (using a ccm cluster) and noticed that the
> distribution of sidecar did not include the conf directory and instead was
> loading sidecar.yaml from the classpath. I noticed the lines in build.gradle:
> {code:java}
> // Config file location should be in file:/// format for local files,
> // when we have the fix for adding /conf directory to classpaht, we can get
> away with below default JvmArg
> {code}
> From this statement I assume the expectation was that eventually the
> sidecar.config system property would be used to locate the sidecar.yaml file.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15288) Read sidecar.yaml from sidecar.config System Property instead of classpath
[
https://issues.apache.org/jira/browse/CASSANDRA-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-15288:
-
Reviewers: Dinesh Joshi, Vinay Chella, Andy Tolbert (was: Andy Tolbert,
Dinesh Joshi, Vinay Chella)
Dinesh Joshi, Vinay Chella, Andy Tolbert (was: Dinesh Joshi,
Vinay Chella)
Status: Review In Progress (was: Patch Available)
> Read sidecar.yaml from sidecar.config System Property instead of classpath
> --
>
> Key: CASSANDRA-15288
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15288
> Project: Cassandra
> Issue Type: Improvement
> Components: Sidecar
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Low
> Labels: pull-request-available
>
> I was experimenting earlier with running multiple sidecar instances against
> multiple local C* nodes (using a ccm cluster) and noticed that the
> distribution of sidecar did not include the conf directory and instead was
> loading sidecar.yaml from the classpath. I noticed the lines in build.gradle:
> {code:java}
> // Config file location should be in file:/// format for local files,
> // when we have the fix for adding /conf directory to classpaht, we can get
> away with below default JvmArg
> {code}
> From this statement I assume the expectation was that eventually the
> sidecar.config system property would be used to locate the sidecar.yaml file.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15288) Read sidecar.yaml from sidecar.config System Property instead of classpath
[
https://issues.apache.org/jira/browse/CASSANDRA-15288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andy Tolbert updated CASSANDRA-15288:
-
Test and Documentation Plan: existing tests should be sufficient.
Status: Patch Available (was: In Progress)
> Read sidecar.yaml from sidecar.config System Property instead of classpath
> --
>
> Key: CASSANDRA-15288
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15288
> Project: Cassandra
> Issue Type: Improvement
> Components: Sidecar
>Reporter: Andy Tolbert
>Assignee: Andy Tolbert
>Priority: Low
> Labels: pull-request-available
>
> I was experimenting earlier with running multiple sidecar instances against
> multiple local C* nodes (using a ccm cluster) and noticed that the
> distribution of sidecar did not include the conf directory and instead was
> loading sidecar.yaml from the classpath. I noticed the lines in build.gradle:
> {code:java}
> // Config file location should be in file:/// format for local files,
> // when we have the fix for adding /conf directory to classpaht, we can get
> away with below default JvmArg
> {code}
> From this statement I assume the expectation was that eventually the
> sidecar.config system property would be used to locate the sidecar.yaml file.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923085#comment-16923085
]
Andy Tolbert commented on CASSANDRA-14825:
--
+1. I tried this out again and left some review comments on the [pull
request|https://github.com/apache/cassandra/pull/284]. The only non-trivial
suggestion I have is to make the topic a clustering column (i.e. {{table_name}}
in {{system_views.describe_table}} should be a clustering column) so you can
make a query such as 'show me all tables in this keyspace'. On the other hand
I can see 'give me the entire schema for this keyspace' and 'give me the schema
for this particular thing' as more valuable usecases, this would be more of a
'nice to have'.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923580#comment-16923580
]
Andy Tolbert commented on CASSANDRA-14825:
--
I do see how {{DESCRIBE}} can have value, but is there any reason why we can't
also offer a virtual table solution? Server providing schema via a select is a
big improvement over what currently exists as it allows users to access schema
via any driver, and doesn't depend on the drivers to build schema which has had
its share of bugs. In addition, the maintenance overhead of the virtual tables
seems pretty low as the code for that is approximately 300 lines. It will
definitely have value until a {{DESCRIBE}}-like implementation is available,
and will still be useful if/when there is something like {{DESCRIBE}}. The PR
also fixes some issues in {{TableCQLHelper}} as well.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923720#comment-16923720
]
Andy Tolbert commented on CASSANDRA-14825:
--
{quote}
I think you misunderstood Robert Stupp's comment above. He's not saying we
should stick to cqlsh-based DESCRIBE, he's mentioning the alternative to the
virtual table approach of having DESCRIBE being a genuine CQL (server-side)
command, one that would return result sets (and that I describe in a number of
comments above).
{quote}
I understand what he's referencing in context of the discussion of providing
server-side {{DESCRIBE}}, that is, not cqlsh {{DESCRIBE}}) support. The
virtual table approach is something that there is an available patch for that
works perfectly fine and will meet a lot of use cases, whereas server-side
{{DESCRIBE}} does not yet exist and may require more discussion. Given that
and that the virtual table implementation is rather concise(300 lines), I felt
it pragmatic to consider doing both.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14825) Expose table schema for drivers
[
https://issues.apache.org/jira/browse/CASSANDRA-14825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923720#comment-16923720
]
Andy Tolbert edited comment on CASSANDRA-14825 at 9/5/19 8:48 PM:
--
{quote}
I think you misunderstood Robert Stupp's comment above. He's not saying we
should stick to cqlsh-based DESCRIBE, he's mentioning the alternative to the
virtual table approach of having DESCRIBE being a genuine CQL (server-side)
command, one that would return result sets (and that I describe in a number of
comments above).
{quote}
I understand what he's referencing in context of the discussion of providing
server-side {{DESCRIBE}}, that is, not cqlsh {{DESCRIBE}} support. The virtual
table approach is something that there is an available patch for that works
perfectly fine and will meet a lot of use cases, whereas server-side
{{DESCRIBE}} does not yet exist and may require more discussion. Given that
and that the virtual table implementation is rather concise(300 lines), I felt
it pragmatic to consider doing both.
was (Author: andrew.tolbert):
{quote}
I think you misunderstood Robert Stupp's comment above. He's not saying we
should stick to cqlsh-based DESCRIBE, he's mentioning the alternative to the
virtual table approach of having DESCRIBE being a genuine CQL (server-side)
command, one that would return result sets (and that I describe in a number of
comments above).
{quote}
I understand what he's referencing in context of the discussion of providing
server-side {{DESCRIBE}}, that is, not cqlsh {{DESCRIBE}}) support. The
virtual table approach is something that there is an available patch for that
works perfectly fine and will meet a lot of use cases, whereas server-side
{{DESCRIBE}} does not yet exist and may require more discussion. Given that
and that the virtual table implementation is rather concise(300 lines), I felt
it pragmatic to consider doing both.
> Expose table schema for drivers
> ---
>
> Key: CASSANDRA-14825
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14825
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/CQL
>Reporter: Chris Lohfink
>Assignee: Chris Lohfink
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently the drivers recreate the CQL for the tables by putting together the
> system table values. This is very difficult to keep up to date and buggy
> enough that its only even supported in Java and Python drivers. Cassandra
> already has some limited output available for snapshots that we could provide
> in a virtual table or new query that the drivers can fetch. This can greatly
> reduce the complexity of drivers while also reducing bugs like
> CASSANDRA-14822 as the underlying schema and properties change.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14655) Upgrade C* to use latest guava (27.0)
[ https://issues.apache.org/jira/browse/CASSANDRA-14655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936222#comment-16936222 ] Andy Tolbert commented on CASSANDRA-14655: -- (y) thanks for bringing this back into light, would be really nice to get this into 4.0 > Upgrade C* to use latest guava (27.0) > - > > Key: CASSANDRA-14655 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14655 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Low > Labels: 4.0-feature-freeze-review-requested > Fix For: 4.x > > > C* currently uses guava 23.3. This JIRA is about changing C* to use latest > guava (26.0). Originated from a discussion in the mailing list. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Tolbert updated CASSANDRA-10190: - Attachment: 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch 0001-Update-six-to-1.12.0.patch > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16945585#comment-16945585
]
Andy Tolbert commented on CASSANDRA-10190:
--
Hi [~ptbannister], this is much needed and great work! I took a look at your
branch today and it looks really good to me. I have some minor suggestions, but
I think things are pretty good as is!
I made patches of my suggestions below ([^0001-Update-six-to-1.12.0.patch],
[^0002-Simplify-version-specific-logic-by-using-six.moves-a.patch]). Note that
other than the import change in {{winpty.py}}, these changes don't really fix
anything, so I would not mind if you chose not to accept them.
Recommended changes:
{{six}}
* I realized that a very old version of six (1.7.3) is being provided via lib
as the cassandra-driver being pulled in claims to require six 1.9.0. We should
consider upgrading that, especially since there are some newer features
(covered below) that may be useful.
* There are some additional features of six that could be used which would
simplify/remove python version checking logic. For example, there's a lot of
code that calls encode/decode if PY2 is used, {{six.ensure_text}} could be used
instead.
{{bin/cqlsh}}
* L85-95: Commented out code probably not needed anymore?
{{bin/cqlsh.py}}
* L50-56: Could make use of
[six.moves|https://six.readthedocs.io/#module-six.moves] to impor in a
version-agnostic way, i.e. from {{six.moves import configparser}}, {{from
six.moves import cStringIO as StringIO}})
* L867: Instead of {{_input}}, could use {{six.moves.input}}.
{{pylib/copyutil.py}}
* L48-54: use {{six.moves}} (see comment ^ {{cqlsh.py}})
* where {{range}} is used in favor of {{xrange}}, it would be good to use
{{six.moves.range}} instead, since when using python2 {{xrange}} will be used
so the existing behavior for python2 will be maintained.
{{pylib/cqlshlib/formatting.py}}
* {{ensure_text}} can be utilized in a places to simplify conversion logic.
{{pylib/cqlshlib/saferscanner.py}}
* Unless i'm mistaken {{Py35SaferScanner}} and {{Py36SaferScanner}} are the
same, {{Py35SaferScanner}} not used, so can be removed.
{{pylib/cqlshlib/sslhandling.py}}
* L21-24, use {{six.moves}}
{{pylib/cqlshlib/test/winpty.py}} & {{pylib/cqlshlib/util.py}}
* import queue won't work with python2, & might as well use cStringIO
The tests all passed for me except the one you mentioned with python3
(TestCqlshOutput#test_user_types). I’m pretty certain this is fixed in newer
python driver versions. The driver bundled with C* is several minor versions
old and is an internal build with some changes for transient replicas (I’m
guessing has to do with the way you express replication when creating a
keyspace with transient replicas). I think it's worth updating the driver to a
newer version, and maybe that can be done in a separate patch unless you think
it should be done here?
> Python 3 support for cqlsh
> --
>
> Key: CASSANDRA-10190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10190
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools
>Reporter: Andrew Pennebaker
>Assignee: Patrick Bannister
>Priority: Normal
> Labels: cqlsh
> Fix For: 4.0, 4.0-alpha
>
> Attachments: 0001-Update-six-to-1.12.0.patch,
> 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch,
> coverage_notes.txt
>
>
> Users who operate in a Python 3 environment may have trouble launching cqlsh.
> Could we please update cqlsh's syntax to run in Python 3?
> As a workaround, users can setup pyenv, and cd to a directory with a
> .python-version containing "2.7". But it would be nice if cqlsh supported
> modern Python versions out of the box.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16945587#comment-16945587
]
Andy Tolbert commented on CASSANDRA-10190:
--
{quote}
I think it's worth updating the driver to a newer version
{quote}
I just read the comments earlier in the thread about the driver, mainly this
one:
{quote}
Finally, extracting the checked in driver is something we'll look into. But
that is a longpole and strictly not needed for Python 3 compatibility.
{quote}
Agree with that, don't think that should hold things up, it can be done out of
band. Noticed that I had already opened up a ticket of updating the python
driver a while back for a different reason ([CASSANDRA-14872]), and there's
probably other 4.0 specific reasons it needs to be updated, so would be good to
do that separately.
> Python 3 support for cqlsh
> --
>
> Key: CASSANDRA-10190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10190
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools
>Reporter: Andrew Pennebaker
>Assignee: Patrick Bannister
>Priority: Normal
> Labels: cqlsh
> Fix For: 4.0, 4.0-alpha
>
> Attachments: 0001-Update-six-to-1.12.0.patch,
> 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch,
> coverage_notes.txt
>
>
> Users who operate in a Python 3 environment may have trouble launching cqlsh.
> Could we please update cqlsh's syntax to run in Python 3?
> As a workaround, users can setup pyenv, and cd to a directory with a
> .python-version containing "2.7". But it would be nice if cqlsh supported
> modern Python versions out of the box.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14872) Update to version of python driver and update cqlsh to use driver metadata for virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16945588#comment-16945588
]
Andy Tolbert commented on CASSANDRA-14872:
--
Newer versions of the python driver also fix some issues around python3, so
upgrading would also benefit [CASSANDRA-10190].
> Update to version of python driver and update cqlsh to use driver metadata
> for virtual tables
> -
>
> Key: CASSANDRA-14872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14872
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL, Legacy/Tools
>Reporter: Andy Tolbert
>Priority: Normal
>
> When virtual tables were implemented ([CASSANDRA-7622]), cqlsh.py was updated
> to parse virtual keyspace metadata by making queries to the
> {{system_virtual_schema}} table and included a TODO:
> {code:python}
> # TODO remove after virtual tables are added to connection metadata
> {code}
> Since python driver 3.15.0 (released in August), the driver now parses
> virtual keyspace metadata. It would be good to update the bundled python
> driver and simplify cqlsh code to utilize its capability to parse virtual
> tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-10190) Python 3 support for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16945587#comment-16945587
]
Andy Tolbert edited comment on CASSANDRA-10190 at 10/7/19 4:58 AM:
---
{quote}
I think it's worth updating the driver to a newer version
{quote}
I just read the comments earlier in the thread about the driver, mainly this
one:
{quote}
Finally, extracting the checked in driver is something we'll look into. But
that is a longpole and strictly not needed for Python 3 compatibility.
{quote}
Agree with that, don't think that should hold things up, it can be done out of
band. Noticed that I had already opened up a ticket for updating the python
driver a while back for a different reason ([CASSANDRA-14872]), and there's
probably other 4.0 specific reasons it needs to be updated, so would be good to
do that separately.
was (Author: andrew.tolbert):
{quote}
I think it's worth updating the driver to a newer version
{quote}
I just read the comments earlier in the thread about the driver, mainly this
one:
{quote}
Finally, extracting the checked in driver is something we'll look into. But
that is a longpole and strictly not needed for Python 3 compatibility.
{quote}
Agree with that, don't think that should hold things up, it can be done out of
band. Noticed that I had already opened up a ticket of updating the python
driver a while back for a different reason ([CASSANDRA-14872]), and there's
probably other 4.0 specific reasons it needs to be updated, so would be good to
do that separately.
> Python 3 support for cqlsh
> --
>
> Key: CASSANDRA-10190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10190
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools
>Reporter: Andrew Pennebaker
>Assignee: Patrick Bannister
>Priority: Normal
> Labels: cqlsh
> Fix For: 4.0, 4.0-alpha
>
> Attachments: 0001-Update-six-to-1.12.0.patch,
> 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch,
> coverage_notes.txt
>
>
> Users who operate in a Python 3 environment may have trouble launching cqlsh.
> Could we please update cqlsh's syntax to run in Python 3?
> As a workaround, users can setup pyenv, and cd to a directory with a
> .python-version containing "2.7". But it would be nice if cqlsh supported
> modern Python versions out of the box.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946196#comment-16946196
]
Andy Tolbert commented on CASSANDRA-10190:
--
Hi [~djoshi],interesting! The 2 new failures seem to be related to copy code,
so it's probably a regression from my commit. Nothing sticks out as obvious to
me as well at the moment, I'll take a look tonight and will get to the bottom
of it. Will also see if I can wrap my head around the {{test_describe}} test.
> Python 3 support for cqlsh
> --
>
> Key: CASSANDRA-10190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10190
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools
>Reporter: Andrew Pennebaker
>Assignee: Patrick Bannister
>Priority: Normal
> Labels: cqlsh
> Fix For: 4.0, 4.0-alpha
>
> Attachments: 0001-Update-six-to-1.12.0.patch,
> 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch,
> coverage_notes.txt
>
>
> Users who operate in a Python 3 environment may have trouble launching cqlsh.
> Could we please update cqlsh's syntax to run in Python 3?
> As a workaround, users can setup pyenv, and cd to a directory with a
> .python-version containing "2.7". But it would be nice if cqlsh supported
> modern Python versions out of the box.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946544#comment-16946544
]
Andy Tolbert commented on CASSANDRA-10190:
--
I see the problem, in {{make_params}}, calling {{ensure_text(query)}} will fail
if using prepared statements, since you aren't operating on a string, but
rather a prepared statement, this causes an unhandled exception. Instead we
should only call {{ensure_text}} if not using a prepared statement.
In addition, I discovered a couple other issues with my commit which I'll
provide fixes for shortly.
> Python 3 support for cqlsh
> --
>
> Key: CASSANDRA-10190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10190
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools
>Reporter: Andrew Pennebaker
>Assignee: Patrick Bannister
>Priority: Normal
> Labels: cqlsh
> Fix For: 4.0, 4.0-alpha
>
> Attachments: 0001-Update-six-to-1.12.0.patch,
> 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch,
> coverage_notes.txt
>
>
> Users who operate in a Python 3 environment may have trouble launching cqlsh.
> Could we please update cqlsh's syntax to run in Python 3?
> As a workaround, users can setup pyenv, and cd to a directory with a
> .python-version containing "2.7". But it would be nice if cqlsh supported
> modern Python versions out of the box.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy Tolbert updated CASSANDRA-10190: - Attachment: 0001-Fix-issues-from-version-specific-logic-commit.patch > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946590#comment-16946590
]
Andy Tolbert commented on CASSANDRA-10190:
--
Added [^0001-Fix-issues-from-version-specific-logic-commit.patch] which fixes
issues from my previous commit and should cause the 2 new failing tests to now
pass. There's still a little more testing I'd like to do though, from the
[{{cqlshlib6-rebase-20190322}}|https://github.com/ptbannister/cassandra-dtest/commits/cqlshlib6-rebase-20190322]
branch, other than the virtual tables related failures, there's one test that
fails because a unicode type instead of str type is given:
{noformat}
TestCqlsh.test_materialized_view
...
> assert "Materialized view 'users_by_state' not found" in err
E assert "Materialized view 'users_by_state' not found" in
":2:Materialized view u'users_by_state' not found\n"
{noformat}
> Python 3 support for cqlsh
> --
>
> Key: CASSANDRA-10190
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10190
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Tools
>Reporter: Andrew Pennebaker
>Assignee: Patrick Bannister
>Priority: Normal
> Labels: cqlsh
> Fix For: 4.0, 4.0-alpha
>
> Attachments:
> 0001-Fix-issues-from-version-specific-logic-commit.patch,
> 0001-Update-six-to-1.12.0.patch,
> 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch,
> coverage_notes.txt
>
>
> Users who operate in a Python 3 environment may have trouble launching cqlsh.
> Could we please update cqlsh's syntax to run in Python 3?
> As a workaround, users can setup pyenv, and cd to a directory with a
> .python-version containing "2.7". But it would be nice if cqlsh supported
> modern Python versions out of the box.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
