[jira] [Comment Edited] (CASSANDRA-11424) Option to leave omitted columns in INSERT JSON unset
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382792#comment-15382792 ] Oded Peer edited comment on CASSANDRA-11424 at 7/18/16 6:34 PM: Thank you for the review Sylvain. It's a great learning experience for me and it will help me do better on my next patch - I wasn't aware of the of the .rst documentation in the source tree. I was glad to learn the syntax in your patch for {code}( { defaultUnset = true; } K_UNSET) ){code}Not knowing this lead to changes in the QueryOptions which is far less elegant than what you propose. Of course I am happy with it. was (Author: odpeer): Thank you for the review Sylvain. It's a great learning experience for me and it will help me do better on my next patch - I wasn't aware of the of the .rst documentation in the source tree. I was glad to learn the syntax in your patch for {code}( { defaultUnset = true; } K_UNSET) ){code}. Not knowing this lead to changes in the QueryOptions which is far less elegant than what you propose. Of course I am happy with it. > Option to leave omitted columns in INSERT JSON unset > > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher >Assignee: Oded Peer > Labels: client-impacting, cql > Fix For: 3.8 > > Attachments: 11424-trunk-V1.txt, 11424-trunk-V2.txt, > 11424-trunk-V3.txt > > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11424) Option to leave omitted columns in INSERT JSON unset
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382792#comment-15382792 ] Oded Peer edited comment on CASSANDRA-11424 at 7/18/16 6:34 PM: Thank you for the review Sylvain. It's a great learning experience for me and it will help me do better on my next patch - I wasn't aware of the of the .rst documentation in the source tree. I was glad to learn the syntax in your patch for {code}( { defaultUnset = true; } K_UNSET) ){code}. Not knowing this lead to changes in the QueryOptions which is far less elegant than what you propose. Of course I am happy with it. was (Author: odpeer): Thank you for the review Sylvain. It's a great learning experience for me and it will help me do better on my next patch - I wasn't aware of the of the .rst documentation in the source tree. I was glad to learn the syntax in your patch for {{ ( { defaultUnset = true; } K_UNSET) }}. Not knowing this lead to changes in the QueryOptions which is far less elegant than what you propose. Of course I am happy with it. > Option to leave omitted columns in INSERT JSON unset > > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher >Assignee: Oded Peer > Labels: client-impacting, cql > Fix For: 3.8 > > Attachments: 11424-trunk-V1.txt, 11424-trunk-V2.txt, > 11424-trunk-V3.txt > > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11424) Option to leave omitted columns in INSERT JSON unset
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382792#comment-15382792 ] Oded Peer commented on CASSANDRA-11424: --- Thank you for the review Sylvain. It's a great learning experience for me and it will help me do better on my next patch - I wasn't aware of the of the .rst documentation in the source tree. I was glad to learn the syntax in your patch for {{ ( { defaultUnset = true; } K_UNSET) }}. Not knowing this lead to changes in the QueryOptions which is far less elegant than what you propose. Of course I am happy with it. > Option to leave omitted columns in INSERT JSON unset > > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher >Assignee: Oded Peer > Labels: client-impacting, cql > Fix For: 3.8 > > Attachments: 11424-trunk-V1.txt, 11424-trunk-V2.txt, > 11424-trunk-V3.txt > > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-11424: -- Attachment: 11424-trunk-V3.txt Thank you for the quick feedback. I see your point. I took the Jira description too literally. I've fixed it and uploaded patch V3. > Add support to "unset" JSON fields in prepared statements > - > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher >Assignee: Oded Peer > Labels: client-impacting, cql > Fix For: 3.8 > > Attachments: 11424-trunk-V1.txt, 11424-trunk-V2.txt, > 11424-trunk-V3.txt > > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-11424: -- Attachment: 11424-trunk-V2.txt Handle 'mixed mode' of explicitly setting a value to null while leaving remaining values default unset > Add support to "unset" JSON fields in prepared statements > - > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher >Assignee: Oded Peer > Labels: client-impacting, cql > Fix For: 3.8 > > Attachments: 11424-trunk-V1.txt, 11424-trunk-V2.txt > > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-11424: -- Fix Version/s: 3.8 Status: Patch Available (was: Open) > Add support to "unset" JSON fields in prepared statements > - > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher >Assignee: Oded Peer > Labels: client-impacting, cql > Fix For: 3.8 > > Attachments: 11424-trunk-V1.txt > > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-11424: -- Attachment: 11424-trunk-V1.txt > Add support to "unset" JSON fields in prepared statements > - > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher >Assignee: Oded Peer > Labels: client-impacting, cql > Attachments: 11424-trunk-V1.txt > > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer reassigned CASSANDRA-11424: - Assignee: Oded Peer > Add support to "unset" JSON fields in prepared statements > - > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher >Assignee: Oded Peer > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364387#comment-15364387 ] Oded Peer commented on CASSANDRA-11424: --- It is rather confusing. As a Cassandra user I would rather have an explicit option. As a Cassandra developer I feel fine with adding new syntax to enhance the new syntax added for JSON. > Add support to "unset" JSON fields in prepared statements > - > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15364037#comment-15364037 ] Oded Peer commented on CASSANDRA-11424: --- [~blerer] you are the CQL expert, what do you think of my proposal above? > Add support to "unset" JSON fields in prepared statements > - > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15277970#comment-15277970 ] Oded Peer commented on CASSANDRA-11424: --- [~thobbs] What do you think of my previous proposal? {noformat}INSERT INTO ks.tab JSON IGNORE_OMITTED '{"k" : "v"}'{noformat} > Add support to "unset" JSON fields in prepared statements > - > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Improvement >Reporter: Ralf Steppacher >Assignee: Tyler Hobbs > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11424) Add support to "unset" JSON fields in prepared statements
[ https://issues.apache.org/jira/browse/CASSANDRA-11424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210140#comment-15210140 ] Oded Peer commented on CASSANDRA-11424: --- Since new syntax was added for JSON select and insert support I feel comfortable adding new constructs for the new syntax. Therefore I suggest adding a new keyword to the JSON insert statement adding an optional keyword "IGNORE_OMITTED" which controls the effect of omitted columns. Insert statements with IGNORE_OMITTED will not create tombstones for omitted columns. > Add support to "unset" JSON fields in prepared statements > - > > Key: CASSANDRA-11424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11424 > Project: Cassandra > Issue Type: Bug >Reporter: Ralf Steppacher > > CASSANDRA-7304 introduced the ability to distinguish between {{NULL}} and > {{UNSET}} prepared statement parameters. > When inserting JSON objects it is not possible to profit from this as a > prepared statement only has one parameter that is bound to the JSON object as > a whole. There is no way to control {{NULL}} vs {{UNSET}} behavior for > columns omitted from the JSON object. > Please extend on CASSANDRA-7304 to include JSON support. > {color:grey} > (My personal requirement is to be able to insert JSON objects with optional > fields without incurring the overhead of creating a tombstone of every column > not covered by the JSON object upon initial(!) insert.) > {color} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8492) Support "IF NOT EXISTS" for "ALTER TABLE ADD COLUMN"
[ https://issues.apache.org/jira/browse/CASSANDRA-8492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14552011#comment-14552011 ] Oded Peer commented on CASSANDRA-8492: -- [~Sachin Janani] Can you add a unit test for your change to AlterTableTest? > Support "IF NOT EXISTS" for "ALTER TABLE ADD COLUMN" > > > Key: CASSANDRA-8492 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8492 > Project: Cassandra > Issue Type: Improvement >Reporter: Peter Mädel >Priority: Minor > Attachments: 8492.patch > > > would enable creation of schema update scripts that can be repeatable > executed without having to worry about invalid query exceptions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7276) Include keyspace and table names in logs where possible
[ https://issues.apache.org/jira/browse/CASSANDRA-7276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526574#comment-14526574 ] Oded Peer commented on CASSANDRA-7276: -- I suggest adding context to the thread name, then it will be printed when exceptions occur without needing to explicitly printing it out when an exception occurs. It's described in http://blog.takipi.com/supercharged-jstack-how-to-debug-your-servers-at-100mph/ > Include keyspace and table names in logs where possible > --- > > Key: CASSANDRA-7276 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7276 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Tyler Hobbs >Assignee: Nitzan Volman >Priority: Minor > Labels: bootcamp, lhf > Fix For: 2.1.x > > Attachments: 2.1-CASSANDRA-7276-v1.txt, > cassandra-2.1-7276-compaction.txt, cassandra-2.1-7276.txt > > > Most error messages and stacktraces give you no clue as to what keyspace or > table was causing the problem. For example: > {noformat} > ERROR [MutationStage:61648] 2014-05-20 12:05:45,145 CassandraDaemon.java > (line 198) Exception in thread Thread[MutationStage:61648,5,main] > java.lang.IllegalArgumentException > at java.nio.Buffer.limit(Unknown Source) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:63) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:72) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:98) > at > org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:35) > at > edu.stanford.ppl.concurrent.SnapTreeMap$1.compareTo(SnapTreeMap.java:538) > at > edu.stanford.ppl.concurrent.SnapTreeMap.attemptUpdate(SnapTreeMap.java:1108) > at > edu.stanford.ppl.concurrent.SnapTreeMap.updateUnderRoot(SnapTreeMap.java:1059) > at edu.stanford.ppl.concurrent.SnapTreeMap.update(SnapTreeMap.java:1023) > at > edu.stanford.ppl.concurrent.SnapTreeMap.putIfAbsent(SnapTreeMap.java:985) > at > org.apache.cassandra.db.AtomicSortedColumns$Holder.addColumn(AtomicSortedColumns.java:328) > at > org.apache.cassandra.db.AtomicSortedColumns.addAllWithSizeDelta(AtomicSortedColumns.java:200) > at org.apache.cassandra.db.Memtable.resolve(Memtable.java:226) > at org.apache.cassandra.db.Memtable.put(Memtable.java:173) > at > org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:893) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:368) > at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:333) > at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:206) > at > org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:56) > at > org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:60) > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) > at java.lang.Thread.run(Unknown Source) > {noformat} > We should try to include info on the keyspace and column family in the error > messages or logs whenever possible. This includes reads, writes, > compactions, flushes, repairs, and probably more. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8755) Replace trivial uses of String.replace/replaceAll/split with StringUtils methods
[ https://issues.apache.org/jira/browse/CASSANDRA-8755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526566#comment-14526566 ] Oded Peer commented on CASSANDRA-8755: -- Java's String.split() method has a fast-path for single character input, avoiding costly regexp creation. See http://stackoverflow.com/a/11002374/248656. > Replace trivial uses of String.replace/replaceAll/split with StringUtils > methods > > > Key: CASSANDRA-8755 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8755 > Project: Cassandra > Issue Type: Improvement >Reporter: Jaroslav Kamenik >Priority: Trivial > Labels: lhf > Attachments: trunk-8755.patch > > > There are places in the code where those regex based methods are used with > plain, not regexp, strings, so StringUtils alternatives should be faster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Attachment: 7304-07.patch Right. Fixed in 7304-07.patch > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-05.patch, > 7304-06.patch, 7304-07.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion > *EDIT 18/12/14 - [~odpeer] Implementation Notes:* > The motivation hasn't changed. > Protocol version 4 specifies that bind variables do not require having a > value when executing a statement. Bind variables without a value are called > 'unset'. The 'unset' bind variable is serialized as the int value '-2' > without following bytes. > \\ > \\ > * An unset bind variable in an EXECUTE or BATCH request > ** On a {{value}} does not modify the value and does not create a tombstone > ** On the {{ttl}} clause is treated as 'unlimited' > ** On the {{timestamp}} clause is treated as 'now' > ** On a map key or a list index throws {{InvalidRequestException}} > ** On a {{counter}} increment or decrement operation does not change the > counter value, e.g. {{UPDATE my_tab SET c = c - ? WHERE k = 1}} does change > the value of counter {{c}} > ** On a tuple field or UDT field throws {{InvalidRequestException}} > * An unset bind variable in a QUERY request > ** On a partition column, clustering column or index column in the {{WHERE}} > clause throws {{InvalidRequestException}} > ** On the {{limit}} clause is treated as 'unlimited' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Attachment: 7304-06.patch Added a new patch 7304-06.patch {quote} There are a few use cases that are not tested and that seems to not work properly: List marker: SELECT * FROM %s WHERE k in ? or SELECT * FROM %s WHERE k = ? AND i IN ? {quote} Fixed. bq. Tuple marker: SELECT * FROM %s WHERE k = ? and (i, j) = ? Fixed. bq. Collection marker containing an unset value. For example: INSERT INTO %s (k, m) VALUES (10, ?) where the value for the map m will be map("k", unset()) The value of the map entry can be an unset value only in testing and internal code, It can't happen from client code. "unset" variables only applies to bind variables in a CQL query. A CQL client can not create an "unset" ByteBuffer as a map value since it is not a bound value. bq. Queries with CONTAINS or CONTAINS KEY conditions Added tests. {quote} There is also a few use cases that are not tested and that I have not tried: Secondary index queries on collection key or value with unset values {quote} Added tests. bq. UPDATE or DELETE queries with unset values in the WHERE clause Added tests. {quote} Nested tulpe with unset values It looks like you missed the following remark from Sylvain In ModificationStatement.executeInternal, the body of the for loop should just be replaced by mutation.apply(). {quote} I didn't replace the for loop body since the {{apply()}} method is not in the {{IMutation}} interface. The {{apply()}} method signature is different in {{Mutation}} and {{CounterMutation}}, one is {{void}} while the other returns a {{Mutation}} instance. I chose to leave it as-is. bq. I do not understand your change in FunctionCall. We cannot know if some function can accept null or not as somebody can create a UDF for which null is a valid input. For unset value, we need to block them in FunctionCall as the existing functions will break otherwise. Since functions do not accept bind variables as input, only column identifiers, and A column value can not be an unset value. I added a comment to {{FunctionCall}} stating why there is no need in checking for unset variables in functions. {quote} The error messages for tuples and UDT do not provides enough information if you have multiple of them in the query (e.g. SELECT * FROM myTable WHERE a = 0 AND (b, c) = (?, ?) AND (d, e) > (?, ?)). I am not sure how we could provide a better message but you might be able to find a way? At least for UDT we should provide the type name in the error message. {quote} I added more information to the bind marker position, and added a test. bq. In Sets.Adder.doAdd, Lists.Appender.doAppend and Maps.Appender.doAppend there are some unused variable. Done. bq. In Sets, Lists, Maps and Constants there are several place where you use some unecessary else. The if either end by a return or by throwing an Exception. I think it's a matter of taste. I changed it and remvoed the unecessary else. bq. I would be in favor to put the tests in the corresponding unit tests rather than in a new one. For example I will put the collection tests in CollectionsTest. I believe that it will help people if all the tests for a collections for example are together. It can serve as a form of documentation. Done. I moved tests to CollectionsTest, TupleTypeTest, UserTypesTest. bq. You should add the feature to the News.txt Done. bq. There are still a lot of whitespaces in your patch. Fixed. > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-05.patch, > 7304-06.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombs
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484950#comment-14484950 ] Oded Peer commented on CASSANDRA-7304: -- [~blerer] Pinging for code review > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-05.patch, > 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion > *EDIT 18/12/14 - [~odpeer] Implementation Notes:* > The motivation hasn't changed. > Protocol version 4 specifies that bind variables do not require having a > value when executing a statement. Bind variables without a value are called > 'unset'. The 'unset' bind variable is serialized as the int value '-2' > without following bytes. > \\ > \\ > * An unset bind variable in an EXECUTE or BATCH request > ** On a {{value}} does not modify the value and does not create a tombstone > ** On the {{ttl}} clause is treated as 'unlimited' > ** On the {{timestamp}} clause is treated as 'now' > ** On a map key or a list index throws {{InvalidRequestException}} > ** On a {{counter}} increment or decrement operation does not change the > counter value, e.g. {{UPDATE my_tab SET c = c - ? WHERE k = 1}} does change > the value of counter {{c}} > ** On a tuple field or UDT field throws {{InvalidRequestException}} > * An unset bind variable in a QUERY request > ** On a partition column, clustering column or index column in the {{WHERE}} > clause throws {{InvalidRequestException}} > ** On the {{limit}} clause is treated as 'unlimited' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Attachment: 7304-05.patch Rebased to trunk > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-05.patch, > 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion > *EDIT 18/12/14 - [~odpeer] Implementation Notes:* > The motivation hasn't changed. > Protocol version 4 specifies that bind variables do not require having a > value when executing a statement. Bind variables without a value are called > 'unset'. The 'unset' bind variable is serialized as the int value '-2' > without following bytes. > \\ > \\ > * An unset bind variable in an EXECUTE or BATCH request > ** On a {{value}} does not modify the value and does not create a tombstone > ** On the {{ttl}} clause is treated as 'unlimited' > ** On the {{timestamp}} clause is treated as 'now' > ** On a map key or a list index throws {{InvalidRequestException}} > ** On a {{counter}} increment or decrement operation does not change the > counter value, e.g. {{UPDATE my_tab SET c = c - ? WHERE k = 1}} does change > the value of counter {{c}} > ** On a tuple field or UDT field throws {{InvalidRequestException}} > * An unset bind variable in a QUERY request > ** On a partition column, clustering column or index column in the {{WHERE}} > clause throws {{InvalidRequestException}} > ** On the {{limit}} clause is treated as 'unlimited' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14383897#comment-14383897 ] Oded Peer commented on CASSANDRA-7304: -- Yes. I will rebase next week. Thanks > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion > *EDIT 18/12/14 - [~odpeer] Implementation Notes:* > The motivation hasn't changed. > Protocol version 4 specifies that bind variables do not require having a > value when executing a statement. Bind variables without a value are called > 'unset'. The 'unset' bind variable is serialized as the int value '-2' > without following bytes. > \\ > \\ > * An unset bind variable in an EXECUTE or BATCH request > ** On a {{value}} does not modify the value and does not create a tombstone > ** On the {{ttl}} clause is treated as 'unlimited' > ** On the {{timestamp}} clause is treated as 'now' > ** On a map key or a list index throws {{InvalidRequestException}} > ** On a {{counter}} increment or decrement operation does not change the > counter value, e.g. {{UPDATE my_tab SET c = c - ? WHERE k = 1}} does change > the value of counter {{c}} > ** On a tuple field or UDT field throws {{InvalidRequestException}} > * An unset bind variable in a QUERY request > ** On a partition column, clustering column or index column in the {{WHERE}} > clause throws {{InvalidRequestException}} > ** On the {{limit}} clause is treated as 'unlimited' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7216) Restricted superuser account request
[ https://issues.apache.org/jira/browse/CASSANDRA-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377554#comment-14377554 ] Oded Peer commented on CASSANDRA-7216: -- [~beobal], I read your description of Roles (http://www.datastax.com/dev/blog/role-based-access-control-in-cassandra). Can you please explain how the following use case is solved using roles? It would serve a good recipe for securing multi-tenant applications. During installation: 1. "superuser" creates "provsioining_user" with permissions to create users and to create keyspaces. "provsioining_user" does not have SELECT permissions on any external keyspaces. 2. Delete "superuser". For security reasons we don't want a superuser in the system. Provision a new tenant: 1. "provsioining_user" creates a keyspace and "tenant_user" for every new tenant, e.g. create keyspace "acme" and "acme_user". 2. "provsioining_user" grants permissions to "tenant_user" to create/drop/alter tables (only tables, not keyspaces), MODIFY and SELECT on the tenant keyspace. I think the Roles should be refined to control this. 3. "tenant_user" creates the tables and indexes for application use. The only Cassandra user that can create/drop/alter/query/modify tables in the tenant keyspace is "tenant_user" > Restricted superuser account request > > > Key: CASSANDRA-7216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7216 > Project: Cassandra > Issue Type: Sub-task >Reporter: Oded Peer >Assignee: Sam Tunnicliffe >Priority: Minor > Fix For: 3.0 > > Attachments: 7216-8650.txt, 7216-POC.txt, 7216.txt > > > I am developing a multi-tenant service. > Every tenant has its own user, keyspace and can access only his keyspace. > As new tenants are provisioned there is a need to create new users and > keyspaces. > Only a superuser can issue CREATE USER requests, so we must have a super user > account in the system. On the other hand super users have access to all the > keyspaces, which poses a security risk. > For tenant provisioning I would like to have a restricted account which can > only create new users, without read access to keyspaces. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14255054#comment-14255054 ] Oded Peer commented on CASSANDRA-7304: -- bq. Will this generate a tombstone because I didn't set the column C? No. This won't create a tombstone. bq. this problem comes from the java driver, but since is a protocol issue I assume is affecting all right? Right, assuming the driver you are using supports prepared statements and does not enforce setting concrete values for all the bind variables. The Java driver [enforces having concrete values for every bind variable|https://datastax-oss.atlassian.net/browse/JAVA-410] as of driver version 2.1.0 bq. the query above *without prepared* statements is not going to generate any tombstone right? Both queries, prepared and simple won't create tombstones. bq. Can some of you provide examples of cases where we can generate tombstones using prepared statements? When you execute the following statement {{INSERT INTO test (A, B) VALUES (1, ?)}} and you do not set a value for the bind variable then Cassandra will set the value of {{B}} to null and create a tombstone for the column. > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion > *EDIT 18/12/14 - [~odpeer] Implementation Notes:* > The motivation hasn't changed. > Protocol version 4 specifies that bind variables do not require having a > value when executing a statement. Bind variables without a value are called > 'unset'. The 'unset' bind variable is serialized as the int value '-2' > without following bytes. > \\ > \\ > * An unset bind variable in an EXECUTE or BATCH request > ** On a {{value}} does not modify the value and does not create a tombstone > ** On the {{ttl}} clause is treated as 'unlimited' > ** On the {{timestamp}} clause is treated as 'now' > ** On a map key or a list index throws {{InvalidRequestException}} > ** On a {{counter}} increment or decrement operation does not change the > counter value, e.g. {{UPDATE my_tab SET c = c - ? WHERE k = 1}} does change > the value of counter {{c}} > ** On a tuple field or UDT field throws {{InvalidRequestException}} > * An unset bind variable in a QUERY request > ** On a partition column, clustering column or index column in the {{WHERE}} > clause throws {{InvalidRequestException}} > ** On the {{limit}} clause is treated as 'unlimited' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14255055#comment-14255055 ] Oded Peer commented on CASSANDRA-7304: -- bq. Will this generate a tombstone because I didn't set the column C? No. This won't create a tombstone. bq. this problem comes from the java driver, but since is a protocol issue I assume is affecting all right? Right, assuming the driver you are using supports prepared statements and does not enforce setting concrete values for all the bind variables. The Java driver [enforces having concrete values for every bind variable|https://datastax-oss.atlassian.net/browse/JAVA-410] as of driver version 2.1.0 bq. the query above *without prepared* statements is not going to generate any tombstone right? Both queries, prepared and simple won't create tombstones. bq. Can some of you provide examples of cases where we can generate tombstones using prepared statements? When you execute the following statement {{INSERT INTO test (A, B) VALUES (1, ?)}} and you do not set a value for the bind variable then Cassandra will set the value of {{B}} to null and create a tombstone for the column. > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion > *EDIT 18/12/14 - [~odpeer] Implementation Notes:* > The motivation hasn't changed. > Protocol version 4 specifies that bind variables do not require having a > value when executing a statement. Bind variables without a value are called > 'unset'. The 'unset' bind variable is serialized as the int value '-2' > without following bytes. > \\ > \\ > * An unset bind variable in an EXECUTE or BATCH request > ** On a {{value}} does not modify the value and does not create a tombstone > ** On the {{ttl}} clause is treated as 'unlimited' > ** On the {{timestamp}} clause is treated as 'now' > ** On a map key or a list index throws {{InvalidRequestException}} > ** On a {{counter}} increment or decrement operation does not change the > counter value, e.g. {{UPDATE my_tab SET c = c - ? WHERE k = 1}} does change > the value of counter {{c}} > ** On a tuple field or UDT field throws {{InvalidRequestException}} > * An unset bind variable in a QUERY request > ** On a partition column, clustering column or index column in the {{WHERE}} > clause throws {{InvalidRequestException}} > ** On the {{limit}} clause is treated as 'unlimited' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14252020#comment-14252020 ] Oded Peer commented on CASSANDRA-7304: -- If a driver sends a value of -2 and is using native protocol version 4 then this will avoid tombstones. If the protocol version will be less than 4 then tombstones will be created even when the value of -2 is set for backward compatibility. A driver that supports this differentiation can only be used after the patch is accepted and committed. > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion > *EDIT 18/12/14 - [~odpeer] Implementation Notes:* > The motivation hasn't changed. > Protocol version 4 specifies that bind variables do not require having a > value when executing a statement. Bind variables without a value are called > 'unset'. The 'unset' bind variable is serialized as the int value '-2' > without following bytes. > \\ > \\ > * An unset bind variable in an EXECUTE or BATCH request > ** On a {{value}} does not modify the value and does not create a tombstone > ** On the {{ttl}} clause is treated as 'unlimited' > ** On the {{timestamp}} clause is treated as 'now' > ** On a map key or a list index throws {{InvalidRequestException}} > ** On a {{counter}} increment or decrement operation does not change the > counter value, e.g. {{UPDATE my_tab SET c = c - ? WHERE k = 1}} does change > the value of counter {{c}} > ** On a tuple field or UDT field throws {{InvalidRequestException}} > * An unset bind variable in a QUERY request > ** On a partition column, clustering column or index column in the {{WHERE}} > clause throws {{InvalidRequestException}} > ** On the {{limit}} clause is treated as 'unlimited' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14251604#comment-14251604 ] Oded Peer commented on CASSANDRA-7304: -- [~0x6e6562] I updated the original summary. > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion > *EDIT 18/12/14 - [~odpeer] Implementation Notes:* > The motivation hasn't changed. > Protocol version 4 specifies that bind variables do not require having a > value when executing a statement. Bind variables without a value are called > 'unset'. The 'unset' bind variable is serialized as the int value '-2' > without following bytes. > \\ > \\ > * An unset bind variable in an EXECUTE or BATCH request > ** On a {{value}} does not modify the value and does not create a tombstone > ** On the {{ttl}} clause is treated as 'unlimited' > ** On the {{timestamp}} clause is treated as 'now' > ** On a map key or a list index throws {{InvalidRequestException}} > ** On a {{counter}} increment or decrement operation does not change the > counter value, e.g. {{UPDATE my_tab SET c = c - ? WHERE k = 1}} does change > the value of counter {{c}} > ** On a tuple field or UDT field throws {{InvalidRequestException}} > * An unset bind variable in a QUERY request > ** On a partition column, clustering column or index column in the {{WHERE}} > clause throws {{InvalidRequestException}} > ** On the {{limit}} clause is treated as 'unlimited' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Description: Currently Cassandra inserts tombstones when a value of a column is bound to NULL in a prepared statement. At higher insert rates managing all these tombstones becomes an unnecessary overhead. This limits the usefulness of the prepared statements since developers have to either create multiple prepared statements (each with a different combination of column names, which at times is just unfeasible because of the sheer number of possible combinations) or fall back to using regular (non-prepared) statements. This JIRA is here to explore the possibility of either: A. Have a flag on prepared statements that once set, tells Cassandra to ignore null columns or B. Have an "UNSET" value which makes Cassandra skip the null columns and not tombstone them Basically, in the context of a prepared statement, a null value means delete, but we don’t have anything that means "ignore" (besides creating a new prepared statement without the ignored column). Please refer to the original conversation on DataStax Java Driver mailing list for more background: https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion *EDIT 18/12/14 - [~odpeer] Implementation Notes:* The motivation hasn't changed. Protocol version 4 specifies that bind variables do not require having a value when executing a statement. Bind variables without a value are called 'unset'. The 'unset' bind variable is serialized as the int value '-2' without following bytes. \\ \\ * An unset bind variable in an EXECUTE or BATCH request ** On a {{value}} does not modify the value and does not create a tombstone ** On the {{ttl}} clause is treated as 'unlimited' ** On the {{timestamp}} clause is treated as 'now' ** On a map key or a list index throws {{InvalidRequestException}} ** On a {{counter}} increment or decrement operation does not change the counter value, e.g. {{UPDATE my_tab SET c = c - ? WHERE k = 1}} does change the value of counter {{c}} ** On a tuple field or UDT field throws {{InvalidRequestException}} * An unset bind variable in a QUERY request ** On a partition column, clustering column or index column in the {{WHERE}} clause throws {{InvalidRequestException}} ** On the {{limit}} clause is treated as 'unlimited' was: Currently Cassandra inserts tombstones when a value of a column is bound to NULL in a prepared statement. At higher insert rates managing all these tombstones becomes an unnecessary overhead. This limits the usefulness of the prepared statements since developers have to either create multiple prepared statements (each with a different combination of column names, which at times is just unfeasible because of the sheer number of possible combinations) or fall back to using regular (non-prepared) statements. This JIRA is here to explore the possibility of either: A. Have a flag on prepared statements that once set, tells Cassandra to ignore null columns or B. Have an "UNSET" value which makes Cassandra skip the null columns and not tombstone them Basically, in the context of a prepared statement, a null value means delete, but we don’t have anything that means "ignore" (besides creating a new prepared statement without the ignored column). Please refer to the original conversation on DataStax Java Driver mailing list for more background: https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone th
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Attachment: 7304-04.patch I appreciate your comments, they are very helpful and I am learning a lot from them. bq. In the spec, instead of changing the meaning of {{\[bytes\]}}, I would rather add a new {{\[value\]}} definition that support 'unset', and use that exclusively in the definition of values for bind variables in QUERY and EXECUTE messages, so as to make it clear that it makes no sense in any other place. I would then add a specific {{CBUtil.readBoundValue()}} to read those. Done bq. Making {{UNSET_CONSTANT_VALUE}} be {{new Value(null)}} is somewhat incorrect, it should be {{new Value(UNSET_BYTE_BUFFER)}} so that we don't lose the the information that it's 'unset' if {{bindAndGet}} is used. For this reason, I'd prefer using {{Constants.UNSET_CONSTANT_VALUE}} (renamed as {{UNSET_VALUE}}) in collections too (instead of adding {{Lists.UNSET_LIST_VALUE}}, ...). Done bq. We can't have an 'unset' value inside a collection since we don't allow bind markers in the first place, and so there is a bit of useless code/validation related to that. I verified the changes aren’t useless with unit tests. bq. There is a bunch of place that don't handle 'UNSET_BYTE_BUFFER' properly: Tuples, ColumnCondition (we might want to reject queries for which all conditions are 'unset' as going through the paxos code for no reason feels like the user is doing something wrong) and SelectStatement where we could get an 'unset' pretty much anywhere where the {{values()}} or {{bound()}} method of a {{Restriction}} is used (and validation might be tricky in SelectStatement: if we have {{SELECT * FROM foo WHERE k1 = ? AND k2 = ? AND k3 = ?}}, then we shouldn't accept an 'unset' for {{k2}} unless {{k3}} is also unset; note that I'd be fine just refusing 'unset' in selects for now to simplify, but we at least need the validation code to reject them). I opted for rejecting unset values in selects. It’s not only to simplify I think it’s the right thing to do. Having a variable assignment or a condition with unset variables is undefined. bq. I'd reject 'unset' indexes in {{UDPATE ... SET l\[?\] = ?}} since it's rejected for map keys. Unless maybe if both the key/index and value are 'unset', but that should be coherent for lists and maps. Done bq. In Constants.Marker.bindAndGet, we should skip validation if 'unset' (even though the validation will never fail because empty values are always accepted, it's still dodgy). Done bq. We should have separate error messages when we reject both {{null}} and {{unset}}. Done bq. I'd prefer rejecting 'unset' inside UDTs (and tuples). Making it equivalent to {{null}} gives it a different meaning than usual and we should avoid that. Done bq. For the limit in SelectStatement, it would make sense to accept unset and to have it mean "no limit" (instead of being rejected). The same applies for the timestamp and ttl in {{Attributes}}. Done bq. In CBUtil.readValue(), we should throw a ProtocolException instead of an IllegalArgumentException. Done bq. I might have put {{UNSET_BYTE_BUFFER}} in {{ByteBufferUtil}} since it's a {{ByteBuffer}}. Done bq. The patch appears to have windows end-of-line and a few weird indentations. Could you check that? My apologies. I switched to Ubuntu. bq. I'd have added an unset() in CQLTester to use in tests to make the tests terser. Done > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-04.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, >
[jira] [Commented] (CASSANDRA-4476) Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE)
[ https://issues.apache.org/jira/browse/CASSANDRA-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234764#comment-14234764 ] Oded Peer commented on CASSANDRA-4476: -- I understand. I created a test that demonstrates the issue. That's a really good catch on your behalf. I can't see a good way to query an index range and return the result in token order for paging. It might be done by fetching the entire table into memory and sorting all the rows by token value, but that's just wrong. Is it OK to close the issue as "won't fix"? > Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE) > > > Key: CASSANDRA-4476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4476 > Project: Cassandra > Issue Type: Improvement > Components: API, Core >Reporter: Sylvain Lebresne >Assignee: Oded Peer >Priority: Minor > Labels: cql > Fix For: 3.0 > > Attachments: 4476-2.patch, 4476-3.patch, 4476-5.patch, > cassandra-trunk-4476.patch > > > Currently, a query that uses 2ndary indexes must have at least one EQ clause > (on an indexed column). Given that indexed CFs are local (and use > LocalPartitioner that order the row by the type of the indexed column), we > should extend 2ndary indexes to allow querying indexed columns even when no > EQ clause is provided. > As far as I can tell, the main problem to solve for this is to update > KeysSearcher.highestSelectivityPredicate(). I.e. how do we estimate the > selectivity of non-EQ clauses? I note however that if we can do that estimate > reasonably accurately, this might provide better performance even for index > queries that both EQ and non-EQ clauses, because some non-EQ clauses may have > a much better selectivity than EQ ones (say you index both the user country > and birth date, for SELECT * FROM users WHERE country = 'US' AND birthdate > > 'Jan 2009' AND birtdate < 'July 2009', you'd better use the birthdate index > first). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-4476) Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE)
[ https://issues.apache.org/jira/browse/CASSANDRA-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14233307#comment-14233307 ] Oded Peer commented on CASSANDRA-4476: -- I am probably missing out something that is obvious to you. {quote}The problem is that (1, 6), (2, 6) are never going to be returned if you keep paging through the query in that fashion.{quote} Why would you need to return these values if you specified a limit of 3 in your query? When doing a range query over a secondary index a view of the index data is created in {{Collationcontroller.collectAllData()}} and iteration is done over the intervalTree of that view. If I understand correctly the values in the interval tree are the index values, not token, which ensures the iterator results are ordered. According to your comment paging is broken in this use case but the test I am running isn't failing. Do you expect the test to fail? > Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE) > > > Key: CASSANDRA-4476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4476 > Project: Cassandra > Issue Type: Improvement > Components: API, Core >Reporter: Sylvain Lebresne >Assignee: Oded Peer >Priority: Minor > Labels: cql > Fix For: 3.0 > > Attachments: 4476-2.patch, 4476-3.patch, 4476-5.patch, > cassandra-trunk-4476.patch > > > Currently, a query that uses 2ndary indexes must have at least one EQ clause > (on an indexed column). Given that indexed CFs are local (and use > LocalPartitioner that order the row by the type of the indexed column), we > should extend 2ndary indexes to allow querying indexed columns even when no > EQ clause is provided. > As far as I can tell, the main problem to solve for this is to update > KeysSearcher.highestSelectivityPredicate(). I.e. how do we estimate the > selectivity of non-EQ clauses? I note however that if we can do that estimate > reasonably accurately, this might provide better performance even for index > queries that both EQ and non-EQ clauses, because some non-EQ clauses may have > a much better selectivity than EQ ones (say you index both the user country > and birth date, for SELECT * FROM users WHERE country = 'US' AND birthdate > > 'Jan 2009' AND birtdate < 'July 2009', you'd better use the birthdate index > first). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-4476) Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE)
[ https://issues.apache.org/jira/browse/CASSANDRA-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-4476: - Attachment: 4476-5.patch [~blerer] Added 4476-5.patch I updated the code to scan a restricted range if there are two expressions on the same column. {quote}I think that you should use isRange or isSlice instead of isRelationalOrderOperator as it is clearer.{quote} I renamed the method. {quote}The name of test class: SecondaryIndexNonEqTest is misleading. CONTAINS an CONTAINS KEY operator are also non eq tests.{quote} I renamed the test. {quote}In getRelationalOrderEstimatedSize I do not understand why you do not return 0 if estimatedKeysForRange return 0. Could you explain?{quote} I added comments to the code since I think it should documented in the code and not in Jira. I hope it is understandable. {quote}Instead of doing some dangerous casting in getRelationalOrderEstimatedSize, you should change the type from bestMeanCount from int to long.{quote} I changed the type of bestMeanCount {quote}In computeNext I do not understand why you do not check for stale data for range queries? Could you explain?{quote} I added comments to the code. {quote}I think it would be nicer to have also an iterator for EQ and use polymorphism instead of if else.{quote} Generally I agree polymorphism is good practice, however I think in this case it would make the code less readable. {quote}The close method of the AbstractScanIterator returned by getSequentialIterator should be called from the close method.{quote} Thanks. I wasn't aware the iterator is closeable, they usually aren't. {quote}The Unit tests are only covering a subset of the possible queries. Could you add more (a > 3 and a <4, a < 3 and a > 4 ...){quote} Added. {quote}When testing for InvalidRequestException you should use assertInvalidMessage{quote} Thanks, I wanted to use such a method but couldn't find it on my own. [~jjordan] I don't understand the problem in your example. The query result seems valid to me. In addition, can you please explain how a query using only secondary indexes such as {{select k from my_table where index1 = 5 and index2 > 10 allow filtering}} retains token order? > Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE) > > > Key: CASSANDRA-4476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4476 > Project: Cassandra > Issue Type: Improvement > Components: API, Core >Reporter: Sylvain Lebresne >Assignee: Oded Peer >Priority: Minor > Labels: cql > Fix For: 3.0 > > Attachments: 4476-2.patch, 4476-3.patch, 4476-5.patch, > cassandra-trunk-4476.patch > > > Currently, a query that uses 2ndary indexes must have at least one EQ clause > (on an indexed column). Given that indexed CFs are local (and use > LocalPartitioner that order the row by the type of the indexed column), we > should extend 2ndary indexes to allow querying indexed columns even when no > EQ clause is provided. > As far as I can tell, the main problem to solve for this is to update > KeysSearcher.highestSelectivityPredicate(). I.e. how do we estimate the > selectivity of non-EQ clauses? I note however that if we can do that estimate > reasonably accurately, this might provide better performance even for index > queries that both EQ and non-EQ clauses, because some non-EQ clauses may have > a much better selectivity than EQ ones (say you index both the user country > and birth date, for SELECT * FROM users WHERE country = 'US' AND birthdate > > 'Jan 2009' AND birtdate < 'July 2009', you'd better use the birthdate index > first). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Attachment: (was: 4476-5.patch) > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Attachment: 4476-5.patch [~blerer] Added 4476-5.patch I updated the code to scan a restricted range if there are two expressions on the same column. {quote}I think that you should use isRange or isSlice instead of isRelationalOrderOperator as it is clearer.{quote} I renamed the method. {quote}The name of test class: SecondaryIndexNonEqTest is misleading. CONTAINS an CONTAINS KEY operator are also non eq tests.{quote} I renamed the test. {quote}In getRelationalOrderEstimatedSize I do not understand why you do not return 0 if estimatedKeysForRange return 0. Could you explain?{quote} I added comments to the code since I think it should documented in the code and not in Jira. I hope it is understandable. {quote}Instead of doing some dangerous casting in getRelationalOrderEstimatedSize, you should change the type from bestMeanCount from int to long.{quote} I changed the type of bestMeanCount {quote}In computeNext I do not understand why you do not check for stale data for range queries? Could you explain?{quote} I added comments to the code. {quote}I think it would be nicer to have also an iterator for EQ and use polymorphism instead of if else.{quote} Generally I agree polymorphism is good practice, however I think in this case it would make the code less readable. {quote}The close method of the AbstractScanIterator returned by getSequentialIterator should be called from the close method.{quote} Thanks. I wasn't aware the iterator is closeable, they usually aren't. {quote}The Unit tests are only covering a subset of the possible queries. Could you add more (a > 3 and a <4, a < 3 and a > 4 ...){quote} Added. {quote}When testing for InvalidRequestException you should use assertInvalidMessage{quote} Thanks, I wanted to use such a method but couldn't find it on my own. [~jjordan] I don't understand the problem in your example. The query result seems valid to me. In addition, can you please explain how a query using only secondary indexes such as {{select k from my_table where index1 = 5 and index2 > 10 allow filtering}} retains token order? > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Issue Comment Deleted] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Comment: was deleted (was: [~blerer] Added 4476-5.patch I updated the code to scan a restricted range if there are two expressions on the same column. {quote}I think that you should use isRange or isSlice instead of isRelationalOrderOperator as it is clearer.{quote} I renamed the method. {quote}The name of test class: SecondaryIndexNonEqTest is misleading. CONTAINS an CONTAINS KEY operator are also non eq tests.{quote} I renamed the test. {quote}In getRelationalOrderEstimatedSize I do not understand why you do not return 0 if estimatedKeysForRange return 0. Could you explain?{quote} I added comments to the code since I think it should documented in the code and not in Jira. I hope it is understandable. {quote}Instead of doing some dangerous casting in getRelationalOrderEstimatedSize, you should change the type from bestMeanCount from int to long.{quote} I changed the type of bestMeanCount {quote}In computeNext I do not understand why you do not check for stale data for range queries? Could you explain?{quote} I added comments to the code. {quote}I think it would be nicer to have also an iterator for EQ and use polymorphism instead of if else.{quote} Generally I agree polymorphism is good practice, however I think in this case it would make the code less readable. {quote}The close method of the AbstractScanIterator returned by getSequentialIterator should be called from the close method.{quote} Thanks. I wasn't aware the iterator is closeable, they usually aren't. {quote}The Unit tests are only covering a subset of the possible queries. Could you add more (a > 3 and a <4, a < 3 and a > 4 ...){quote} Added. {quote}When testing for InvalidRequestException you should use assertInvalidMessage{quote} Thanks, I wanted to use such a method but couldn't find it on my own. [~jjordan] I don't understand the problem in your example. The query result seems valid to me. In addition, can you please explain how a query using only secondary indexes such as {{select k from my_table where index1 = 5 and index2 > 10 allow filtering}} retains token order?) > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian >Assignee: Oded Peer > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-4476) Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE)
[ https://issues.apache.org/jira/browse/CASSANDRA-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-4476: - Attachment: 4476-3.patch Added new patch 4476-3.patch # I switched my development env to avoid these types of errors in the future. # Right, I fixed it. Instead of comparing to {{EQ}} I added a method to {{Operator}} that identifies a relational operator with the notion of order. # The simple index selection algorithm is described in Sylvain’s comment from 4/Dec/13. I improved the algorithm in this patch to estimate the amount of rows the slice operator returns. # I see it as a trade-off between code complexity and query performance. As Sylvain explained in his earlier comment ??more than one indexed column means ALLOW FILTERING, for which all bets are off in terms of performance anyway??. While it is good to strive and deliver the optimal performance altogether I think the use case you are describing is rare. Jonathan Ellis described “When Not to Use Secondary Indexes” in a blog post ??Do not use secondary indexes to query a huge volume of records for a small number of results?? so for the proper use of indexed queries this shouldn't have a significant effect but it would make the code more complex. # I added comments. # I thought the purpose of the test was obvious from the class name. The bug description in the method name was meant to put it into context. I guess it wasn't obvious. I changed the names. # I chose to use an instance variable to be consistent with {{usePrepared}} usage. I understand your point and I have made this an explicit variable to a new {{execute}} method In addition as part of committing the patch a change the wiki page describing Secondary Indexes needs to be changed (http://wiki.apache.org/cassandra/SecondaryIndexes). How are wiki changes usually handled? > Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE) > > > Key: CASSANDRA-4476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4476 > Project: Cassandra > Issue Type: Improvement > Components: API, Core >Reporter: Sylvain Lebresne >Assignee: Oded Peer >Priority: Minor > Labels: cql > Fix For: 3.0 > > Attachments: 4476-2.patch, 4476-3.patch, cassandra-trunk-4476.patch > > > Currently, a query that uses 2ndary indexes must have at least one EQ clause > (on an indexed column). Given that indexed CFs are local (and use > LocalPartitioner that order the row by the type of the indexed column), we > should extend 2ndary indexes to allow querying indexed columns even when no > EQ clause is provided. > As far as I can tell, the main problem to solve for this is to update > KeysSearcher.highestSelectivityPredicate(). I.e. how do we estimate the > selectivity of non-EQ clauses? I note however that if we can do that estimate > reasonably accurately, this might provide better performance even for index > queries that both EQ and non-EQ clauses, because some non-EQ clauses may have > a much better selectivity than EQ ones (say you index both the user country > and birth date, for SELECT * FROM users WHERE country = 'US' AND birthdate > > 'Jan 2009' AND birtdate < 'July 2009', you'd better use the birthdate index > first). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-4476) Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE)
[ https://issues.apache.org/jira/browse/CASSANDRA-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-4476: - Attachment: 4476-2.patch Added new patch - 4476-2.patch: * ORDER BY is already refused for queries using secondary indexes in {{SelectStatement.verifyOrderingIsAllowed()}}, I added a test to verify this behavior * I added a test for paging. It includes a small modification of {{CQLTester}} > Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE) > > > Key: CASSANDRA-4476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4476 > Project: Cassandra > Issue Type: Improvement > Components: API, Core >Reporter: Sylvain Lebresne >Assignee: Oded Peer >Priority: Minor > Labels: cql > Fix For: 3.0 > > Attachments: 4476-2.patch, cassandra-trunk-4476.patch > > > Currently, a query that uses 2ndary indexes must have at least one EQ clause > (on an indexed column). Given that indexed CFs are local (and use > LocalPartitioner that order the row by the type of the indexed column), we > should extend 2ndary indexes to allow querying indexed columns even when no > EQ clause is provided. > As far as I can tell, the main problem to solve for this is to update > KeysSearcher.highestSelectivityPredicate(). I.e. how do we estimate the > selectivity of non-EQ clauses? I note however that if we can do that estimate > reasonably accurately, this might provide better performance even for index > queries that both EQ and non-EQ clauses, because some non-EQ clauses may have > a much better selectivity than EQ ones (say you index both the user country > and birth date, for SELECT * FROM users WHERE country = 'US' AND birthdate > > 'Jan 2009' AND birtdate < 'July 2009', you'd better use the birthdate index > first). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-4476) Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE)
[ https://issues.apache.org/jira/browse/CASSANDRA-4476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-4476: - Attachment: cassandra-trunk-4476.patch > Support 2ndary index queries with only inequality clauses (LT, LTE, GT, GTE) > > > Key: CASSANDRA-4476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4476 > Project: Cassandra > Issue Type: Improvement > Components: API, Core >Reporter: Sylvain Lebresne >Priority: Minor > Labels: cql > Fix For: 2.1.2 > > Attachments: cassandra-trunk-4476.patch > > > Currently, a query that uses 2ndary indexes must have at least one EQ clause > (on an indexed column). Given that indexed CFs are local (and use > LocalPartitioner that order the row by the type of the indexed column), we > should extend 2ndary indexes to allow querying indexed columns even when no > EQ clause is provided. > As far as I can tell, the main problem to solve for this is to update > KeysSearcher.highestSelectivityPredicate(). I.e. how do we estimate the > selectivity of non-EQ clauses? I note however that if we can do that estimate > reasonably accurately, this might provide better performance even for index > queries that both EQ and non-EQ clauses, because some non-EQ clauses may have > a much better selectivity than EQ ones (say you index both the user country > and birth date, for SELECT * FROM users WHERE country = 'US' AND birthdate > > 'Jan 2009' AND birtdate < 'July 2009', you'd better use the birthdate index > first). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14194519#comment-14194519 ] Oded Peer commented on CASSANDRA-7304: -- [~slebresne] Pinging for code review > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8189) tombstone WARN in log file when creating a table with many columns
[ https://issues.apache.org/jira/browse/CASSANDRA-8189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14186475#comment-14186475 ] Oded Peer commented on CASSANDRA-8189: -- I think the right course of action would be not to add the columns to the record if the value is 'null'. I would do the same for "index_type" instead of inserting the string "null" value. > tombstone WARN in log file when creating a table with many columns > -- > > Key: CASSANDRA-8189 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8189 > Project: Cassandra > Issue Type: Bug >Reporter: Oded Peer >Assignee: Aleksey Yeschenko >Priority: Minor > Attachments: LargeTableTest.java > > > I created a table with 500 columns, creating the table resulted in a warning > in the log file. > bq. WARN 10:13:17 Read 2504 live and 1003 tombstoned cells in > system.schema_columns (see tombstone_warn_threshold). 2147483647 columns was > requested, slices=[table_0-table_0:!], > delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647} > See the attached unit test. > I analyzed the table to understand what are the tombstone columns and it > appears that Cassandra creates deleted columns for "index_type" and > "index_name" if they are not present for a column. > "index_type" get the String value "null" if it is not present. > A side effect is that these deleted columns are not removed. Since > system.schema_columns rarely changes compaction never happens for this table > and the sstable will retain the deleted columns. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-8189) tombstone WARN in log file when creating a table with many columns
Oded Peer created CASSANDRA-8189: Summary: tombstone WARN in log file when creating a table with many columns Key: CASSANDRA-8189 URL: https://issues.apache.org/jira/browse/CASSANDRA-8189 Project: Cassandra Issue Type: Bug Reporter: Oded Peer Priority: Minor Attachments: LargeTableTest.java I created a table with 500 columns, creating the table resulted in a warning in the log file. bq. WARN 10:13:17 Read 2504 live and 1003 tombstoned cells in system.schema_columns (see tombstone_warn_threshold). 2147483647 columns was requested, slices=[table_0-table_0:!], delInfo={deletedAt=-9223372036854775808, localDeletion=2147483647} See the attached unit test. I analyzed the table to understand what are the tombstone columns and it appears that Cassandra creates deleted columns for "index_type" and "index_name" if they are not present for a column. "index_type" get the String value "null" if it is not present. A side effect is that these deleted columns are not removed. Since system.schema_columns rarely changes compaction never happens for this table and the sstable will retain the deleted columns. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Attachment: 7304-03.patch 7304-03.patch contents: * Updated the v4 spec doc * Handled backward compatilbilty in {{CBUtil.readValue}} by inspecting the version protocol * Reverted modifications to CQLTester * Handled collections and counters * Since a UDT value is always written in its entirety Cassandra can't preserve a pre-existing value by 'not setting' the new value. unset values in UDT are treated as null values. * Added tests for collections and counters and UDT > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Sub-task >Reporter: Drew Kutcharian > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-03.patch, 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133692#comment-14133692 ] Oded Peer commented on CASSANDRA-7304: -- Pinging for code review > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian > Labels: cql, protocolv4 > Fix For: 3.0 > > Attachments: 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-7910) wildcard prepared statements are incorrect after a column is added to the table
Oded Peer created CASSANDRA-7910: Summary: wildcard prepared statements are incorrect after a column is added to the table Key: CASSANDRA-7910 URL: https://issues.apache.org/jira/browse/CASSANDRA-7910 Project: Cassandra Issue Type: Bug Components: Core Reporter: Oded Peer Priority: Minor Attachments: PreparedStatementAfterAddColumnTest.java 1. Prepare a statement with a wildcard in the select clause. 2. Alter the table - add a column 3. execute the prepared statement Expected result - get all the columns including the new column Actual result - get the columns except the new column Attached a test using cassandra-unit -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Attachment: 7304-2.patch > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian > Labels: cql, protocolv4 > Attachments: 7304-2.patch, 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051522#comment-14051522 ] Oded Peer commented on CASSANDRA-7304: -- bq. Do note that this is specific to the Java driver, not a server issue. Let me verify your intent: you want to change the Java driver to throw an exception if not all the bind variables are set, then change the Cassandra code to make unset variables a no-op and finally change the Java driver code to allow unset variables. IMHO this addresses safety in the java driver for a very short time period. I would skip the exception-throwing phase and go straight to implementing unset variables in the Java driver and add it to the release notes. > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian > Labels: cql > Attachments: 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14051224#comment-14051224 ] Oded Peer commented on CASSANDRA-7304: -- Having two steps can still break backwards compatibility for clients skipping over the first step (advancing from the existing version to the step-2 version). On second thought step 1 by itself breaks backwards compatibility by forcing client to change their code. I don't see an elegant way to change existing behavior without breaking backwards compatibility. As I see it you can either keep the existing syntax and semantics in-tact and add new syntax for the new feature (IGNORE NULLS), or break backwards compatibility and release this feature in a major version. > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian > Labels: cql > Attachments: 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048588#comment-14048588 ] Oded Peer commented on CASSANDRA-7304: -- Aren't you concerned about backward compatibility? If clients assume unset bind variables in prepared statements deletes the column value then ignoring unset bind variables breaks that assumption. That said, assuming you differentiate between 'null' and 'unset' values using the size of the value, would it be acceptable to mark unset values with a new class (return "UnsetByteBuffer") and using instanceof to ignore the bind variable? > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian > Labels: cql > Attachments: 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-7304: - Attachment: 7304.patch > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian > Attachments: 7304.patch > > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14038504#comment-14038504 ] Oded Peer commented on CASSANDRA-7304: -- +1 I like {{IGNORE NULLS}} at the end as well > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037158#comment-14037158 ] Oded Peer commented on CASSANDRA-7304: -- I think it's confusing since it's applicable to regular statements as well. The following statements all do the same thing: UPDATE table SET column = 3 WHERE key = 2; UPDATE table USING IGNORE_NULLS true SET column = 3 WHERE key = 2; UPDATE table USING IGNORE_NULLS false SET column = 3 WHERE key = 2; I would like to avoid having more than one way to perform an action > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7304) Ability to distinguish between NULL and UNSET values in Prepared Statements
[ https://issues.apache.org/jira/browse/CASSANDRA-7304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14034913#comment-14034913 ] Oded Peer commented on CASSANDRA-7304: -- Since values are byte arrays I don't think it's possible to add special values like unset". I am thinking of two ways to solve this: 1. Add a new flag to the QueryOptions, instructing Cassandra to ignore null values in prepared statements. The default being to create tombstones for backward compatibility. The advantage of this approach is that it can defined by Cassandra and by the driver to be applied only to prepared statements. The disadvantage I see in this is that since flags is a represented in a byte this will take up the last Flag slot and make a total of 8 flags. 2. Add a new boolean option to the INSERT and UPDATE commands named IGNORE_NULLS which instructs Cassandra to ignore null values instead of marking them as tombstones. The advantage of this is that it is explicit and more natural as part of the language syntax. The disadvantage is that it is not limited to prepared statements and will be confusing. I prefer the first option. > Ability to distinguish between NULL and UNSET values in Prepared Statements > --- > > Key: CASSANDRA-7304 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7304 > Project: Cassandra > Issue Type: Improvement >Reporter: Drew Kutcharian > > Currently Cassandra inserts tombstones when a value of a column is bound to > NULL in a prepared statement. At higher insert rates managing all these > tombstones becomes an unnecessary overhead. This limits the usefulness of the > prepared statements since developers have to either create multiple prepared > statements (each with a different combination of column names, which at times > is just unfeasible because of the sheer number of possible combinations) or > fall back to using regular (non-prepared) statements. > This JIRA is here to explore the possibility of either: > A. Have a flag on prepared statements that once set, tells Cassandra to > ignore null columns > or > B. Have an "UNSET" value which makes Cassandra skip the null columns and not > tombstone them > Basically, in the context of a prepared statement, a null value means delete, > but we don’t have anything that means "ignore" (besides creating a new > prepared statement without the ignored column). > Please refer to the original conversation on DataStax Java Driver mailing > list for more background: > https://groups.google.com/a/lists.datastax.com/d/topic/java-driver-user/cHE3OOSIXBU/discussion -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-5964) cqlsh raises a ValueError when connecting to Cassandra running in Eclipse
[ https://issues.apache.org/jira/browse/CASSANDRA-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oded Peer updated CASSANDRA-5964: - Attachment: 5964-v2.patch This is a proposed fix to the issue - adding the resources directory as an Eclipse source directory > cqlsh raises a ValueError when connecting to Cassandra running in Eclipse > - > > Key: CASSANDRA-5964 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5964 > Project: Cassandra > Issue Type: Bug >Reporter: Greg DeAngelis >Assignee: Dave Brosius >Priority: Minor > Fix For: 2.0.1 > > Attachments: 5964-v2.patch, 5964.txt > > > The release_version is set to 'Unknown' in system.local so the version > parsing logic fails. > Traceback (most recent call last): > File "./cqlsh", line 2027, in > main(*read_options(sys.argv[1:], os.environ)) > File "./cqlsh", line 2013, in main > display_float_precision=options.float_precision) > File "./cqlsh", line 486, in __init__ > self.get_connection_versions() > File "./cqlsh", line 580, in get_connection_versions > self.cass_ver_tuple = tuple(map(int, vers['build'].split('-', > 1)[0].split('.')[:3])) > ValueError: invalid literal for int() with base 10: 'Unknown' -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-5964) cqlsh raises a ValueError when connecting to Cassandra running in Eclipse
[ https://issues.apache.org/jira/browse/CASSANDRA-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010947#comment-14010947 ] Oded Peer edited comment on CASSANDRA-5964 at 5/28/14 9:18 AM: --- [~mishail] Alexey Filippov is right. Regenerating {{version.properties}} creates the {{version.properties}} file in {{${build.src.resources}/org/apache/cassandra/config}}, however, {{${build.src.resources}}} is not configured as an Eclipse src directory causing {{version.properties}} not to be copied to {{${build.classes.main}}}. was (Author: odpeer): [~mishail] Alexey Filippov is right. Regenerating {{version.properties}} creates the {{version.properties file}} in {{${build.src.resources}/org/apache/cassandra/config}}, however, {{${build.src.resources}}} is not configured as an Eclipse src directory causing {{version.properties}} not to be copied to {{${build.classes.main}}}. > cqlsh raises a ValueError when connecting to Cassandra running in Eclipse > - > > Key: CASSANDRA-5964 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5964 > Project: Cassandra > Issue Type: Bug >Reporter: Greg DeAngelis >Assignee: Dave Brosius >Priority: Minor > Fix For: 2.0.1 > > Attachments: 5964.txt > > > The release_version is set to 'Unknown' in system.local so the version > parsing logic fails. > Traceback (most recent call last): > File "./cqlsh", line 2027, in > main(*read_options(sys.argv[1:], os.environ)) > File "./cqlsh", line 2013, in main > display_float_precision=options.float_precision) > File "./cqlsh", line 486, in __init__ > self.get_connection_versions() > File "./cqlsh", line 580, in get_connection_versions > self.cass_ver_tuple = tuple(map(int, vers['build'].split('-', > 1)[0].split('.')[:3])) > ValueError: invalid literal for int() with base 10: 'Unknown' -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-5964) cqlsh raises a ValueError when connecting to Cassandra running in Eclipse
[ https://issues.apache.org/jira/browse/CASSANDRA-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14010947#comment-14010947 ] Oded Peer commented on CASSANDRA-5964: -- [~mishail] Alexey Filippov is right. Regenerating {{version.properties}} creates the {{version.properties file}} in {{${build.src.resources}/org/apache/cassandra/config}}, however, {{${build.src.resources}}} is not configured as an Eclipse src directory causing {{version.properties}} not to be copied to {{${build.classes.main}}}. > cqlsh raises a ValueError when connecting to Cassandra running in Eclipse > - > > Key: CASSANDRA-5964 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5964 > Project: Cassandra > Issue Type: Bug >Reporter: Greg DeAngelis >Assignee: Dave Brosius >Priority: Minor > Fix For: 2.0.1 > > Attachments: 5964.txt > > > The release_version is set to 'Unknown' in system.local so the version > parsing logic fails. > Traceback (most recent call last): > File "./cqlsh", line 2027, in > main(*read_options(sys.argv[1:], os.environ)) > File "./cqlsh", line 2013, in main > display_float_precision=options.float_precision) > File "./cqlsh", line 486, in __init__ > self.get_connection_versions() > File "./cqlsh", line 580, in get_connection_versions > self.cass_ver_tuple = tuple(map(int, vers['build'].split('-', > 1)[0].split('.')[:3])) > ValueError: invalid literal for int() with base 10: 'Unknown' -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7216) Restricted superuser account request
[ https://issues.apache.org/jira/browse/CASSANDRA-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14002909#comment-14002909 ] Oded Peer commented on CASSANDRA-7216: -- My problem is with granting the permissions. If the user managing other users can grant any permission to any user what prevents the attacker from creating a new user and granting permissions to read existing keyspaces? > Restricted superuser account request > > > Key: CASSANDRA-7216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7216 > Project: Cassandra > Issue Type: Improvement >Reporter: Oded Peer >Assignee: Dave Brosius >Priority: Minor > Fix For: 3.0 > > Attachments: 7216-POC.txt, 7216.txt > > > I am developing a multi-tenant service. > Every tenant has its own user, keyspace and can access only his keyspace. > As new tenants are provisioned there is a need to create new users and > keyspaces. > Only a superuser can issue CREATE USER requests, so we must have a super user > account in the system. On the other hand super users have access to all the > keyspaces, which poses a security risk. > For tenant provisioning I would like to have a restricted account which can > only create new users, without read access to keyspaces. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7216) Restricted superuser account request
[ https://issues.apache.org/jira/browse/CASSANDRA-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14001608#comment-14001608 ] Oded Peer commented on CASSANDRA-7216: -- [~dbrosius] If the user-admin role can grant any permission to any user what prevents the attacker from creating a new user and granting permissions to read existing keyspaces? > Restricted superuser account request > > > Key: CASSANDRA-7216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7216 > Project: Cassandra > Issue Type: Improvement >Reporter: Oded Peer >Assignee: Dave Brosius >Priority: Minor > Fix For: 3.0 > > Attachments: 7216.txt > > > I am developing a multi-tenant service. > Every tenant has its own user, keyspace and can access only his keyspace. > As new tenants are provisioned there is a need to create new users and > keyspaces. > Only a superuser can issue CREATE USER requests, so we must have a super user > account in the system. On the other hand super users have access to all the > keyspaces, which poses a security risk. > For tenant provisioning I would like to have a restricted account which can > only create new users, without read access to keyspaces. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7216) Restricted superuser account request
[ https://issues.apache.org/jira/browse/CASSANDRA-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14001523#comment-14001523 ] Oded Peer commented on CASSANDRA-7216: -- [~iamaleksey] For my case keyspaces and users are related. I tried your suggestion, using a custom IAuthenticator to grant permissions upon user creation. I granted CREATE and ALTER on the root resource so the new user will be able to create the tenant keyspace, and I granted CREATE through MODIFY on the tenant's keyspace. Even though all the users can create tables in every keyspace only a single user can query the tables, this seems reasonable from security perspective. > Restricted superuser account request > > > Key: CASSANDRA-7216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7216 > Project: Cassandra > Issue Type: Improvement >Reporter: Oded Peer >Assignee: Dave Brosius >Priority: Minor > Fix For: 3.0 > > Attachments: 7216.txt > > > I am developing a multi-tenant service. > Every tenant has its own user, keyspace and can access only his keyspace. > As new tenants are provisioned there is a need to create new users and > keyspaces. > Only a superuser can issue CREATE USER requests, so we must have a super user > account in the system. On the other hand super users have access to all the > keyspaces, which poses a security risk. > For tenant provisioning I would like to have a restricted account which can > only create new users, without read access to keyspaces. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7216) Restricted superuser account request
[ https://issues.apache.org/jira/browse/CASSANDRA-7216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13997366#comment-13997366 ] Oded Peer commented on CASSANDRA-7216: -- I can have just a single super user, however as tightly as I control this user it still poses a security threat. This has implications in security audits, including external audits done by customers and partners. I got to know the permissions better in Cassandra and it appears that in addition to creating keyspaces and users the restricted superuser account also needs to GRANT permissions to the newly-created user to access and modify the newly-created keyspace. If the restricted superuser account has GRANT permissions to any keyspace it still poses security threat since it can create users with permissions to any arbitrary keyspace. What we are trying to find an analogy of the postgres security model in Cassandra. In postgres objects have a single 'owner'. For most kinds of objects, the initial state is that only the owner can do anything with the object. [http://www.postgresql.org/docs/9.0/static/privileges.html]. Thus, in postgres, we have a restricted admin user used in the tenant provisioning process that can only create users. These newly-created users create database objects as their 'owner' and only the user creating the objects can use them. > Restricted superuser account request > > > Key: CASSANDRA-7216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7216 > Project: Cassandra > Issue Type: Improvement >Reporter: Oded Peer >Priority: Minor > > I am developing a multi-tenant service. > Every tenant has its own user, keyspace and can access only his keyspace. > As new tenants are provisioned there is a need to create new users and > keyspaces. > Only a superuser can issue CREATE USER requests, so we must have a super user > account in the system. On the other hand super users have access to all the > keyspaces, which poses a security risk. > For tenant provisioning I would like to have a restricted account which can > only create new users, without read access to keyspaces. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7216) Restricted superuser account request
Oded Peer created CASSANDRA-7216: Summary: Restricted superuser account request Key: CASSANDRA-7216 URL: https://issues.apache.org/jira/browse/CASSANDRA-7216 Project: Cassandra Issue Type: Improvement Reporter: Oded Peer Priority: Minor I am developing a multi-tenant service. Every tenant has its own user, keyspace and can access only his keyspace. As new tenants are provisioned there is a need to create new users and keyspaces. Only a superuser can issue CREATE USER requests, so we must have a super user account in the system. On the other hand super users have access to all the keyspaces, which poses a security risk. For tenant provisioning I would like to have a restricted account which can only create new users, without read access to keyspaces. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-6829) nodes sporadically shutting down
Oded Peer created CASSANDRA-6829: Summary: nodes sporadically shutting down Key: CASSANDRA-6829 URL: https://issues.apache.org/jira/browse/CASSANDRA-6829 Project: Cassandra Issue Type: Bug Components: Core Environment: Windows Azure VMs. The VMs OS is SUSE Enterprise. I striped 2 logical volumes for each VM, one for data and one for commitlog, and formatted them as XFS. Oracle Java 1.7_45 Datastax Enterprise 4.0 (Cassandra version 2.0.5.22) Reporter: Oded Peer I deployed a Datastax 4.0 Cassandra cluster in Windows Azure and started load tests. After a while some of the nodes announce shutdown and stop responding to client requests. The error preceding the shutdown is "FSWriteError in /mnt/dsedata/lib/cassandra/poc/event_log/poc-event_log-tmp-jb-581-Data.db" "Caused by: java.io.IOException: Input/output error". The storage I'm using in my VMs is Azure Blob storage. The VMs OS is SUSE Enterprise. I striped 2 logical volumes for each VM, one for data and one for commitlog, and formatted them as XFS. I am using Oracle Java 1.7_45 Restarting the Cassandra process resolves the problem for a short while (minutes) afterwards the problem occurs again. I noticed that it happens only in tmp files of a specific table. See the errors from 3 random nodes: (1) ERROR [CompactionExecutor:48] 2014-03-09 11:38:45,188 CassandraDaemon.java (line 192) Exception in thread Thread[CompactionExecutor:48,1,main] FSWriteError in /mnt/dsedata/lib/cassandra/poc/event_log/poc-event_log-tmp-jb-409-Data.db (2) ERROR [CompactionExecutor:37] 2014-03-10 10:04:30,828 CassandraDaemon.java (line 196) Exception in thread Thread[CompactionExecutor:37,1,main] FSWriteError in /mnt/dsedata/lib/cassandra/poc/event_log/poc-event_log-tmp-jb-946-Data.db (3) ERROR [CompactionExecutor:48] 2014-03-10 10:23:39,248 CassandraDaemon.java (line 196) Exception in thread Thread[CompactionExecutor:48,1,main] FSWriteError in /mnt/dsedata/lib/cassandra/poc/event_log/poc-event_log-tmp-jb-874-Data.db The table is a wide-row table created as: CREATE TABLE event_log ( time_slice bigint, distribution_key int, event_id text, ... 300 columns ... PRIMARY KEY ((time_slice, distribution_key), event_id) ) compaction={'class': 'SizeTieredCompactionStrategy'} AND compression={'sstable_compression': 'LZ4Compressor'}; CREATE INDEX EVENT_LOG_2IX ON event_log (event_id); 'time_slice' represents a 5 minute time-period such as MMddHHmm where 'mm' is between 00 and 55 with increments of 5. The Data files under the 'data' directory got to be very big in a very short time after the test started. For example: 1.5G Mar 10 10:50 /mnt/dsedata/lib/cassandra/poc/event_log/poc-event_log-jb-968-Data.db 3.0G Mar 10 11:41 /mnt/dsedata/lib/cassandra/poc/event_log/poc-event_log-tmp-jb-970-Data.db Full stack trace: ERROR [CompactionExecutor:37] 2014-03-10 10:04:30,828 CassandraDaemon.java (line 196) Exception in thread Thread[CompactionExecutor:37,1,main] FSWriteError in /mnt/dsedata/lib/cassandra/poc/event_log/poc-event_log-tmp-jb-946-Data.db at org.apache.cassandra.io.compress.CompressedSequentialWriter.close(CompressedSequentialWriter.java:270) at org.apache.cassandra.io.sstable.SSTableWriter.close(SSTableWriter.java:356) at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:324) at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:204) at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48) at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60) at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59) at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.io.IOException: Input/output error at sun.nio.ch.FileDispatcherImpl.force0(Native Method) at sun.nio.ch.FileDispatcherImpl.force(FileDispatcherImpl.java:76) at sun.nio.ch.FileChannelImpl.force(FileChannelImpl.java:376) at org.apache.cassandra.io.compress.CompressionMetadata$Writer.close(CompressionMetadata.java:366) at org.apache.cassandra.io.compress.CompressedSequentialWriter.close(CompressedSequentialWriter.java:266) ... 13 more