[jira] [Commented] (CASSANDRA-6477) Global indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14352988#comment-14352988 ] Rustam Aliyev commented on CASSANDRA-6477: -- +1 for replacing DENORMALISED with INCLUDE DynamoDB has similar capability which is referred as Projection (http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Projection.html) and there are 3 supported projection types: ALL, KEYS_ONLY and *INCLUDE*. Global indexes -- Key: CASSANDRA-6477 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477 Project: Cassandra Issue Type: New Feature Components: API, Core Reporter: Jonathan Ellis Assignee: Carl Yeksigian Labels: cql Fix For: 3.0 Local indexes are suitable for low-cardinality data, where spreading the index across the cluster is a Good Thing. However, for high-cardinality data, local indexes require querying most nodes in the cluster even if only a handful of rows is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7857) Ability to froze UDT
[
https://issues.apache.org/jira/browse/CASSANDRA-7857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14117712#comment-14117712
]
Rustam Aliyev commented on CASSANDRA-7857:
--
We just need to find keyword which 1) most people familiar with 2) best
describes UDT differences.
{{frozen}} can be misleading because 1) not many people familiar with it 2) it
has quite different meaning in Ruby and Eiffel.
I think {{serialized}} is what most of people are familiar with and what best
(although not exactly) describes difference between those UDT types.
Ability to froze UDT
Key: CASSANDRA-7857
URL: https://issues.apache.org/jira/browse/CASSANDRA-7857
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.1.0
Attachments: 7857-v2.txt, 7857.txt
Currently, UDT are serialized into a single value. For 3.0, we want to change
that somewhat and allow updating individual subfields: CASSANDRA-7423 (and
ultimately, we'll probably allow querying subpart of UDT to some extend).
Also for 3.0, we want to allow some nesting of collections (CASSANDRA-7826).
However, migrating the currently serialized UDT would be challenging. Besides
that, even with nested collections, we probably won't be able to support
nesting within map keys and sets without serializing (at the very least, not
initially). Also, it can be useful in some specific case to have UDT or
collections for PK columns, even if those are serialized.
So we need a better way to distinguish when a composite types (collections
UDT) are serialized (which imply you can't update subpart of the value, you
have to rewrite it fully) and when they are not. The suggestion is then to
introduce a new keyword, {{frozen}}, to indicate that a type is serialized:
{noformat}
CREATE TYPE foo (a int, b int);
CREATE TABLE bar (
k frozenfoo PRIMARY KEY,
m mapfrozensetint, text
)
{noformat}
A big advantage is that it makes the downside (you can't update the value
without rewriting it all) clear and upfront.
Now, as of 2.1, we only support frozen UDT, and so we should make this clear
by 1) adding the frozen keyword and 2) don't allow use of UDT unless they are
frozen (since that's all we really support). This is what this ticket
proposes to do. And this should be done in 2.1.0 or this will be a breaking
change.
We will have a follow-up ticket that will extend {{frozen}} to collection,
but this is less urgent since this will be strictly an improvement.
I'll note that in term of syntax, {{serialized}} was suggested as an
alternative to {{frozen}}. I personally have a minor preference for
{{serialized}} but it was argued that it had a sequential connotation which
{{frozen}} don't have. Changing that is still up for discussion, but we need
to reach a decision quickly.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7642) Adaptive Consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080706#comment-14080706
]
Rustam Aliyev commented on CASSANDRA-7642:
--
Yes. I think I finally found a simple way to explain the problem :).
Robert, regarding DowngradingConsistencyRetryPolicy see my first comment on top
- it's useless.
Adaptive Consistency
Key: CASSANDRA-7642
URL: https://issues.apache.org/jira/browse/CASSANDRA-7642
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Rustam Aliyev
Fix For: 3.0
h4. Problem
At minimum, application requires consistency level of X, which must be fault
tolerant CL. However, when there is no failure it would be advantageous to
use stronger consistency Y (YX).
h4. Suggestion
Application defines minimum (X) and maximum (Y) consistency levels. C* can
apply adaptive consistency logic to use Y whenever possible and downgrade to
X when failure occurs.
Implementation should not negatively impact performance. Therefore, state has
to be maintained globally (not per request).
h4. Example
{{MIN_CL=LOCAL_QUORUM}}
{{MAX_CL=EACH_QUORUM}}
h4. Use Case
Consider a case where user wants to maximize their uptime and consistency.
They designing a system using C* where transactions are read/written with
LOCAL_QUORUM and distributed across 2 DCs. Occasional inconsistencies between
DCs can be tolerated. R/W with LOCAL_QUORUM is satisfactory in most of the
cases.
Application requires new transactions to be read back right after they were
generated. Write and read could be done through different DCs (no
stickiness). In some cases when user writes into DC1 and reads immediately
from DC2, replication delay may cause problems. Transaction won't show up on
read in DC2, user will retry and create duplicate transaction. Occasional
duplicates are fine and the goal is to minimize number of dups.
Therefore, we want to perform writes with stronger consistency (EACH_QUORUM)
whenever possible without compromising on availability. Using adaptive
consistency they should be able to define:
{{Read CL = LOCAL_QUORUM}}
{{Write CL = ADAPTIVE (MIN:LOCAL_QUORUM, MAX:EACH_QUORUM)}}
Similar scenario can be described for {{Write CL = ADAPTIVE (MIN:QUORUM,
MAX:ALL)}} case.
h4. Criticism
# This functionality can/should be implemented by user himself.
bq. It will be hard for an average user to implement topology monitoring and
state machine. Moreover, this is a pattern which repeats.
# Transparent downgrading violates the CL contract, and that contract
considered be just about the most important element of Cassandra's runtime
behavior.
bq.Fully transparent downgrading without any contract is dangerous. However,
would it be problem if we specify explicitly only two discrete CL levels -
MIN_CL and MAX_CL?
# If you have split brain DCs (partitioned in CAP), you have to sacrifice
either consistency or availability, and auto downgrading sacrifices the
consistency in dangerous ways if the application isn't designed to handle it.
And if the application is designed to handle it, then it should be able to
handle it in normal circumstances, not just degraded/extraordinary ones.
bq. Agreed. Application should be designed for MIN_CL. In that case, MAX_CL
will not be causing much harm, only adding flexibility.
# It might be a better idea to loudly downgrade, instead of silently
downgrading, meaning that the client code does an explicit retry with lower
consistency on failure and takes some other kind of action to attempt to
inform either users or operators of the problem. The silent part of the
downgrading which could be dangerous.
bq. There are certainly cases where user should be informed when consistency
changes in order to perform custom action. For this purpose we could
allow/require user to register callback function which will be triggered when
consistency level changes. Best practices could be enforced by requiring
callback.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7642) Adaptive Consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080746#comment-14080746
]
Rustam Aliyev commented on CASSANDRA-7642:
--
That will be better than nothing, but that means at least 3 DCs. Depending on
the cluster size this can be very expensive requirement (imagine case with
hundreds of nodes in each DC).
I'd like to see if we can come up with solution which will work with 2 DCs.
With AC one can read and write with (MIN:LOCAL_Q, MAX:EACH_Q) and achieve same
goal at no extra cost.
Adaptive Consistency
Key: CASSANDRA-7642
URL: https://issues.apache.org/jira/browse/CASSANDRA-7642
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Rustam Aliyev
Fix For: 3.0
h4. Problem
At minimum, application requires consistency level of X, which must be fault
tolerant CL. However, when there is no failure it would be advantageous to
use stronger consistency Y (YX).
h4. Suggestion
Application defines minimum (X) and maximum (Y) consistency levels. C* can
apply adaptive consistency logic to use Y whenever possible and downgrade to
X when failure occurs.
Implementation should not negatively impact performance. Therefore, state has
to be maintained globally (not per request).
h4. Example
{{MIN_CL=LOCAL_QUORUM}}
{{MAX_CL=EACH_QUORUM}}
h4. Use Case
Consider a case where user wants to maximize their uptime and consistency.
They designing a system using C* where transactions are read/written with
LOCAL_QUORUM and distributed across 2 DCs. Occasional inconsistencies between
DCs can be tolerated. R/W with LOCAL_QUORUM is satisfactory in most of the
cases.
Application requires new transactions to be read back right after they were
generated. Write and read could be done through different DCs (no
stickiness). In some cases when user writes into DC1 and reads immediately
from DC2, replication delay may cause problems. Transaction won't show up on
read in DC2, user will retry and create duplicate transaction. Occasional
duplicates are fine and the goal is to minimize number of dups.
Therefore, we want to perform writes with stronger consistency (EACH_QUORUM)
whenever possible without compromising on availability. Using adaptive
consistency they should be able to define:
{{Read CL = LOCAL_QUORUM}}
{{Write CL = ADAPTIVE (MIN:LOCAL_QUORUM, MAX:EACH_QUORUM)}}
Similar scenario can be described for {{Write CL = ADAPTIVE (MIN:QUORUM,
MAX:ALL)}} case.
h4. Criticism
# This functionality can/should be implemented by user himself.
bq. It will be hard for an average user to implement topology monitoring and
state machine. Moreover, this is a pattern which repeats.
# Transparent downgrading violates the CL contract, and that contract
considered be just about the most important element of Cassandra's runtime
behavior.
bq.Fully transparent downgrading without any contract is dangerous. However,
would it be problem if we specify explicitly only two discrete CL levels -
MIN_CL and MAX_CL?
# If you have split brain DCs (partitioned in CAP), you have to sacrifice
either consistency or availability, and auto downgrading sacrifices the
consistency in dangerous ways if the application isn't designed to handle it.
And if the application is designed to handle it, then it should be able to
handle it in normal circumstances, not just degraded/extraordinary ones.
bq. Agreed. Application should be designed for MIN_CL. In that case, MAX_CL
will not be causing much harm, only adding flexibility.
# It might be a better idea to loudly downgrade, instead of silently
downgrading, meaning that the client code does an explicit retry with lower
consistency on failure and takes some other kind of action to attempt to
inform either users or operators of the problem. The silent part of the
downgrading which could be dangerous.
bq. There are certainly cases where user should be informed when consistency
changes in order to perform custom action. For this purpose we could
allow/require user to register callback function which will be triggered when
consistency level changes. Best practices could be enforced by requiring
callback.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7642) Adaptive Consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080774#comment-14080774
]
Rustam Aliyev commented on CASSANDRA-7642:
--
Ah, I see what you mean. In case with 2 DCs, it will work (assuming it will
fail when CL drops below LOCAL_Q).
However, for DC number 2, it would suffer most of the points discussed in the
Criticism section of this ticket - no CL contract (app may not be able to
handle certain CL levels), unpredictable, transparent and silent downgrading,
etc.
Adaptive Consistency
Key: CASSANDRA-7642
URL: https://issues.apache.org/jira/browse/CASSANDRA-7642
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Rustam Aliyev
Fix For: 3.0
h4. Problem
At minimum, application requires consistency level of X, which must be fault
tolerant CL. However, when there is no failure it would be advantageous to
use stronger consistency Y (YX).
h4. Suggestion
Application defines minimum (X) and maximum (Y) consistency levels. C* can
apply adaptive consistency logic to use Y whenever possible and downgrade to
X when failure occurs.
Implementation should not negatively impact performance. Therefore, state has
to be maintained globally (not per request).
h4. Example
{{MIN_CL=LOCAL_QUORUM}}
{{MAX_CL=EACH_QUORUM}}
h4. Use Case
Consider a case where user wants to maximize their uptime and consistency.
They designing a system using C* where transactions are read/written with
LOCAL_QUORUM and distributed across 2 DCs. Occasional inconsistencies between
DCs can be tolerated. R/W with LOCAL_QUORUM is satisfactory in most of the
cases.
Application requires new transactions to be read back right after they were
generated. Write and read could be done through different DCs (no
stickiness). In some cases when user writes into DC1 and reads immediately
from DC2, replication delay may cause problems. Transaction won't show up on
read in DC2, user will retry and create duplicate transaction. Occasional
duplicates are fine and the goal is to minimize number of dups.
Therefore, we want to perform writes with stronger consistency (EACH_QUORUM)
whenever possible without compromising on availability. Using adaptive
consistency they should be able to define:
{{Read CL = LOCAL_QUORUM}}
{{Write CL = ADAPTIVE (MIN:LOCAL_QUORUM, MAX:EACH_QUORUM)}}
Similar scenario can be described for {{Write CL = ADAPTIVE (MIN:QUORUM,
MAX:ALL)}} case.
h4. Criticism
# This functionality can/should be implemented by user himself.
bq. It will be hard for an average user to implement topology monitoring and
state machine. Moreover, this is a pattern which repeats.
# Transparent downgrading violates the CL contract, and that contract
considered be just about the most important element of Cassandra's runtime
behavior.
bq.Fully transparent downgrading without any contract is dangerous. However,
would it be problem if we specify explicitly only two discrete CL levels -
MIN_CL and MAX_CL?
# If you have split brain DCs (partitioned in CAP), you have to sacrifice
either consistency or availability, and auto downgrading sacrifices the
consistency in dangerous ways if the application isn't designed to handle it.
And if the application is designed to handle it, then it should be able to
handle it in normal circumstances, not just degraded/extraordinary ones.
bq. Agreed. Application should be designed for MIN_CL. In that case, MAX_CL
will not be causing much harm, only adding flexibility.
# It might be a better idea to loudly downgrade, instead of silently
downgrading, meaning that the client code does an explicit retry with lower
consistency on failure and takes some other kind of action to attempt to
inform either users or operators of the problem. The silent part of the
downgrading which could be dangerous.
bq. There are certainly cases where user should be informed when consistency
changes in order to perform custom action. For this purpose we could
allow/require user to register callback function which will be triggered when
consistency level changes. Best practices could be enforced by requiring
callback.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7642) Adaptive Consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14079101#comment-14079101
]
Rustam Aliyev commented on CASSANDRA-7642:
--
There's DowngradingConsistencyRetryPolicy in the driver which is very
primitive:
https://github.com/tfredrich/cassandra-java-driver/blob/master/driver-core/src/main/java/com/datastax/driver/core/policies/DowngradingConsistencyRetryPolicy.java#L74-L81
The main problem is that it's not topology aware and will retry for every
single request which in case of failure will impact performance. It also
suffers from all the points mentioned in Criticism section above. Therefore,
not production suitable.
Adaptive Consistency logic should monitor topology and maintain global adaptive
CL state based on that. For example, if one of the DCs is unavailable, then
globally downgrade to MIN_CL and potentially notify client (through callback).
So the AC logic is not as simple as RetryPolicy (and in fact it's not a retry).
Yet, technically this can be done on the driver side. However, given the fact
that now we have 4 official DataStax drivers and much more community drivers,
implementing this in each of them would require much more work and will be more
error prone than server side implementation.
Adaptive Consistency
Key: CASSANDRA-7642
URL: https://issues.apache.org/jira/browse/CASSANDRA-7642
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Rustam Aliyev
Fix For: 3.0
h4. Problem
At minimum, application requires consistency level of X, which must be fault
tolerant CL. However, when there is no failure it would be advantageous to
use stronger consistency Y (YX).
h4. Suggestion
Application defines minimum (X) and maximum (Y) consistency levels. C* can
apply adaptive consistency logic to use Y whenever possible and downgrade to
X when failure occurs.
Implementation should not negatively impact performance. Therefore, state has
to be maintained globally (not per request).
h4. Example
{{MIN_CL=LOCAL_QUORUM}}
{{MAX_CL=EACH_QUORUM}}
h4. Use Case
Consider a case where user wants to maximize their uptime and consistency.
They designing a system using C* where transactions are read/written with
LOCAL_QUORUM and distributed across 2 DCs. Occasional inconsistencies between
DCs can be tolerated. R/W with LOCAL_QUORUM is satisfactory in most of the
cases.
Application requires new transactions to be read back right after they were
generated. Write and read could be done through different DCs (no
stickiness). In some cases when user writes into DC1 and reads immediately
from DC2, replication delay may cause problems. Transaction won't show up on
read in DC2, user will retry and create duplicate transaction. Occasional
duplicates are fine and the goal is to minimize number of dups.
Therefore, we want to perform writes with stronger consistency (EACH_QUORUM)
whenever possible without compromising on availability. Using adaptive
consistency they should be able to define:
{{Read CL = LOCAL_QUORUM}}
{{Write CL = ADAPTIVE (MIN:LOCAL_QUORUM, MAX:EACH_QUORUM)}}
Similar scenario can be described for {{Write CL = ADAPTIVE (MIN:QUORUM,
MAX:ALL)}} case.
h4. Criticism
# This functionality can/should be implemented by user himself.
bq. It will be hard for an average user to implement topology monitoring and
state machine. Moreover, this is a pattern which repeats.
# Transparent downgrading violates the CL contract, and that contract
considered be just about the most important element of Cassandra's runtime
behavior.
bq.Fully transparent downgrading without any contract is dangerous. However,
would it be problem if we specify explicitly only two discrete CL levels -
MIN_CL and MAX_CL?
# If you have split brain DCs (partitioned in CAP), you have to sacrifice
either consistency or availability, and auto downgrading sacrifices the
consistency in dangerous ways if the application isn't designed to handle it.
And if the application is designed to handle it, then it should be able to
handle it in normal circumstances, not just degraded/extraordinary ones.
bq. Agreed. Application should be designed for MIN_CL. In that case, MAX_CL
will not be causing much harm, only adding flexibility.
# It might be a better idea to loudly downgrade, instead of silently
downgrading, meaning that the client code does an explicit retry with lower
consistency on failure and takes some other kind of action to attempt to
inform either users or operators of the problem. The silent part of the
downgrading which could be dangerous.
bq. There are certainly cases where user should be informed when consistency
changes in order to perform custom action. For this purpose we could
[jira] [Commented] (CASSANDRA-7637) Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
[
https://issues.apache.org/jira/browse/CASSANDRA-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14079105#comment-14079105
]
Rustam Aliyev commented on CASSANDRA-7637:
--
There's already so many places where inequality and equality operators just
return Bad Request. From user experience point of view, it's not very
different. Error message/docs can refine usage.
Good thing about LIKE keyword is that it can be extended to other use cases in
future (e.g. ALLOW FILTERING).
Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
--
Key: CASSANDRA-7637
URL: https://issues.apache.org/jira/browse/CASSANDRA-7637
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Rustam Aliyev
Fix For: 3.0
Currently, if I want to perform range query on lexical type I need to do
something like this:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute 'interests.food.' AND
attribute 'interests.food.z';
{code}
This is very efficient range query. Yet, many users who are not familiar with
Thrift and storage level implementation are unaware of this trick.
Therefore, it would be convenient to introduce CQL keyword which will do this
more simply:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute START_WITH('interests.food.');
{code}
Keyword would have same restrictions as other inequality search operators
plus some type restrictions.
Allowed types would be:
* {{ascii}}
* {{text}} / {{varchar}}
* {{maptext, *}} (same for ascii) (?)
* {{settext}} (same for ascii) (?)
(?) may require more work, therefore optional
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7642) Adaptive Consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14079397#comment-14079397
]
Rustam Aliyev commented on CASSANDRA-7642:
--
Jack, that could work. Basically, you suggest to move blocking for MAX CL from
server side to client side and let client handle scenarios when it's not
achieved.
However, how will client track which transaction reached MAX CL? AFAIK, there's
no transaction ID or anything like that in protocol. Even if there would be,
managing state of the transactions on the client side would require quite a bit
of work and would lead to mistakes.
The point here is that failure rate in production isn't high, let's say 0.1%
(for 99.9% SLA). And AC should be an easy way to provide MAX CL for 99.9% and
MIN CL for 0.1% without writing any code.
Adaptive Consistency
Key: CASSANDRA-7642
URL: https://issues.apache.org/jira/browse/CASSANDRA-7642
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Rustam Aliyev
Fix For: 3.0
h4. Problem
At minimum, application requires consistency level of X, which must be fault
tolerant CL. However, when there is no failure it would be advantageous to
use stronger consistency Y (YX).
h4. Suggestion
Application defines minimum (X) and maximum (Y) consistency levels. C* can
apply adaptive consistency logic to use Y whenever possible and downgrade to
X when failure occurs.
Implementation should not negatively impact performance. Therefore, state has
to be maintained globally (not per request).
h4. Example
{{MIN_CL=LOCAL_QUORUM}}
{{MAX_CL=EACH_QUORUM}}
h4. Use Case
Consider a case where user wants to maximize their uptime and consistency.
They designing a system using C* where transactions are read/written with
LOCAL_QUORUM and distributed across 2 DCs. Occasional inconsistencies between
DCs can be tolerated. R/W with LOCAL_QUORUM is satisfactory in most of the
cases.
Application requires new transactions to be read back right after they were
generated. Write and read could be done through different DCs (no
stickiness). In some cases when user writes into DC1 and reads immediately
from DC2, replication delay may cause problems. Transaction won't show up on
read in DC2, user will retry and create duplicate transaction. Occasional
duplicates are fine and the goal is to minimize number of dups.
Therefore, we want to perform writes with stronger consistency (EACH_QUORUM)
whenever possible without compromising on availability. Using adaptive
consistency they should be able to define:
{{Read CL = LOCAL_QUORUM}}
{{Write CL = ADAPTIVE (MIN:LOCAL_QUORUM, MAX:EACH_QUORUM)}}
Similar scenario can be described for {{Write CL = ADAPTIVE (MIN:QUORUM,
MAX:ALL)}} case.
h4. Criticism
# This functionality can/should be implemented by user himself.
bq. It will be hard for an average user to implement topology monitoring and
state machine. Moreover, this is a pattern which repeats.
# Transparent downgrading violates the CL contract, and that contract
considered be just about the most important element of Cassandra's runtime
behavior.
bq.Fully transparent downgrading without any contract is dangerous. However,
would it be problem if we specify explicitly only two discrete CL levels -
MIN_CL and MAX_CL?
# If you have split brain DCs (partitioned in CAP), you have to sacrifice
either consistency or availability, and auto downgrading sacrifices the
consistency in dangerous ways if the application isn't designed to handle it.
And if the application is designed to handle it, then it should be able to
handle it in normal circumstances, not just degraded/extraordinary ones.
bq. Agreed. Application should be designed for MIN_CL. In that case, MAX_CL
will not be causing much harm, only adding flexibility.
# It might be a better idea to loudly downgrade, instead of silently
downgrading, meaning that the client code does an explicit retry with lower
consistency on failure and takes some other kind of action to attempt to
inform either users or operators of the problem. The silent part of the
downgrading which could be dangerous.
bq. There are certainly cases where user should be informed when consistency
changes in order to perform custom action. For this purpose we could
allow/require user to register callback function which will be triggered when
consistency level changes. Best practices could be enforced by requiring
callback.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7642) Adaptive Consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14079405#comment-14079405
]
Rustam Aliyev commented on CASSANDRA-7642:
--
bq. Then, if the request timed out, and you see at least 1 in the 'received'
field of the WriteTimeoutException, you have your 'min' condition satisfied.
That would be performance degradation. That's basically what
DowngradingConsistencyRetryPolicy does.
Adaptive Consistency
Key: CASSANDRA-7642
URL: https://issues.apache.org/jira/browse/CASSANDRA-7642
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Rustam Aliyev
Fix For: 3.0
h4. Problem
At minimum, application requires consistency level of X, which must be fault
tolerant CL. However, when there is no failure it would be advantageous to
use stronger consistency Y (YX).
h4. Suggestion
Application defines minimum (X) and maximum (Y) consistency levels. C* can
apply adaptive consistency logic to use Y whenever possible and downgrade to
X when failure occurs.
Implementation should not negatively impact performance. Therefore, state has
to be maintained globally (not per request).
h4. Example
{{MIN_CL=LOCAL_QUORUM}}
{{MAX_CL=EACH_QUORUM}}
h4. Use Case
Consider a case where user wants to maximize their uptime and consistency.
They designing a system using C* where transactions are read/written with
LOCAL_QUORUM and distributed across 2 DCs. Occasional inconsistencies between
DCs can be tolerated. R/W with LOCAL_QUORUM is satisfactory in most of the
cases.
Application requires new transactions to be read back right after they were
generated. Write and read could be done through different DCs (no
stickiness). In some cases when user writes into DC1 and reads immediately
from DC2, replication delay may cause problems. Transaction won't show up on
read in DC2, user will retry and create duplicate transaction. Occasional
duplicates are fine and the goal is to minimize number of dups.
Therefore, we want to perform writes with stronger consistency (EACH_QUORUM)
whenever possible without compromising on availability. Using adaptive
consistency they should be able to define:
{{Read CL = LOCAL_QUORUM}}
{{Write CL = ADAPTIVE (MIN:LOCAL_QUORUM, MAX:EACH_QUORUM)}}
Similar scenario can be described for {{Write CL = ADAPTIVE (MIN:QUORUM,
MAX:ALL)}} case.
h4. Criticism
# This functionality can/should be implemented by user himself.
bq. It will be hard for an average user to implement topology monitoring and
state machine. Moreover, this is a pattern which repeats.
# Transparent downgrading violates the CL contract, and that contract
considered be just about the most important element of Cassandra's runtime
behavior.
bq.Fully transparent downgrading without any contract is dangerous. However,
would it be problem if we specify explicitly only two discrete CL levels -
MIN_CL and MAX_CL?
# If you have split brain DCs (partitioned in CAP), you have to sacrifice
either consistency or availability, and auto downgrading sacrifices the
consistency in dangerous ways if the application isn't designed to handle it.
And if the application is designed to handle it, then it should be able to
handle it in normal circumstances, not just degraded/extraordinary ones.
bq. Agreed. Application should be designed for MIN_CL. In that case, MAX_CL
will not be causing much harm, only adding flexibility.
# It might be a better idea to loudly downgrade, instead of silently
downgrading, meaning that the client code does an explicit retry with lower
consistency on failure and takes some other kind of action to attempt to
inform either users or operators of the problem. The silent part of the
downgrading which could be dangerous.
bq. There are certainly cases where user should be informed when consistency
changes in order to perform custom action. For this purpose we could
allow/require user to register callback function which will be triggered when
consistency level changes. Best practices could be enforced by requiring
callback.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7642) Adaptive Consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080042#comment-14080042
]
Rustam Aliyev commented on CASSANDRA-7642:
--
Many interesting points and ideas. But it sounds like discussion is getting a
bit too broad. Just to reinforce the main point:
bq. Writes - how many nodes do I wait to hear back from?
If possible EACH_Q (99.9%), but app designed to work with LOCAL_Q (0.1%). For
details see use case above (it's simplified version of real use case).
It is in fact niche use case and from what I can see mostly would be useful in
multi-DC setups.
Adaptive Consistency
Key: CASSANDRA-7642
URL: https://issues.apache.org/jira/browse/CASSANDRA-7642
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Rustam Aliyev
Fix For: 3.0
h4. Problem
At minimum, application requires consistency level of X, which must be fault
tolerant CL. However, when there is no failure it would be advantageous to
use stronger consistency Y (YX).
h4. Suggestion
Application defines minimum (X) and maximum (Y) consistency levels. C* can
apply adaptive consistency logic to use Y whenever possible and downgrade to
X when failure occurs.
Implementation should not negatively impact performance. Therefore, state has
to be maintained globally (not per request).
h4. Example
{{MIN_CL=LOCAL_QUORUM}}
{{MAX_CL=EACH_QUORUM}}
h4. Use Case
Consider a case where user wants to maximize their uptime and consistency.
They designing a system using C* where transactions are read/written with
LOCAL_QUORUM and distributed across 2 DCs. Occasional inconsistencies between
DCs can be tolerated. R/W with LOCAL_QUORUM is satisfactory in most of the
cases.
Application requires new transactions to be read back right after they were
generated. Write and read could be done through different DCs (no
stickiness). In some cases when user writes into DC1 and reads immediately
from DC2, replication delay may cause problems. Transaction won't show up on
read in DC2, user will retry and create duplicate transaction. Occasional
duplicates are fine and the goal is to minimize number of dups.
Therefore, we want to perform writes with stronger consistency (EACH_QUORUM)
whenever possible without compromising on availability. Using adaptive
consistency they should be able to define:
{{Read CL = LOCAL_QUORUM}}
{{Write CL = ADAPTIVE (MIN:LOCAL_QUORUM, MAX:EACH_QUORUM)}}
Similar scenario can be described for {{Write CL = ADAPTIVE (MIN:QUORUM,
MAX:ALL)}} case.
h4. Criticism
# This functionality can/should be implemented by user himself.
bq. It will be hard for an average user to implement topology monitoring and
state machine. Moreover, this is a pattern which repeats.
# Transparent downgrading violates the CL contract, and that contract
considered be just about the most important element of Cassandra's runtime
behavior.
bq.Fully transparent downgrading without any contract is dangerous. However,
would it be problem if we specify explicitly only two discrete CL levels -
MIN_CL and MAX_CL?
# If you have split brain DCs (partitioned in CAP), you have to sacrifice
either consistency or availability, and auto downgrading sacrifices the
consistency in dangerous ways if the application isn't designed to handle it.
And if the application is designed to handle it, then it should be able to
handle it in normal circumstances, not just degraded/extraordinary ones.
bq. Agreed. Application should be designed for MIN_CL. In that case, MAX_CL
will not be causing much harm, only adding flexibility.
# It might be a better idea to loudly downgrade, instead of silently
downgrading, meaning that the client code does an explicit retry with lower
consistency on failure and takes some other kind of action to attempt to
inform either users or operators of the problem. The silent part of the
downgrading which could be dangerous.
bq. There are certainly cases where user should be informed when consistency
changes in order to perform custom action. For this purpose we could
allow/require user to register callback function which will be triggered when
consistency level changes. Best practices could be enforced by requiring
callback.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7642) Adaptive Consistency
[
https://issues.apache.org/jira/browse/CASSANDRA-7642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14080464#comment-14080464
]
Rustam Aliyev commented on CASSANDRA-7642:
--
Here's better version of use case which highlights the goal better. DC concept
in C* is essentially a Replication Group (RG) and I will use this term to draw
a parallel between RG and simple C* replica:
In context of RG, today C* allows ONE (LOCAL_QUORUM) and ALL (EACH_QUORUM).
This means, if I want to achieve strong consistency across RGs, I can only do
R+W as ONE+ALL or ALL+ONE. This is consistent with known R + W N formula and
this isn't fault tolerant.
Today we don't support R+W as QUORUM+QUORUM in context of RGs which would
guarantee strong consistency and at the same time be fault tolerant. *This is
important for the type of applications which are completely stateless across
RGs.*
We could have some sort of QUORUM_QUORUM for that, where it would require
QUORUM of RGs to respond. However, that will require at least 3 RGs which can
be expensive.
The main goal of AC is to solve this problem with 2 RGs.
Adaptive Consistency
Key: CASSANDRA-7642
URL: https://issues.apache.org/jira/browse/CASSANDRA-7642
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Rustam Aliyev
Fix For: 3.0
h4. Problem
At minimum, application requires consistency level of X, which must be fault
tolerant CL. However, when there is no failure it would be advantageous to
use stronger consistency Y (YX).
h4. Suggestion
Application defines minimum (X) and maximum (Y) consistency levels. C* can
apply adaptive consistency logic to use Y whenever possible and downgrade to
X when failure occurs.
Implementation should not negatively impact performance. Therefore, state has
to be maintained globally (not per request).
h4. Example
{{MIN_CL=LOCAL_QUORUM}}
{{MAX_CL=EACH_QUORUM}}
h4. Use Case
Consider a case where user wants to maximize their uptime and consistency.
They designing a system using C* where transactions are read/written with
LOCAL_QUORUM and distributed across 2 DCs. Occasional inconsistencies between
DCs can be tolerated. R/W with LOCAL_QUORUM is satisfactory in most of the
cases.
Application requires new transactions to be read back right after they were
generated. Write and read could be done through different DCs (no
stickiness). In some cases when user writes into DC1 and reads immediately
from DC2, replication delay may cause problems. Transaction won't show up on
read in DC2, user will retry and create duplicate transaction. Occasional
duplicates are fine and the goal is to minimize number of dups.
Therefore, we want to perform writes with stronger consistency (EACH_QUORUM)
whenever possible without compromising on availability. Using adaptive
consistency they should be able to define:
{{Read CL = LOCAL_QUORUM}}
{{Write CL = ADAPTIVE (MIN:LOCAL_QUORUM, MAX:EACH_QUORUM)}}
Similar scenario can be described for {{Write CL = ADAPTIVE (MIN:QUORUM,
MAX:ALL)}} case.
h4. Criticism
# This functionality can/should be implemented by user himself.
bq. It will be hard for an average user to implement topology monitoring and
state machine. Moreover, this is a pattern which repeats.
# Transparent downgrading violates the CL contract, and that contract
considered be just about the most important element of Cassandra's runtime
behavior.
bq.Fully transparent downgrading without any contract is dangerous. However,
would it be problem if we specify explicitly only two discrete CL levels -
MIN_CL and MAX_CL?
# If you have split brain DCs (partitioned in CAP), you have to sacrifice
either consistency or availability, and auto downgrading sacrifices the
consistency in dangerous ways if the application isn't designed to handle it.
And if the application is designed to handle it, then it should be able to
handle it in normal circumstances, not just degraded/extraordinary ones.
bq. Agreed. Application should be designed for MIN_CL. In that case, MAX_CL
will not be causing much harm, only adding flexibility.
# It might be a better idea to loudly downgrade, instead of silently
downgrading, meaning that the client code does an explicit retry with lower
consistency on failure and takes some other kind of action to attempt to
inform either users or operators of the problem. The silent part of the
downgrading which could be dangerous.
bq. There are certainly cases where user should be informed when consistency
changes in order to perform custom action. For this purpose we could
allow/require user to register callback function which will be triggered when
consistency level changes. Best practices could be enforced by requiring
callback.
--
This
[jira] [Created] (CASSANDRA-7637) Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
Rustam Aliyev created CASSANDRA-7637:
Summary: Add CQL3 keyword for efficient lexical range queries
(e.g. START_WITH)
Key: CASSANDRA-7637
URL: https://issues.apache.org/jira/browse/CASSANDRA-7637
Project: Cassandra
Issue Type: Improvement
Reporter: Rustam Aliyev
Currently, if I want to perform range query on lexical type I need to do
something like this:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute 'interests.food.' AND
attribute 'interests.food.z';
{code}
This is very efficient range query. Yet, many users who are not familiar with
Thrift and storage level implementation are unaware of this trick.
Therefore, it would be convenient to introduce CQL keyword which will do this
more simply:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute START_WITH('interests.food.');
{code}
Keyword would have same restrictions as other inequality search operators plus
some type restrictions.
Allowed types would be:
* {{ascii}}
* {{text}} / {{varchar}}
* {{inet}} (?)
* {{maptext, *}} (same for ascii) (?)
* {{settext}} (same for ascii) (?)
(?) may require more work, therefore optional
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7637) Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
[
https://issues.apache.org/jira/browse/CASSANDRA-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rustam Aliyev updated CASSANDRA-7637:
-
Component/s: API
Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
--
Key: CASSANDRA-7637
URL: https://issues.apache.org/jira/browse/CASSANDRA-7637
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Rustam Aliyev
Currently, if I want to perform range query on lexical type I need to do
something like this:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute 'interests.food.' AND
attribute 'interests.food.z';
{code}
This is very efficient range query. Yet, many users who are not familiar with
Thrift and storage level implementation are unaware of this trick.
Therefore, it would be convenient to introduce CQL keyword which will do this
more simply:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute START_WITH('interests.food.');
{code}
Keyword would have same restrictions as other inequality search operators
plus some type restrictions.
Allowed types would be:
* {{ascii}}
* {{text}} / {{varchar}}
* {{inet}} (?)
* {{maptext, *}} (same for ascii) (?)
* {{settext}} (same for ascii) (?)
(?) may require more work, therefore optional
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7637) Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
[
https://issues.apache.org/jira/browse/CASSANDRA-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rustam Aliyev updated CASSANDRA-7637:
-
Issue Type: New Feature (was: Improvement)
Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
--
Key: CASSANDRA-7637
URL: https://issues.apache.org/jira/browse/CASSANDRA-7637
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Rustam Aliyev
Currently, if I want to perform range query on lexical type I need to do
something like this:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute 'interests.food.' AND
attribute 'interests.food.z';
{code}
This is very efficient range query. Yet, many users who are not familiar with
Thrift and storage level implementation are unaware of this trick.
Therefore, it would be convenient to introduce CQL keyword which will do this
more simply:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute START_WITH('interests.food.');
{code}
Keyword would have same restrictions as other inequality search operators
plus some type restrictions.
Allowed types would be:
* {{ascii}}
* {{text}} / {{varchar}}
* {{inet}} (?)
* {{maptext, *}} (same for ascii) (?)
* {{settext}} (same for ascii) (?)
(?) may require more work, therefore optional
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7642) Adaptive Consistency
Rustam Aliyev created CASSANDRA-7642:
Summary: Adaptive Consistency
Key: CASSANDRA-7642
URL: https://issues.apache.org/jira/browse/CASSANDRA-7642
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Rustam Aliyev
Fix For: 3.0
h4. Problem
At minimum, application requires consistency level of X, which must be fault
tolerant CL. However, when there is no failure it would be advantageous to use
stronger consistency Y (YX).
h4. Suggestion
Application defines minimum (X) and maximum (Y) consistency levels. C* can
apply adaptive consistency logic to use Y whenever possible and downgrade to X
when failure occurs.
Implementation should not negatively impact performance. Therefore, state has
to be maintained globally (not per request).
h4. Example
{{MIN_CL=LOCAL_QUORUM}}
{{MAX_CL=EACH_QUORUM}}
h4. Use Case
Consider a case where user wants to maximize their uptime and consistency. They
designing a system using C* where transactions are read/written with
LOCAL_QUORUM and distributed across 2 DCs. Occasional inconsistencies between
DCs can be tolerated. R/W with LOCAL_QUORUM is satisfactory in most of the
cases.
Application requires new transactions to be read back right after they were
generated. Write and read could be done through different DCs (no stickiness).
In some cases when user writes into DC1 and reads immediately from DC2,
replication delay may cause problems. Transaction won't show up on read in DC2,
user will retry and create duplicate transaction. Occasional duplicates are
fine and the goal is to minimize number of dups.
Therefore, we want to perform writes with stronger consistency (EACH_QUORUM)
whenever possible without compromising on availability. Using adaptive
consistency they should be able to define:
{{Read CL = LOCAL_QUORUM}}
{{Write CL = ADAPTIVE (MIN:LOCAL_QUORUM, MAX:EACH_QUORUM)}}
Similar scenario can be described for {{Write CL = ADAPTIVE (MIN:QUORUM,
MAX:ALL)}} case.
h4. Criticism
# This functionality can/should be implemented by user himself.
bq. It will be hard for an average user to implement topology monitoring and
state machine. Moreover, this is a pattern which repeats.
# Transparent downgrading violates the CL contract, and that contract
considered be just about the most important element of Cassandra's runtime
behavior.
bq.Fully transparent downgrading without any contract is dangerous. However,
would it be problem if we specify explicitly only two discrete CL levels -
MIN_CL and MAX_CL?
# If you have split brain DCs (partitioned in CAP), you have to sacrifice
either consistency or availability, and auto downgrading sacrifices the
consistency in dangerous ways if the application isn't designed to handle it.
And if the application is designed to handle it, then it should be able to
handle it in normal circumstances, not just degraded/extraordinary ones.
bq. Agreed. Application should be designed for MIN_CL. In that case, MAX_CL
will not be causing much harm, only adding flexibility.
# It might be a better idea to loudly downgrade, instead of silently
downgrading, meaning that the client code does an explicit retry with lower
consistency on failure and takes some other kind of action to attempt to inform
either users or operators of the problem. The silent part of the downgrading
which could be dangerous.
bq. There are certainly cases where user should be informed when consistency
changes in order to perform custom action. For this purpose we could
allow/require user to register callback function which will be triggered when
consistency level changes. Best practices could be enforced by requiring
callback.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7637) Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
[
https://issues.apache.org/jira/browse/CASSANDRA-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rustam Aliyev updated CASSANDRA-7637:
-
Fix Version/s: 3.0
Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
--
Key: CASSANDRA-7637
URL: https://issues.apache.org/jira/browse/CASSANDRA-7637
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Rustam Aliyev
Fix For: 3.0
Currently, if I want to perform range query on lexical type I need to do
something like this:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute 'interests.food.' AND
attribute 'interests.food.z';
{code}
This is very efficient range query. Yet, many users who are not familiar with
Thrift and storage level implementation are unaware of this trick.
Therefore, it would be convenient to introduce CQL keyword which will do this
more simply:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute START_WITH('interests.food.');
{code}
Keyword would have same restrictions as other inequality search operators
plus some type restrictions.
Allowed types would be:
* {{ascii}}
* {{text}} / {{varchar}}
* {{inet}} (?)
* {{maptext, *}} (same for ascii) (?)
* {{settext}} (same for ascii) (?)
(?) may require more work, therefore optional
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7637) Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
[
https://issues.apache.org/jira/browse/CASSANDRA-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rustam Aliyev updated CASSANDRA-7637:
-
Description:
Currently, if I want to perform range query on lexical type I need to do
something like this:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute 'interests.food.' AND
attribute 'interests.food.z';
{code}
This is very efficient range query. Yet, many users who are not familiar with
Thrift and storage level implementation are unaware of this trick.
Therefore, it would be convenient to introduce CQL keyword which will do this
more simply:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute START_WITH('interests.food.');
{code}
Keyword would have same restrictions as other inequality search operators plus
some type restrictions.
Allowed types would be:
* {{ascii}}
* {{text}} / {{varchar}}
* {{maptext, *}} (same for ascii) (?)
* {{settext}} (same for ascii) (?)
(?) may require more work, therefore optional
was:
Currently, if I want to perform range query on lexical type I need to do
something like this:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute 'interests.food.' AND
attribute 'interests.food.z';
{code}
This is very efficient range query. Yet, many users who are not familiar with
Thrift and storage level implementation are unaware of this trick.
Therefore, it would be convenient to introduce CQL keyword which will do this
more simply:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute START_WITH('interests.food.');
{code}
Keyword would have same restrictions as other inequality search operators plus
some type restrictions.
Allowed types would be:
* {{ascii}}
* {{text}} / {{varchar}}
* {{inet}} (?)
* {{maptext, *}} (same for ascii) (?)
* {{settext}} (same for ascii) (?)
(?) may require more work, therefore optional
Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
--
Key: CASSANDRA-7637
URL: https://issues.apache.org/jira/browse/CASSANDRA-7637
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Rustam Aliyev
Fix For: 3.0
Currently, if I want to perform range query on lexical type I need to do
something like this:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute 'interests.food.' AND
attribute 'interests.food.z';
{code}
This is very efficient range query. Yet, many users who are not familiar with
Thrift and storage level implementation are unaware of this trick.
Therefore, it would be convenient to introduce CQL keyword which will do this
more simply:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute START_WITH('interests.food.');
{code}
Keyword would have same restrictions as other inequality search operators
plus some type restrictions.
Allowed types would be:
* {{ascii}}
* {{text}} / {{varchar}}
* {{maptext, *}} (same for ascii) (?)
* {{settext}} (same for ascii) (?)
(?) may require more work, therefore optional
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7637) Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
[
https://issues.apache.org/jira/browse/CASSANDRA-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078599#comment-14078599
]
Rustam Aliyev commented on CASSANDRA-7637:
--
Agree, LIKE would be more SQL compatible way to do it. It will be very limited
functionality for LIKE, only {{%}} wildcard in the end of an expression. But I
think it's still better.
I'm also removing inet type - I think there are better ways to do range queries
on that one.
Btw, in your example above it should be {{%}} in the end.
Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
--
Key: CASSANDRA-7637
URL: https://issues.apache.org/jira/browse/CASSANDRA-7637
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Rustam Aliyev
Fix For: 3.0
Currently, if I want to perform range query on lexical type I need to do
something like this:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute 'interests.food.' AND
attribute 'interests.food.z';
{code}
This is very efficient range query. Yet, many users who are not familiar with
Thrift and storage level implementation are unaware of this trick.
Therefore, it would be convenient to introduce CQL keyword which will do this
more simply:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute START_WITH('interests.food.');
{code}
Keyword would have same restrictions as other inequality search operators
plus some type restrictions.
Allowed types would be:
* {{ascii}}
* {{text}} / {{varchar}}
* {{inet}} (?)
* {{maptext, *}} (same for ascii) (?)
* {{settext}} (same for ascii) (?)
(?) may require more work, therefore optional
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7637) Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
[
https://issues.apache.org/jira/browse/CASSANDRA-7637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078599#comment-14078599
]
Rustam Aliyev edited comment on CASSANDRA-7637 at 7/29/14 11:47 PM:
Agree, LIKE would be more SQL compatible way to do it. It will be very limited
functionality for LIKE, only {{%}} wildcard in the end of an expression. But I
think it's still better.
Regarding wildcards, if we try maximize SQL compatibility, I'd choose {{%}}
instead of {{*}}.
I'm also removing inet type - I think there are better ways to do range queries
on that one.
was (Author: rstml):
Agree, LIKE would be more SQL compatible way to do it. It will be very limited
functionality for LIKE, only {{%}} wildcard in the end of an expression. But I
think it's still better.
I'm also removing inet type - I think there are better ways to do range queries
on that one.
Btw, in your example above it should be {{%}} in the end.
Add CQL3 keyword for efficient lexical range queries (e.g. START_WITH)
--
Key: CASSANDRA-7637
URL: https://issues.apache.org/jira/browse/CASSANDRA-7637
Project: Cassandra
Issue Type: New Feature
Components: API
Reporter: Rustam Aliyev
Fix For: 3.0
Currently, if I want to perform range query on lexical type I need to do
something like this:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute 'interests.food.' AND
attribute 'interests.food.z';
{code}
This is very efficient range query. Yet, many users who are not familiar with
Thrift and storage level implementation are unaware of this trick.
Therefore, it would be convenient to introduce CQL keyword which will do this
more simply:
{code}
SELECT * FROM profile WHERE profile_id = 123 AND
attribute START_WITH('interests.food.');
{code}
Keyword would have same restrictions as other inequality search operators
plus some type restrictions.
Allowed types would be:
* {{ascii}}
* {{text}} / {{varchar}}
* {{maptext, *}} (same for ascii) (?)
* {{settext}} (same for ascii) (?)
(?) may require more work, therefore optional
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6477) Global indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14032102#comment-14032102
]
Rustam Aliyev commented on CASSANDRA-6477:
--
In addition to performance, one of the key advantages of application-maintained
global indexes is flexibility. I think it's important to preserve it in
built-in global indexes. Few cases I think important to consider:
# Composite index. Global index can be based on more than one column.
# Range query on indexed elements. With high cardinality global index it would
be efficient to allow range query on elements to make consecutive multiget
efficient. For example, indexing time-series data by type and then looking up
with {{... TYPE=type1 and ID minTimeuuid('2013-02-02 10:00+')}}
# Reverse key index. Should be able to define index clustering key (i.e.
indexed elements) order (ASC, DESC). Helpful when used with range queries above.
# Function based index. In this case, index is defined by transformation
function. For example, lowercase(value) or arithmetic function like (field1 *
field2).
# Storing data in index. Typically, global indexes have following structure
where values are nulls:
{code}
idx_table {
index_value1 : {
el_id1 : null,
el_id5 : null,
...
}
}
{code}
However, sometimes it's efficient and convenient to keep some information in
values. For example, let's assume that elements above contains tens of fields.
However, in 90% cases application uses only one of those e.g. hash. In that
case, it's efficient to scan index and retrieve hash values directly from index
instead of doing additional lookup to original table. Above table would looks
like:
{code}
idx_table {
index_value1 : {
el_id1 : 74335a7c9229...,
el_id5 : 28b986fa29eb...,
...
}
}
{code}
Traditional RDBMS support most of these indexes. For function based indexes we
could create a bunch of functions in CQL3 (e.g. Math.*, LOWERCASE(), etc.)
similar to other RDBMS.
Alternatively, we can achieve greater flexibility by storing optional Java 8
lambda functions. Lambda function will take mutated row as an input and return
2 vars:
# non-empty set of indexes (required)
# map of id - value which will be used to lookup stored index values
(optional). If element not found, null is stored.
{{CREATE INDEX}} statement has to define produced index CQL type and optionally
stored index values:
{code}
CREATE GLOBAL INDEX account_by_email_idx ON accounts ( LAMBDA(row - { return
row.email.toLowerCase(); }) ) WITH INDEX_TYPE = {'text'};
{code}
More examples:
# Lowercase email: {code} row - { return row.email.toLowerCase(); } {code}
# Distance between coordinates: {code} row - { return
Math.sqrt((row.x1-row.x2)*(row.x1-row.x2) + (row.y1-row.y2)*(row.y1-row.y2)); }
{code}
# Conditional index: {code} row - { return row.price 0 ? paid : free; }
{code}
# Indexes with values (item 5 above) may require some special return type (e.g.
{{IndexWithValues}}). In the example above, message length will be stored in
the index: {code} row - { return new IndexWithValues(row.type,
row.message.length()); } {code}
Querying these indexes is another caveat. Consider distance between coordinates
example above - what would be SELECT statement for this index? With
application-maintained global indexes, application can just lookup in index
using given value. Same applies to indexes with stored values.
Without these, built-in global indexes will be very limited and once again,
application-maintained global indexes would remain as go to solution.
Global indexes
--
Key: CASSANDRA-6477
URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
Project: Cassandra
Issue Type: New Feature
Components: API, Core
Reporter: Jonathan Ellis
Fix For: 3.0
Local indexes are suitable for low-cardinality data, where spreading the
index across the cluster is a Good Thing. However, for high-cardinality
data, local indexes require querying most nodes in the cluster even if only a
handful of rows is returned.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-6477) Global indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14032102#comment-14032102
]
Rustam Aliyev edited comment on CASSANDRA-6477 at 6/16/14 3:57 AM:
---
In addition to performance, one of the key advantages of application-maintained
global indexes is flexibility. I think it's important to preserve it in
built-in global indexes. Few cases I think important to consider:
# Composite index. Global index can be based on more than one column.
# Range query on indexed elements. With high cardinality global index it would
be efficient to allow range query on elements to make consecutive multiget
efficient. For example, indexing time-series data by type and then looking up
with {{... TYPE=type1 and ID minTimeuuid('2013-02-02 10:00+')}}
# Reverse key index. Should be able to define index clustering key (i.e.
indexed elements) order (ASC, DESC). Helpful when used with range queries
above. E.g. {code}CREATE GLOBAL INDEX dpt_emp_idx ON employee(department_id
DESC, name ASC); {code}
# Function based index. In this case, index is defined by transformation
function. For example, lowercase(value) or arithmetic function like (field1 *
field2).
# Storing data in index. Typically, global indexes have following structure
where values are nulls:
{code}
idx_table {
index_value1 : {
el_id1 : null,
el_id5 : null,
...
}
}
{code}
However, sometimes it's efficient and convenient to keep some information in
values. For example, let's assume that elements above contains tens of fields.
However, in 90% cases application uses only one of those e.g. hash. In that
case, it's efficient to scan index and retrieve hash values directly from index
instead of doing additional lookup to original table. Above table would looks
like:
{code}
idx_table {
index_value1 : {
el_id1 : 74335a7c9229...,
el_id5 : 28b986fa29eb...,
...
}
}
{code}
Traditional RDBMS support most of these indexes. For function based indexes we
could create a bunch of functions in CQL3 (e.g. Math.*, LOWERCASE(), etc.)
similar to other RDBMS.
Alternatively, we can achieve greater flexibility by storing optional Java 8
lambda functions. Lambda function will take mutated row as an input and return
2 vars:
# non-empty set of indexes (required)
# map of id - value which will be used to lookup stored index values
(optional). If element not found, null is stored.
{{CREATE INDEX}} statement has to define produced index CQL type and optionally
stored index values:
{code}
CREATE GLOBAL INDEX account_by_email_idx ON accounts ( LAMBDA(row - { return
row.email.toLowerCase(); }) ) WITH INDEX_TYPE = {'text'};
{code}
More examples:
# Lowercase email: {code} row - { return row.email.toLowerCase(); } {code}
# Distance between coordinates: {code} row - { return
Math.sqrt((row.x1-row.x2)*(row.x1-row.x2) + (row.y1-row.y2)*(row.y1-row.y2)); }
{code}
# Conditional index: {code} row - { return row.price 0 ? paid : free; }
{code}
# Indexes with values (item 5 above) may require some special return type (e.g.
{{IndexWithValues}}). In the example above, message length will be stored in
the index: {code} row - { return new IndexWithValues(row.type,
row.message.length()); } {code}
Querying these indexes is another caveat. Consider distance between coordinates
example above - what would be SELECT statement for this index? With
application-maintained global indexes, application can just lookup in index
using given value. Same applies to indexes with stored values.
Without these, built-in global indexes will be very limited and once again,
application-maintained global indexes would remain as go to solution.
was (Author: rstml):
In addition to performance, one of the key advantages of application-maintained
global indexes is flexibility. I think it's important to preserve it in
built-in global indexes. Few cases I think important to consider:
# Composite index. Global index can be based on more than one column.
# Range query on indexed elements. With high cardinality global index it would
be efficient to allow range query on elements to make consecutive multiget
efficient. For example, indexing time-series data by type and then looking up
with {{... TYPE=type1 and ID minTimeuuid('2013-02-02 10:00+')}}
# Reverse key index. Should be able to define index clustering key (i.e.
indexed elements) order (ASC, DESC). Helpful when used with range queries above.
# Function based index. In this case, index is defined by transformation
function. For example, lowercase(value) or arithmetic function like (field1 *
field2).
# Storing data in index. Typically, global indexes have following structure
where values are nulls:
{code}
idx_table {
index_value1 : {
el_id1 : null,
el_id5 : null,
...
}
}
{code}
However, sometimes it's
[jira] [Commented] (CASSANDRA-3237) refactor super column implmentation to use composite column names instead
[ https://issues.apache.org/jira/browse/CASSANDRA-3237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13447255#comment-13447255 ] Rustam Aliyev commented on CASSANDRA-3237: -- Is there any page/post describing how exactly SC will be implemented using Composite Columns? refactor super column implmentation to use composite column names instead - Key: CASSANDRA-3237 URL: https://issues.apache.org/jira/browse/CASSANDRA-3237 Project: Cassandra Issue Type: Improvement Reporter: Matthew F. Dennis Assignee: Vijay Priority: Minor Labels: ponies Fix For: 1.3 Attachments: cassandra-supercolumn-irc.log super columns are annoying. composite columns offer a better API and performance. people should use composites over super columns. some people are already using super columns. C* should implement the super column API in terms of composites to reduce code, complexity and testing as well as increase performance. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-47) SSTable compression
[ https://issues.apache.org/jira/browse/CASSANDRA-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13002685#comment-13002685 ] Rustam Aliyev commented on CASSANDRA-47: Just wanted to share this benchmark from QuickLZ: http://www.quicklz.com/bench.html LZF performance quite impressive in terms of both - compress/decompress speeds and compression ratio. There's another LZF implementation available under Apache 2 license: https://github.com/ning/compress SSTable compression --- Key: CASSANDRA-47 URL: https://issues.apache.org/jira/browse/CASSANDRA-47 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Priority: Minor Fix For: 0.8 We should be able to do SSTable compression which would trade CPU for I/O (almost always a good trade). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1832) mx4j does not load
[
https://issues.apache.org/jira/browse/CASSANDRA-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12968833#action_12968833
]
Rustam Aliyev commented on CASSANDRA-1832:
--
I did a bit more tests and here are some results which might help:
1. JMX port set to 9090 in cassandra-env.sh
2. On the machine where another service running on 8080 we get exception above
3. On the machine where no service running on 8080 we don't get any exception
and MX4J runs on port 9090
Seems like something checks for port 8080 even though it is configured to run
on 9090.
mx4j does not load
--
Key: CASSANDRA-1832
URL: https://issues.apache.org/jira/browse/CASSANDRA-1832
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 0.7.0 rc 1
Environment: CentOS 5.5
Reporter: Rustam Aliyev
Priority: Minor
Fix For: 0.7.0
Adding mx4j-tools.jar (latest) to the library causes following exception:
{code}
WARN 20:22:25,123 Could not start register mbean in JMX
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.cassandra.utils.Mx4jTool.maybeLoad(Mx4jTool.java:67)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:169)
at
org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:55)
at
org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:216)
at
org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:134)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:365)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.init(ServerSocket.java:185)
at
mx4j.tools.adaptor.PlainAdaptorServerSocketFactory.createServerSocket(PlainAdaptorServerSocketFactory.java:24)
at
mx4j.tools.adaptor.http.HttpAdaptor.createServerSocket(HttpAdaptor.java:672)
at mx4j.tools.adaptor.http.HttpAdaptor.start(HttpAdaptor.java:478)
... 9 more
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1832) mx4j does not load
[
https://issues.apache.org/jira/browse/CASSANDRA-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12969035#action_12969035
]
Rustam Aliyev commented on CASSANDRA-1832:
--
You right Ran, I checked this machine again and I have another service
listening on 8081.
For some reason I thought that MX4J uses same port.
With config options we can close it now.
mx4j does not load
--
Key: CASSANDRA-1832
URL: https://issues.apache.org/jira/browse/CASSANDRA-1832
Project: Cassandra
Issue Type: Bug
Components: Tools
Affects Versions: 0.7.0 rc 1
Environment: CentOS 5.5
Reporter: Rustam Aliyev
Assignee: Jonathan Ellis
Priority: Minor
Fix For: 0.7.0
Attachments: CASSANDRA-1832.patch
Adding mx4j-tools.jar (latest) to the library causes following exception:
{code}
WARN 20:22:25,123 Could not start register mbean in JMX
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.cassandra.utils.Mx4jTool.maybeLoad(Mx4jTool.java:67)
at
org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:169)
at
org.apache.cassandra.thrift.CassandraDaemon.setup(CassandraDaemon.java:55)
at
org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:216)
at
org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:134)
Caused by: java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:365)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.init(ServerSocket.java:185)
at
mx4j.tools.adaptor.PlainAdaptorServerSocketFactory.createServerSocket(PlainAdaptorServerSocketFactory.java:24)
at
mx4j.tools.adaptor.http.HttpAdaptor.createServerSocket(HttpAdaptor.java:672)
at mx4j.tools.adaptor.http.HttpAdaptor.start(HttpAdaptor.java:478)
... 9 more
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
