[jira] [Resolved] (CASSANDRA-4560) Add tracing support for CQL3 bind variables
[ https://issues.apache.org/jira/browse/CASSANDRA-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves resolved CASSANDRA-4560. Resolution: Fixed > Add tracing support for CQL3 bind variables > --- > > Key: CASSANDRA-4560 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4560 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2.1 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459019#comment-13459019 ] David Alves edited comment on CASSANDRA-4584 at 9/20/12 6:48 AM: - separated logging for prepared query submission, execution and regular query execution, this pretty much includes CASSANDRA-4560 which is now included in this patch. was (Author: dr-alves): separated logging for prepared query submission, execution and regular query execution, this pretty much includes CASSANDRA-4560 which is not included in this patch. > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Labels: cql3 > Fix For: 1.2.0 beta 2 > > Attachments: 4584.patch, 4584.patch, 4584.patch, 4584.patch, > 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4584: --- Attachment: 4584.patch > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Labels: cql3 > Fix For: 1.2.0 beta 2 > > Attachments: 4584.patch, 4584.patch, 4584.patch, 4584.patch, > 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459019#comment-13459019 ] David Alves commented on CASSANDRA-4584: separated logging for prepared query submission, execution and regular query execution, this pretty much includes CASSANDRA-4560 which is not included in this patch. > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Labels: cql3 > Fix For: 1.2.0 beta 2 > > Attachments: 4584.patch, 4584.patch, 4584.patch, 4584.patch, > 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4560) Add tracing support for CQL3 bind variables
[ https://issues.apache.org/jira/browse/CASSANDRA-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4560: --- Attachment: (was: 4560.patch) > Add tracing support for CQL3 bind variables > --- > > Key: CASSANDRA-4560 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4560 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2.1 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4584: --- Attachment: 4584.patch moved the initial logging of cql queries into QueryProcessor in the cql3 case because sessions were being started before and we needed to log params. > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Labels: cql3 > Fix For: 1.2.0 beta 2 > > Attachments: 4584.patch, 4584.patch, 4584.patch, 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4585) Add cqlsh support for tracing results
[ https://issues.apache.org/jira/browse/CASSANDRA-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4585: --- Attachment: 4585.patch added pretty printing to the uuid. the result can be used directly to query the sessions/events table. > Add cqlsh support for tracing results > - > > Key: CASSANDRA-4585 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4585 > Project: Cassandra > Issue Type: Sub-task > Components: Tools >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4585.patch, 4585.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4560) Add tracing support for CQL3 bind variables
[ https://issues.apache.org/jira/browse/CASSANDRA-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4560: --- Attachment: 4560.patch simple patch that exposes bound variable names by exposing ParsedStatement.Prepared in ClientState instead of exposing CQLStatement directly. > Add tracing support for CQL3 bind variables > --- > > Key: CASSANDRA-4560 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4560 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Priority: Minor > Fix For: 1.2.1 > > Attachments: 4560.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4560) Add tracing support for CQL3 bind variables
[ https://issues.apache.org/jira/browse/CASSANDRA-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4560: --- Assignee: David Alves > Add tracing support for CQL3 bind variables > --- > > Key: CASSANDRA-4560 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4560 > Project: Cassandra > Issue Type: Sub-task >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2.1 > > Attachments: 4560.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4585) Add cqlsh support for tracing results
[ https://issues.apache.org/jira/browse/CASSANDRA-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458830#comment-13458830 ] David Alves commented on CASSANDRA-4585: we need it to be able to query the trace tables so i'm planning on adding it, just need some more time > Add cqlsh support for tracing results > - > > Key: CASSANDRA-4585 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4585 > Project: Cassandra > Issue Type: Sub-task > Components: Tools >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4585.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4585) Add cqlsh support for tracing results
[ https://issues.apache.org/jira/browse/CASSANDRA-4585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4585: --- Attachment: 4585.patch requires http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/issues/detail?id=24 to go in. just missing pretty printing the uuid. > Add cqlsh support for tracing results > - > > Key: CASSANDRA-4585 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4585 > Project: Cassandra > Issue Type: Sub-task > Components: Tools >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: Aleksey Yeschenko > Fix For: 1.2.0 > > Attachments: 4585.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458806#comment-13458806 ] David Alves commented on CASSANDRA-4584: moving cqlsh to another issue makes this one pretty much done (before review that is) > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4584.patch, 4584.patch, 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4584: --- Attachment: 4584.patch reverted changes to cqlsh as those go into CASSANDRA-4585 > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4584.patch, 4584.patch, 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4584: --- Attachment: 4584.patch moves enabling tracing into queryprocessor, still missing uuid pretty printing in cqlsh. > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4584.patch, 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458746#comment-13458746 ] David Alves commented on CASSANDRA-4584: bq. But honestly I'm not sure there is a point in disallowing tracing for some query, especially since that's more work than doing it for all queries. Besides, for any query it could be interesting to know how long it took to parse it, to validate it and to execute it. I see no reason why tracing should not be enabled for schema querying/altering statements, but since "trace" is keyword we only know wether to trace *after* we parse the query, which will make it difficult to track parse times. > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458492#comment-13458492 ] David Alves commented on CASSANDRA-4584: the non-schema-tracing behavior is legacy from the original patch, but I can't actually think of a reason not to trace schema altering/querying statements, so I'll add it where you suggest (and also add it to thrift for that matter). > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458471#comment-13458471 ] David Alves commented on CASSANDRA-4584: Allowing tracing only on Modification/Select and Truncate statements was deliberate as it mimics what is done in the thrift interface (cf/keyspace methods do not enable tracing), but I hadn't thought of placing it in QueryProcessor, maybe it makes more sense there. Still you raise a point should schema altering methods be traced? > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458458#comment-13458458 ] David Alves commented on CASSANDRA-4584: Note: patch needs an updated version of lib/cql-internal-only-1.2.0.zip that can be built with the patch in http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/issues/detail?id=24 > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458454#comment-13458454 ] David Alves edited comment on CASSANDRA-4584 at 9/19/12 5:37 PM: - patch that adds the keyword "trace" to cql3. tracing is forced for a query when the keyword is prepended (e.g. "trace select * from "Standard1";") the CqlResult from the query includes the session uuid so that clients may use it to query the events/sessions tables adds support for showing said uuid in cqlsh *but* couldn't figure out how to format it in intelligible form (only gibberish is printed, uuid is returned as bytes, tried using the formatting methods but ultimately gave up, need help). also altered cassandra-dbapi2 so that cursor now includes the uuid from the reponse (submitted patch in http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/issues/detail?id=24) was (Author: dr-alves): patch that adds the keyword "trace" to cql3. tracing is forced for a query when the keyword is prepended (e.g. "trace select * from "Standard1";) the CqlResult from the query includes the session uuid so that clients may use it to query the events/sessions tables adds support for showing said uuid in cqlsh *but* couldn't figure out how to format it in intelligible form (only gibberish is printed, uuid is returned as bytes, tried using the formatting methods but ultimately gave up, need help). also altered cassandra-dbapi2 so that cursor now includes the uuid from the reponse (submitted patch in http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/issues/detail?id=24) > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4584: --- Attachment: 4584.patch patch that adds the keyword "trace" to cql3. tracing is forced for a query when the keyword is prepended (e.g. "trace select * from "Standard1";) the CqlResult from the query includes the session uuid so that clients may use it to query the events/sessions tables adds support for showing said uuid in cqlsh *but* couldn't figure out how to format it in intelligible form (only gibberish is printed, uuid is returned as bytes, tried using the formatting methods but ultimately gave up, need help). also altered cassandra-dbapi2 so that cursor now includes the uuid from the reponse (submitted patch in http://code.google.com/a/apache-extras.org/p/cassandra-dbapi2/issues/detail?id=24) > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 4584.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4672) _TRACE verb is not droppable which causes an AssertionError
[ https://issues.apache.org/jira/browse/CASSANDRA-4672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4672: --- Attachment: 4672.patch attaching trivial fix > _TRACE verb is not droppable which causes an AssertionError > --- > > Key: CASSANDRA-4672 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4672 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.2.0 >Reporter: David Alves >Assignee: David Alves >Priority: Trivial > Fix For: 1.2.0 > > Attachments: 4672.patch > > > When a big enough statement is traced (like select *) an assertion error is > fired because the _TRACE verb is not droppable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4672) _TRACE verb is not droppable which causes an AssertionError
[ https://issues.apache.org/jira/browse/CASSANDRA-4672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4672: --- Assignee: David Alves > _TRACE verb is not droppable which causes an AssertionError > --- > > Key: CASSANDRA-4672 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4672 > Project: Cassandra > Issue Type: Bug > Components: Core >Affects Versions: 1.2.0 >Reporter: David Alves >Assignee: David Alves >Priority: Trivial > Fix For: 1.2.0 > > > When a big enough statement is traced (like select *) an assertion error is > fired because the _TRACE verb is not droppable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4672) _TRACE verb is not droppable which causes an AssertionError
David Alves created CASSANDRA-4672: -- Summary: _TRACE verb is not droppable which causes an AssertionError Key: CASSANDRA-4672 URL: https://issues.apache.org/jira/browse/CASSANDRA-4672 Project: Cassandra Issue Type: Bug Components: Core Affects Versions: 1.2.0 Reporter: David Alves Priority: Trivial Fix For: 1.2.0 When a big enough statement is traced (like select *) an assertion error is fired because the _TRACE verb is not droppable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1337) parallelize fetching rows for low-cardinality indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456908#comment-13456908 ] David Alves commented on CASSANDRA-1337: cool, that means things are probably kosher at least semantically, right? I mean the bug present in v1 is no longer present and all other related tests pass. > parallelize fetching rows for low-cardinality indexes > - > > Key: CASSANDRA-1337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1337 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2.1 > > Attachments: 1137-bugfix.patch, 1337.patch, 1337-v4.patch, > ASF.LICENSE.NOT.GRANTED--0001-CASSANDRA-1337-scan-concurrently-depending-on-num-rows.txt, > CASSANDRA-1337.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > currently, we read the indexed rows from the first node (in partitioner > order); if that does not have enough matching rows, we read the rows from the > next, and so forth. > we should use the statistics fom CASSANDRA-1155 to query multiple nodes in > parallel, such that we have a high chance of getting enough rows w/o having > to do another round of queries (but, if our estimate is incorrect, we do need > to loop and do more rounds until we have enough data or we have fetched from > each node). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4615) cannot set log4j level on modules/classes
[ https://issues.apache.org/jira/browse/CASSANDRA-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448208#comment-13448208 ] David Alves commented on CASSANDRA-4615: not as long as the file appender has an INFO threshold, no. > cannot set log4j level on modules/classes > - > > Key: CASSANDRA-4615 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4615 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.2.0 beta 1 >Reporter: Brandon Williams >Assignee: David Alves >Priority: Minor > > For example, setting log4j.logger.org.apache.cassandra.db=DEBUG in the config > has no effect. Perhaps there is something else that needs to be set as well, > but we should at least document that there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-4615) cannot set log4j level on modules/classes
[ https://issues.apache.org/jira/browse/CASSANDRA-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448107#comment-13448107 ] David Alves edited comment on CASSANDRA-4615 at 9/5/12 9:15 AM: right, you need to also down the threshold for the appender you want (probably the R one in this case). I'll try and find find a better way to do that, but without actually using a different logger for tracing (and therefore changing quite a lot of code) it will probably depend on some weird hack to the logging system. was (Author: dr-alves): right, you need to also up the threshold for the appender you want (probably the R one in this case). I'll try and find find a better way to do that, but without actually using a different logger for tracing (and therefore changing quite a lot of code) it will probably depend on some weird hack to the logging system. > cannot set log4j level on modules/classes > - > > Key: CASSANDRA-4615 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4615 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.2.0 beta 1 >Reporter: Brandon Williams >Assignee: David Alves >Priority: Minor > > For example, setting log4j.logger.org.apache.cassandra.db=DEBUG in the config > has no effect. Perhaps there is something else that needs to be set as well, > but we should at least document that there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4615) cannot set log4j level on modules/classes
[ https://issues.apache.org/jira/browse/CASSANDRA-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448107#comment-13448107 ] David Alves commented on CASSANDRA-4615: right, you need to also up the threshold for the appender you want (probably the R one in this case). I'll try and find find a better way to do that, but without actually using a different logger for tracing (and therefore changing quite a lot of code) it will probably depend on some weird hack to the logging system. > cannot set log4j level on modules/classes > - > > Key: CASSANDRA-4615 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4615 > Project: Cassandra > Issue Type: Bug >Affects Versions: 1.2.0 beta 1 >Reporter: Brandon Williams >Assignee: David Alves >Priority: Minor > > For example, setting log4j.logger.org.apache.cassandra.db=DEBUG in the config > has no effect. Perhaps there is something else that needs to be set as well, > but we should at least document that there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4615) cannot set log4j level on modules/classes
[
https://issues.apache.org/jira/browse/CASSANDRA-4615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448098#comment-13448098
]
David Alves commented on CASSANDRA-4615:
The way it is setup right now, in order for tracing appender to be used in
debug level, the root logger was set to DEBUG and the other appenders were set
to INFO. This unfortunately has the side effect of not being able to lower only
a particular logger to debug (because the root is already at debug).
The other strategy would be to have an alternative logger for tracing, but this
cannot be done at the conf file level only, the actual instance of Logger would
have to be a different one. I mean instead of Logger logger =
LoggerFactory.getLogger(CassandraServer.class); we would have to use a
particular one, like Logger logger = LoggerFactory.getLogger("tracing-logger");.
I chose the first one because it didn't mean changing code (just the conf file).
> cannot set log4j level on modules/classes
> -
>
> Key: CASSANDRA-4615
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4615
> Project: Cassandra
> Issue Type: Bug
>Affects Versions: 1.2.0 beta 1
>Reporter: Brandon Williams
>Assignee: David Alves
>Priority: Minor
>
> For example, setting log4j.logger.org.apache.cassandra.db=DEBUG in the config
> has no effect. Perhaps there is something else that needs to be set as well,
> but we should at least document that there.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-4584) Add CQL syntax to enable request tracing
[ https://issues.apache.org/jira/browse/CASSANDRA-4584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves reassigned CASSANDRA-4584: -- Assignee: David Alves (was: Sylvain Lebresne) > Add CQL syntax to enable request tracing > > > Key: CASSANDRA-4584 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4584 > Project: Cassandra > Issue Type: Sub-task >Affects Versions: 1.2.0 beta 1 >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1337) parallelize fetching rows for low-cardinality indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-1337: --- Attachment: 1337-v4.patch Corrected 2ndary index cfactor calc and (slightly) optimized gathering rows. > parallelize fetching rows for low-cardinality indexes > - > > Key: CASSANDRA-1337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1337 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2.1 > > Attachments: 1137-bugfix.patch, 1337.patch, 1337-v4.patch, > ASF.LICENSE.NOT.GRANTED--0001-CASSANDRA-1337-scan-concurrently-depending-on-num-rows.txt, > CASSANDRA-1337.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > currently, we read the indexed rows from the first node (in partitioner > order); if that does not have enough matching rows, we read the rows from the > next, and so forth. > we should use the statistics fom CASSANDRA-1155 to query multiple nodes in > parallel, such that we have a high chance of getting enough rows w/o having > to do another round of queries (but, if our estimate is incorrect, we do need > to loop and do more rounds until we have enough data or we have fetched from > each node). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-1337) parallelize fetching rows for low-cardinality indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446542#comment-13446542 ] David Alves edited comment on CASSANDRA-1337 at 9/1/12 5:09 PM: Clean rehash that addresses Sylvain's (very helpful) comments, including an implementation for the CQL3 case. It estimates concurrency factor the following ways: Estimate Rows: - Primary Indexes - uses cfs's estimated keys divided by RF - 2ndary indexes - uses the mean col count of the most selective index to estimate the total num keys Estimate Cols (CQL3): - IdentityFilter - uses the estimated keys + mean col count to estimate total cols - NamesFilter - assumes cols with names are present and uses estimated keys to calculate to estimate total cols - Other filters - as ylvain mentioned because we have no idea on the selectivity of the col filter we cannot estimate how many cols will be returned per node so we revert to concurrecy factor = 1. Reimplemented parallel the parallel execution part to make it a lot cleaner IMO (previous implementation was forcefully adapting from the initial sequential execution which made it difficult to read) Notes: - cql_test.py dtest is failing in the same place as trunk ,need to look into it to make sure Sylvain's dtest passes - not sure whether to wait on read repair results for all handlers or just for the ones we actually use was (Author: dr-alves): Clean rehash that addressed Sylvain's (very helpful comments) including implementing for the CQL3 case. It estimated concurrency factor the following ways: - Primary Indexes + Thrift - divides cfs by RF - 2ndary indexes + Thrift - uses the mean col count of the most selective index to estimate the number of keys - CQL3 + IdentityFilter - uses the estimated keys + mean col count to estimate cols per node - CQL3 + Names filter - assumes cols with names are present and uses estimated keys to calculate cols per node - CQL3 - Other filters - as sylvain mentioned because we have no idea on the selectivity of the col filter we cannot estimate how many cols will be returned per node so we revert to concurrecy factor = 1. Reimplemented parallel the parallel execution part to make it a lot cleaner IMO (previous implementation was adapting sequential execution which made it difficult to read) cql_test.py dtest is failing in the same place as trunk ,need to look into it to make sure Sylvain's dtest passes > parallelize fetching rows for low-cardinality indexes > - > > Key: CASSANDRA-1337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1337 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2.1 > > Attachments: 1137-bugfix.patch, 1337.patch, > ASF.LICENSE.NOT.GRANTED--0001-CASSANDRA-1337-scan-concurrently-depending-on-num-rows.txt, > CASSANDRA-1337.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > currently, we read the indexed rows from the first node (in partitioner > order); if that does not have enough matching rows, we read the rows from the > next, and so forth. > we should use the statistics fom CASSANDRA-1155 to query multiple nodes in > parallel, such that we have a high chance of getting enough rows w/o having > to do another round of queries (but, if our estimate is incorrect, we do need > to loop and do more rounds until we have enough data or we have fetched from > each node). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-1337) parallelize fetching rows for low-cardinality indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446542#comment-13446542 ] David Alves edited comment on CASSANDRA-1337 at 9/1/12 5:09 PM: Clean rehash that addresses Sylvain's (very helpful) comments, including an implementation for the CQL3 case. It estimates concurrency factor the following ways: Estimate Rows: - Primary Indexes - uses cfs's estimated keys divided by RF - 2ndary indexes - uses the mean col count of the most selective index to estimate the total num keys Estimate Cols (CQL3): - IdentityFilter - uses the estimated keys + mean col count to estimate total cols - NamesFilter - assumes cols with names are present and uses estimated keys to calculate to estimate total cols - Other filters - as Sylvain mentioned because we have no idea on the selectivity of the col filter we cannot estimate how many cols will be returned per node so we revert to concurrecy factor = 1. Reimplemented parallel the parallel execution part to make it a lot cleaner IMO (previous implementation was forcefully adapting from the initial sequential execution which made it difficult to read) Notes: - cql_test.py dtest is failing in the same place as trunk ,need to look into it to make sure Sylvain's dtest passes - not sure whether to wait on read repair results for all handlers or just for the ones we actually use was (Author: dr-alves): Clean rehash that addresses Sylvain's (very helpful) comments, including an implementation for the CQL3 case. It estimates concurrency factor the following ways: Estimate Rows: - Primary Indexes - uses cfs's estimated keys divided by RF - 2ndary indexes - uses the mean col count of the most selective index to estimate the total num keys Estimate Cols (CQL3): - IdentityFilter - uses the estimated keys + mean col count to estimate total cols - NamesFilter - assumes cols with names are present and uses estimated keys to calculate to estimate total cols - Other filters - as ylvain mentioned because we have no idea on the selectivity of the col filter we cannot estimate how many cols will be returned per node so we revert to concurrecy factor = 1. Reimplemented parallel the parallel execution part to make it a lot cleaner IMO (previous implementation was forcefully adapting from the initial sequential execution which made it difficult to read) Notes: - cql_test.py dtest is failing in the same place as trunk ,need to look into it to make sure Sylvain's dtest passes - not sure whether to wait on read repair results for all handlers or just for the ones we actually use > parallelize fetching rows for low-cardinality indexes > - > > Key: CASSANDRA-1337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1337 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2.1 > > Attachments: 1137-bugfix.patch, 1337.patch, > ASF.LICENSE.NOT.GRANTED--0001-CASSANDRA-1337-scan-concurrently-depending-on-num-rows.txt, > CASSANDRA-1337.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > currently, we read the indexed rows from the first node (in partitioner > order); if that does not have enough matching rows, we read the rows from the > next, and so forth. > we should use the statistics fom CASSANDRA-1155 to query multiple nodes in > parallel, such that we have a high chance of getting enough rows w/o having > to do another round of queries (but, if our estimate is incorrect, we do need > to loop and do more rounds until we have enough data or we have fetched from > each node). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1337) parallelize fetching rows for low-cardinality indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-1337: --- Attachment: 1337.patch Clean rehash that addressed Sylvain's (very helpful comments) including implementing for the CQL3 case. It estimated concurrency factor the following ways: - Primary Indexes + Thrift - divides cfs by RF - 2ndary indexes + Thrift - uses the mean col count of the most selective index to estimate the number of keys - CQL3 + IdentityFilter - uses the estimated keys + mean col count to estimate cols per node - CQL3 + Names filter - assumes cols with names are present and uses estimated keys to calculate cols per node - CQL3 - Other filters - as sylvain mentioned because we have no idea on the selectivity of the col filter we cannot estimate how many cols will be returned per node so we revert to concurrecy factor = 1. Reimplemented parallel the parallel execution part to make it a lot cleaner IMO (previous implementation was adapting sequential execution which made it difficult to read) cql_test.py dtest is failing in the same place as trunk ,need to look into it to make sure Sylvain's dtest passes > parallelize fetching rows for low-cardinality indexes > - > > Key: CASSANDRA-1337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1337 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2.1 > > Attachments: 1137-bugfix.patch, 1337.patch, > ASF.LICENSE.NOT.GRANTED--0001-CASSANDRA-1337-scan-concurrently-depending-on-num-rows.txt, > CASSANDRA-1337.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > currently, we read the indexed rows from the first node (in partitioner > order); if that does not have enough matching rows, we read the rows from the > next, and so forth. > we should use the statistics fom CASSANDRA-1155 to query multiple nodes in > parallel, such that we have a high chance of getting enough rows w/o having > to do another round of queries (but, if our estimate is incorrect, we do need > to loop and do more rounds until we have enough data or we have fetched from > each node). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4599) Event tracing always records thread name as 'TracingStage'
[
https://issues.apache.org/jira/browse/CASSANDRA-4599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13446451#comment-13446451
]
David Alves commented on CASSANDRA-4599:
lgtm, +1
> Event tracing always records thread name as 'TracingStage'
> --
>
> Key: CASSANDRA-4599
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4599
> Project: Cassandra
> Issue Type: Bug
>Reporter: Yuki Morishita
>Assignee: Yuki Morishita
>Priority: Trivial
> Fix For: 1.2.0 beta 1
>
> Attachments: 4599.txt
>
>
> Since LoggingEvent#getThreadName gets current thread name when accessed, name
> of tracing thread ('TracingStage') is always logged to events CF.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-1123: --- Attachment: 1123-v9.patch Fixes the NPE. Problem was the result.getColumnCount call inside the logging statement (result maybe null). > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 beta 1 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch, 1123-v6.txt, 1123-v7.patch, 1123-v8.patch, 1123-v9.patch, > 1123-v9.txt, 1123-v9.txt > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444209#comment-13444209 ] David Alves commented on CASSANDRA-1123: +1, wfm > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 beta 1 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch, 1123-v6.txt, 1123-v7.patch, 1123-v8.patch, 1123-v9.txt, > 1123-v9.txt > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4406) Update stress for CQL3
[ https://issues.apache.org/jira/browse/CASSANDRA-4406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4406: --- Attachment: 4406.patch addresses Pavel's review issues namely: -adds support for both cql version (through -L and -L3) -CQLReader now correctly forms both cql2 and cql3 queries. also readded support for time uuid col names in cql2 > Update stress for CQL3 > -- > > Key: CASSANDRA-4406 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4406 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Affects Versions: 1.1.0 >Reporter: Sylvain Lebresne >Assignee: David Alves > Labels: stress > Fix For: 1.2.0 > > Attachments: 4406.patch, 4406.patch, 4406.patch > > > Stress does not support CQL3. We should add support for it so that: > # we can benchmark CQL3 > # we can benchmark CASSANDRA-2478 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-1123: --- Attachment: 1123-v8.patch Moves the error handling out of debuggableTPE into a parent TPE that is also used as the bare TPE for tracing, debuggable TPE now does only tracing plus the blocking queue. > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 beta 1 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch, 1123-v6.txt, 1123-v7.patch, 1123-v8.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13443229#comment-13443229 ] David Alves edited comment on CASSANDRA-1123 at 8/29/12 2:50 AM: - maybe I'm missing something, but I think we're covered: - since the responsibility of logging such an event would always fall on the writing thread my point was that any log event generated by that thread would already go to all appenders but the tracing one because the TL state doen't get propagated there (for instance if we used try/catch in runMayThrow or if we logged in afterExecute). - CassandraDaemon already sets a default exception handler that logs to error. - The only thing that is not done is the test-whether-exception-handler-is-set-and-if-not-log dance that DTPE does but since one such handler is set this shouldn't be a problem, correct? All this being said nothing like making sure by testing. will post back. was (Author: dr-alves): maybe I'm missing something, but I think we're covered: - since the responsibility of logging such an event would always fall on the writing thread my point was that any log event generated by that thread would already go to all appenders but the tracing one because the TL state doen't get propagated there (for instance if we used try/catch in runMayThrow). - CassandraDaemon already sets a default exception handler that logs to error. - The only thing that is not done is the test-whether-exception-handler-is-set-and-if-not-log dance that DTPE does but since one such handler is set this shouldn't be a problem, correct? All this being said nothing like making sure by testing. will post back. > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 beta 1 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch, 1123-v6.txt, 1123-v7.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13443229#comment-13443229 ] David Alves commented on CASSANDRA-1123: maybe I'm missing something, but I think we're covered: - since the responsibility of logging such an event would always fall on the writing thread my point was that any log event generated by that thread would already go to all appenders but the tracing one because the TL state doen't get propagated there (for instance if we used try/catch in runMayThrow). - CassandraDaemon already sets a default exception handler that logs to error. - The only thing that is not done is the test-whether-exception-handler-is-set-and-if-not-log dance that DTPE does but since one such handler is set this shouldn't be a problem, correct? All this being said nothing like making sure by testing. will post back. > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 beta 1 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch, 1123-v6.txt, 1123-v7.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442995#comment-13442995 ] David Alves commented on CASSANDRA-1123: (previous results were with regard to writes/sec rate), int_op_rate/key_rate see the same results, avg latency does not see a difference > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 beta 1 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch, 1123-v6.txt, 1123-v7.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442983#comment-13442983 ] David Alves commented on CASSANDRA-1123: ran some smoke tests (locally): 0.1% tracing probability under full load causes ~5.5% slowdown (in writes), without any tracing event being discarded. > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 beta 1 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch, 1123-v6.txt, 1123-v7.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442949#comment-13442949 ] David Alves commented on CASSANDRA-1123: bq. Hmm, can we do something clever like http://stackoverflow.com/questions/4046228/log4j-excluding-the-logging-of-some-classes to just log o.a.c.tracing to stdout + R so we don't just error out silently when something goes wrong w/ tracing? after I had done this I realized that there was no point. all log events from everywhere (including o.a.c.tracing) but the ones coming from tracing stage executions are already present in all logs (tracing, rolling and stdout) if at the debug level. tracing writes do not inherit the debuggableTPE, which means log events coming from a tracing write do not go the tracing appender but *do* go the rest of the appenders by default already. > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 beta 1 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch, 1123-v6.txt, 1123-v7.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-1123: --- Attachment: 1123-v7.patch - Fixed NPE in TracingAppender because of inlining isTracing() (the instance may not be built at the time the appender is started) - Added settraceprobability to nodetool - Removed tracing settings from stress (assumed that was the goal, happy to readd if needed) > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 beta 1 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch, 1123-v6.txt, 1123-v7.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13442806#comment-13442806 ] David Alves commented on CASSANDRA-1123: bq. Hmm, can we do something clever like http://stackoverflow.com/questions/4046228/log4j-excluding-the-logging-of-some-classes to just log o.a.c.tracing to stdout + R so we don't just error out silently when something goes wrong w/ tracing? We can, I'm taking care of that on the next patch along with nodetool. Although, just to be clear, using a bare TPE is still the way to go because we still wouldn't want to propagate tracing to tracing mutation threads. nice job on the DTPE cleanup. > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 beta 1 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch, 1123-v6.txt > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-1123: --- Attachment: 1123.patch final touches: - JMX enabled tracing - simplified enabling tracing - completed cli and stress > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2.0 > > Attachments: 1123-3.patch.gz, 1123.patch, 1123.patch, 1123.patch, > 1123.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1123) Allow tracing query details
[
https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Alves updated CASSANDRA-1123:
---
Attachment: 1123.patch
Attached is a patch that implements almost everything mentioned:
- Trace is implemented asynchronously with a new Stage that has a threadpool
with a single thread that refuses to execute when the queue if full ( as
suggested a warn is logged, experiments say that under huge load 0.001
traceProbability still works without rejecting events). (also this threadpool
is the only one that does not propagate trace events).
- Allows to enable tracing/disable tracing from cli. Also enable tracing has
two parameters traceProbability (the prop that any single request from that
client gets traced) and maxTraceNumber to allow to set a maximum number of
traces to do (-1 set this to Integer.MAX_INT which is also the default)
- Adds the possibility to enable tracing in stress (using -tr probability
[optionally maxNumTraces]
- TraceEvents can be build using a fluent builder (TraceEventBuilder) that is
also able to deserialize events (both from thrift and from IColumns).
- All requests in CassandraServer start a tracing session when tracing is
enabled and all parameters are stored along with the request details. This is
done using a ThriftType that is able to serialize and deserialize thrift
objects into Cassandra.
- User Request/Reply, Stage Start/Finish and Message Request/Reply are traced
along with specific custom requests (such as apply_mutation, and
get_column_family)
- Cli contains two other commands to explore traces:
show tracing summary [request_name] - this displays a summary for a request
type:
{quote}
Summary for sessions of request: batch_mutate
Total Sessions: 500
Total Events: 3190
==
|Avg.| StdDev. | Max. |Min.|
99% | Unit |
=
| Latency | 22.48 | 28.68 | 293.90 | 0.56 |
150.38| msec |
|---|
| Batch Rows | 1.00 | 0.00 | 1.00 | 1.00 |
1.00| amount/req |
|---|
| Mutations| 5.00 | 0.00 | 5.00 | 5.00 |
5.00| amount/req |
|---|
| Deletions| 0.00 | 0.00 | 0.00 | 0.00 |
0.00| amount/req |
|---|
| Columns | 5.00 | 0.00 | 5.00 | 5.00 |
5.00| amount/req |
|---|
| Counters | 0.00 | 0.00 | 0.00 | 0.00 |
0.00| amount/req |
|---|
| Super Col. | 0.00 | 0.00 | 0.00 | 0.00 |
0.00| amount/req |
|---|
| Written Bytes| 170.00 | 0.00 | 170.00 | 170.00 |
170.00| amount/req |
|---|
Quickest Request sessionId: 135b0c60-eac0-11e1--fe8ebeead9ff
Slowest Request sessionId: f09c3e10-eabf-11e1--fe8ebeead9ff
{quote}
explain trace session [sessionId] - displays the complete set of events of a
tracing session along with a to-scale timeline for its execution.
{quote}
Session Summary: f09c3e10-eabf-11e1--fe8ebeead9ff
Request: batch_mutate
Coordinator: /127.0.0.1
Interacting nodes: 2 {[/127.0.0.1, /127.0.0.2]}
Duration: 293901000 nano seconds (293.901 msecs)
Consistency Level: ONE
Request Timeline:
||--|
Se;/Msg
/Se
|-||
Msg,apply_mutation;St:[Mutation];/St:[Mutation]
Caption:
Se - Session start (start of request: batch_mutate)
/Se - Session end (end of request: batch_mutate)
St[StageName] - Begin of Stage with StageName (e.g. Mutatio
[jira] [Comment Edited] (CASSANDRA-1123) Allow tracing query details
[
https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437816#comment-13437816
]
David Alves edited comment on CASSANDRA-1123 at 8/20/12 11:28 PM:
--
Attached is a patch that implements almost everything mentioned:
- Trace is implemented asynchronously with a new Stage that has a threadpool
with a single thread that refuses to execute when the queue if full ( as
suggested a warn is logged, experiments say that under huge load 0.001
traceProbability still works without rejecting events). (also this threadpool
is the only one that does not propagate trace events).
- Allows to enable tracing/disable tracing from cli. Also enable tracing has
two parameters traceProbability (the prop that any single request from that
client gets traced) and maxTraceNumber to allow to set a maximum number of
traces to do (-1 set this to Integer.MAX_INT which is also the default)
- Adds the possibility to enable tracing in stress (using -tr probability
[optionally maxNumTraces]
- TraceEvents can be build using a fluent builder (TraceEventBuilder) that is
also able to deserialize events (both from thrift and from IColumns).
- All requests in CassandraServer start a tracing session when tracing is
enabled and all parameters are stored along with the request details. This is
done using a ThriftType that is able to serialize and deserialize thrift
objects into Cassandra.
- User Request/Reply, Stage Start/Finish and Message Request/Reply are traced
along with specific custom requests (such as apply_mutation, and
get_column_family)
- Cli contains two other commands to explore traces:
show tracing summary [request_name] - this displays a summary for a request
type:
{code}
Summary for sessions of request: batch_mutate
Total Sessions: 500
Total Events: 3190
==
|Avg.| StdDev. | Max. |Min.|
99% | Unit |
=
| Latency | 22.48 | 28.68 | 293.90 | 0.56 |
150.38| msec |
|---|
| Batch Rows | 1.00 | 0.00 | 1.00 | 1.00 |
1.00| amount/req |
|---|
| Mutations| 5.00 | 0.00 | 5.00 | 5.00 |
5.00| amount/req |
|---|
| Deletions| 0.00 | 0.00 | 0.00 | 0.00 |
0.00| amount/req |
|---|
| Columns | 5.00 | 0.00 | 5.00 | 5.00 |
5.00| amount/req |
|---|
| Counters | 0.00 | 0.00 | 0.00 | 0.00 |
0.00| amount/req |
|---|
| Super Col. | 0.00 | 0.00 | 0.00 | 0.00 |
0.00| amount/req |
|---|
| Written Bytes| 170.00 | 0.00 | 170.00 | 170.00 |
170.00| amount/req |
|---|
Quickest Request sessionId: 135b0c60-eac0-11e1--fe8ebeead9ff
Slowest Request sessionId: f09c3e10-eabf-11e1--fe8ebeead9ff
{code}
explain trace session [sessionId] - displays the complete set of events of a
tracing session along with a to-scale timeline for its execution.
{code}
Session Summary: f09c3e10-eabf-11e1--fe8ebeead9ff
Request: batch_mutate
Coordinator: /127.0.0.1
Interacting nodes: 2 {[/127.0.0.1, /127.0.0.2]}
Duration: 293901000 nano seconds (293.901 msecs)
Consistency Level: ONE
Request Timeline:
||--|
Se;/Msg
/Se
|-||
Msg,apply_mutation;St:[Mutation];/St:[Mutation]
Caption:
Se - Session start (start of request: batch_mutate)
/Se - Session end (end of request
[jira] [Commented] (CASSANDRA-4555) select statement with indexed column causes node to OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437238#comment-13437238
]
David Alves commented on CASSANDRA-4555:
+1 applies cleanly and solves the problem.
> select statement with indexed column causes node to OOM
> ---
>
> Key: CASSANDRA-4555
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4555
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Affects Versions: 1.2.0
> Environment: MAC OSx
>Reporter: David Alves
>Assignee: Jonathan Ellis
> Fix For: 1.2.0
>
> Attachments: 4555.txt
>
>
> After creating a keyspace, table and index on a clean ccm 3 node cluster
> based on trunk, when a select statement with an index expression is executed
> in cqlsh one of the nodes OOM's and goes down.
> The steps to reproduce the problem are:
> create a 3 node cluster from trunk (I used ccm)
> execute the following statements in cqlsh:
> {noformat}
> CREATE KEYSPACE trace WITH strategy_class = 'SimpleStrategy'
> AND strategy_options:replication_factor = '1';
> CREATE TABLE trace.trace_events(sessionId timeuuid,
> coordinator inet,
> eventId timeuuid,
> description text,
> duration bigint,
> happened_at timestamp,
> name text,
> payload_types map,
> payload map,
> sourceinet,
> type text,
> PRIMARY KEY (sessionId, coordinator, eventId));
> CREATE INDEX idx_name ON trace.trace_events (name);
> {noformat}
> Executing the following statement causes node2 to OOM:
> select * from trace_events where name = 'batch_mutate';
> In my case node2 goes down with:
> {noformat}
> ERROR [Thread-9] 2012-08-17 19:42:55,741 CassandraDaemon.java (line 131)
> Exception in thread Thread[Thread-9,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.cassandra.dht.Token$TokenSerializer.deserialize(Token.java:97)
> at
> org.apache.cassandra.dht.AbstractBounds$AbstractBoundsSerializer.deserialize(AbstractBounds.java:161)
> at
> org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:299)
> at
> org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:181)
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94)
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:181)
> at
> org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:122)
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:69)
> INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,746 ThriftServer.java
> (line 221) Stop listening to thrift clients
> INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,748 Gossiper.java
> (line 1054) Announcing shutdown
> INFO [StorageServiceShutdownHook] 2012-08-17 19:42:56,749
> MessagingService.java (line 657) Waiting for messaging service to quiesce
> INFO [ACCEPT-/127.0.0.2] 2012-08-17 19:42:56,751 MessagingService.java (line
> 849) MessagingService shutting down server thread.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4555) select statement with indexed column causes node to OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13437231#comment-13437231
]
David Alves commented on CASSANDRA-4555:
With and without data. It OOMs because it tries to allocate a huge byte[] on
TokenSerializer.deserialize().
> select statement with indexed column causes node to OOM
> ---
>
> Key: CASSANDRA-4555
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4555
> Project: Cassandra
> Issue Type: Bug
> Environment: MAC OSx
>Reporter: David Alves
>
> After creating a keyspace, table and index on a clean ccm 3 node cluster
> based on trunk, when a select statement with an index expression is executed
> in cqlsh one of the nodes OOM's and goes down.
> The steps to reproduce the problem are:
> create a 3 node cluster from trunk (I used ccm)
> execute the following statements in cqlsh:
> {noformat}
> CREATE KEYSPACE trace WITH strategy_class = 'SimpleStrategy'
> AND strategy_options:replication_factor = '1';
> CREATE TABLE trace.trace_events(sessionId timeuuid,
> coordinator inet,
> eventId timeuuid,
> description text,
> duration bigint,
> happened_at timestamp,
> name text,
> payload_types map,
> payload map,
> sourceinet,
> type text,
> PRIMARY KEY (sessionId, coordinator, eventId));
> CREATE INDEX idx_name ON trace.trace_events (name);
> {noformat}
> Executing the following statement causes node2 to OOM:
> select * from trace_events where name = 'batch_mutate';
> In my case node2 goes down with:
> {noformat}
> ERROR [Thread-9] 2012-08-17 19:42:55,741 CassandraDaemon.java (line 131)
> Exception in thread Thread[Thread-9,5,main]
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.cassandra.dht.Token$TokenSerializer.deserialize(Token.java:97)
> at
> org.apache.cassandra.dht.AbstractBounds$AbstractBoundsSerializer.deserialize(AbstractBounds.java:161)
> at
> org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:299)
> at
> org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:181)
> at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94)
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:181)
> at
> org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:122)
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:69)
> INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,746 ThriftServer.java
> (line 221) Stop listening to thrift clients
> INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,748 Gossiper.java
> (line 1054) Announcing shutdown
> INFO [StorageServiceShutdownHook] 2012-08-17 19:42:56,749
> MessagingService.java (line 657) Waiting for messaging service to quiesce
> INFO [ACCEPT-/127.0.0.2] 2012-08-17 19:42:56,751 MessagingService.java (line
> 849) MessagingService shutting down server thread.
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4555) select statement with indexed column causes node to OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Alves updated CASSANDRA-4555:
---
Description:
After creating a keyspace, table and index on a clean ccm 3 node cluster based
on trunk, when a select statement with an index expression is executed in cqlsh
one of the nodes OOM's and goes down.
The steps to reproduce the problem are:
create a 3 node cluster from trunk (I used ccm)
execute the following statements in cqlsh:
{noformat}
CREATE KEYSPACE trace WITH strategy_class = 'SimpleStrategy'
AND strategy_options:replication_factor = '1';
CREATE TABLE trace.trace_events(sessionId timeuuid,
coordinator inet,
eventId timeuuid,
description text,
duration bigint,
happened_at timestamp,
name text,
payload_types map,
payload map,
sourceinet,
type text,
PRIMARY KEY (sessionId, coordinator, eventId));
CREATE INDEX idx_name ON trace.trace_events (name);
{noformat}
Executing the following statement causes node2 to OOM:
select * from trace_events where name = 'batch_mutate';
In my case node2 goes down with:
{noformat}
ERROR [Thread-9] 2012-08-17 19:42:55,741 CassandraDaemon.java (line 131)
Exception in thread Thread[Thread-9,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.dht.Token$TokenSerializer.deserialize(Token.java:97)
at
org.apache.cassandra.dht.AbstractBounds$AbstractBoundsSerializer.deserialize(AbstractBounds.java:161)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:299)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:181)
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94)
at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:181)
at
org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:122)
at
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:69)
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,746 ThriftServer.java
(line 221) Stop listening to thrift clients
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,748 Gossiper.java (line
1054) Announcing shutdown
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:56,749
MessagingService.java (line 657) Waiting for messaging service to quiesce
INFO [ACCEPT-/127.0.0.2] 2012-08-17 19:42:56,751 MessagingService.java (line
849) MessagingService shutting down server thread.
{noformat}
was:
After creating a keyspace, table and index on a clean ccm 3 node cluster based
on trunk, when a select statement with an index expression is executed in cqlsh
one of the nodes OOM's and goes down.
The steps to reproduce the problem are:
create a 3 node cluster from trunk (I used ccm)
execute the following statements in cqlsh:
{noformat}
CREATE KEYSPACE trace WITH strategy_class = 'SimpleStrategy'
AND strategy_options:replication_factor = '1';
CREATE TABLE trace.trace_events(sessionId timeuuid,
coordinator inet,
eventId timeuuid,
description text,
duration bigint,
happened_at timestamp,
name text,
payload_types map,
payload map,
sourceinet,
type text,
PRIMARY KEY (sessionId, coordinator, eventId));
CREATE INDEX idx_name ON trace.trace_events (name);
{noformat}
Executing the following statement causes node2 to OOM:
select * from trace_events where name = 'batch_mutate';
In make case node2 goes down with:
{noformat}
ERROR [Thread-9] 2012-08-17 19:42:55,741 CassandraDaemon.java (line 131)
Exception in thread Thread[Thread-9,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.dht.Token$TokenSerializer.deserialize(Token.java:97)
at
org.apache.cassandra.dht.AbstractBounds$AbstractBoundsSerializer.deserialize(AbstractBounds.java:161)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:299)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:181)
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94)
at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:181)
at
org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:122)
at
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:69)
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,746 ThriftServer.java
(line 221) Stop listening to thrift clients
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,748 Gossiper.java (line
1054) Announcing shutdown
INFO [StorageServiceShutdownH
[jira] [Updated] (CASSANDRA-4555) select statement with indexed column causes node to OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Alves updated CASSANDRA-4555:
---
Description:
After creating a keyspace, table and index on a clean ccm 3 node cluster based
on trunk, when a select statement with an index expression is executed in cqlsh
one of the nodes OOM's and goes down.
The steps to reproduce the problem are:
create a 3 node cluster from trunk (I used ccm)
execute the following statements in cqlsh:
{noformat}
CREATE KEYSPACE trace WITH strategy_class = 'SimpleStrategy'
AND strategy_options:replication_factor = '1';
CREATE TABLE trace.trace_events(sessionId timeuuid,
coordinator inet,
eventId timeuuid,
description text,
duration bigint,
happened_at timestamp,
name text,
payload_types map,
payload map,
sourceinet,
type text,
PRIMARY KEY (sessionId, coordinator, eventId));
CREATE INDEX idx_name ON trace.trace_events (name);
{noformat}
Executing the following statement causes node2 to OOM:
select * from trace_events where name = 'batch_mutate';
In make case node2 goes down with:
{noformat}
ERROR [Thread-9] 2012-08-17 19:42:55,741 CassandraDaemon.java (line 131)
Exception in thread Thread[Thread-9,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.dht.Token$TokenSerializer.deserialize(Token.java:97)
at
org.apache.cassandra.dht.AbstractBounds$AbstractBoundsSerializer.deserialize(AbstractBounds.java:161)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:299)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:181)
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94)
at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:181)
at
org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:122)
at
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:69)
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,746 ThriftServer.java
(line 221) Stop listening to thrift clients
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,748 Gossiper.java (line
1054) Announcing shutdown
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:56,749
MessagingService.java (line 657) Waiting for messaging service to quiesce
INFO [ACCEPT-/127.0.0.2] 2012-08-17 19:42:56,751 MessagingService.java (line
849) MessagingService shutting down server thread.
{noformat}
was:
After creating a keyspace, table and index on a clean ccm 3 node cluster based
on trunk, when a select statement with an index is expression is executed in
cqlsh one of the nodes OOM's and goes down.
The steps to reproduce the problem are:
create a 3 node cluster from trunk (I used ccm)
execute the following statements in cqlsh:
CREATE KEYSPACE trace WITH strategy_class = 'SimpleStrategy'
AND strategy_options:replication_factor = '1';
CREATE TABLE trace.trace_events(sessionId timeuuid,
coordinator inet,
eventId timeuuid,
description text,
duration bigint,
happened_at timestamp,
name text,
payload_types map,
payload map,
sourceinet,
type text,
PRIMARY KEY (sessionId, coordinator, eventId));
CREATE INDEX idx_name ON trace.trace_events (name);
Executing the following statement causes node2 to OOM:
select * from trace_events where name = 'batch_mutate';
In make case node2 goes down with:
{quote}
ERROR [Thread-9] 2012-08-17 19:42:55,741 CassandraDaemon.java (line 131)
Exception in thread Thread[Thread-9,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.dht.Token$TokenSerializer.deserialize(Token.java:97)
at
org.apache.cassandra.dht.AbstractBounds$AbstractBoundsSerializer.deserialize(AbstractBounds.java:161)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:299)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:181)
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94)
at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:181)
at
org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:122)
at
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:69)
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,746 ThriftServer.java
(line 221) Stop listening to thrift clients
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,748 Gossiper.java (line
1054) Announcing shutdown
INFO [StorageServiceShutdownHook] 2012-08-17 19:42
[jira] [Updated] (CASSANDRA-4555) select statement with indexed column causes node to OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-4555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Alves updated CASSANDRA-4555:
---
Description:
After creating a keyspace, table and index on a clean ccm 3 node cluster based
on trunk, when a select statement with an index is expression is executed in
cqlsh one of the nodes OOM's and goes down.
The steps to reproduce the problem are:
create a 3 node cluster from trunk (I used ccm)
execute the following statements in cqlsh:
CREATE KEYSPACE trace WITH strategy_class = 'SimpleStrategy'
AND strategy_options:replication_factor = '1';
CREATE TABLE trace.trace_events(sessionId timeuuid,
coordinator inet,
eventId timeuuid,
description text,
duration bigint,
happened_at timestamp,
name text,
payload_types map,
payload map,
sourceinet,
type text,
PRIMARY KEY (sessionId, coordinator, eventId));
CREATE INDEX idx_name ON trace.trace_events (name);
Executing the following statement causes node2 to OOM:
select * from trace_events where name = 'batch_mutate';
In make case node2 goes down with:
{quote}
ERROR [Thread-9] 2012-08-17 19:42:55,741 CassandraDaemon.java (line 131)
Exception in thread Thread[Thread-9,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.dht.Token$TokenSerializer.deserialize(Token.java:97)
at
org.apache.cassandra.dht.AbstractBounds$AbstractBoundsSerializer.deserialize(AbstractBounds.java:161)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:299)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:181)
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94)
at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:181)
at
org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:122)
at
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:69)
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,746 ThriftServer.java
(line 221) Stop listening to thrift clients
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,748 Gossiper.java (line
1054) Announcing shutdown
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:56,749
MessagingService.java (line 657) Waiting for messaging service to quiesce
INFO [ACCEPT-/127.0.0.2] 2012-08-17 19:42:56,751 MessagingService.java (line
849) MessagingService shutting down server thread.
{quote}
was:
After creating a keyspace, table and index on a clean ccm 3 node cluster based
on trunk, when a select statement with an index is expression is executed in
cqlsh one of the nodes OOM's and goes down.
The statements to reproduce the problem are:
create a 3 node cluster from trunk (I used ccm)
execute the following statements in cqlsh:
CREATE KEYSPACE trace WITH strategy_class = 'SimpleStrategy'
AND strategy_options:replication_factor = '1';
CREATE TABLE trace.trace_events(sessionId timeuuid,
coordinator inet,
eventId timeuuid,
description text,
duration bigint,
happened_at timestamp,
name text,
payload_types map,
payload map,
sourceinet,
type text,
PRIMARY KEY (sessionId, coordinator, eventId));
CREATE INDEX idx_name ON trace.trace_events (name);
Executing the following statement causes node2 to OOM:
select * from trace_events where name = 'batch_mutate';
In make case node2 goes down with:
{quote}
ERROR [Thread-9] 2012-08-17 19:42:55,741 CassandraDaemon.java (line 131)
Exception in thread Thread[Thread-9,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.dht.Token$TokenSerializer.deserialize(Token.java:97)
at
org.apache.cassandra.dht.AbstractBounds$AbstractBoundsSerializer.deserialize(AbstractBounds.java:161)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:299)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:181)
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94)
at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:181)
at
org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:122)
at
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:69)
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,746 ThriftServer.java
(line 221) Stop listening to thrift clients
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,748 Gossiper.java (line
1054) Announcing shutdown
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:56,749
MessagingSe
[jira] [Created] (CASSANDRA-4555) select statement with indexed column causes node to OOM
David Alves created CASSANDRA-4555:
--
Summary: select statement with indexed column causes node to OOM
Key: CASSANDRA-4555
URL: https://issues.apache.org/jira/browse/CASSANDRA-4555
Project: Cassandra
Issue Type: Bug
Environment: MAC OSx
Reporter: David Alves
After creating a keyspace, table and index on a clean ccm 3 node cluster based
on trunk, when a select statement with an index is expression is executed in
cqlsh one of the nodes OOM's and goes down.
The statements to reproduce the problem are:
create a 3 node cluster from trunk (I used ccm)
execute the following statements in cqlsh:
CREATE KEYSPACE trace WITH strategy_class = 'SimpleStrategy'
AND strategy_options:replication_factor = '1';
CREATE TABLE trace.trace_events(sessionId timeuuid,
coordinator inet,
eventId timeuuid,
description text,
duration bigint,
happened_at timestamp,
name text,
payload_types map,
payload map,
sourceinet,
type text,
PRIMARY KEY (sessionId, coordinator, eventId));
CREATE INDEX idx_name ON trace.trace_events (name);
Executing the following statement causes node2 to OOM:
select * from trace_events where name = 'batch_mutate';
In make case node2 goes down with:
{quote}
ERROR [Thread-9] 2012-08-17 19:42:55,741 CassandraDaemon.java (line 131)
Exception in thread Thread[Thread-9,5,main]
java.lang.OutOfMemoryError: Java heap space
at
org.apache.cassandra.dht.Token$TokenSerializer.deserialize(Token.java:97)
at
org.apache.cassandra.dht.AbstractBounds$AbstractBoundsSerializer.deserialize(AbstractBounds.java:161)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:299)
at
org.apache.cassandra.db.RangeSliceCommandSerializer.deserialize(RangeSliceCommand.java:181)
at org.apache.cassandra.net.MessageIn.read(MessageIn.java:94)
at
org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:181)
at
org.apache.cassandra.net.IncomingTcpConnection.handleModernVersion(IncomingTcpConnection.java:122)
at
org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:69)
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,746 ThriftServer.java
(line 221) Stop listening to thrift clients
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:55,748 Gossiper.java (line
1054) Announcing shutdown
INFO [StorageServiceShutdownHook] 2012-08-17 19:42:56,749
MessagingService.java (line 657) Waiting for messaging service to quiesce
INFO [ACCEPT-/127.0.0.2] 2012-08-17 19:42:56,751 MessagingService.java (line
849) MessagingService shutting down server thread.
{quote}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-1337) parallelize fetching rows for low-cardinality indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424071#comment-13424071 ] David Alves edited comment on CASSANDRA-1337 at 7/27/12 7:09 PM: - patch that addresses the bugs raised by sylvain. (StoragProxyTest and cql_test.py both pass) namely: - local path counts as one less handler - enough check moved out of the remote branch - estimatedKeysPerRange take into account replication factor - columns.maxIsColumns sets concurrency to 1 still working on the dtest that proves (or disproves that this works) I'd like to move the rest of the issues raised by sylvain to another ticket. was (Author: dr-alves): patch that addresses the bugs raised by sylvain. (StoragProxyTest and cql_test.py both pass) namely: - local path counts as one less handler - enough check moved out of the remote branch - estimatedKeysPerRange take into account replication factor - columns.maxIsColumns sets concurrency to 1 still working on the dtest that proves (or disproves that this works) but both StorageProxyTest and the regression test created by Sylvain pass I'd like to move the rest of the issues raised by sylvain to another ticket. > parallelize fetching rows for low-cardinality indexes > - > > Key: CASSANDRA-1337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1337 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: > 0001-CASSANDRA-1337-scan-concurrently-depending-on-num-rows.txt, > 1137-bugfix.patch, CASSANDRA-1337.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > currently, we read the indexed rows from the first node (in partitioner > order); if that does not have enough matching rows, we read the rows from the > next, and so forth. > we should use the statistics fom CASSANDRA-1155 to query multiple nodes in > parallel, such that we have a high chance of getting enough rows w/o having > to do another round of queries (but, if our estimate is incorrect, we do need > to loop and do more rounds until we have enough data or we have fetched from > each node). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-1337) parallelize fetching rows for low-cardinality indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424071#comment-13424071 ] David Alves edited comment on CASSANDRA-1337 at 7/27/12 7:07 PM: - patch that addresses the bugs raised by sylvain. (StoragProxyTest and cql_test.py both pass) namely: - local path counts as one less handler - enough check moved out of the remote branch - estimatedKeysPerRange take into account replication factor - columns.maxIsColumns sets concurrency to 1 still working on the dtest that proves (or disproves that this works) but both StorageProxyTest and the regression test created by Sylvain pass I'd like to move the rest of the issues raised by sylvain to another ticket. was (Author: dr-alves): patch that addresses the bugs raised by sylvain. (StoragProxyTest and cql_test.py both pass) namely: - local path counts as one less handler - enough check moven out of the remote branch - estimatedKeysPerRange take into account replication factor - columns.maxIsColumns sets concurrency to 1 still working on the dtest that proves (or disproves that this works) I'd like to move the rest of the issues raised by sylvain to another ticket. > parallelize fetching rows for low-cardinality indexes > - > > Key: CASSANDRA-1337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1337 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: > 0001-CASSANDRA-1337-scan-concurrently-depending-on-num-rows.txt, > 1137-bugfix.patch, CASSANDRA-1337.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > currently, we read the indexed rows from the first node (in partitioner > order); if that does not have enough matching rows, we read the rows from the > next, and so forth. > we should use the statistics fom CASSANDRA-1155 to query multiple nodes in > parallel, such that we have a high chance of getting enough rows w/o having > to do another round of queries (but, if our estimate is incorrect, we do need > to loop and do more rounds until we have enough data or we have fetched from > each node). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1337) parallelize fetching rows for low-cardinality indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-1337: --- Attachment: 1137-bugfix.patch patch that addresses the bugs raised by sylvain. (StoragProxyTest and cql_test.py both pass) namely: - local path counts as one less handler - enough check moven out of the remote branch - estimatedKeysPerRange take into account replication factor - columns.maxIsColumns sets concurrency to 1 still working on the dtest that proves (or disproves that this works) I'd like to move the rest of the issues raised by sylvain to another ticket. > parallelize fetching rows for low-cardinality indexes > - > > Key: CASSANDRA-1337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1337 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: > 0001-CASSANDRA-1337-scan-concurrently-depending-on-num-rows.txt, > 1137-bugfix.patch, CASSANDRA-1337.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > currently, we read the indexed rows from the first node (in partitioner > order); if that does not have enough matching rows, we read the rows from the > next, and so forth. > we should use the statistics fom CASSANDRA-1155 to query multiple nodes in > parallel, such that we have a high chance of getting enough rows w/o having > to do another round of queries (but, if our estimate is incorrect, we do need > to loop and do more rounds until we have enough data or we have fetched from > each node). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4406) Update stress for CQL3
[ https://issues.apache.org/jira/browse/CASSANDRA-4406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4406: --- Attachment: 4406.patch removed whitespace > Update stress for CQL3 > -- > > Key: CASSANDRA-4406 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4406 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Affects Versions: 1.1.0 >Reporter: Sylvain Lebresne >Assignee: David Alves > Labels: stress > Fix For: 1.2 > > Attachments: 4406.patch, 4406.patch > > > Stress does not support CQL3. We should add support for it so that: > # we can benchmark CQL3 > # we can benchmark CASSANDRA-2478 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4406) Update stress for CQL3
[ https://issues.apache.org/jira/browse/CASSANDRA-4406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421931#comment-13421931 ] David Alves commented on CASSANDRA-4406: What I couldn't figure out was how to use UUID column names (required when the TimeUUID comparator is set with the U option), that option currently throws an UnsupportedOperationException. > Update stress for CQL3 > -- > > Key: CASSANDRA-4406 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4406 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Affects Versions: 1.1.0 >Reporter: Sylvain Lebresne >Assignee: David Alves > Labels: stress > Fix For: 1.2 > > Attachments: 4406.patch > > > Stress does not support CQL3. We should add support for it so that: > # we can benchmark CQL3 > # we can benchmark CASSANDRA-2478 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4406) Update stress for CQL3
[ https://issues.apache.org/jira/browse/CASSANDRA-4406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4406: --- Attachment: 4406.patch this patch does the following: - updates cql version to 3.0 - adds column names and types to the cfdef - uses those column names to build the statement (prepared statement previsouly used ?=? pairs for columns now uses "C.." = ? pairs) > Update stress for CQL3 > -- > > Key: CASSANDRA-4406 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4406 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Affects Versions: 1.1.0 >Reporter: Sylvain Lebresne >Assignee: David Alves > Labels: stress > Fix For: 1.2 > > Attachments: 4406.patch > > > Stress does not support CQL3. We should add support for it so that: > # we can benchmark CQL3 > # we can benchmark CASSANDRA-2478 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421828#comment-13421828 ] David Alves commented on CASSANDRA-3564: the patch includes the nodetool functionality to flush everything, we'd still want that right? I can trim it to just that (and open a different ticket?). > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch, 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421792#comment-13421792 ] David Alves commented on CASSANDRA-3564: One last thing... since we're only changing debian/init this will work only on debian flavored deployments maybe we should go ahead place the flag on the cassandra.yaml and make the shutdown hook follow that option inetead, this would make the option valid for all deployments (I'd still add the automated wait/kill for debian deployments, non debian deployments would have to take care of killing themselves). wdyt? > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch, 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421770#comment-13421770 ] David Alves commented on CASSANDRA-3564: makes sense, will add a 10 min wiat period, still what should be the default setting? flush everything or just the non durable tables? > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch, 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421770#comment-13421770 ] David Alves edited comment on CASSANDRA-3564 at 7/24/12 9:26 PM: - makes sense, will add a 10 min wait period, still what should be the default setting? flush everything or just the non durable tables? was (Author: dr-alves): makes sense, will add a 10 min wiat period, still what should be the default setting? flush everything or just the non durable tables? > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch, 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421757#comment-13421757 ] David Alves commented on CASSANDRA-3564: I suggest that we add flag that when enabled waits for the full flush for all tables and maybe we don't even need the wait period since the user explicitely told that he wanted a full flush and should be willing to wait. When the flag disabled reverts to the current setting (flush only non-durable tables). Even in the default setting (say the admin remembered he wanted to flush everything *after* the daemon started) he can always use nodetool to do so. > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch, 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421713#comment-13421713 ] David Alves commented on CASSANDRA-3564: You mean send SIGINT/SIGTERM before waiting and SIGKILL after right? SIGINT/SIGTERM don't do a global flush because the shutdown hooks still only flush tables that have durable_writes to false, so either we used nodetool to explicitly call flushAllTables of do in cassandra daemon which is being called from jsvc anyway. > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch, 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-3564: --- Attachment: 3564.patch Patch that includes killing/waiting in the do_stop function of the init debian script. Script does not actually call the flushTablesAndExit() instead jsvc's stop() hook does, after killing the RPC and native servers. Node is given 100 secs to finish flush, I'm not sure this is a good default setting. > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch, 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3706) Back up configuration files on startup
[ https://issues.apache.org/jira/browse/CASSANDRA-3706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420729#comment-13420729 ] David Alves commented on CASSANDRA-3706: How far is this from going in? Any thing I can do to help? CASSANDRA-1123 will also need distributed system tables and I'd like to minimize duplicate code. I can split the patch into generic dist system tables and specific configuration backup + rebase and use the common part in 1123. wdyt? > Back up configuration files on startup > -- > > Key: CASSANDRA-3706 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3706 > Project: Cassandra > Issue Type: New Feature > Components: Tools >Reporter: Jonathan Ellis >Assignee: Dave Brosius >Priority: Minor > Labels: lhf > Fix For: 1.2 > > Attachments: save_configuration.diff, save_configuration_2.diff, > save_configuration_3.diff, save_configuration_4.diff, > save_configuration_6.diff, save_configuration_7.diff, > save_configuration_8.diff, save_configuration_9.diff > > > Snapshot can backup user data, but it's also nice to be able to have > known-good configurations saved as well in case of accidental snafus or even > catastrophic loss of a cluster. If we check for changes to cassandra.yaml, > cassandra-env.sh, and maybe log4j-server.properties on startup, we can back > them up to a columnfamily that can then be handled by normal snapshot/backup > procedures. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420392#comment-13420392 ] David Alves commented on CASSANDRA-1123: What should we do wrt to reads then? also store the complete results (what is returned to the client) and the whole row (required to cover the examples you mentioned), or do we summarize in that case? In this case not summarizing can be costly (lots of net traffic). > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz, 1123.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13420323#comment-13420323 ] David Alves commented on CASSANDRA-1123: Right, but for that example storing the args as they are wouldn't help either, which was the issue to begin with. Yet of course your point is valid. In order to accomodate the use case you mention I think we could get away with storing the number and size of both argument and results in the "coordinator" node *and* store the actual size of the whole row (num and size of cols) in the nodes that are performing the reads. wdyt? > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz, 1123.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419889#comment-13419889 ] David Alves commented on CASSANDRA-1123: bq. Since every get_slice call will have the same arg count I don't see this being very useful. I actually meant storing both the size and length of the arguments *and* the results. would that be sufficient? > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz, 1123.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419741#comment-13419741 ] David Alves edited comment on CASSANDRA-1123 at 7/21/12 10:49 AM: -- i get the point, the original paper mentioned that that was the case because the data was stored outside of the cluster. Still there is the question of authentication, even though access control is not very thorough AFAIK, the principle behind it is per keyspace access correct? this would mean that we're storing data belonging to a keyspace (that might have access control) in another keyspace (that must be outside accessible). I'm happy to do it either way, but maybe we could instead store the number and length of the arguments. additionally this would decrease tracing network bandwidth usage. was (Author: dr-alves): i get the point, the original paper mentioned that that was the case because the data was stored outside of the cluster. Still there is the question of authentication, even though access control is not completely implemented, the principle behind it is per keyspace access correct? this would meant that we're storing data belonging to a keyspace (that might have access control) in another keyspace (that must be outside accessible). am I wrong? > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz, 1123.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419741#comment-13419741 ] David Alves commented on CASSANDRA-1123: i get the point, the original paper mentioned that that was the case because the data was stored outside of the cluster. Still there is the question of authentication, even though access control is not completely implemented, the principle behind it is per keyspace access correct? this would meant that we're storing data belonging to a keyspace (that might have access control) in another keyspace (that must be outside accessible). am I wrong? > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz, 1123.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13419737#comment-13419737 ] David Alves commented on CASSANDRA-1123: Thank you for the thorough review. bq. Inclined to think we should include the parameters along w/ the String request type on session start. (Object... + toString would be adequate.) Maybe even use the new List type to store the arguments (CASSANDRA-3647). In one of the papers this same issue raised privacy concerns that are probably even more valid for CASSANDRA since it's an open source project. IMO we should at the very least make this optional if not drop it all together. bq. Tracing should be asynchronous. StorageProxy.mutate waits for a response, this is not what we want. Suggest a simple ExecutorService + queue. (If queue gets full, throw out the tracing events and log a WARN.). bq. Would like tracing to log.debug the event as well. This will cut down on duplicate debug/trace code, but also give us a fallback if we can't log it remotely. This will also cut down on log spam for when we enable debug level globally – only logging requests at debug where tracing was explicitly enabled will be a huge improvement. +1, nice idea, was looking into doing something similar. bq. get_slice uses startSession instead of startSessionIfRequested. bq. There's a no-op initialization of trace context in StorageService. bq. Naming: system_enable_query_details -> system_enable_query__tracing. TraceSession, TraceSessionState -> TraceSessionContext, TraceSessionContextThreadLocalState. endSession -> stopSession. getQueryDetails -> isTracingEnabled. Most already corrected in current version. bq. CFMetaData definitions should be with the other hardcoded ones in CFMetaData. bq. Let's move helpers that are only used by test code like EVENT_TYPE into the Test class. will do! bq. Still think threadlocals are not the way to go, and this will become more clear as you try to add useful trace entries. I think you'll end up w/ a trace session "registry" like we have for MessagingService that we'll look up by session id. In that vein, I'm not sure what the afterExecute business is supposed to be doing. That stuff runs on the executor's thread, not the submitter's. Current version uses this to trace pre and post stage execution (which are the only trace events at the moment). I've just finished updating the cli and I'd like a chance to do some runs as is to get a sense of the usefulness of the current setup. bq. Finally, a more generic keyspace name like dsystem would be nice for all distributed system tables. (We're thinking of using one for CASSANDRA-3706, for instance.) will look into/synchronize. > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz, 1123.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13416363#comment-13416363 ] David Alves edited comment on CASSANDRA-1123 at 7/17/12 9:00 PM: - First patch that implements most of what was suggested, with accompanying test. Still to define/discuss: - aaraon original version "replaced" the logger which meant there were already a lot of things being traced. As is this patch is pretty "bare" in terms of trace events in the sense that stuff that goes through CassandraServer gets a new trace session but not much else is traced (even though sessions get propagated across threads and nodes). Maybe we could add events automatically when a session is propagated across stages and across nodes, therefore getting a per-stage and per-node view of how things are executed (I think this could be done rather unintrusively)? - I haven't added the log to the logger option yet, as I'm not completely sure how useful it is if events are generated per stage/node. was (Author: dr-alves): First patch that implements most of what was suggested, with accompanying test. Still to define/discuss: - aaraons "replaced" the logger with meant there were already a lot of things being logged. As is this patch is pretty "bare" in terms of trace events in the sense that stuff that goes through CassandraServer gets a new trace session but not much else is traced (even though session gets propagated across threads and nodes). Maybe we could add events automatically when a session is propagated across stages and across nodes? - I haven't added the log to the logger option yet, as I'm not completely sure how useful it is if events are generated per stage/node. > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz, 1123.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-1123: --- Attachment: 1123.patch First patch that implements most of what was suggested, with accompanying test. Still to define/discuss: - aaraons "replaced" the logger with meant there were already a lot of things being logged. As is this patch is pretty "bare" in terms of trace events in the sense that stuff that goes through CassandraServer gets a new trace session but not much else is traced (even though session gets propagated across threads and nodes). Maybe we could add events automatically when a session is propagated across stages and across nodes? - I haven't added the log to the logger option yet, as I'm not completely sure how useful it is if events are generated per stage/node. > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz, 1123.patch > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4054) SStableImport and SStableExport does not serialize row level deletion
[ https://issues.apache.org/jira/browse/CASSANDRA-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4054: --- Attachment: 4054.patch patch without abbreviated names > SStableImport and SStableExport does not serialize row level deletion > - > > Key: CASSANDRA-4054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4054 > Project: Cassandra > Issue Type: Bug > Components: Tools >Affects Versions: 0.5 >Reporter: Zhu Han >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 4054.patch, 4054.patch, 4054.patch > > > SSTableImport and SSTableExport does not serialize/de-serialize the row-level > deletion info to/from the json file. This brings back the deleted data after > restore from the json file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4054) SStableImport and SStableExport does not serialize row level deletion
[ https://issues.apache.org/jira/browse/CASSANDRA-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410387#comment-13410387 ] David Alves commented on CASSANDRA-4054: Yuki is the implementation on the latest patch what you had in mind? wrt to pulling deletionInfo up and abandoning metadata +0, I'm happy either way. > SStableImport and SStableExport does not serialize row level deletion > - > > Key: CASSANDRA-4054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4054 > Project: Cassandra > Issue Type: Bug > Components: Tools >Affects Versions: 0.5 >Reporter: Zhu Han >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 4054.patch, 4054.patch > > > SSTableImport and SSTableExport does not serialize/de-serialize the row-level > deletion info to/from the json file. This brings back the deleted data after > restore from the json file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4413) upgrade guava to 12.0
[ https://issues.apache.org/jira/browse/CASSANDRA-4413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4413: --- Attachment: 4413.tgz binary patch (compressed) that upgrades guava to v 12.0, and updates build.xml. > upgrade guava to 12.0 > - > > Key: CASSANDRA-4413 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4413 > Project: Cassandra > Issue Type: Task > Components: Core >Reporter: David Alves >Assignee: David Alves >Priority: Trivial > Fix For: 1.2 > > Attachments: 4413.tgz > > > Immediately useful because of StopWatch but also other goodies like > FluentIterable or LoadingCache (from 11.0). Depends on jdk 1.6 but also does > cassandra. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-4413) upgrade guava to 12.0
David Alves created CASSANDRA-4413: -- Summary: upgrade guava to 12.0 Key: CASSANDRA-4413 URL: https://issues.apache.org/jira/browse/CASSANDRA-4413 Project: Cassandra Issue Type: Task Components: Core Reporter: David Alves Assignee: David Alves Priority: Trivial Fix For: 1.2 Immediately useful because of StopWatch but also other goodies like FluentIterable or LoadingCache (from 11.0). Depends on jdk 1.6 but also does cassandra. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1123) Allow tracing query details
[ https://issues.apache.org/jira/browse/CASSANDRA-1123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406836#comment-13406836 ] David Alves commented on CASSANDRA-1123: was about to start tracing with StopWatch, but it's only available from 10.0. upgrade? > Allow tracing query details > --- > > Key: CASSANDRA-1123 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1123 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: Jonathan Ellis >Assignee: David Alves > Fix For: 1.2 > > Attachments: 1123-3.patch.gz > > > In the spirit of CASSANDRA-511, it would be useful to tracing on queries to > see where latency is coming from: how long did row cache lookup take? key > search in the index? merging the data from the sstables? etc. > The main difference vs setting debug logging is that debug logging is too big > of a hammer; by turning on the flood of logging for everyone, you actually > distort the information you're looking for. This would be something you > could set per-query (or more likely per connection). > We don't need to be as sophisticated as the techniques discussed in the > following papers but they are interesting reading: > http://research.google.com/pubs/pub36356.html > http://www.usenix.org/events/osdi04/tech/full_papers/barham/barham_html/ > http://www.usenix.org/event/nsdi07/tech/fonseca.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406093#comment-13406093 ] David Alves edited comment on CASSANDRA-3564 at 7/3/12 9:58 PM: I'm trying to wrap this up... I changed nodecmd to handle the expected exception nicely (the EOF one because thrift was closed on the other side). the "cassandra" shell script does not deal with PID reading/killing. So I was wondering should we include this functionality in "cassandra", i.e., call nodetool, read/check pid wait and kill, or should we deal with it within nodetool (maybe make the flushAndExit function return the PID to nodetool and have that kill the process if required). was (Author: dr-alves): I'm trying to wrap this up... I changed nodecmd to handle the expected exception nicely (the EOF one because thrift was closed on the other side). the "cassandra" shell script does not deal with PID reading/killing. So I was wondering should we include this call nodetool, read/check pid and kill funcionality in "cassandra" or should we deal with it within nodetool (maybe make the flushAndExit function return the PID to nodetool and have that kill the process if required). > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406093#comment-13406093 ] David Alves commented on CASSANDRA-3564: I'm trying to wrap this up... I changed nodecmd to handle the expected exception nicely (the EOF one because thrift was closed on the other side). the "cassandra" shell script does not deal with PID reading/killing. So I was wondering should we include this call nodetool, read/check pid and kill funcionality in "cassandra" or should we deal with it within nodetool (maybe make the flushAndExit function return the PID to nodetool and have that kill the process if required). > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2181) sstable2json should return better error message if the usage is wrong
[
https://issues.apache.org/jira/browse/CASSANDRA-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13406013#comment-13406013
]
David Alves commented on CASSANDRA-2181:
Shotaro, any news on this?
> sstable2json should return better error message if the usage is wrong
> -
>
> Key: CASSANDRA-2181
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2181
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Environment: linux
>Reporter: Shotaro Kamio
>Assignee: David Alves
>Priority: Minor
> Fix For: 1.2
>
> Attachments: 2181.patch
>
>
> These errors are not user friendly.
> (Cassandra 0.7.2)
> $ bin/sstable2json PATH_TO/Order-f-7-Data.db -k aaa -x 0
> WARN 21:55:34,383 Schema definitions were defined both locally and in
> cassandra.yaml. Definitions in cassandra.yaml were ignored.
> {
> "aaa": {Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:108)
> at
> org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:178)
> at
> org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:310)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:444)
> $ bin/sstable2json PATH_TO/Order-f-7-Data.db -k aaa
> WARN 21:55:49,603 Schema definitions were defined both locally and in
> cassandra.yaml. Definitions in cassandra.yaml were ignored.
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:284)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:444)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2293) Rewrite nodetool help
[ https://issues.apache.org/jira/browse/CASSANDRA-2293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404514#comment-13404514 ] David Alves commented on CASSANDRA-2293: just to be clear, this ticket is supposed to remove all help related stuff from NodeCmd, move it to a yaml file and use an approach similar to cli, correct? > Rewrite nodetool help > - > > Key: CASSANDRA-2293 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2293 > Project: Cassandra > Issue Type: Improvement > Components: Core, Documentation & website >Affects Versions: 0.8 beta 1 >Reporter: Aaron Morton >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > > Once CASSANDRA-2008 is through and we are happy with the approach I would > like to write similar help for nodetool. > Both command line help of the form "nodetool help" and "nodetool help > command." -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-4054) SStableImport and SStableExport does not serialize row level deletion
[ https://issues.apache.org/jira/browse/CASSANDRA-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404319#comment-13404319 ] David Alves edited comment on CASSANDRA-4054 at 6/29/12 11:36 PM: -- the only point I can think of for the metadata container is to ease maintenance/compatibility down the line (i.e. writing/parsing metadata can change without having impact on the overall format), then again the less verbose the output the better (previous format had that in mind and that's why I used "meta" and "cols" instead of "metadata" and "columns"). I'm happy either way. wdyt? was (Author: dr-alves): the only point I can think of for the metadata container is to ease maintenance compatibility down the line (i.e. writing/parsing metadata can change without having impact on the overall format), then again the less verbose the output the better (previous format had that in mind and that's why I used "meta" and "cols" instead of "metadata" and "columns"). I'm happy wither way. wdyt? > SStableImport and SStableExport does not serialize row level deletion > - > > Key: CASSANDRA-4054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4054 > Project: Cassandra > Issue Type: Bug > Components: Tools >Affects Versions: 0.5 >Reporter: Zhu Han >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 4054.patch, 4054.patch > > > SSTableImport and SSTableExport does not serialize/de-serialize the row-level > deletion info to/from the json file. This brings back the deleted data after > restore from the json file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4054) SStableImport and SStableExport does not serialize row level deletion
[ https://issues.apache.org/jira/browse/CASSANDRA-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404319#comment-13404319 ] David Alves commented on CASSANDRA-4054: the only point I can think of for the metadata container is to ease maintenance compatibility down the line (i.e. writing/parsing metadata can change without having impact on the overall format), then again the less verbose the output the better (previous format had that in mind and that's why I used "meta" and "cols" instead of "metadata" and "columns"). I'm happy wither way. wdyt? > SStableImport and SStableExport does not serialize row level deletion > - > > Key: CASSANDRA-4054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4054 > Project: Cassandra > Issue Type: Bug > Components: Tools >Affects Versions: 0.5 >Reporter: Zhu Han >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 4054.patch, 4054.patch > > > SSTableImport and SSTableExport does not serialize/de-serialize the row-level > deletion info to/from the json file. This brings back the deleted data after > restore from the json file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4054) SStableImport and SStableExport does not serialize row level deletion
[ https://issues.apache.org/jira/browse/CASSANDRA-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-4054: --- Attachment: 4054.patch additionally implements Yuki's suggestion. also deals with formatting and whitespace issues. > SStableImport and SStableExport does not serialize row level deletion > - > > Key: CASSANDRA-4054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-4054 > Project: Cassandra > Issue Type: Bug > Components: Tools >Affects Versions: 0.5 >Reporter: Zhu Han >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 4054.patch, 4054.patch > > > SSTableImport and SSTableExport does not serialize/de-serialize the row-level > deletion info to/from the json file. This brings back the deleted data after > restore from the json file. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3633) update stress to support prepared statements
[
https://issues.apache.org/jira/browse/CASSANDRA-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Alves updated CASSANDRA-3633:
---
Attachment: 3633.patch
updates Eric's 3633.stress branch to use binary encoding.
> update stress to support prepared statements
>
>
> Key: CASSANDRA-3633
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3633
> Project: Cassandra
> Issue Type: Sub-task
> Components: API, Core
>Reporter: Eric Evans
>Assignee: David Alves
>Priority: Minor
> Labels: cql
> Fix For: 1.1.2
>
> Attachments: 3633.patch
>
>
> The {{stress}} utility needs to be updated for testing prepared statements.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3047) implementations of IPartitioner.describeOwnership() are not DC aware
[ https://issues.apache.org/jira/browse/CASSANDRA-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-3047: --- Attachment: 3047.patch corrected some bugs in NodeCmd output. > implementations of IPartitioner.describeOwnership() are not DC aware > > > Key: CASSANDRA-3047 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3047 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Aaron Morton >Assignee: David Alves >Priority: Trivial > Fix For: 1.1.2 > > Attachments: 3047.patch, 3047.patch, CASSANDRA-3047.patch, > CASSANDRA-3047.patch, CASSANDRA-3047.patch, CASSANDRA-3047.patch > > > see http://www.mail-archive.com/user@cassandra.apache.org/msg16375.html > When a cluster the multiple rings approach to tokens the output from nodetool > ring is incorrect. > When it uses the interleaved token approach (e.g. dc1, dc2, dc1, dc2) it will > be correct. > It's a bit hacky but could we special case (RP) tokens that are off by 1 and > calculate the ownership per dc ? I guess another approach would be to add > some parameters so the partitioner can be told about the token assignment > strategy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3047) implementations of IPartitioner.describeOwnership() are not DC aware
[ https://issues.apache.org/jira/browse/CASSANDRA-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Alves updated CASSANDRA-3047: --- Attachment: 3047.patch applies on top of 3881. addresses most suggestions, code is a lot less verbose and returned map is now endpoint->ownership. > implementations of IPartitioner.describeOwnership() are not DC aware > > > Key: CASSANDRA-3047 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3047 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Aaron Morton >Assignee: David Alves >Priority: Trivial > Fix For: 1.1.2 > > Attachments: 3047.patch, CASSANDRA-3047.patch, CASSANDRA-3047.patch, > CASSANDRA-3047.patch, CASSANDRA-3047.patch > > > see http://www.mail-archive.com/user@cassandra.apache.org/msg16375.html > When a cluster the multiple rings approach to tokens the output from nodetool > ring is incorrect. > When it uses the interleaved token approach (e.g. dc1, dc2, dc1, dc2) it will > be correct. > It's a bit hacky but could we special case (RP) tokens that are off by 1 and > calculate the ownership per dc ? I guess another approach would be to add > some parameters so the partitioner can be told about the token assignment > strategy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3881) reduce computational complexity of processing topology changes
[
https://issues.apache.org/jira/browse/CASSANDRA-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402965#comment-13402965
]
David Alves commented on CASSANDRA-3881:
hi sam
yes I was using that ctor to test StorageService.effectiveOwnership, included
in the CASSANDRA-3047 patch.
I worked around it since it makes sense that it makes sense that TokenMetadata
receives token->endpoint mappings through {code}updateNormalTokens{code}, in
order to build topology. The thing was that while previously the ctor
{code}TokenMetadata(BiMap tokenToEndpointMap){code} was
usable, now it is not, at least not without getting topology from another
TokenMetadata instance.
> reduce computational complexity of processing topology changes
> --
>
> Key: CASSANDRA-3881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3881
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Peter Schuller
>Assignee: Sam Overton
> Labels: vnodes
>
> This constitutes follow-up work from CASSANDRA-3831 where a partial
> improvement was committed, but the fundamental issue was not fixed. The
> maximum "practical" cluster size was significantly improved, but further work
> is expected to be necessary as cluster sizes grow.
> _Edit0: Appended patch information._
> h3. Patches
> ||Compare||Raw diff||Description||
> |[00_snitch_topology|https://github.com/acunu/cassandra/compare/refs/top-bases/p/3881/00_snitch_topology...p/3881/00_snitch_topology]|[00_snitch_topology.patch|https://github.com/acunu/cassandra/compare/refs/top-bases/p/3881/00_snitch_topology...p/3881/00_snitch_topology.diff]|Adds
> some functionality to TokenMetadata to track which endpoints and racks exist
> in a DC.|
> |[01_calc_natural_endpoints|https://github.com/acunu/cassandra/compare/refs/top-bases/p/3881/01_calc_natural_endpoints...p/3881/01_calc_natural_endpoints]|[01_calc_natural_endpoints.patch|https://github.com/acunu/cassandra/compare/refs/top-bases/p/3881/01_calc_natural_endpoints...p/3881/01_calc_natural_endpoints.diff]|Rewritten
> O(logN) implementation of calculateNaturalEndpoints using the topology
> information from the tokenMetadata.|
>
> _Note: These are branches managed with TopGit. If you are applying the patch
> output manually, you will either need to filter the TopGit metadata files
> (i.e. {{wget -O - | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}),
> or remove them afterward ({{rm .topmsg .topdeps}})._
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3047) implementations of IPartitioner.describeOwnership() are not DC aware
[ https://issues.apache.org/jira/browse/CASSANDRA-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402664#comment-13402664 ] David Alves commented on CASSANDRA-3047: ok, I'll leave the generic ground work here and do a specific vnodes follow up on another ticket. > implementations of IPartitioner.describeOwnership() are not DC aware > > > Key: CASSANDRA-3047 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3047 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Aaron Morton >Assignee: David Alves >Priority: Trivial > Fix For: 1.1.2 > > Attachments: CASSANDRA-3047.patch, CASSANDRA-3047.patch, > CASSANDRA-3047.patch, CASSANDRA-3047.patch > > > see http://www.mail-archive.com/user@cassandra.apache.org/msg16375.html > When a cluster the multiple rings approach to tokens the output from nodetool > ring is incorrect. > When it uses the interleaved token approach (e.g. dc1, dc2, dc1, dc2) it will > be correct. > It's a bit hacky but could we special case (RP) tokens that are off by 1 and > calculate the ownership per dc ? I guess another approach would be to add > some parameters so the partitioner can be told about the token assignment > strategy. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3047) implementations of IPartitioner.describeOwnership() are not DC aware
[
https://issues.apache.org/jira/browse/CASSANDRA-3047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402644#comment-13402644
]
David Alves commented on CASSANDRA-3047:
thanks for the review and for the clues. code is much more succinct with
topology and getRangesForEndpoint.
final question, since we're changing the mapping to a per endpoint basis should
we keep showing the tokens, or do something that would work straight out with
vnodes such as (per dc):
{code}
Address DC RackStatus State LoadOwns
Num. Ranges RangeIds
127.0.0.4 DC3 RAC1Up Normal 52.76 KB75.00%
1 0
127.0.0.5 DC3 RAC1Up Normal 54.46 KB75.00%
1 1
127.0.0.6 DC3 RAC2Up Normal 52.78 KB75.00%
1 2
127.0.0.7 DC3 RAC2Up Normal 52.78 KB75.00%
1 3
{code}
where range id is the position in the token ring starting from 0 and then allow
to get the token delimiters for a range with another command (or get all for an
endpoint?).
> implementations of IPartitioner.describeOwnership() are not DC aware
>
>
> Key: CASSANDRA-3047
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3047
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
>Reporter: Aaron Morton
>Assignee: David Alves
>Priority: Trivial
> Fix For: 1.1.2
>
> Attachments: CASSANDRA-3047.patch, CASSANDRA-3047.patch,
> CASSANDRA-3047.patch, CASSANDRA-3047.patch
>
>
> see http://www.mail-archive.com/user@cassandra.apache.org/msg16375.html
> When a cluster the multiple rings approach to tokens the output from nodetool
> ring is incorrect.
> When it uses the interleaved token approach (e.g. dc1, dc2, dc1, dc2) it will
> be correct.
> It's a bit hacky but could we special case (RP) tokens that are off by 1 and
> calculate the ownership per dc ? I guess another approach would be to add
> some parameters so the partitioner can be told about the token assignment
> strategy.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3881) reduce computational complexity of processing topology changes
[
https://issues.apache.org/jira/browse/CASSANDRA-3881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402392#comment-13402392
]
David Alves commented on CASSANDRA-3881:
Hi
small nit:
I was using the ctor {code}public TokenMetadata(BiMap
tokenToEndpointMap){code} in non-commited code. That ctor has now become
{code}public TokenMetadata(BiMap tokenToEndpointMap,
Topology topology){code} since the ctor in Topology is protected this ctor in
TokenMetadata can't be called outside of protected scope from now on. I suggest
either broadening the scope of the ctor in Topology or restricting the scope in
TokenMetadata.
> reduce computational complexity of processing topology changes
> --
>
> Key: CASSANDRA-3881
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3881
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Peter Schuller
>Assignee: Sam Overton
> Labels: vnodes
>
> This constitutes follow-up work from CASSANDRA-3831 where a partial
> improvement was committed, but the fundamental issue was not fixed. The
> maximum "practical" cluster size was significantly improved, but further work
> is expected to be necessary as cluster sizes grow.
> _Edit0: Appended patch information._
> h3. Patches
> ||Compare||Raw diff||Description||
> |[00_snitch_topology|https://github.com/acunu/cassandra/compare/refs/top-bases/p/3881/00_snitch_topology...p/3881/00_snitch_topology]|[00_snitch_topology.patch|https://github.com/acunu/cassandra/compare/refs/top-bases/p/3881/00_snitch_topology...p/3881/00_snitch_topology.diff]|Adds
> some functionality to TokenMetadata to track which endpoints and racks exist
> in a DC.|
> |[01_calc_natural_endpoints|https://github.com/acunu/cassandra/compare/refs/top-bases/p/3881/01_calc_natural_endpoints...p/3881/01_calc_natural_endpoints]|[01_calc_natural_endpoints.patch|https://github.com/acunu/cassandra/compare/refs/top-bases/p/3881/01_calc_natural_endpoints...p/3881/01_calc_natural_endpoints.diff]|Rewritten
> O(logN) implementation of calculateNaturalEndpoints using the topology
> information from the tokenMetadata.|
>
> _Note: These are branches managed with TopGit. If you are applying the patch
> output manually, you will either need to filter the TopGit metadata files
> (i.e. {{wget -O - | filterdiff -x*.topdeps -x*.topmsg | patch -p1}}),
> or remove them afterward ({{rm .topmsg .topdeps}})._
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-1337) parallelize fetching rows for low-cardinality indexes
[ https://issues.apache.org/jira/browse/CASSANDRA-1337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399093#comment-13399093 ] David Alves commented on CASSANDRA-1337: Thanks for the review Vijay. I have nothing to add... > parallelize fetching rows for low-cardinality indexes > - > > Key: CASSANDRA-1337 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1337 > Project: Cassandra > Issue Type: Improvement >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: > 0001-CASSANDRA-1337-scan-concurrently-depending-on-num-rows.txt, > CASSANDRA-1337.patch > > Original Estimate: 8h > Remaining Estimate: 8h > > currently, we read the indexed rows from the first node (in partitioner > order); if that does not have enough matching rows, we read the rows from the > next, and so forth. > we should use the statistics fom CASSANDRA-1155 to query multiple nodes in > parallel, such that we have a high chance of getting enough rows w/o having > to do another round of queries (but, if our estimate is incorrect, we do need > to loop and do more rounds until we have enough data or we have fetched from > each node). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13399090#comment-13399090 ] David Alves commented on CASSANDRA-3564: bq. Sure, but this is where is gets a little more complicated, since after receiving the error the init script needs to check that the pid is really gone, and if not get more forceful. was about to implement this when a problem came to mind. There's no time limit on how long a flush will take and shutdown hooks can take as long as needed to finish up so we have no idea when to "get more forceful". There's two ways we could deal with this IMO: - we poll-check on the pid, nodetool side, and give it an umbrella grace period to do everything, after that we SIGKILL it - we simply inform that it might take a while. > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398571#comment-13398571 ] David Alves commented on CASSANDRA-3564: bq. Sure, but this is where is gets a little more complicated, since after receiving the error the init script needs to check that the pid is really gone, and if not get more forceful. hum... handn't thought about that. you're right. any script/code comes to mind that has similar functionality? (for inspiration :)) > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3564) flush before shutdown so restart is faster
[ https://issues.apache.org/jira/browse/CASSANDRA-3564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398546#comment-13398546 ] David Alves commented on CASSANDRA-3564: thanks for the review bq. The catch with flushandexit is, you have no way of knowing if it worked or not (from nodetool's perspective) since either way, you're going to get an error (EOF because the jvm shutdown, or EOF because ???) right, I thought about that (might even deserve some special error handling) but the only alternative that I could think was to make the call async so the RPC method would return fine. But even that wouldn't provide any meaningful feedback since the user would still have no idea whether things flushed ok so I left it as is. If its alright, I'll finish up by adding the cassandra stop command to the shell script and handle the broken rpc call. > flush before shutdown so restart is faster > -- > > Key: CASSANDRA-3564 > URL: https://issues.apache.org/jira/browse/CASSANDRA-3564 > Project: Cassandra > Issue Type: New Feature > Components: Packaging >Reporter: Jonathan Ellis >Assignee: David Alves >Priority: Minor > Fix For: 1.2 > > Attachments: 3564.patch > > > Cassandra handles flush in its shutdown hook for durable_writes=false CFs > (otherwise we're *guaranteed* to lose data) but leaves it up to the operator > otherwise. I'd rather leave it that way to offer these semantics: > - cassandra stop = shutdown nicely [explicit flush, then kill -int] > - kill -INT = shutdown faster but don't lose any updates [current behavior] > - kill -KILL = lose most recent writes unless durable_writes=true and batch > commits are on [also current behavior] > But if it's not reasonable to use nodetool from the init script then I guess > we can just make the shutdown hook flush everything. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (CASSANDRA-2181) sstable2json should return better error message if the usage is wrong
[
https://issues.apache.org/jira/browse/CASSANDRA-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13398205#comment-13398205
]
David Alves edited comment on CASSANDRA-2181 at 6/21/12 5:50 AM:
-
i'm not able to reproduce the issue.
the only error I'm hitting is a malformed hex key and that one is pretty clear.
I've tried passing a missing key and an existing one as argument and both times
the exporter does what it should.
the patch changes usage and adds a check to avoid letting the user use -k and
-x at the same time
Shotaro Kamio: could you help me to try and figure it out? can you still
reproduce this error?
was (Author: dr-alves):
i'm not able to reproduce the issue.
the only error I'm hitting is a malformed hex key and that one is pretty clear.
I've tried passing a missing key and an existing one as argument and both times
the exporter does what it should.
the patch changes usage and adds a check to avoid letting the user use -k and
-x at the same time
> sstable2json should return better error message if the usage is wrong
> -
>
> Key: CASSANDRA-2181
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2181
> Project: Cassandra
> Issue Type: Improvement
> Components: Tools
> Environment: linux
>Reporter: Shotaro Kamio
>Assignee: David Alves
>Priority: Minor
> Fix For: 1.2
>
> Attachments: 2181.patch
>
>
> These errors are not user friendly.
> (Cassandra 0.7.2)
> $ bin/sstable2json PATH_TO/Order-f-7-Data.db -k aaa -x 0
> WARN 21:55:34,383 Schema definitions were defined both locally and in
> cassandra.yaml. Definitions in cassandra.yaml were ignored.
> {
> "aaa": {Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.cassandra.db.columniterator.SSTableSliceIterator.hasNext(SSTableSliceIterator.java:108)
> at
> org.apache.cassandra.tools.SSTableExport.serializeRow(SSTableExport.java:178)
> at
> org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:310)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:444)
> $ bin/sstable2json PATH_TO/Order-f-7-Data.db -k aaa
> WARN 21:55:49,603 Schema definitions were defined both locally and in
> cassandra.yaml. Definitions in cassandra.yaml were ignored.
> Exception in thread "main" java.lang.NullPointerException
> at
> org.apache.cassandra.tools.SSTableExport.export(SSTableExport.java:284)
> at org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:444)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
