[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14527111#comment-14527111 ] Matt Stump commented on CASSANDRA-9193: --- I spent some time exploring hanging a JS REPL off of C* last night to see if that what I could do in terms of dynamically rewriting functionality of a running StorageProxy. I can interrogate or modify variables, and call methods, but I can't dynamically rewrite code. I think this may be a JVM limitation. I can extend a Java class overriding a method, but I can't modify the method of an already instantiated object unless it was already being resolved via invoke dynamic (I think). We could still have a JS REPL, but it's usefulness is limited. The vision of having the power of something like a Lisp or Erlang REPL hang off of the server process is mostly dead. So that leaves us with three possibilities, the known injection points as originally mentioned, the UDF route as outlined by [~slebresne], or a combination of the two. > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495157#comment-14495157 ] Albert P Tobey commented on CASSANDRA-9193: --- Javascript makes sense since Nashorn ships with Java 8. No dependencies to add to C*. Looks like you can get at a lot of the stuff we'd need as soon as a REPL or some way to run scripts is available: http://moduscreate.com/javascript-and-the-jvm/ > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[
https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495451#comment-14495451
]
Patrick McFadin commented on CASSANDRA-9193:
I don't think this is too far away from CASSANDRA-7526 with addition of
metadata.
I have used something similar on F5 network appliances in a feature called
iRules. It was a feature to run a trigger based on a network event. My favorite
part was how the user specified actions. You assigned a rule to a network port
and wrote a collection of actions on events. If I were to translate that to a
Cassandra use case, you would assign a rule set to a Table. Inside the rule set
would be actions on events. Something like this pseudo code:
{
onRequest {
if(consistencyLevel == ALL) {
log.WARN ("Dude. Seriously?")
}
}
onResponse {
if (consistencyError) {
...do something...
}
if (data.size > 50) {
log.WARN ("Dude. Seriously?")
}
}
}
Not proposing syntax but you get the idea. Could be a very powerful tool for
troubleshooting and insight.
> Facility to write dynamic code to selectively trigger trace or log for queries
> --
>
> Key: CASSANDRA-9193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9193
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Matt Stump
>
> I want the equivalent of dtrace for Cassandra. I want the ability to
> intercept a query with a dynamic script (assume JS) and based on logic in
> that script trigger the statement for trace or logging.
> Examples
> - Trace only INSERT statements to a particular CF.
> - Trace statements for a particular partition or consistency level.
> - Log statements that fail to reach the desired consistency for read or write.
> - Log If the request size for read or write exceeds some threshold
> At some point in the future it would be helpful to also do things such as log
> partitions greater than X bytes or Z cells when performing compaction.
> Essentially be able to inject custom code dynamically without a reboot to the
> different stages of C*.
> The code should be executed synchronously as part of the monitored task, but
> we should provide the ability to log or execute CQL asynchronously from the
> provided API.
> Further down the line we could use this functionality to modify/rewrite
> requests or tasks dynamically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495458#comment-14495458 ] Jon Haddad commented on CASSANDRA-9193: --- Going on record as loving this idea. > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503054#comment-14503054 ] Matt Stump commented on CASSANDRA-9193: --- I've got it so I can run JS when events are triggered, but the problem is that the partition key is serialized to a ByteBuffer fairly high in the stack above StorageProxy. StorageProxy doesn't have the context of a CQL3 query so if the query utilizes a composite partition key it's hard to check for equality if you wanted to filter by partition key. We have three options: # Only allow JS introspection for CQL3 queries # Expose functionality to compose composite partition keys in JS so that we can check for equality # Require that the user know the fully serialized partition key beforehand. I'm leaning towards option 1 and 3. Offer multiple injection points, one at the CQL3 query processing level, and maybe later an injection point further down the stack which would require knowledge of the fully serialized partition key. > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503069#comment-14503069 ] Sylvain Lebresne commented on CASSANDRA-9193: - Hard to have an opinion without having more context on how this is working, but I do feel like we shouldn't ask users to work on serialized stuffs in general. That is, people should have the ability to filter on CQL partition key column values and the handling of composite versus non-composite partition keys should be transparent. bq. StorageProxy doesn't have the context of a CQL3 query It kind of does, through the CFMetaData (which will tell you if the partition key is composite or not). > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[
https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503087#comment-14503087
]
Matt Stump commented on CASSANDRA-9193:
---
It's actually pretty simple the Nashorn API makes this pretty easy. For context
here is a little snippet that I had running in {{StorageProxy.read}}. I'm
evaluating a script, then executing a named function passing parameters to the
JS function. The idea would be to pull a script from a CF for the injection
point, eval and when we reach that point in the code invoke a known function
name adhering to an interface. We would pass the request parameters. The JS
would have full visibility into the internals of C*. Each injection point
would have a different context (scope) so that we don't have to worry about
namespace collision between scripts.
{code}
String script = ""
+ "var printReadCommands = function(readCommands,
consistency, state) {"
+ "for each (var rc in readCommands) {"
+ "print(\"rc class definition: \" +
Object.prototype.toString.call(rc) + \"\\n\");"
+ "print(\"KEYSPACE \" + rc.ksName);"
+ "print(\" CF \" + rc.cfName);"
+ "print(\" KEY \" + rc.key);"
+ "print(\"\\n\");"
+ "}"
+ "};";
jsEngine.eval(script);
Invocable invocable = (Invocable) jsEngine;
Object result = invocable.invokeFunction("printReadCommands",
commands, consistencyLevel, state);
{code}
> Facility to write dynamic code to selectively trigger trace or log for queries
> --
>
> Key: CASSANDRA-9193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9193
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Matt Stump
>
> I want the equivalent of dtrace for Cassandra. I want the ability to
> intercept a query with a dynamic script (assume JS) and based on logic in
> that script trigger the statement for trace or logging.
> Examples
> - Trace only INSERT statements to a particular CF.
> - Trace statements for a particular partition or consistency level.
> - Log statements that fail to reach the desired consistency for read or write.
> - Log If the request size for read or write exceeds some threshold
> At some point in the future it would be helpful to also do things such as log
> partitions greater than X bytes or Z cells when performing compaction.
> Essentially be able to inject custom code dynamically without a reboot to the
> different stages of C*.
> The code should be executed synchronously as part of the monitored task, but
> we should provide the ability to log or execute CQL asynchronously from the
> provided API.
> Further down the line we could use this functionality to modify/rewrite
> requests or tasks dynamically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[
https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503098#comment-14503098
]
Matt Stump commented on CASSANDRA-9193:
---
The above example demonstrates the serialized partition key dilemma pretty
well. A {{ReadCommand}} has ByteBuffer for key, which can be a serialized
composite. I suppose I could provide a function that given a CFMetaData and a
key byte buffer return an array of java objects to JS, one object per key
component.
> Facility to write dynamic code to selectively trigger trace or log for queries
> --
>
> Key: CASSANDRA-9193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9193
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Matt Stump
>
> I want the equivalent of dtrace for Cassandra. I want the ability to
> intercept a query with a dynamic script (assume JS) and based on logic in
> that script trigger the statement for trace or logging.
> Examples
> - Trace only INSERT statements to a particular CF.
> - Trace statements for a particular partition or consistency level.
> - Log statements that fail to reach the desired consistency for read or write.
> - Log If the request size for read or write exceeds some threshold
> At some point in the future it would be helpful to also do things such as log
> partitions greater than X bytes or Z cells when performing compaction.
> Essentially be able to inject custom code dynamically without a reboot to the
> different stages of C*.
> The code should be executed synchronously as part of the monitored task, but
> we should provide the ability to log or execute CQL asynchronously from the
> provided API.
> Further down the line we could use this functionality to modify/rewrite
> requests or tasks dynamically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503111#comment-14503111 ] Albert P Tobey commented on CASSANDRA-9193: --- Maybe just steal this? https://github.com/datastax/nodejs-driver/blob/master/lib/tokenizer.js > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[
https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14504597#comment-14504597
]
Sylvain Lebresne commented on CASSANDRA-9193:
-
I don't know, I'm sorry to be difficult but I don't really like the idea of
having people write functions that manipulate internal data structures (like
ReadCommand). Those stuffs are way too volatile to make a user-facing API
depend on it (please don't get me wrong, I'm not saying we haven't done such
things in the past, I'm saying that every single time we've done it, we've come
to regret it and it has been a bad user experience).
I also really agree that this should reuse stuffs from CASSANDRA-7526 as much
as possible rather than being JS specific.
Anyway, I'm going to go out on a limb here since I haven't though too much
about this, but the kind of interface I would prefer would be something along
the lines of:
{noformat}
CREATE FUNCTION myFunction(r request) LANGUAGE JAVA AS
$$
if (!r.isSelect())
return;
if (r.consistency().equals("ALL"))
logger.warn("WHAT!?");
if (r.size() > 50)
logger.warn("Ouch!");
// If the request has 'c1 = 3' in the WHERE clause
if (r.selects("c1", 3))
r.trace();
$$;
INJECT myFunction ON REQUEST TO myKs.myCf;
{noformat}
I'm not attached to any of the specifics here, it's just to say that it would
be ideal in my book if 1) we could reuse UDFs and 2) having it directly doable
from CQL. I'm also her implying we'd add some custom {{request}} object that
we'd pass to the function and would also to get the different parameters of a
request.
> Facility to write dynamic code to selectively trigger trace or log for queries
> --
>
> Key: CASSANDRA-9193
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9193
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Matt Stump
>
> I want the equivalent of dtrace for Cassandra. I want the ability to
> intercept a query with a dynamic script (assume JS) and based on logic in
> that script trigger the statement for trace or logging.
> Examples
> - Trace only INSERT statements to a particular CF.
> - Trace statements for a particular partition or consistency level.
> - Log statements that fail to reach the desired consistency for read or write.
> - Log If the request size for read or write exceeds some threshold
> At some point in the future it would be helpful to also do things such as log
> partitions greater than X bytes or Z cells when performing compaction.
> Essentially be able to inject custom code dynamically without a reboot to the
> different stages of C*.
> The code should be executed synchronously as part of the monitored task, but
> we should provide the ability to log or execute CQL asynchronously from the
> provided API.
> Further down the line we could use this functionality to modify/rewrite
> requests or tasks dynamically.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14512043#comment-14512043 ] Matt Stump commented on CASSANDRA-9193: --- [~slebresne] I'm not ready to disagree yet, but I want to make two points and see what you guys think. # The consumer of this facility would primarily be advanced users that are attempting to gain visibility into Cassandra internals. Right now the field doesn't have a good way to perform targeted analysis for a variety of tasks of which queries are only a subset. I can see wanting to use this to troubleshoot compaction for example. # I want to back port this to older versions of Cassandra so that I can include it in DSE. The reason being is that the field needs this now and it's going to take a while before whatever branch of 3.X this lands in becomes readily deployed. If I'm dependent on UDFs then back porting becomes impossible. > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14514386#comment-14514386 ] Sylvain Lebresne commented on CASSANDRA-9193: - bq. The consumer of this facility would primarily be advanced users that are attempting to gain visibility into Cassandra internals I don't entirely disagree, but at the same time, the more easy this is to use, the more likely this will be to people. In particular, tying this too much to the internals will imo make it painful in practice because those can change a lot and without warnings. bq. I want to back port this to older versions of Cassandra \[...\] The reason being is that the field needs this now and it's going to take a while before whatever branch of 3.X this lands in becomes readily deployed. While I understand the need, this is pretty much out of scope for anyting pre-3.0. I also really think for anything user-facing like this, we should try to make it as clean, generally useful and easy to maintain moving forward as possible, and this even if this mean taking more time to deliver it. > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14514538#comment-14514538 ] Rich Rein commented on CASSANDRA-9193: -- >From prior experience, this a tremendously helpful facility. If access to objects (and possibly lazy loading), result sets, and synchronization is thought out, the scripting is exactly this easy. Matt, do you have any resistance to making this an option on a API request to micro trace the internals? > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14514622#comment-14514622 ] Benedict commented on CASSANDRA-9193: - I completely agree with [~slebresne] here. The internal APIs are really not a good thing to expose in an API like this. Even full time developers get behind on how they function at times, and the interfaces can vary dramatically. Expecting anyone to interface with these is a bad idea, and likely to result in ill will. Conversely, having the same scripts function across C* versions would make this much more useful to teams deploying multiple clusters. I can also see this being a really powerful tool used much more widely than those conversant with the codebase. > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14516839#comment-14516839 ] Robert Stupp commented on CASSANDRA-9193: - TBH using JSR233 stuff for logging is not a good idea. Using a scripting language (either JavaScript or any other) will add a significant amount of overhead to each request. And even worse, that penalty increases when JS is invoked concurrently. IMO we know for sure that implementations using internal 2.1 APIs of will no longer work in 3.0. Having such a thing could possibly break the upgrade from 2.1 to 3.0. I like the idea to pass a clean interface (as proposed by Sylvain) and probably a logger instance to a (Java) UDF. Such a logger could be named according to the keyspace+table names so its logging level can be easily configured using JMX and logger config files. > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14517534#comment-14517534 ] Patrick McFadin commented on CASSANDRA-9193: I'm sensing this debate shifting to more about a potential developer centric feature when it's almost 100% not. This is all about the operators/DBAs that need insight. System log and JMX mostly work in a postmortem but less so in a real-time diagnostic scenario. The two most important features are that it's expressive and complete. I don't think that negative side effect is a deal killer. I would even say if there wasn't any potential negative side-effects then it's not complete enough. If you look at the original kernel of an idea for this Jira, "dtrace for Cassandra" it holds up. Dtrace can add a lot of overhead. You can do horrible things to your OS. Despite of all the downsides, put in the hands of the right operator, dtrace can save hours of troubleshooting and potentially save the day. As scary as it might be, this seems to me like a step in the right direction for product maturity. > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14518074#comment-14518074 ] Jonathan Ellis commented on CASSANDRA-9193: --- Just to make sure we're on the same page: Adding overhead for traced requests: okay Adding overhead for non-traced requests: not okay > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14518079#comment-14518079 ] Patrick McFadin commented on CASSANDRA-9193: +1 from me > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9193) Facility to write dynamic code to selectively trigger trace or log for queries
[ https://issues.apache.org/jira/browse/CASSANDRA-9193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520385#comment-14520385 ] Robert Stupp commented on CASSANDRA-9193: - Are we talking about traced requests only? Or something that's always executed, which would only work with Java-UDFs where Hotspot could ”remove” never executed code? Both variants have a charm. The 1st would always trigger - basically enriching trace output. The 2nd (”injected") variant would trigger on ”certain conditions” coded in the UDF. For traced requests: wouldn't it be easier to add the logged stuff directly to the trace tables so an operator (or developer) can see all output from all involved nodes? > Facility to write dynamic code to selectively trigger trace or log for queries > -- > > Key: CASSANDRA-9193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9193 > Project: Cassandra > Issue Type: New Feature >Reporter: Matt Stump > > I want the equivalent of dtrace for Cassandra. I want the ability to > intercept a query with a dynamic script (assume JS) and based on logic in > that script trigger the statement for trace or logging. > Examples > - Trace only INSERT statements to a particular CF. > - Trace statements for a particular partition or consistency level. > - Log statements that fail to reach the desired consistency for read or write. > - Log If the request size for read or write exceeds some threshold > At some point in the future it would be helpful to also do things such as log > partitions greater than X bytes or Z cells when performing compaction. > Essentially be able to inject custom code dynamically without a reboot to the > different stages of C*. > The code should be executed synchronously as part of the monitored task, but > we should provide the ability to log or execute CQL asynchronously from the > provided API. > Further down the line we could use this functionality to modify/rewrite > requests or tasks dynamically. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
