[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480823#comment-16480823 ] Aleksey Yeschenko commented on CASSANDRA-7622: -- CI passed ([here|https://circleci.com/gh/iamaleksey/cassandra/295] and [here|https://circleci.com/gh/iamaleksey/cassandra/296]). Committed as [0d464cd25ffbb5734f96c3082f9cc35011de366|https://github.com/apache/cassandra/commit/0d464cd25ffbb5734f96c3082f9cc35011de3667] with nits addressed. Thanks [~cnlwsu] and [~blerer] for you work and patience. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.0 > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16480683#comment-16480683
]
Benjamin Lerer commented on CASSANDRA-7622:
---
Sorry, for the delay of my review.
As discussed with [~iamaleksey] offline, I have 2 concerns with the final patch.
# It feels wrong to me that we have to reconstruct the {{VirtualKeyspace}},
{{VirtualTable}} and {{VirtualColumns}} dataset everytime they are queried
taking into account that for the moment they will never change after startup.
As Aleksey pointed out it is really cheap but still it is bothering me. Now, I
might be the only one with that feeling and we can still change that easily
later if needed.
# We will probably use virtual tables for different functionalities in the
future (e.g. system views, foreign data mapper ...). For the user they should
appear as different functionalities even if they are implemented on top of
virtual tables. So, I believe that we should modify the schema change
validation errors to display the name of the specific functionality instead of
{{virtual tables}}. Aleksey proposed to fix that as part of CASSANDRA-13426
which is fine with me.
Outside of that I only found 2 small nits that can be fixed on commit.
* The message change
[here|https://github.com/iamaleksey/cassandra/commit/020202e698c654123fe90bffd01f46c412311794#diff-ac79f1ada9707ae283908e7ca8844cacL157]
look like a mistake to me.
* The class javadoc in {{VirtualTableReadQuery}},
{{VirtualTablePartitionRangeReadQuery}} and
{{VirtualTableSinglePartitionReadQuery}} are referring to system views instead
of virtual tables.
+1
Thanks a lot to everybody for the hard work. I will be really happy to see that
ticket committed and I know that I am not the only one :-).
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.0
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479221#comment-16479221
]
Aleksey Yeschenko commented on CASSANDRA-7622:
--
[~blerer] and I had a chat, and he had a concern about having to build up the
whole dataset from scratch on every request, vs. maintaining one that would
persist between requests. The idea here is to use that model, and
{{SimpleDataSet}} for simpler, smaller virtual tables where it's not expensive
to do, and the complexity of adding extra code and hooks to keep the virtual
table's view up to date is unjustified.
For bigger and more complex virtual tables - think metrics - one would still
inherit {{AbstractVirtualTable}}, but wouldn't use {{SimpleDataSet}} that the
simpler virtual tables use, instead providing a custom implementation, that
would be more akin to Benjamin's original {{MapBasedSystemView}}. In fact, it's
likely that we'll take at least a portion of that code to implement {{DataSet}}
for use in the table metrics virtual table.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.0
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478605#comment-16478605 ] Chris Lohfink commented on CASSANDRA-7622: -- I tested it out and made some poc tables and it works great and easy to use. Resounding +1 from me and thanks for all the effort! Once interface set I'll start creating some tickets to expose some metrics/internals that we can discuss in them. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.0 > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16478230#comment-16478230
]
Aleksey Yeschenko commented on CASSANDRA-7622:
--
[~blerer] thanks.
I took [~cnlwsu]'s and [~jjirsa]'s original patch as the base, read path
integration from [~blerer]'s, almost as is, and rewrote the bits that I wasn't
happy with in either patch - namely schema and metadata handling and write path
integration.
The end result, ready for review, is
[here|https://github.com/iamaleksey/cassandra/commits/7622-4.0] - without any
system views in it. And
[here|https://github.com/iamaleksey/cassandra/commits/7622-4.0-more] lives
another commit, with extracted sample system views by [~cnlwsu] - compaction
stats, settings, and ring state - included as further example of the interface.
I would prefer if we reviewed and committed them as separate tickets, one JIRA
per table, and everything else we might want to provide, such as metrics,
connected clients, size estimates, etc.
Some notes:
1. I did go with the name 'virtual table' and 'virtual keyspace' for internal
classes and methods, but with 'system_views' for the name of the keyspace where
the initial system views will live. I'm using 'virtual table/virtual keyspace'
internally because the concept itself is more generic than just system views,
and we will provide other virtual keyspaces and tables that don't quite fit
'system view' definition. Class renames that went with it were the only change
to [~blerer]'s read path integration code, otherwise it was taken as is (read
command / read query changes).
2. Virtual tables can now only live in virtual keyspaces, and cannot be mixed
with regular tables. Their metadata is also kept separately from {{Schema}},
and {{Keyspace}} / {{ColumnFamilyStore}} classes have been left essentially
untouched.
3. Schema of virtual keyspaces and tables itself is now exposed by
{{system_virtual_schema}} keyspace tables, that are themselves virtual. This
simplifies the evolution of virtual keyspaces/tables implementations, as
changing schema of the tables, dropping and adding new ones, becomes
essentially hassle free - there is nothing to clean up or migrate, only what
is, is. If and when we eventually decide to expose creation of virtual
keyspaces via DDL, the way I see it, we'd create a {{CREATE VIRTUAL KEYSPACE ..
WITH }} statement, and persist keyspace name / implementation class name
in something like {{system_schema.virtual_keyspaces}} table.
4. Write path integration was rewritten to terminate at statement / mutation
collector levels, with a new specialised {{VirtualMutation}} class to round it
up. The end result is not having to special-cases any code in {{Keyspace}} or
{{StorageProxy}} or {{CassandraKeyspaceWriteHandler}} or {{Mutation}}, which in
my opinion is cleaner than what we had in either of the two combined patches.
5. {{MapBasedSystemView}} from [~blerer]'s patch was roughly translated into
{{AbstractVirtualTable}} that is a bit more generic.
[~cnlwsu] and [~blerer], can you please have a look at the proposed combined
patch, and let me know if you have any concerns or objections, or see any
problems with it? Once those are addressed, I would like to commit the
interface patch, and have [~cnlwsu] move on to implementing the actual system
views we want to expose, in separate tickets. Thanks (:
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.0
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467652#comment-16467652 ] Benjamin Lerer commented on CASSANDRA-7622: --- [here|https://github.com/apache/cassandra/compare/trunk...blerer:7622-trunk] is the patch I promised. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467507#comment-16467507 ] Aleksey Yeschenko commented on CASSANDRA-7622: -- Pushed part of virtual/read metadata separation [here|https://github.com/iamaleksey/cassandra/commits/7622-4.0]. I'm now sufficiently satisfied with it to move on to CQL integration review/edit. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16464082#comment-16464082
]
Aleksey Yeschenko commented on CASSANDRA-7622:
--
Pushed another commit that untangles things further, with eventual goal of
nothing knowing about virtual tables except {{Schema}} and some CQL statements.
Also streamlines metadata initialisation of the virtual tables - there was no
need to reinvent a {{TableMetadata.Builder}}, now the tables just use it
instead.
Part 2 will complete metadata segregation, will look into CQL statements
integration next.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462426#comment-16462426
]
Aleksey Yeschenko commented on CASSANDRA-7622:
--
Pushed the first review commit
[here|https://github.com/iamaleksey/cassandra/commits/7622-4.0].
It's the first commit in the series with the goal of separating virtual
keyspaces from real keyspaces on metadata level, with more to follow. This one
adds a new virtual keyspace - {{system_virtual_schema}} - with virtual tables
{{keyspaces}}, {{tables}}, and {{columns}} that mimic those of
{{system_schema}} to the extent that is reasonable to duplicate, and omits any
metadata that doesn't make any sense in context of virtual keyspaces
(properties like {{durable}} or {{replication}} for keyspaces, and most of the
param fields for tables). Among other things, this way evolution of arbitrary
virtual keyspaces becomes trivial, as you no longer have to worry with
incompatible previous schema stored persistently - because none of it is.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462172#comment-16462172 ] Benjamin Lerer commented on CASSANDRA-7622: --- [~cnlwsu], [~rustyrazorblade], [~bdeggleston] I agree my proposed schema has some issue but I am also not convinced by the other one :(. In my opinion, we should be able to find something better. I am fine letting that discussion for a follow up ticket. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462165#comment-16462165
]
Benjamin Lerer commented on CASSANDRA-7622:
---
{quote}Further down the road (in the 6.x/7.x timeframe), leaving these as
virtual tables instead of system views may allow future uses like
[https://wiki.postgresql.org/wiki/Foreign_data_wrappers]
{quote}
I think we should be careful to not mix up everything. Even if we reuse the
same mechanism internally, the way we expose functionalities to the users is
also extremely important. If we take the example of ProgreSQL, they have 2
different functionalities {{System Views}} and {{Foreign Data Wrappers}}. There
is a clear distinction between the 2 features from the outside. Mixing up
everything under the {{Virtual Table}} hat is not in my opinion the right
approach.
{quote}But on the other side, I can come up with the following
counter-arguments:
* if we do ever allow user to create their own, then "System View" becomes imo
somewhat misleading/wrong. As said above, I'm actually yet to be convinced that
we should do the former, but not to the point that I'm against future
proofing.{quote}
I am also not convinced that we should do it. I would really love to avoid
another {{Trigger}} like feature.
{quote} * we've used "virtual tables" for a long time now. I worry that it's
too late to change the naming now, that even if we somewhat officially decide
to rename to System Views, people will informally continue to refer to "virtual
tables" and more confusion than anything will follow.{quote}
[~slebresne] For {{Materialized Views}} we initially started with the {{Global
Index}} name. We only changed it later on and it was in my opinion a good
decision because people understood straight away what it was.
C* is now used by a huge amount of people. Only a tiny fraction of them is
following that ticket or the NGCC discussions. We should choose what is the
best for them.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456749#comment-16456749
]
Jeff Jirsa commented on CASSANDRA-7622:
---
{quote}
No disagreement here. We don't want it to be exposed in regular DDL. However,
we would like this to be at least a little bit generic, so that one could
plug-in extra virtual keyspaces on C* startup, similar to how some folks/forks
add extra system keyspaces and system tables. There are some use cases for
virtual tables that we want to experiment with (Jeff Jirsa gave a few examples
in his NGCC talk) that are compelling enough to at least allow this possibility.
{quote}
Just to expand on this: the metrics/config stuff I think is a no brainer and
not really contentious. There are a few other things I think are useful to
operators that we'll likely be experimenting with (privately) in the near
future, as a proof-of-concept before we attempt to suggest they're suitable or
relevent for the larger audience. An example in this space is something like a
table that inspects unfiltered cells of a partition (a la {{sstabledump}} , but
over CQL) for troubleshooting purposes ('which sstables have this key' and 'why
does this partition have so many tombstones?' type virtual table
implementations). Further down the road (in the 6.x/7.x timeframe), leaving
these as virtual tables instead of system views may allow future uses like
https://wiki.postgresql.org/wiki/Foreign_data_wrappers , so long term I'd like
to consider leaving the door open for that if possible, but I'm happy to set
that desire aside if necessary in order to get the first two steps
(metrics/config and an extension point for the debugging tables/views) in
properly.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456583#comment-16456583 ] Aleksey Yeschenko commented on CASSANDRA-7622: -- bq. If by "plug-in", you mean "patch-in" (but easily so and without being too limited), then sure, that's just good coding and no issue on that from me. Patch in. Think adding an extra line [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/Schema.java#L80] kind of way. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456579#comment-16456579 ] Sylvain Lebresne commented on CASSANDRA-7622: - bq. we would like this to be at least a little bit generic, so that one could plug-in extra virtual keyspaces on C* startup, similar to how some folks/forks add extra system keyspaces and system tables. If by "plug-in", you mean "patch-in" (but easily so and without being too limited), then sure, that's just good coding and no issue on that from me. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456572#comment-16456572 ] Aleksey Yeschenko commented on CASSANDRA-7622: -- bq. For instance, I do have a major high level concerns on the current patch in that I am strongly in favor of not shipping syntax for creating virtual tables, at least for the initial commit. I am completely sold on virtual tables as a mean to expose configuration infos and metrics, but I'm a lot less sold about letting user create their own. No disagreement here. We don't want it to be exposed in regular DDL. However, we would like this to be at least a little bit generic, so that one could plug-in extra virtual keyspaces on C* startup, similar to how some folks/forks add extra system keyspaces and system tables. There are some use cases for virtual tables that we want to experiment with ([~jjirsa] gave a few examples in his NGCC talk) that are compelling enough to at least allow this possibility. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456500#comment-16456500 ] Chris Lohfink commented on CASSANDRA-7622: -- Its a one line change so easy to make so if you really want it I can make the change. Personally I like not requiring it but I understand your point, it was more of an attempt to make it as accessible as possible, especially given that theres no performance or impact at all and the error message is a little off putting. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456479#comment-16456479
]
Sylvain Lebresne commented on CASSANDRA-7622:
-
bq. it makes querying interesting things a lot easier.
To be a bit facetious, 2 words easier isn't exactly "a lot". But don't get me
wrong, I understand the sentiment, I just worry that from new user points of
view, not needing {{ALLOW FILTERING}} in case where any other query would
require might confuse the hell out of them. Might be unfounded fear, but don't
want to not at least raise it.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456443#comment-16456443
]
Chris Lohfink commented on CASSANDRA-7622:
--
> I do think it would be worth separating the discussion on the system
> view/virtual table mechanism from the one on the exact schema of metrics
> tables for the sake of focus
agreed, I removed table_stats from system_view and it can be discussed in
another ticket. That leaves {{settings}}, {{compaction_stats}}, {{ring_state}}.
Thats 2 essentially copies of the nodetool output and a new table that lets you
see current state of the configuration (with a few updatable).
> I am strongly in favor of not shipping syntax for creating virtual tables
Removing CQL syntax change and {{CreateTableStatement}} changes. Will build the
tablemetadata from builder instead in the {{SystemViewKeyspace}}.
> The patch seems to disable the {{ALLOW FILTERING}} check for virtual tables
The patch allows each virtual table to enable or disable the restrictions that
normally would require {{ALLOW FILTERING}} to make
{code:java}
/**
* If the table allows unrestricted queries (ie filter on clustering key
with no partition). Since These tables are
* not backed by the C* data model, this restriction isnt always necessary.
*/
public boolean allowFiltering()
{
return true;
}
{code}
I will change default to have same behavior as normal C* tables, but I actually
really would like to remove those restrictions on queries for the
{{SystemView}} tables since they dont need them and it makes querying
interesting things a lot easier.
> naming is important
agreed but its hard to have 100% foresight, the goal of this ticket shifted
dramatically through its lifetime even. I went with {{VirtualTable}} being the
base abstract class for a custom backend, but the one implementation provided
is a {{SystemView}} to make it easier to make internal system state/metrics
tables.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456198#comment-16456198
]
Sylvain Lebresne commented on CASSANDRA-7622:
-
Fwiw, I do think it would be worth separating the discussion on the {{system
view}}/{{virtual table}} mechanism from the one on the exact schema of metrics
tables for the sake of focus. Surel
y we can find 1 or 2 simple (yet useful to have) tables/views with
uncontroversial schema to start with.
Because I do think there is enough discussion to be had on the mechanism itself
(both on how it is exposed to user in general, and on the implementation in
particular).
For instance, I do have a major high level concerns on the current patch in
that I am *strongly* in favor of not shipping syntax for creating virtual
tables, at least for the initial commit. I am completely sold on virtual tables
as a mean to expose configuration infos and metrics, but I'm a lot less sold
about letting user create their own.
_At the very least_, the latter adds a lot of risk and difficulties:
* we're more or less committing to an API which could limit us moving forward,
so we need to be a bit careful
* there is security/foot-shooting risks. How do we mitigate them?
* It increase the test and documentation surface area quite a bit (assuming we
care about testing/documenting things right).
* We have more stuff to discuss from the get go (creation syntax for instance;
If we were to do this, then I'm definitively not a fan of not providing the
schema at all in the {{CREATE}} for instance).
Adding to that the usual question we should always ask of "are we not feature
creeping?".
All this to say that we should discuss the compelling use cases that makes
adding user creation worth it. I haven't seen that yet. And more generally, I
see no reason for not moving that as a 2nd step. Surely everyone agrees doing
things incrementally is better, and there is a very clear and logical path for
being incremental here.
And so I'm strongly in favor for the first version of this to be 98%[1]
transparent for end users: it should look like the addition of a bunch of new
system tables, but how those tables are implemented underneath should be
largely an implementation detail.
I can make the additional high level comments:
* I'd really rather not have the mechanism tied to a particular keyspace. If
keyspaces are added out of this, this should be because they make sense
"organization"-wise. That is, I could see use adding {{system_configurations}}
and {{system_metrics}} keyspaces, less fan of a
{{system_view}}/{{virtual_tables}} one.
* The patch seems to disable the {{ALLOW FILTERING}} check for virtual tables.
I'm conflicted on this. On the one side, I understand it's technically safe
(because served from memory and we know it will never grow big enough to be an
issue anyway), but on the other side I wonder if creating an inconsistency API
wise won't confuse users more than anything, especially when {{ALLOW
FILTERING}} is probably not something that well understood in the first place.
Part of me would prefer that we explain {{ALLOW FILTERING}} better and keep
things consistent.
[1]: the 2% remaining is that: 1) we should probably at least add a flag in the
schema of those tables for future proofing and 2) those tables won't have
sstables on disk, so we cannot entirely hide the fact those tables have a
different implementation.
bq. I am fine with renaming virtualtable to systemview, its irrelevant at this
point and not exposed externally currently anyway. Ill do the rename in
followup if its important.
I wouldn't dismiss that discussion as unimportant (it's ok to not have much
personal opinion of course), because naming is important and even if that name
doesn't appear in any user visible string (not true in you patch btw), we'll
likely use it in doc and discussions. And we have a really bad track record of
confusing users with inconsistent or changing naming so let's learn and
improve, not repeat our mistakes.
Anyway, on the one side I think [~blerer] argument that system views is
somewhat of a standard is compelling (provided what we do here is indeed close
to what those other DB do). But on the other side, I can come up with the
following counter-arguments:
* if we do ever allow user to create their own, then "System View" becomes imo
somewhat misleading/wrong. As said above, I'm actually yet to be convinced that
we should do the former, but not to the point that I'm against future proofing.
* we've used "virtual tables" for a long time now. I worry that it's too late
to change the naming now, that even if we somewhat officially decide to rename
to System Views, people will informally continue to refer to "virtual tables"
and more confusion than anything will follow.
As said above, I'm in favor of making this as transparent f
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16455767#comment-16455767
]
Chris Lohfink commented on CASSANDRA-7622:
--
While waiting for other implementation, made some changes on some feedback out
of band:
* Use same QueryPager as other ReadCommands
* Remove virtual tables from the system_schema.tables. Moved into
\{{system_schemas.virtual_tables}}. The drivers will need to add the
TableMetadata from that in order to have it in own metadata, for cqlsh I added
a temp override until it exists in driver to make easier to play with.
* Added the missed support for IN on partitions (so can do relations or IN
clause for partition key, or both on parts of key). I think this can be done
better but seems to function on things Ive thought of.
* Some minor cleaning/renaming
||branch||tests||
|[trunk|https://github.com/clohfink/cassandra/tree/7622]|[tests|[https://circleci.com/gh/clohfink/cassandra/92|https://circleci.com/gh/clohfink/cassandra/104]]|
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454907#comment-16454907 ] Blake Eggleston commented on CASSANDRA-7622: I’m pretty heavily in favor of the schema as proposed by Chris. One of the big benefits of virtual tables is improving operator quality of life. Exposing metrics in a table with hundreds of columns would negate many of the ergonomic benefits of them being available through cql in the first place. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454730#comment-16454730 ] Jon Haddad commented on CASSANDRA-7622: --- For what it's worth, I wrote an internal TLP tool that takes every JMX metric and stores it in a sqlite DB, and I'm doing it the same with [~cnlwsu] has listed above. Yes, it requires grabbing the key you want up front, but as [~cnlwsu] notes, in every practical use of these tables so does the alternative proposed by [~blerer]. Assuming there's a trivial way to grab all the possible metrics that can be used from a different view, it makes sense to me to expose the metrics in the way [~cnlwsu] has, so I'm in favor of his approach as an operator. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454698#comment-16454698
]
Chris Lohfink commented on CASSANDRA-7622:
--
To numerate issues I see with that schema:
# I really dislike idea of expecting users to use EXPAND ON to make a query
without it becoming unreadable _screenfuls_ of
{{|--+++---...}}'s. Thats the biggest issue I have with
that schema. For most purposes this schema is not user friendly in cqlsh unless
you know exactly what you want beforehand.
# if not sure what table or metric, but want to know what metrics are "bad"
you cant do it like: \{{ SELECT * FROM system_view.table_stats WHERE p99th >
10 AND count > 100;}} like: !screenshot-1.png|width=1062,height=93!
# Schema changes from time to time. Not knowing whats there is fine and
expected, but to find out you have to dump the schema with a DESC TABLE or DESC
SYSTEM_VIEW or something, which will be hundreds of lines long, then follow up
with a 2nd query to find the values. So a user would have to be familiar with
DESC TABLE, schema formats and EXPAND commands in cqlsh to get most out of this.
# To know the type of the metric you have to refer to said schema, but its
virutally always double or bigint so meh
# Each metric in C* will actually have to broken into up to 14 different
metrics in the table (count, 99th percentile, mean, rates etc), x 2 for latency
metrics if including both raw and decaying. So you cant just query to see the
"write latency metrics" without a 2 line long column definition, cant even use
range queries to collect them since they are columns
# The order of the columns in the schema is not necessarily the order that
will be displayed on client side. Really thats Ok though, just might be
unexpected
# Theres also a case where different tables have different metrics (ie view
only metrics) so you either dont have them or have NULLs exactly like this
implementation
So while not as useful from a query perspective, an option is to actually just
have a ((keyspace, table), metric) with a single value metric that is a text
json representation or a Map. I didn't like that at first but it
certainly could be a nice view for casual browsing.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
> Attachments: screenshot-1.png
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16454691#comment-16454691 ] Dinesh Joshi commented on CASSANDRA-7622: - [~cnlwsu] nice! > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > Attachments: screenshot-1.png > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16453748#comment-16453748
]
Benjamin Lerer commented on CASSANDRA-7622:
---
{quote}I don't think I see the benefit to that schema, you have to have a
schema change or dynamic schema for whenever a new metric is added? \{quote}
At the end of the day you have 2 alternatives. Either you expect the users to
know the type of every metric and face a big bunch of {{NULL}} values for some
queries or you follow the approach that the database industry has chosen which
could result in a schema change with new versions.
There are no perfect solutions.
Now, users have lived with the second alternative for years, are used to it and
in my opinion is far more user friendly.
{quote}
Reminder there have been cases where metrics dont appear until execution (not
currently but in past), so its not unheard of for the metric set to change at
runtime.
{quote}That is a bug and should not impact the design.
{quote}
And requiring {{EXPAND ON;}} to explore what metrics exist I think would be a
high bar for a lot of people who don't even know the command exists, which is
also session wide not just for the query.\{quote}
It is a really useful feature. There are a lot of case where your results do
not fit the display. Maybe we should simply advertise it more.
We should also change CQLSH to support it a the query level like the *\G* of
the MySQL console.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452700#comment-16452700
]
Jeff Jirsa commented on CASSANDRA-7622:
---
{quote}
Around mid-February, I wanted to come back to that ticket. At that time it was
assigned to Jeff and I knew he was not actively working on it.
{quote}
I dunno why you'd think that, I gave a talk about it at NGCC in October where I
indicated it was something we were actively working on.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452667#comment-16452667
]
Chris Lohfink commented on CASSANDRA-7622:
--
I don't think I see the benefit to that schema, you have to have a schema
change or dynamic schema for whenever a new metric is added? And requiring
{{EXPAND ON;}} to explore what metrics exist I think would be a high bar for a
lot of people who don't even know the command exists, which is also session
wide not just for the query.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16452213#comment-16452213
]
Benjamin Lerer commented on CASSANDRA-7622:
---
{quote}You have 1 cell per attribute of each metric? theres currently 63
metrics with between 1 and 15 values - literally hundreds of cells per row
which isnt viewable on a sane display.
{quote}
This is a display problem not a schema problem. If you take the case of CQLSH
you can simply use {{EXPAND ON}} to print the output vertically.
{quote}I do have to say though, this ticket has been 4 years of bikeshedding,
kinda unfortunate timing to wait until theres an implementation in review to do
it from scratch instead of giving feedback.
{quote}
All my apologies.
I can understand your frustration. Now, the story is a bit different from what
you seem believe. In 2015, I decided to work on that ticket and assigned it to
me. Due to some priority changes, I did not manage to work on it and got
unassigned. Around mid-February, I wanted to come back to that ticket. At that
time it was assigned to Jeff and I knew he was not actively working on it. I
decided to create a patch on the DSE side. One month later the patch was in
review on our side and at the same time you updated this ticket.
I had some offline discussion with [~iamaleksey] this morning and he also has
some concerns about my patch. I will try to port the refactoring part of the
patch and the API used to plug {{SystemViews}} for next week. Once, it is done
it will be easier for everybody to judge what is good or bad about it.
Hopefully, we can end up with a solution that satisfy everybody.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451767#comment-16451767 ] ASF GitHub Bot commented on CASSANDRA-7622: --- Github user clohfink closed the pull request at: https://github.com/apache/cassandra/pull/205 > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450123#comment-16450123 ] Chris Lohfink commented on CASSANDRA-7622: -- How do you create new tables? have a snippet or something? ie https://github.com/clohfink/cassandra/blob/945309719a447c863bb60d3976d6a73d7b2f6863/src/java/org/apache/cassandra/db/virtual/CompactionStats.java 2 weeks isnt long to wait, and im all for dropping this patch if the other implementation is better, but I am not really convinced its any better, just different, and in a way that is in some ways worse sounding to me (I am not convinced a custom DDL for querying is a good idea). I do have to say though, this ticket has been 4 years of bikeshedding, kinda unfortunate timing to wait until theres an implementation in review to do it from scratch instead of giving feedback. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450066#comment-16450066
]
Chris Lohfink commented on CASSANDRA-7622:
--
I am fine with renaming virtualtable to systemview, its irrelevant at this
point and not exposed externally currently anyway. Ill do the rename in
followup if its important.
If you want it to display "SYSTEM VIEW" in cqlsh when do a describe, thats
purely a client side/driver change (if virtual or systemview whatever flag set,
use that instead of CREATE TABLE), you dont need to change CQL to do that, and
changing protocol wont make that difference unless your planning on a whole new
place to store the schema and share that information with driver/cqlsh. I think
that can be captured in followup jira or sub task to improve UX with
cqlsh/drivers.
Its supposed range queries and about any slice/dicing, but I did miss short
cutting the IN restriction, I'll fix that.
As for the schema... You have 1 cell per attribute of each metric? theres
currently 63 metrics with between 1 and 15 values - literally hundreds of cells
_per row_ which isnt viewable on a sane display. you can just as easily grab a
single value efficiently with impl:
{code}
SELECT one_min_rate FROM system_info.table_stats WHERE keyspace_name = 'system'
AND table_name = 'local' AND metric = 'writeLatency';
{code}
or see all parts of the metric:
{code}
SELECT * FROM system_info.table_stats WHERE keyspace_name = 'system' AND
table_name = 'local' AND metric= 'writeLatency';
{code}
If we want to fine tune the table_stats schema, we can just remove it from this
patch, and move to another patch to discuss that particular tables schema.
Thats not really what this ticket is about. There are a lot of tables to
create, lets just get the capability to create them in. If your implementation
is very similar with a ReadQuery replacement its at a high level the same
thing. In next patch I did refactor to look a bit cleaner and reuse existing
AbstractQueryPager logic more vs doing its own from scratch QueryPager.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450043#comment-16450043 ] Aleksey Yeschenko commented on CASSANDRA-7622: -- [~blerer] 1. Do these tables live in a special keyspace? 2. What's the DDL for it? Is it at all exposed to users or hard-coded internally? 3. How's it represented in schema tables? Does it highjack system_schema.tables with shim props, or is there a separate schema table for system views? > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450017#comment-16450017
]
Benjamin Lerer commented on CASSANDRA-7622:
---
{quote}you can do arithmetic operations, searches and aggregations on them but
the type was wrong{quote}
Sorry, my comment was misleading. I just wanted to mention the fact that
aggregates and arithmetic operations do not work on {{TEXT}} values.
{quote}What would you think the table schema should look like?{quote}
I asked myself that question a lot. Due to the CQL limitations, I do not think
that there is a perfect solution. In the end, my preferred schema is:
{code}
SYSTEM VIEW sv_table_metrics (keyspace TEXT,
table TEXT,
memtable_on_heap_size BIGINT,
memtable_off_heap_size BIGINT,
[...]
PRIMARY KEY (keyspace, table));
{code}
That approach in the case of the {{table}} and {{keyspace}} metrics result in
tables with a big number of columns (even if we mitigate that fact by using
user define types for histograms, meters and timers) but it allow to easily
select different subset of data. You can query based on {{keyspaces}},
{{tables}}, {{metrics}} and {{metric fields}}. At the same time you can easily
select a specific metric value for a given table in an efficient way.
{quote}I am not fussy about naming. However, using the same terminology does
confuse users as they may expect the same feature set from Cassandra as they
got in their relational database. I would personally avoid it.{quote}
Based on my experience working on CQL tickets and my interaction with users or
discussions with evangelists I came up with 2 conclusions.
# If the feature is the similar to one that they know from the relational
world, people prefer when you use the same name. It is easier for them to
recognize it and to understand how it should be used.
# If the feature has a different behavior that what is used in the relational
world you should be careful and use a different naming or it will backfire.
In this case, there is no real difference between us an the relational world.
Due to that, I think it would be a mistake to not reuse the name.
The {{Virtual Table}} name is in my opinion the really confusing one. It just
make me think to some form of pluggable storage. Coming from the SQL world, it
is not the name I would use in google to figure out how to access system
information in Cassandra.
{quote}do you have a design or code that you can share? It would be great if
you can post it. Is there a timeline around when you'll post it?{quote}
At the high level there are some similarities between [~cnlwsu] patch and ours.
We have introduced some {{ReadQuery}} subclasses that delegate calls to
{{SystemViews}} and slightly refactored the CQL layer to allow it to work on
top of all {{ReadQuery}} implementations. The advantage of that approach is
that the existing CQL functionalities are automatically supported on top the
{{SystemViews}} and the conditional logic require for adding support for
{{SystemViews}} is much lower.
[~cnlwsu] current patch does not support some range queries or multi-partition
queries for example. It will fire an {{[Invalid query] message="IN restrictions
are not supported on indexed columns"}} for multi-partition queries. We avoided
that kind of risks/problems with our approach.
That reduced the logic of our {{SystemView}} implementation to just fetching
the requesting data or updating them.
Our current code has been designed for DSE. So I need to modify it to make it
work on top of C*. As I am also quite busy with some other tasks, it would
probably take 2 weeks before I finish the port.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail:
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448699#comment-16448699 ] Dinesh Joshi commented on CASSANDRA-7622: - [~blerer] do you have a design or code that you can share? It would be great if you can post it. Is there a timeline around when you'll post it? > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448475#comment-16448475 ] Chris Lohfink commented on CASSANDRA-7622: -- I think theres 2 things here # blanket table exposing ALL metrics (so that jmx isnt required) # provide a good ux for exploring them I think for #1 we still need a table like the one in patch, otherwise every time a metric is added a curated table would also need to be updated which might be prohibitive for 1 off metrics that while critical in some scenarios are so off the normal path that it would be wasted effort and overwhelming. For #2 I was thinking of basically having an equivalent of the nodetool views++ (tablestats, tablehistograms, info, netstats, clientlist etc but also some new things), but that needs to be broken into sub tasks since working on it until the mechanism for making them is nailed down is kinda waste of time. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448391#comment-16448391 ] Chris Lohfink commented on CASSANDRA-7622: -- > gauge value are simply returned as TEXT which prevent you to do arithmetic > operations or aggregation will fix that. you can do arithmetic operations, searches and aggregations on them but the type was wrong. > all the C* metrics. Streaming metrics, Cache metrics ... all of them should > be exposed in the best possible way for an admin. Planned on it, but that makes patch too big. once the implementation is done can make sub tasks for others. What would you think the table schema should look like? > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448357#comment-16448357
]
Benjamin Lerer commented on CASSANDRA-7622:
---
{quote}table metrics (all of them)
{quote}
The schema of this table could be much better in my opinion. The user needs to
know what metric type is corresponding to the metric he is looking at to be
able to fetch only the columns that are meaningful for that metric. Otherwise
he will just get a bunch of {{NULL}} values. {{Gauge}} value are simply
returned as {{TEXT}} which prevent you to do arithmetic operations or
aggregation if you want to. As an admin I would expect a more powerful schema.
When I am mentioning all metrics I am refering to all the C* metrics. Streaming
metrics, Cache metrics ... all of them should be exposed in the best possible
way for an admin.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448330#comment-16448330
]
Benjamin Lerer commented on CASSANDRA-7622:
---
{quote}You do not get {{CREATE TABLE USING CompactionStats compaction_stats}},
it still displays the schema how you would expect with a {{create table}}
output because thats the way cqlsh is created to take a schema and display
it.\{quote}
Do not get me wrong. Both approach have some issues in my opinion even if
CREATE TABLE USING CompactionStats compaction_stats is less misleading.
The problem of displaying the table as a normal one is that it let the user
believe that it is a normal table which is not the case in many ways.
{quote} While I think its possible to use that shim to do other things I very
much doubt it would ever be used as such.\{quote}
Then we do not need that logic. It is wrong in my opinion to add some logic
just in case we might need it one day.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448295#comment-16448295
]
Chris Lohfink commented on CASSANDRA-7622:
--
You do not get {{CREATE TABLE USING CompactionStats compaction_stats}}, it
still displays the schema how you would expect with a {{create table}}
output because thats the way cqlsh is created to take a schema and display it.
This ticket does not add pluggable storage, it creates a system_info keyspace
that has some pre-created (with more to come if it ever gets through) tables
displaying internal information, like compaction stats, table metrics (all of
them), all configuration settings (with some settable), and ring state. It
makes it (hopefully) easy to make more of these as well with any kind of
querying you want (normal cql query limitations do not apply, with exception of
some of the order by ones).
This patch creates a shim that sits between the parsing of the query and
actually executing it. The only implementation is that shim is one for this
system_info. While I think its possible to use that shim to do other things I
very much doubt it would ever be used as such.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448021#comment-16448021
]
Benjamin Lerer commented on CASSANDRA-7622:
---
I have always been frustrated by that JIRA ticket and that frustration has only
grown with time. The root of my frustration is in fact the {{Virtual Table}}
name.
What every administrator would like to have when it start using C* is a way to
access the system information through CQL (ask [~pmcfadin] about it ;)). Every
major relational database provide that functionality. Oracle has [Dynamic
Performance
Views|https://docs.oracle.com/cd/B19306_01/server.102/b14237/dynviews_1.htm],
MSSQL has [System Views/Dynamic Management
Views|https://technet.microsoft.com/en-us/library/ms177862(v=sql.110).aspx] and
PostgreSQL has [System
Views|https://www.postgresql.org/docs/9.5/static/views-overview.html].
{{Virtual Table}} is a form of pluggable storage as used by
[SQLite|https://www.sqlite.org/vtab.html].
The idea of this ticket was lets add some form of pluggable storage and use it
to expose system information. At first glance, it looks like killing two bird
with one stone but after some time thinking about it, I tend to believe that we
could simply end up with 2 broken features.
Adding support for pluggable storages is a complex challenge that requires a
large number of changes all around the code base as outlined in the
{{Rocksandra}} discussion. On the other hand, as system information
(configuration, metrics, ring state ...) are locals, exposing them to the user
require much less changes to the code base but more reflection towards user
usability.
It should be easy for administrator to navigate through system information
tables/views but the {{Virtual Table}} syntax does not play well with it. If an
administrator wants an idea of what columns are returned by the
{{compaction_stats}} table and use {{DESCRIBE}} for it, what it will get will
be {{CREATE TABLE USING CompactionStats compaction_stats}}.
The tables should also be meaningful and easy to use. As we work a lot with the
different metrics (counters, gauges, timers, histograms...) we tend to assume
that it is also the case of the users but it is not so we should not rely on
the user knowing it.
As there is already some effort going on for a proper pluggable storage
solution, I came to the conclusion that we should drop that idea of {{Virtual
Table}} and simply expose the system information through what we could call
{{System Views}}. It will make the transition easier for people coming from the
relational word and will help us to focus on what is really important for users
which is the usability of the all thing.
[~adelapena] and I have been working on a patch on our side to expose most of
the system information through system views. We have done our best to make the
exposed views as useful as possible. Taking into account the fact that metrics
have been added without any constraint, making them fit to table was not always
easy :(.
I do not want to hijacke that ticket but if our approach make sense and people
are interested. I am willing to port our patch to C*.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434732#comment-16434732
]
Dinesh Joshi commented on CASSANDRA-7622:
-
Hey [~cnlwsu],
Here's my feedback on the code so far -
* {{Schema#getVirtualTable}} - {{containsKey()}} followed by a {{get()}} is
unnecessary. Using {{get}} will have the same effect.
* {{VirtualSchema}} - the initializers for {{key}} and {{clustering}} fields
are unnecessary as you're overwriting them in the constructor. It would be a
good idea to make the fields final.
* {{VirtualTable#classFromName}}, {{TableMetaData#virtualClass}} - shouldn't
the error read differently? VirtualTable strategy class instead of Compaction
strategy class? Also there is no {{AbstractVirtualColumnFamilyStore}}. Am I
missing something here?
* {{InMemoryVirtualTable$SimpleVirtualCommand}} - Use finals for fields
* {{InMemoryVirtualTable$ResultReadState}} - Use finals for fields
* {{InMemoryVirtualTable$ResultReadState}} - line 258 - isn't the if check
redundant?
Nits -
* {{InMemoryVirtualTable}} - get rid of unused import
{{org.apache.cassandra.db.marshal.AbstractType;}}
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425952#comment-16425952 ] Chris Lohfink commented on CASSANDRA-7622: -- Well with my proposed implementation there is only a single system non-replicated keyspace containing virtual tables and you cant create them so thats not really an issue. When we add support to allow people create virtual tables that could easily be limited to local replicated or a keyspace flagged with virtual. With this implementation worth noting that theres nothing preventing an implementor from using the keyspace's replication factor and distributing accordingly, but the in memory virtual table used in the system keyspace are all setup for local replication, so the limitation (on a local RF) on create could be placed on that implementation of VirtualTable. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425902#comment-16425902 ] Jeremiah Jordan commented on CASSANDRA-7622: Since we specify replication at the keyspace level, I also think it would be very confusing to have non-replicated virtual tables mixed in to some other keyspace. Those replication settings are used by drivers and other tools to figure out how to handle the tables that use them, it makes the most sense to me for virtual tables to follow the same logic, rather than becoming even more special. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425769#comment-16425769 ] Chris Lohfink commented on CASSANDRA-7622: -- With the implementation provided there are no if-else conditions in the Keyspace.apply because the handoff to the virtual logic is handled essentially as soon as the query has been parsed and identified as going to that virtual table. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425756#comment-16425756
]
Aleksey Yeschenko commented on CASSANDRA-7622:
--
The right level of abstraction, for both this and pluggable storage, is on
keyspace level. The basic write unit we have is a {{Mutation}}, and you have to
accept this. What would not be acceptable is if-else conditions in
{{Keyspace.apply()}} depending on what the kind of table this is.
Additionally, all tables in a keyspace by definition share the same replication
strategy and params, and I just don't see how you can mix and match regular and
virtual ones in the same keyspace.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425639#comment-16425639 ] Jon Haddad commented on CASSANDRA-7622: --- I agree with [~burmanm]. Requiring the keyspace to be a virtual one seems unnecessary and even weird, from a user perspective. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425478#comment-16425478 ] Michael Burman commented on CASSANDRA-7622: --- My personal view would be to avoid keyspace per type of origin and instead concentrate on the data itself. Monitoring "virtual tables" for example should probably live in the SystemKeyspace (or monitoring, but I'm not sure if we need so many internal keyspaces). Although this opinion is affected by the fact that I got so used to using SQL Server's DMVs, but they do their job quite well (they use the table naming to indicate information type also - but that's outside this ticket). > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16425076#comment-16425076
]
Dinesh Joshi commented on CASSANDRA-7622:
-
[~cnlwsu], I spent some time looking at the patch. My initial thoughts on the
implementation are that we should probably have a separate {{VirtualKeyspace}}
like we have a {{SystemKeyspace}}. This will allow us to isolate / namespace
{{VirtualTables}}. We can always change this if needed. One of the advantage, I
think, is that this keyspace can become available for serving sooner (we would
ofcourse need to change the bootstrap for this to work). If we're exposing
metrics or other internal state then this would be very helpful. What do you
think?
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411783#comment-16411783 ] Jason Brown commented on CASSANDRA-7622: ftr, [~bdeggleston], [~dikanggu], and I are working on the pluggable storage engine right now. It's not our top priority, but we should get the vast majority of it completed for 4.0 (probably within a month or two .. no guarantees on that, though). > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411645#comment-16411645 ] Aleksey Yeschenko commented on CASSANDRA-7622: -- [~cnlwsu] that's a fair point. Still, I want to outline the purist vision anyway - if you don't mind. Maybe then we can meet somewhere in the middle, or at least make sure that we don't commit to anything sticky that would be hard to undo/redo. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411617#comment-16411617 ] Chris Lohfink commented on CASSANDRA-7622: -- I was actually thinking of after pluggable storage jira, giving this another shot as it will definitely provide a cleaner interface to attach this too. Waiting possibly year(s) for that to get finished, merged and then rebuilding this however would be unfortunate. Maybe just for the operational purpose virtual tables being able to expose the internals via cql would be great. Even if it means a bit of work to rework them later with a better interface. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16411590#comment-16411590 ] Aleksey Yeschenko commented on CASSANDRA-7622: -- I like what this exposes, but I'm not sure I'm a fan of the abstraction level it operates on. Doing this in a way that I would consider proper would involve quite a bit more change - starting with exposing schema itself as a virtual keyspace. This is probably the first genuine use case for hypothetical pluggable storage engine implementation, now that I think of it. I'll make a more detail comment some time next week. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement > Components: CQL >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410535#comment-16410535 ] sankalp kohli commented on CASSANDRA-7622: -- [~djoshi3] Can do it [~jjirsa] > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410528#comment-16410528 ] Jeff Jirsa commented on CASSANDRA-7622: --- [~spo...@gmail.com] any chance you're interested in reviewing? (Failing that, [~jasobrown] , I know your queue is huge, but interested in throwing this onto the tail of it ?) > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399353#comment-16399353 ] Chris Lohfink commented on CASSANDRA-7622: -- Oh I see you mean you want to be able to grant custom privileges within a table. Currently we would need a custom IAuthorizer, to check if its a virtual table and hand some custom options to the vtable is a nice idea but yeah - better as a follow on ticket. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399241#comment-16399241 ] Chris Lohfink commented on CASSANDRA-7622: -- Since the RBAC is applied on statement before execute, and execute is where the virtual table shim sits it works as it does with other tables. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398915#comment-16398915 ] Stefan Podkowinski commented on CASSANDRA-7622: --- I was rather skeptical at the beginning of the discussion on this issue. But I like the way we moved forward here and the proposed implementation looks promising. Definitely worth the effort. Please keep in mind that it should also be possible to grant permissions based on our RBAC model to virtual tables, too (could be done in another ticket though). I've not looked into it in detail and it might not be a big deal to implement, but it should be possible by working with GRANTs on table level. This requires that any vtables will be known from the start and not will not be registered dynamically. E.g. I'd like to be able to grant select permissions for a certain role on the table 'tablemetrics', instead of having dynamic table names, if that's planed at all. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398815#comment-16398815
]
Chris Lohfink commented on CASSANDRA-7622:
--
1)
If they extend the inmemoryvirtualtable it handles {{LIMIT}}, {{PER PARTITION
LIMIT}} , {{GROUP BY}}, {{DISTINCT}} etc (ie all the system_info tables). With
it the methods an implementor handles is:
{code}
public void mutate(DecoratedKey partitionKey, Row row)
public void read(StatementRestrictions restrictions, QueryOptions options,
ResultBuilder result)
{code}
ie the compactionstats
{code}
public void read(StatementRestrictions restrictions, QueryOptions options,
ResultBuilder result)
{
UUID hostId = StorageService.instance.getLocalHostUUID();
for (Map c :
CompactionManager.instance.getCompactions())
{
result.row(hostId, UUID.fromString(c.get("compactionId")))
.column(TASK_TYPE, c.get("taskType"))
.column(KEYSPACE_NAME, c.get("keyspace"))
.column(TABLE_NAME, c.get("columnFamily"))
.column(BYTES_COMPACTED, c.get("completed"))
.column(BYTES_TOTAL, c.get("total"))
.endRow();
}
}
{code}
Which provides a builder interface to generate the resultset.
An implementation of VirtualTable and writing own ReadQuery will only be needed
or do something more efficient over large datasets or in distributed fashion. I
dont expect anyone to do that, the interface just works at that level of the
query best otherwise the patch would have too be a rewrite the entire
columfamilystore with all its implications.
2,3) Ill start working on more tests
4) fixed
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398785#comment-16398785
]
Benjamin Lerer commented on CASSANDRA-7622:
---
I unfortunately does not have the time to review that patch but I quickly went
through it and here are my initial concerns
* Paging is a logic that always been difficult to get right as it is query
dependent and require different handling based on different options: {{LIMIT}},
{{PER PARTITION LIMIT}} , {{GROUP BY}} , ... Moreover, it seems to me that
forcing the person implementing the Virtual table to have to take care of it is
a bit risky.
* The tests covers only a minimal amount of the queries supported by normal
tables. It would be nice to have a more complete set of tests for each of the 3
types of queries: {{single partition queries}}, {{multi-partition queries}} and
{{range queries}}. With a combination of {{LIMIT}}, {{ORDER BY}}, {{PER
PARTITION LIMIT}} with and without paging.
* For paging tests I would prefer the use of {{CQLTester.executeNetWithPaging}}
with a paging size that vary to cover all the possible cases.
* It seems to me that the patch does not prevent to create MV, index or trigger
on top of virtual table
I
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397707#comment-16397707 ] Patrick McFadin commented on CASSANDRA-7622: +1 to static tables for now. Getting just CQL support for a virtual table is good progress. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397683#comment-16397683 ] Chris Lohfink commented on CASSANDRA-7622: -- [branch|https://github.com/apache/cassandra/pull/205] [tests|https://circleci.com/gh/clohfink/cassandra/68] > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397665#comment-16397665
]
Chris Lohfink commented on CASSANDRA-7622:
--
Only internally can create now:
{code}
cassandra_maint@cqlsh> CREATE TABLE USING
'org.apache.cassandra.db.virtual.RingState' basic.ring_state;
InvalidRequest: Error from server: code=2200 [Invalid query] message="AdHoc
create statements for virtual tabels are disabled."
{code}
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397045#comment-16397045 ] Jeff Jirsa commented on CASSANDRA-7622: --- At NGCC I tried to make an argument for their general availabiluty, guarded behind some experimental flag that gave us the opportunity to change the API or remove them later I believe there are some general purpose use cases (for example, analogous to foreign data wrappers) that I expect real users to appreciate, but I also realize that it’s a young feature that is likely going to change/evolve I agree we should guard it. I’d like to see it in arbitrary keyspaces, but I don’t necessarily feel it needs to happen immediately , maybe in v5 > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397044#comment-16397044
]
Benjamin Lerer commented on CASSANDRA-7622:
---
I would simply not support any {{CREATE/ALTER/DROP}} statements for Virtual
table and just allow them to be created internally.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397003#comment-16397003 ] Chris Lohfink commented on CASSANDRA-7622: -- Something like disable create statements outside system_info? Maybe a flag (property, yaml something) to enable creating if acknowledge that api will likely change? > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396815#comment-16396815 ] Benjamin Lerer commented on CASSANDRA-7622: --- One thing, I have some trouble with is to allow people to create their own Virtual Table. Once those things are given you cannot remove them easily and we will have to live with them for a really long time. The API being quite flexible I can imagine a lot of way of abusing of it. All the use cases I have ever seen mentioned are about providing access to some local server information. I would really be in favor of limiting the patch to that. It is easier to open that functionality later on that having to try to block it or remove it. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396619#comment-16396619
]
Chris Lohfink commented on CASSANDRA-7622:
--
Made some updates for NPs
> _can_ i drop a VT?
yes, can drop and create with own classes on classpath. Cannot drop
system_info though (same protections as other system created local keyspaces
exist)
> null class a legal state?
yes - if not a virtual table
> builder for statics
That was kinda awkward - trying to make more builder based in static block to
define the virtual tables schema like:
{code:java}
static
{
Map definitions = new HashMap<>();
definitions.put(KEYSPACE_NAME, CQL3Type.Native.TEXT);
definitions.put(TABLE_NAME, CQL3Type.Native.TEXT);
definitions.put(METRIC, CQL3Type.Native.TEXT);
definitions.put(VALUE, CQL3Type.Native.TEXT);
...
schemaBuilder(definitions)
.addKey(KEYSPACE_NAME)
.addKey(TABLE_NAME)
.addClustering(METRIC)
.register();
}
{code}
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396386#comment-16396386 ] Nate McCall commented on CASSANDRA-7622: [~cnlwsu] (inadvertently) commented on the PR directly: [https://github.com/apache/cassandra/pull/205] > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396177#comment-16396177 ] ASF GitHub Bot commented on CASSANDRA-7622: --- Github user zznate commented on a diff in the pull request: https://github.com/apache/cassandra/pull/205#discussion_r173958314 --- Diff: src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java --- @@ -267,12 +267,18 @@ public boolean updatesRegularRows() // columns is if we set some static columns, and in that case no clustering // columns should be given. So in practice, it's enough to check if we have // either the table has no clustering or if it has at least one of them set. -return metadata().clusteringColumns().isEmpty() || restrictions.hasClusteringColumnsRestrictions(); +return !metadata().isVirtual() && --- End diff -- nit: why use the method if we have it locally (and it's public anyhoo). The object reference is being used elsewhere herein already. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396176#comment-16396176
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173963715
--- Diff: src/java/org/apache/cassandra/db/marshal/AbstractType.java ---
@@ -529,4 +529,9 @@ public void checkComparable()
{
return testAssignment(receiver.type);
}
+
+public ByteBuffer unsafeDecompose(Object object)
--- End diff --
I dunno... that a pretty shoot-self-in-foot thing to add the the
marshalling API. What's the reasoning?
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396181#comment-16396181
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173964951
--- Diff: src/java/org/apache/cassandra/db/virtual/SystemInfoKeyspace.java
---
@@ -0,0 +1,63 @@
+package org.apache.cassandra.db.virtual;
+
+import static java.lang.String.format;
+
+import java.util.List;
+
+import org.apache.cassandra.cql3.statements.CreateTableStatement;
+import org.apache.cassandra.schema.KeyspaceMetadata;
+import org.apache.cassandra.schema.KeyspaceParams;
+import org.apache.cassandra.schema.SchemaConstants;
+import org.apache.cassandra.schema.TableId;
+import org.apache.cassandra.schema.TableMetadata;
+import org.apache.cassandra.schema.Tables;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.collect.ImmutableList;
+
+public class SystemInfoKeyspace
+{
+private static final Logger logger =
LoggerFactory.getLogger(SystemInfoKeyspace.class);
+
+private SystemInfoKeyspace() {}
+
+public static final String SETTINGS = "settings";
--- End diff --
yay! statics!
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396174#comment-16396174
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173962979
--- Diff: src/java/org/apache/cassandra/db/VirtualTable.java ---
@@ -0,0 +1,148 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.db;
+
+import static java.lang.String.format;
+
+import java.lang.reflect.InvocationTargetException;
+import java.util.List;
+
+import org.apache.cassandra.cql3.Operation;
+import org.apache.cassandra.cql3.QueryOptions;
+import org.apache.cassandra.cql3.UpdateParameters;
+import org.apache.cassandra.cql3.statements.SelectStatement;
+import org.apache.cassandra.db.filter.DataLimits;
+import org.apache.cassandra.db.partitions.PartitionUpdate;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.exceptions.CassandraException;
+import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.schema.TableMetadata;
+import org.apache.cassandra.transport.messages.ResultMessage;
+import org.apache.cassandra.utils.FBUtilities;
+
+import com.google.common.collect.Lists;
+
+/**
+ * Base requirements for a VirtualTable. This is required to provide
metadata about the virtual table, such as the
+ * partition and clustering keys, and provide a ReadQuery for a
SelectStatement.
+ */
+public abstract class VirtualTable
+{
+protected final TableMetadata metadata;
+protected String keyspace;
+protected String name;
+
+public VirtualTable(TableMetadata metadata)
+{
+this.metadata = metadata;
+}
+
+public String getTableName()
+{
+return this.metadata.name;
+}
+
+/**
+ * Is this table writable?
+ *
+ * @return True if UPDATE is supported
+ */
+public boolean writable()
+{
+return false;
+}
+
+/**
+ * If the table allows unrestricted queries (ie filter on clustering
key with no partition). Since These tables are
+ * not backed by the C* data model, this restriction isnt always
necessary.
+ */
+public boolean allowFiltering()
+{
+return true;
+}
+
+/**
+ * Return some implementation of a ReadQuery for a given select
statement and query options.
+ *
+ * @param selectStatement
+ * @param options
+ * @param limits
+ * @param nowInSec
+ * @return ReadQuery
+ */
+public abstract ReadQuery getQuery(SelectStatement selectStatement,
QueryOptions options, DataLimits limits,
+int nowInSec);
+
+/**
+ * Execute an update operation.
+ *
+ * @param partitionKey
+ *partition key for the update.
+ * @param params
+ *parameters of the update.
+ */
+public void mutate(DecoratedKey partitionKey, Row row) throws
CassandraException
+{
+// this should not be called unless writable is overridden
+throw new InvalidRequestException("Not Implemented");
+}
+
+public static Class classFromName(String name)
+{
+String className = name.contains(".")
+? name
+: "org.apache.cassandra.db.virtual." + name;
+Class strategyClass =
FBUtilities.classForName(className, "virtual table");
+
+if (!VirtualTable.class.isAssignableFrom(strategyClass))
+{
+throw new ConfigurationException(format("Compaction strategy
class %s is not derived from
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396184#comment-16396184
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173964748
--- Diff: src/java/org/apache/cassandra/db/virtual/Settings.java ---
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.db.virtual;
+
+import java.lang.reflect.Array;
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Consumer;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.cql3.CQL3Type;
+import org.apache.cassandra.cql3.ColumnIdentifier;
+import org.apache.cassandra.cql3.QueryOptions;
+import org.apache.cassandra.cql3.restrictions.StatementRestrictions;
+import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.InMemoryVirtualTable;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.exceptions.CassandraException;
+import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.schema.ColumnMetadata;
+import org.apache.cassandra.schema.TableMetadata;
+import org.apache.cassandra.service.StorageProxy;
+import org.apache.cassandra.service.StorageService;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.collect.ImmutableMap;
+
+public class Settings extends InMemoryVirtualTable
+{
+private static final Logger logger =
LoggerFactory.getLogger(Settings.class);
+private static Map> WRITABLES =
ImmutableMap.>builder()
+ .put("batch_size_warn_threshold_in_kb", v ->
DatabaseDescriptor.setBatchSizeWarnThresholdInKB(Integer.parseInt(v)))
+ .put("batch_size_fail_threshold_in_kb", v ->
DatabaseDescriptor.setBatchSizeFailThresholdInKB(Integer.parseInt(v)))
+
+ .put("compaction_throughput_mb_per_sec", v ->
StorageService.instance.setCompactionThroughputMbPerSec(Integer.parseInt(v)))
+ .put("concurrent_compactors", v ->
StorageService.instance.setConcurrentCompactors(Integer.parseInt(v)))
+ .put("concurrent_validations", v ->
StorageService.instance.setConcurrentValidators(Integer.parseInt(v)))
+
+ .put("tombstone_warn_threshold", v ->
DatabaseDescriptor.setTombstoneWarnThreshold(Integer.parseInt(v)))
+ .put("tombstone_failure_threshold", v ->
DatabaseDescriptor.setTombstoneFailureThreshold(Integer.parseInt(v)))
+
+ .put("hinted_handoff_enabled", v ->
StorageProxy.instance.setHintedHandoffEnabled(Boolean.parseBoolean(v)))
+ .put("hinted_handoff_throttle_in_kb", v ->
StorageService.instance.setHintedHandoffThrottleInKB(Integer.parseInt(v)))
+
+ .put("incremental_backups", v ->
DatabaseDescriptor.setIncrementalBackupsEnabled(Boolean.parseBoolean(v)))
+
+ .put("inter_dc_stream_throughput_outbound_megabits_per_sec", v ->
StorageService.instance.setInterDCStreamThroughputMbPerSec(Integer.parseInt(v)))
+ .put("stream_throughput_outbound_megabits_per_sec", v ->
StorageService.instance.setStreamThroughputMbPerSec(Integer.parseInt(v)))
+
+ .put("truncate_request_timeout_in_ms", v ->
StorageService.instance.setTruncateRpcTimeout(Long.parseLong(v)))
+ .put("cas_contention_timeout_in_ms", v ->
StorageService.instance.setCasContentionTimeout(Long.parseLong(v)))
+ .put("counter_write_request_timeout_in_ms", v ->
StorageService.instance.setCounterWriteRpcTimeout(Long.parseLong(v)))
+ .put("write_request_timeout_in_ms", v ->
StorageService.instance.setWriteRpcTimeout(Long.parseLong(v)))
+ .put("range_request_timeout_in_ms", v ->
StorageService.inst
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396185#comment-16396185 ] ASF GitHub Bot commented on CASSANDRA-7622: --- Github user zznate commented on a diff in the pull request: https://github.com/apache/cassandra/pull/205#discussion_r173966104 --- Diff: src/java/org/apache/cassandra/schema/TableMetadata.java --- @@ -141,6 +147,15 @@ private TableMetadata(Builder builder) params = builder.params.build(); isView = builder.isView; +isVirtual = builder.isVirtual; +if (isVirtual) +virtualKlass = builder.virtualKlass; +else +virtualKlass = null; + +assert isVirtual == flags.contains(Flag.VIRTUAL) || !Strings.isNullOrEmpty(virtualKlass) && !flags.contains(Flag.VIRTUAL) --- End diff -- Same as above w. the asserts. Let's just draw a line in the sand and stop doing this. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396182#comment-16396182
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173965027
--- Diff: src/java/org/apache/cassandra/db/virtual/TableStats.java ---
@@ -0,0 +1,166 @@
+package org.apache.cassandra.db.virtual;
+
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.nio.ByteBuffer;
+import java.util.*;
+
+import org.apache.cassandra.cql3.CQL3Type;
+import org.apache.cassandra.cql3.QueryOptions;
+import org.apache.cassandra.cql3.restrictions.StatementRestrictions;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.InMemoryVirtualTable;
+import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.metrics.LatencyMetrics;
+import org.apache.cassandra.schema.TableMetadata;
+import org.apache.cassandra.utils.EstimatedHistogram;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.codahale.metrics.Counting;
+import com.codahale.metrics.Gauge;
+import com.codahale.metrics.Metric;
+import com.codahale.metrics.Sampling;
+import com.codahale.metrics.Snapshot;
+import com.codahale.metrics.Timer;
+import com.google.common.collect.ImmutableSet;
+
+public class TableStats extends InMemoryVirtualTable
+{
+private static final Logger logger =
LoggerFactory.getLogger(TableStats.class);
+private static final Field[] FIELDS =
org.apache.cassandra.metrics.TableMetrics.class.getFields();
+static
+{
+Arrays.sort(FIELDS, Comparator.comparing(Field::getName));
+}
+private static final Collection EH_GAUGES = ImmutableSet.of(
+"estimatedPartitionSizeHistogram",
+"estimatedColumnCountHistogram");
+private CompositeType keyType;
+
+public static Map columns()
--- End diff --
same as above. Initialize statically.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396179#comment-16396179
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173964217
--- Diff: src/java/org/apache/cassandra/db/virtual/CompactionStats.java ---
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.db.virtual;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.UUID;
+
+import org.apache.cassandra.cql3.CQL3Type;
+import org.apache.cassandra.cql3.QueryOptions;
+import org.apache.cassandra.cql3.restrictions.StatementRestrictions;
+import org.apache.cassandra.db.InMemoryVirtualTable;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.schema.TableMetadata;
+import org.apache.cassandra.service.StorageService;
+
+public class CompactionStats extends InMemoryVirtualTable
+{
+
+public static Map columns()
+{
+Map definitions = new HashMap<>();
+definitions.put("host_id", CQL3Type.Native.UUID);
+definitions.put("compaction_id", CQL3Type.Native.UUID);
+definitions.put("task_type", CQL3Type.Native.TEXT);
+definitions.put("keyspace_name", CQL3Type.Native.TEXT);
+definitions.put("table_name", CQL3Type.Native.TEXT);
+definitions.put("bytes_compacted", CQL3Type.Native.TEXT);
+definitions.put("bytes_total", CQL3Type.Native.TEXT);
+return definitions;
+}
+
+public static Key primaryKey()
+{
+return createKey()
+.addKey("host_id")
+.addClustering("compaction_id");
+}
+
+public CompactionStats(TableMetadata metadata)
+{
+super(metadata);
+}
+
+public void read(StatementRestrictions restrictions, QueryOptions
options, ResultBuilder result)
+{
+UUID hostId = StorageService.instance.getLocalHostUUID();
+for (Map c :
CompactionManager.instance.getCompactions())
+{
+result.row(hostId, UUID.fromString(c.get("compactionId")))
--- End diff --
My $0.02 is that if we are duping the same strings everywhere, we should do
some statics we don't have a whole nudist colony of naked literals.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396175#comment-16396175
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173965848
--- Diff: src/java/org/apache/cassandra/schema/Schema.java ---
@@ -713,20 +727,24 @@ private void dropView(ViewMetadata metadata)
private void dropTable(TableMetadata metadata)
{
-ColumnFamilyStore cfs =
Keyspace.open(metadata.keyspace).getColumnFamilyStore(metadata.name);
-assert cfs != null;
-// make sure all the indexes are dropped, or else.
-cfs.indexManager.markAllIndexesRemoved();
-
CompactionManager.instance.interruptCompactionFor(Collections.singleton(metadata),
true);
-if (DatabaseDescriptor.isAutoSnapshot())
-
cfs.snapshot(Keyspace.getTimestampedSnapshotNameWithPrefix(cfs.name,
ColumnFamilyStore.SNAPSHOT_DROP_PREFIX));
-
CommitLog.instance.forceRecycleAllSegments(Collections.singleton(metadata.id));
--- End diff --
I'll go back and look through comments, but _can_ i drop a VT?
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396180#comment-16396180
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173964414
--- Diff: src/java/org/apache/cassandra/db/virtual/Settings.java ---
@@ -0,0 +1,161 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.db.virtual;
+
+import java.lang.reflect.Array;
+import java.lang.reflect.Field;
+import java.lang.reflect.Modifier;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.function.Consumer;
+
+import org.apache.cassandra.config.Config;
+import org.apache.cassandra.config.DatabaseDescriptor;
+import org.apache.cassandra.cql3.CQL3Type;
+import org.apache.cassandra.cql3.ColumnIdentifier;
+import org.apache.cassandra.cql3.QueryOptions;
+import org.apache.cassandra.cql3.restrictions.StatementRestrictions;
+import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.InMemoryVirtualTable;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.exceptions.CassandraException;
+import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.schema.ColumnMetadata;
+import org.apache.cassandra.schema.TableMetadata;
+import org.apache.cassandra.service.StorageProxy;
+import org.apache.cassandra.service.StorageService;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.collect.ImmutableMap;
+
+public class Settings extends InMemoryVirtualTable
+{
+private static final Logger logger =
LoggerFactory.getLogger(Settings.class);
+private static Map> WRITABLES =
ImmutableMap.>builder()
+ .put("batch_size_warn_threshold_in_kb", v ->
DatabaseDescriptor.setBatchSizeWarnThresholdInKB(Integer.parseInt(v)))
+ .put("batch_size_fail_threshold_in_kb", v ->
DatabaseDescriptor.setBatchSizeFailThresholdInKB(Integer.parseInt(v)))
+
+ .put("compaction_throughput_mb_per_sec", v ->
StorageService.instance.setCompactionThroughputMbPerSec(Integer.parseInt(v)))
+ .put("concurrent_compactors", v ->
StorageService.instance.setConcurrentCompactors(Integer.parseInt(v)))
+ .put("concurrent_validations", v ->
StorageService.instance.setConcurrentValidators(Integer.parseInt(v)))
+
+ .put("tombstone_warn_threshold", v ->
DatabaseDescriptor.setTombstoneWarnThreshold(Integer.parseInt(v)))
+ .put("tombstone_failure_threshold", v ->
DatabaseDescriptor.setTombstoneFailureThreshold(Integer.parseInt(v)))
+
+ .put("hinted_handoff_enabled", v ->
StorageProxy.instance.setHintedHandoffEnabled(Boolean.parseBoolean(v)))
+ .put("hinted_handoff_throttle_in_kb", v ->
StorageService.instance.setHintedHandoffThrottleInKB(Integer.parseInt(v)))
+
+ .put("incremental_backups", v ->
DatabaseDescriptor.setIncrementalBackupsEnabled(Boolean.parseBoolean(v)))
+
+ .put("inter_dc_stream_throughput_outbound_megabits_per_sec", v ->
StorageService.instance.setInterDCStreamThroughputMbPerSec(Integer.parseInt(v)))
+ .put("stream_throughput_outbound_megabits_per_sec", v ->
StorageService.instance.setStreamThroughputMbPerSec(Integer.parseInt(v)))
+
+ .put("truncate_request_timeout_in_ms", v ->
StorageService.instance.setTruncateRpcTimeout(Long.parseLong(v)))
+ .put("cas_contention_timeout_in_ms", v ->
StorageService.instance.setCasContentionTimeout(Long.parseLong(v)))
+ .put("counter_write_request_timeout_in_ms", v ->
StorageService.instance.setCounterWriteRpcTimeout(Long.parseLong(v)))
+ .put("write_request_timeout_in_ms", v ->
StorageService.instance.setWriteRpcTimeout(Long.parseLong(v)))
+ .put("range_request_timeout_in_ms", v ->
StorageService.inst
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396178#comment-16396178
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173963909
--- Diff: src/java/org/apache/cassandra/db/virtual/CompactionStats.java ---
@@ -0,0 +1,75 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.db.virtual;
+
+import java.util.HashMap;
+import java.util.Map;
+import java.util.UUID;
+
+import org.apache.cassandra.cql3.CQL3Type;
+import org.apache.cassandra.cql3.QueryOptions;
+import org.apache.cassandra.cql3.restrictions.StatementRestrictions;
+import org.apache.cassandra.db.InMemoryVirtualTable;
+import org.apache.cassandra.db.compaction.CompactionManager;
+import org.apache.cassandra.schema.TableMetadata;
+import org.apache.cassandra.service.StorageService;
+
+public class CompactionStats extends InMemoryVirtualTable
+{
+
+public static Map columns()
--- End diff --
change this to a static initializer if it's never going to change.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396172#comment-16396172
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173962324
--- Diff: src/java/org/apache/cassandra/db/Keyspace.java ---
@@ -331,8 +337,11 @@ private Keyspace(String keyspaceName, boolean
loadSSTables)
this.viewManager = new ViewManager(this);
for (TableMetadata cfm : metadata.tablesAndViews())
{
-logger.trace("Initializing {}.{}", getName(), cfm.name);
-initCf(Schema.instance.getTableMetadataRef(cfm.id),
loadSSTables);
+logger.info("Initializing {}.{}", getName(), cfm.name);
--- End diff --
Change from trace() to info() intentional?
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396173#comment-16396173
]
ASF GitHub Bot commented on CASSANDRA-7622:
---
Github user zznate commented on a diff in the pull request:
https://github.com/apache/cassandra/pull/205#discussion_r173957234
--- Diff:
src/java/org/apache/cassandra/cql3/statements/CreateTableStatement.java ---
@@ -195,11 +217,27 @@ public TableMetadata toTableMetadata()
private final Multiset definedNames =
HashMultiset.create(1);
private final boolean ifNotExists;
+private final boolean isVirtual;
+private final String klass;
public RawStatement(CFName name, boolean ifNotExists)
{
super(name);
this.ifNotExists = ifNotExists;
+this.isVirtual = false;
+this.klass = null;
+
+assert isVirtual == (klass != null);
--- End diff --
Sorta OT, but we should just stop doing this and use `Preconditions` to
throw IAEs. I hate the fact that a user can disable so much of our error
checking w. a command line arg.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396183#comment-16396183 ] ASF GitHub Bot commented on CASSANDRA-7622: --- Github user zznate commented on a diff in the pull request: https://github.com/apache/cassandra/pull/205#discussion_r173966458 --- Diff: src/java/org/apache/cassandra/schema/TableMetadata.java --- @@ -552,22 +601,25 @@ public boolean equals(Object o) TableMetadata tm = (TableMetadata) o; return keyspace.equals(tm.keyspace) -&& name.equals(tm.name) -&& id.equals(tm.id) -&& partitioner.equals(tm.partitioner) -&& params.equals(tm.params) -&& flags.equals(tm.flags) -&& isView == tm.isView -&& columns.equals(tm.columns) -&& droppedColumns.equals(tm.droppedColumns) -&& indexes.equals(tm.indexes) -&& triggers.equals(tm.triggers); +&& name.equals(tm.name) +&& id.equals(tm.id) +&& partitioner.equals(tm.partitioner) +&& params.equals(tm.params) +&& flags.equals(tm.flags) +&& isView == tm.isView +&& columns.equals(tm.columns) +&& droppedColumns.equals(tm.droppedColumns) +&& indexes.equals(tm.indexes) +&& triggers.equals(tm.triggers) +&& isVirtual == tm.isVirtual +&& (virtualKlass == null || virtualKlass.equals(tm.virtualKlass)); --- End diff -- null class a legal state? > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396132#comment-16396132
]
Chris Lohfink commented on CASSANDRA-7622:
--
Default metrics included in this patch would be
{code:java}
Keyspace system_info
table_stats ring_state settings compaction_stats
{code}
looks like:
{code:java}
UPDATE system_info.settings SET value = 'false' WHERE setting =
'hinted_handoff_enabled';
SELECT * FROM system_info.settings WHERE writable = True;
setting | value | writable
--++--
batch_size_fail_threshold_in_kb | 50 | True
batch_size_warn_threshold_in_kb | 5 | True
cas_contention_timeout_in_ms | 1000 | True
compaction_throughput_mb_per_sec | 16 | True
concurrent_compactors | 2 | True
concurrent_validations | 2147483647 | True
counter_write_request_timeout_in_ms | 5000 | True
hinted_handoff_enabled | false | True
hinted_handoff_throttle_in_kb | 1024 | True
incremental_backups | false | True
inter_dc_stream_throughput_outbound_megabits_per_sec |200 | True
phi_convict_threshold |8.0 | True
range_request_timeout_in_ms | 1 | True
read_request_timeout_in_ms | 5000 | True
request_timeout_in_ms | 1 | True
stream_throughput_outbound_megabits_per_sec |200 | True
tombstone_failure_threshold | 10 | True
tombstone_warn_threshold | 1000 | True
truncate_request_timeout_in_ms | 6 | True
write_request_timeout_in_ms | 2000 | True
{code}
Querying a little more flexible since its an inmemory resultset, ie:
{code:java}
SELECT keyspace_name, table_name, metric, median, p99th FROM
system_info.table_stats WHERE keyspace_name = 'system' and p99th > 0;
keyspace_name | table_name | metric | median
| p99th
---+-+-++
system | batches |rangeLatency |
1.5232e+05 | 3.7902e+05
system | compaction_history | estimatedColumnCountHistogram |
6 | 7
system | compaction_history | estimatedPartitionSizeHistogram |
124 |149
system | local | estimatedColumnCountHistogram |
17 |310
system | local | estimatedPartitionSizeHistogram |
215 | 6866
system | prepared_statements | estimatedColumnCountHistogram |
1 | 2
system | prepared_statements | estimatedPartitionSizeHistogram |
103 |149
system | size_estimates | estimatedColumnCountHistogram |
1109 | 2299
system | size_estimates | estimatedPartitionSizeHistogram |
42510 | 88148
system | size_estimates |writeLatency |
4.5483e+05 | 6.5495e+05
system |sstable_activity | estimatedColumnCountHistogram |
2 | 2
system |sstable_activity | estimatedPartitionSizeHistogram |
72 | 86
system |sstable_activity |writeLatency |
61214 | 1.0578e+05
{code}
Will add more tables eventually but this is starter set along with a "no op"
table for cassandra-stress for just benchmarking the networking/cql processing.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395892#comment-16395892
]
Chris Lohfink commented on CASSANDRA-7622:
--
PR has an abstract class thats more or less
{code}public ReadQuery getQuery(SelectStatement selectStatement, QueryOptions
options, DataLimits limits, int nowInSec)
public void mutate(DecoratedKey partitionKey, Row row){code}
This ultimately allows either supporting local or distributed since you can
make a ReadQuery for coordination that collects from replicas (never done here
though, but its possible). The {{system_info}} keyspace added for system
metrics locally. To make it easier for simpler local tables created an
InMemoryVirtualTable implementation that these all extend. Examples seen in
[https://github.com/clohfink/cassandra/tree/7622-trunk/src/java/org/apache/cassandra/db/virtual]
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Chris Lohfink
>Priority: Major
> Fix For: 4.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395886#comment-16395886 ] ASF GitHub Bot commented on CASSANDRA-7622: --- GitHub user clohfink opened a pull request: https://github.com/apache/cassandra/pull/205 Add virtual tables for CASSANDRA-7622 You can merge this pull request into a Git repository by running: $ git pull https://github.com/clohfink/cassandra 7622-trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/cassandra/pull/205.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #205 commit 170e71e2b0b6e26e7092f9a35b0f166676d073f6 Author: Chris Lohfink Date: 2018-03-12T20:53:39Z Add virtual tables for CASSANDRA-7622 > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Chris Lohfink >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363273#comment-16363273 ] Kurt Greaves commented on CASSANDRA-7622: - bq. I'm not really sure a lot of ops people would even notice this new feature as long as we don't pull the plug on JMX. New users would still likely use it over JMX because it would hopefully be more user friendly (JMX is pretty ugly). If we could make it perform better than JMX that would be a big win (this seems unlikely though). Once people implement the tools to gather metrics from it the cost to users for crossing over to the vTables would be pretty minimal. There's no reason to only expose config in cqlsh. Ops people would want to do reading and writing to config properties from their own programs, and limiting it to cqlsh would be nasty. Being able to use CQL would be ideal for config management. On another note I can think of a couple examples for "replicated vTables" such as whole cluster state. At the moment if you want to do any proper monitoring on cluster state (down nodes) you have to aggregate all this information yourself from every node in the cluster. With replicated vTables we could easily store cluster state in a table and update it, making monitoring much easier for everyone. This also goes for things like repairs, where we could better expose repairs happening across the cluster, nodes, and ranges involved. We could probably even use it to better manage repairs that fail (e.g automatically kill off validations that are no longer important). Being able to read and modify config I'd still say is the biggest win however. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa >Priority: Major > Fix For: 4.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437553#comment-15437553 ] T Jake Luciani commented on CASSANDRA-7622: --- Regarding exposing metrics as CQL this is a FB project to look at for ideas https://osquery.io/ > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324565#comment-15324565 ] Jon Haddad commented on CASSANDRA-7622: --- Those tools aren't the main concern. They'd need to wait for the virtual tables to be mainstream & stable, so don't expect them to be using VT till at least a year or two from now. Regarding the simplified, non-replicated, read only version, see above where I mention a follow up for 2 months after the initial version. We're not on a yearly release cycle anymore so the need to do *everything* up front is no longer an issue. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324531#comment-15324531 ] Stefan Podkowinski commented on CASSANDRA-7622: --- bq. Getting access to metrics in a read only, non JMX fashion would be awesome from an operational perspective and be 100% worth it by itself. [~rustyrazorblade], as much as I share your aversion towards JMX, I'm not really sure a lot of ops people would even notice this new feature as long as we don't pull the plug on JMX. All existing monitoring solutions are based on JMX (which indirectly includes solutions on top of jolokia) and I don't expect a lot of enthusiasm among vendors to adopt virtual tables instead. So JMX will be here to stay, which begs the question who we have in mind for this feature? Just starting with a simplified, non-replicated, read-only version of virtual tables is also raising some red flags for me. We should be able to at least answer how advanced use cases could be implemented based on the current query execution model. If we can't, virtual tables are probably just a dead end road for any further steps we want to take to improve operational aspects. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15321103#comment-15321103
]
Chris Lohfink commented on CASSANDRA-7622:
--
There is no {{WHERE node='1.2.3.4'}} for system.compaction_history table and
people get along just fine. Why have routing at all? It also adds a lot of
complexity and possibly instability. Local only would be easy for operations to
be accessed just fine from cqlsh or the driver (ie driver can check load or
latency to weigh coordinators in load balancing policy) or a careful user. The
main use case is viewing (possibly setting) configs and viewing metrics so
system-like thing seems right to me.
We could take the metric attributes from the registry to create the table, and
partition/composite key with meaningful names just fine. What i envisioned was:
{code}
cqlsh:system_metrics> describe keyspaces;
testks system_schema system_auth system system_distributed system_traces
system_metrics
cqlsh:system_metrics> describe tables; # use the type attrib as the name,
camelcase -> lowercased, underscored
thread_pools table storage index dropped_message ...
cqlsh:system_metrics> expand on;
cqlsh:system_metrics> select * from table where keyspace = 'system' and scope =
'paxos';
# considering just the metrics
#
org.apache.cassandra.metrics:type=Table,keyspace=system,scope=paxos,name=BytesFlushed
#
org.apache.cassandra.metrics:type=Table,keyspace=system,scope=paxos,name=WriteLatency
@ Row 1
-+--
keyspace| system # partition key
scope | paxos# composite
bytes_flushed | {'Count': 0}
write_latency | {'Count': 1, '999thPercentile': 1302, ...}
...
{code}
metrics that only have a scope will have that as the partition key alone (ie
client_request table would have 'read' as a partition key and failures,
latency, timouts, unavailable, total_latency as fields).
{code}
# org.apache.cassandra.metrics:type=ClientRequest,scope=Read,name=Latency
cqlsh:system_metrics> select * from client_requests where scope = 'read';
@ Row 1
-+--
scope | read# partition key
failures | {'Count': 0}
latency | {'Count': 1, '999thPercentile': 1302, ...}
...
{code}
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15321005#comment-15321005 ] Robert Stupp commented on CASSANDRA-7622: - bq. If true this is something that needs to be changed, as it affects reading system tables. Nothing to be fixed. There's a whitelist policy on the cluster object (wrongly recalled it's just an initial contact point). > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320996#comment-15320996 ] Jeremiah Jordan commented on CASSANDRA-7622: Another thing to keep in mind, if we are not going to do the CQL based virtual table thing and only do a system version, we need to make sure that things like custom secondary indexes can expose stuff through this, the current CassandraMetricsRegistry lets them do this fairly easily. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320979#comment-15320979 ] Aleksey Yeschenko commented on CASSANDRA-7622: -- My overall concern after reading through the comments again is premature overgeneralisation, which is something I'm strongly against. Need more specific use cases before we generalise. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320976#comment-15320976 ] Brian Hess commented on CASSANDRA-7622: +1 > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320968#comment-15320968
]
Aleksey Yeschenko commented on CASSANDRA-7622:
--
bq. That certainly doesn't prevent us for supporting SHOW VARIABLES as
syntactic sugar for the equivalent select statement. This might even make sense
as a cqlsh thing (rather than a CQL one).
Fair enough. I'm just questioning if there are any important use cases outside
of configuration that we actually need a generalized virtual table mechanism at
all. If we don't, then we might as well just add special purpose
{{SHOW}}/{{SET}} only and call it a day. If we do, then {{SHOW}}/{{SET}} in
cqlsh on top of a virtual table wfm.
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[
https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320961#comment-15320961
]
Sylvain Lebresne commented on CASSANDRA-7622:
-
bq. Sure, but SHOW is a common enough command in DB world
Imo, the general mechanism of virtual tables should be just an implementation
detail for exposing a table. That means even in the case of {{SHOW VARIABLES}},
we'd really be adding a new system table with those variables, and we'll be
able to query it through normal select. That certainly doesn't prevent us for
supporting {{SHOW VARIABLES}} as syntactic sugar for the equivalent select
statement. This might even make sense as a cqlsh thing (rather than a CQL one).
> Implement virtual tables
>
>
> Key: CASSANDRA-7622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7622
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Tupshin Harper
>Assignee: Jeff Jirsa
> Fix For: 3.x
>
>
> There are a variety of reasons to want virtual tables, which would be any
> table that would be backed by an API, rather than data explicitly managed and
> stored as sstables.
> One possible use case would be to expose JMX data through CQL as a
> resurrection of CASSANDRA-3527.
> Another is a more general framework to implement the ability to expose yaml
> configuration information. So it would be an alternate approach to
> CASSANDRA-7370.
> A possible implementation would be in terms of CASSANDRA-7443, but I am not
> presupposing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320954#comment-15320954 ] Sylvain Lebresne commented on CASSANDRA-7622: - Drivers, at least the java one, _have_ everything they need to allow forcing which node it routes queries to (it may not be exposed too conveniently so far, but we can let them do better there). And given that system table are already local and we're only talking about adding more system table (just with a different backing implementation), I don't think we need to invent anything new here. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320939#comment-15320939 ] Aleksey Yeschenko commented on CASSANDRA-7622: -- bq. But the node just added as the initial contact point. When that node fails, the statement would be silently executed against another node. If true this is something that needs to be changed, as it affects reading system tables. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320933#comment-15320933 ] Brian Hess commented on CASSANDRA-7622: For metrics, Postgres appears to use a SELECT syntax. For system settings they use SHOW/SET. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7622) Implement virtual tables
[ https://issues.apache.org/jira/browse/CASSANDRA-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320913#comment-15320913 ] Brian Hess commented on CASSANDRA-7622: I think this is a bit narrow of thinking. I can see many use cases for programmatic access to these values. If SHOW VARIABLE is useful in cqlsh it could have special cqlsh commands that convert into this. If you want to mix regular CQL and these, you'd need different Sessions as you'd need separate Cluster objects (so you can have a WhiteListPolicy for the one node of interest). Or if you want to look at the metrics for multiple nodes you would need one Cluster object per node. > Implement virtual tables > > > Key: CASSANDRA-7622 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7622 > Project: Cassandra > Issue Type: Improvement >Reporter: Tupshin Harper >Assignee: Jeff Jirsa > Fix For: 3.x > > > There are a variety of reasons to want virtual tables, which would be any > table that would be backed by an API, rather than data explicitly managed and > stored as sstables. > One possible use case would be to expose JMX data through CQL as a > resurrection of CASSANDRA-3527. > Another is a more general framework to implement the ability to expose yaml > configuration information. So it would be an alternate approach to > CASSANDRA-7370. > A possible implementation would be in terms of CASSANDRA-7443, but I am not > presupposing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
