Re: Status of CEP-1

[guo Maxwell]

Have the same question ： what ‘s the plan ？

Jeff Jirsa 于2024年10月2日 周三上午10:43写道：

>
>
> On Oct 1, 2024, at 7:26 PM, Josh McKenzie  wrote:
>
> However it is used by a number of other features as a dependency such as
> analytics, backup/restore, repair, metrics, and CDC
>
> It seems like a natural pressure relief valve for moving operations out of
> a core C* node that are well served out of process.
>
>
> Yea, but the point of the foundation is to RELEASE software for the public
> good, and the link asserting consensus was dec2018, so its’ 5.5 years and
> no releases.
>
> What’s the plan here?
>
>
>
>
>

Re: [DISCUSS] Introduce CREATE TABLE LIKE grammer

[guo Maxwell]

Hello, everyone ,
I have finished the doc for CEP-43 for CREATE_TABLE_LIKE
<https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-43++Apache+Cassandra+CREATE+TABLE++LIKE>
as
said before, looking forward to your suggestions.

Štefan Miklošovič  于2024年9月25日周三 03:51写道：

> I am sorry I do not follow what you mean, maybe an example would help.
>
> On Tue, Sep 24, 2024 at 6:18 PM guo Maxwell  wrote:
>
>>
>> If there are multiple schema information changes in one ddl statement,
>> will there be schema conflicts in extreme cases?
>> For example, our statement contains both table creation and index
>> creation.
>>
>> guo Maxwell 于2024年9月24日 周二下午8:12写道：
>>
>>> +1 on splitting this task  and adding the ability to copy tables through
>>> different keyspaces in the future.
>>>
>>> Štefan Miklošovič  于2024年9月23日周一 22:05写道：
>>>
>>>> If we have this table
>>>>
>>>> CREATE TABLE ks.tb2 (
>>>> id int PRIMARY KEY,
>>>> name text
>>>> );
>>>>
>>>> I can either specify name of an index on my own like this:
>>>>
>>>> CREATE INDEX name_index ON ks.tb2 (name) ;
>>>>
>>>> or I can let Cassandra to figure that name on its own:
>>>>
>>>> CREATE INDEX ON ks.tb2 (name) ;
>>>>
>>>> in that case it will name that index "tb2_name_idx".
>>>>
>>>> Hence, I would expect that when we do
>>>>
>>>> ALTER TABLE ks.to_copy LIKE ks.tb2 WITH INDICES;
>>>>
>>>> Then ks.to_copy table will have an index which is called
>>>> "to_copy_name_idx" without me doing anything.
>>>>
>>>> For types, we do not need to do anything when we deal with the same
>>>> keyspace. For simplicity, I mentioned that we might deal with the same
>>>> keyspace scenario only for now and iterate on that in the future.
>>>>
>>>> On Mon, Sep 23, 2024 at 8:53 AM guo Maxwell 
>>>> wrote:
>>>>
>>>>> Hello everyone,
>>>>>
>>>>> Cep is being written, and I encountered some problems during the
>>>>> process. I would like to discuss them with you. If you read the 
>>>>> description
>>>>> of this CASSANDRA-7662
>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-7662>, we will find
>>>>> that initially the original creator of this jira did not intend to
>>>>> implement structural copying of indexes, views, and triggers  only the
>>>>> column and its data type.
>>>>>
>>>>> However, after investigating some db related syntax and function
>>>>> implementation, I found that it may be necessary for us to provide some
>>>>> rich syntax to support the replication of indexes, views, etc.
>>>>>
>>>>> In order to support selective copy of the basic structure of the table
>>>>> (columns and types), table options, table-related indexes, views, 
>>>>> triggers,
>>>>> etc. We need some new syntax, it seems that the syntax of pg is relatively
>>>>> comprehensive, it use the keyword INCLUDING/EXCLUDING to flexibly control
>>>>> the removal and retention of indexes, table information, etc. see pg
>>>>> create table like
>>>>> <https://www.postgresql.org/docs/8.1/sql-createtable.html> , the new
>>>>> created index name is different from the original table's index name , 
>>>>> seenewly
>>>>> copied index names are different from original
>>>>> <https://github.com/postgres/postgres/blob/master/doc/src/sgml/ref/create_table.sgml#L749>
>>>>> , the name is based on some rule.
>>>>> Mysql is relatively simple and copies columns and indexes by default.
>>>>> see mysql create table like
>>>>> <https://dev.mysql.com/doc/refman/8.4/en/create-table-like.html> and
>>>>> the newly created index name is the same with the original table's index
>>>>> name.
>>>>>
>>>>> So for Casandra, I hope it can also support the information copy of
>>>>> index and even view/trigger. And I also hope to be able to flexibly decide
>>>>> which information is copied like pg.
>>>>>
>>>>> Besides, I think the copy can happen between different keyspaces. And
>>>>> UDT needs to be taken into 

Re: [REVIEW REQUEST] Exposing the status of a cleanup command on a virtual table

[guo Maxwell]

Hi,

Can we make some schema  adjustments to System.compaction_history table to
meet our needs?
For example : If there is no task execution, what about  just recording 0
bytes_in, 0 bytes_out for this compaction history . Other status can be
placed in compaction_properties,
and you can even add a column to represent the status. I am actually more
inclined to add a column .

I think there may be many situations that are similar to the special
situation of compaction execution that you described, such as the view
building  with no data or building with a non-successful state or other
module.
I think it is not possible to implement a separate solution for each
particular situation.

Štefan Miklošovič  于2024年9月27日周五 23:54写道：

> Thank you Maxim for summarizing it like this for better visibility of what
> you are suggesting to contribute.
>
> I am OK with that in general but:
>
> 1) The table should be limited in its size (I think the original patch you
> had was already done with space-limitation in mind). Since we want to put
> this on a heap, I think we should definitely make it capped on size.
> 2) If both of them are limited in their size, can it not happen that there
> would be an entry e.g. in compaction_operation_linked_stats which would not
> have its counterpart in compaction_operation_status (or vice versa)? Could
> not it be done in such a way that if an entry in
> compaction_operation_status is dropped (as new entries would come in, old
> would be discarded), the dropping of that entry would automatically drop
> all related rows in compaction_operations_linked_tasks?
>
> For the record, we were also investigating if existing
> system.compaction_history could not be somehow used for this but that
> appears to be problematic as we would need to probably change the schema /
> add new columns etc. and this is not so simple as it is a system table. We
> might use the compaction_properties column which is a map and put all the
> details there, however that starts to be quite uncomfortable on querying
> and even if we somehow did it, it still does not solve all the issues the
> two table approach seems to address.
>
> On Fri, Sep 27, 2024 at 4:21 PM Maxim Muzafarov  wrote:
>
>> Hello everyone,
>>
>> I still need a few more eyes on [1][2], but this time I'm going to try
>> and do some marketing for the feature I'm talking about, so...
>>
>>
>> We are trying to bridge the gap between the API that is called and the
>> compaction process that MAY or MAY NOT be called as a result, and make
>> users aware of what is happening inside the cluster with their running
>> commands. Currently, this can only be viewed by reading logs, which is
>> not a convenient way for both operators and audit subsystems of the
>> node internals.
>>
>> What we want to do is store the history of running operations for the
>> compaction manager in a small collection in the java heap and fill
>> this gap with virtual tables on top of this data collection, namely:
>>
>> - compaction_operations_status - has (operation_type, operation_id)
>> primary key and exposes the status of the cleanup command as a whole.
>> It may or may not trigger the compaction process and the compaction
>> may or may not appear in the sstable_tasks virtual table (active
>> compactions);
>> - compaction_operations_linked_tasks - has (operation_type,
>> operation_id, compaction_id) as its primary key and shows the
>> relationship between the user-triggered operation and the compaction
>> process invoked as a result;
>>
>> The CASSANDRA-19670 [1] issue covers only the cleanup command and
>> demonstrates the approach; all other commands, which can be identified
>> by the OperationType class, could be implemented in follow-up issues.
>>
>>
>> Examples:
>>
>> - The definition of these new virtual tables looks like:
>> https://gist.github.com/Mmuzaf/2d3006f5b654d54e7cabc343cd73a2a3
>>
>> - The output when we run the cleanup command, but it doesn't trigger
>> the compaction:
>>
>> https://gist.github.com/user-attachments/assets/9089d5c1-70d4-475f-9cf7-cc16dff48699
>>
>>
>> [1] https://issues.apache.org/jira/browse/CASSANDRA-19760
>> [2] https://github.com/apache/cassandra/pull/3412/files
>>
>> On Mon, 15 Jul 2024 at 21:06, Maxim Muzafarov  wrote:
>> >
>> > Hello everyone,
>> >
>> > I would like to gently ask for help in reviewing the following issue
>> > that we've been facing for a while:
>> > https://issues.apache.org/jira/browse/CASSANDRA-19760
>> >
>> > When a cleanup command is called, the compaction process under the
>> > hood is triggered accordingly. However, if there is nothing to compact
>> > or the cleanup command returns with a status other than SUCCESSFUL,
>> > there is no way to get the execution results of the command that was
>> > run. This is especially true when using any kind of
>> > automation/scripting on top of JMX or as a nodetool wrapper.
>> >
>> > I propose to keep these history results in memory for some time and
>> > expose them via a virtual ta

Re: 【DISCUSS】The configuration of Commitlog archiving

[guo Maxwell]

Hello，are there any new updates？🤔

guo Maxwell 于2024年9月18日 周三下午4:06写道：

> Do you have any new updates  on this DISCUSS ?
>
> - The reason this pattern is popular is it allows extension of
> functionality ahead of the database. Some people copy to a NAS/SAN. Some
> people copy to S3. Some people copy to their own object storage that isn’t
> s3 compatible. “Compress and move” is super limiting, because “move” varies
> remarkably between environments.
>
> Yes, it is indeed very flexible to use this way, but would it be more
> appropriate to decouple the file archiving to heterogeneous storage and
> leave it to other systems to handle it specifically? And we only do
> compression and copying (file linking like sstable incremental backup)?
>
>
> Štefan Miklošovič  于2024年9月5日周四 04:18写道：
>
>>
>> On Wed, Sep 4, 2024 at 8:34 PM Jon Haddad  wrote:
>>
>>> I thought about this a bit over the last few days, and there's actually
>>> quite a few problems present that would need to be addressed.
>>>
>>> *Insecure JMX*
>>>
>>> First off - if someone has access to JMX, the entire system is already
>>> compromised.  A bad actor can mess with the cluster topology, truncate
>>> tables, and do a ton of other disruptive stuff.  But if we're going to go
>>> down this path I think we should apply your logic consistently to avoid
>>> creating a "solution" that has the same "problem" as we do now.  I use
>>> quotes because I'm not entirely convinced the root cause of the problem is
>>> enabling some shell access, but I'll entertain it for the sake of the
>>> discussion.
>>>
>>> *Dynamic Configuration and Shell Scripts*
>>>
>>> Let's pretend that somehow an open JMX isn't already a *massive*
>>> security flaw by itself.  Once an attacker has control of a system, the
>>> next phase of the attack relies on them dynamically changing the
>>> configuration to point to a different shell script, or to execute arbitrary
>>> shell scripts.
>>>
>> I agree with the general idea that we don't want this - so in my mind the
>>> necessary solution here is to disable the ability to change the commit log
>>> archiving behavior at runtime.
>>>
>>> The idea that commit log archiving (and many other config settings)
>>> would be dynamically configurable is a massive security flaw that should be
>>> disallowed.  If you want to take this a step further and claim there's a
>>> flaw with shell scripts in general, I'll even entertain that for a minute,
>>> but we need to examine if the proposed solution of moving code to Java
>>> actually solves the problem.
>>>
>>> *Dynamic Configuration and Java Code*
>>>
>>> Let's say we've removed the ability to use shell scripts, and we've
>>> gotten people to rewrite their shell code with java code, but we've left
>>> the dynamic configuration in.  Going back to my original email, I mentioned
>>> copying commit logs off the node and into an object store.  If someone is
>>> able to change the parameter dynamically at runtime, they could just as
>>> easily point to a public S3 bucket and commit logs would be archived there
>>> which is just as bad as the shell version.  So if we are to convert this
>>> functionality to Java, we should also be making best practice
>>> recommendations on what users should and should not do.
>>>
>>
>> I think what you meant here is that if we allowed people to provide a
>> pluggable way how stuff is copied over and they would code it up, put that
>> JAR on the class path, Cassandra (re)started etc, then someone might
>> reconfigure this custom solution in runtime? Yeah, we do not want this. We
>> can make it pluggable, but not reconfigurable. To have it pluggable and not
>> reconfigurable, then to replace it with something else, an attacker would
>> basically need to restart Cassandra with a rogue JAR on the class path. In
>> order to do that, I think that at this point it would be beyond any
>> salvation and the system is completely compromised anyway.
>>
>>
>>>
>>>
>>> *Apply All Operational Best Practices*
>>>
>>> There's been a variety of examples of how a user can further compromise
>>> a machine once they have JMX, working in tandem with shell scripts, but I
>>> hope at this point you can see that the issue is fundamentally more complex
>>> than simply disallowing shell scripts.  The issue

Re: [DISCUSS] Introduce CREATE TABLE LIKE grammer

[guo Maxwell]

If there are multiple schema information changes in one ddl statement, will
there be schema conflicts in extreme cases?
For example, our statement contains both table creation and index creation.

guo Maxwell 于2024年9月24日 周二下午8:12写道：

> +1 on splitting this task  and adding the ability to copy tables through
> different keyspaces in the future.
>
> Štefan Miklošovič  于2024年9月23日周一 22:05写道：
>
>> If we have this table
>>
>> CREATE TABLE ks.tb2 (
>> id int PRIMARY KEY,
>> name text
>> );
>>
>> I can either specify name of an index on my own like this:
>>
>> CREATE INDEX name_index ON ks.tb2 (name) ;
>>
>> or I can let Cassandra to figure that name on its own:
>>
>> CREATE INDEX ON ks.tb2 (name) ;
>>
>> in that case it will name that index "tb2_name_idx".
>>
>> Hence, I would expect that when we do
>>
>> ALTER TABLE ks.to_copy LIKE ks.tb2 WITH INDICES;
>>
>> Then ks.to_copy table will have an index which is called
>> "to_copy_name_idx" without me doing anything.
>>
>> For types, we do not need to do anything when we deal with the same
>> keyspace. For simplicity, I mentioned that we might deal with the same
>> keyspace scenario only for now and iterate on that in the future.
>>
>> On Mon, Sep 23, 2024 at 8:53 AM guo Maxwell  wrote:
>>
>>> Hello everyone,
>>>
>>> Cep is being written, and I encountered some problems during the
>>> process. I would like to discuss them with you. If you read the description
>>> of this CASSANDRA-7662
>>> <https://issues.apache.org/jira/browse/CASSANDRA-7662>, we will find
>>> that initially the original creator of this jira did not intend to
>>> implement structural copying of indexes, views, and triggers  only the
>>> column and its data type.
>>>
>>> However, after investigating some db related syntax and function
>>> implementation, I found that it may be necessary for us to provide some
>>> rich syntax to support the replication of indexes, views, etc.
>>>
>>> In order to support selective copy of the basic structure of the table
>>> (columns and types), table options, table-related indexes, views, triggers,
>>> etc. We need some new syntax, it seems that the syntax of pg is relatively
>>> comprehensive, it use the keyword INCLUDING/EXCLUDING to flexibly control
>>> the removal and retention of indexes, table information, etc. see pg
>>> create table like
>>> <https://www.postgresql.org/docs/8.1/sql-createtable.html> , the new
>>> created index name is different from the original table's index name , 
>>> seenewly
>>> copied index names are different from original
>>> <https://github.com/postgres/postgres/blob/master/doc/src/sgml/ref/create_table.sgml#L749>
>>> , the name is based on some rule.
>>> Mysql is relatively simple and copies columns and indexes by default.
>>> see mysql create table like
>>> <https://dev.mysql.com/doc/refman/8.4/en/create-table-like.html> and
>>> the newly created index name is the same with the original table's index
>>> name.
>>>
>>> So for Casandra, I hope it can also support the information copy of
>>> index and even view/trigger. And I also hope to be able to flexibly decide
>>> which information is copied like pg.
>>>
>>> Besides, I think the copy can happen between different keyspaces. And
>>> UDT needs to be taken into account.
>>>
>>> But as we know the index/view/trigger name are all under keyspace level,
>>> so it seems that the newly created index name (or view name/ trigger name)
>>> must be different from the original tables' ,otherwise  names would clash .
>>>
>>> So regarding the above problem, one idea I have is that for newly
>>> created types, indexes and views under different keyspaces and the same
>>> keyspace, we first generate random names for them, and then we can add the
>>> ability of modifying the names(for types/indexes/views/triggers) so that
>>> users can manually change the names.
>>>
>>>
>>> guo Maxwell  于2024年9月20日周五 08:06写道：
>>>
>>>> No，I think still need some discuss on grammar detail after I finish the
>>>> first version
>>>>
>>>> Patrick McFadin 于2024年9月20日 周五上午2:24写道：
>>>>
>>>>> Is this CEP ready for a VOTE thread?
>>>>>
>>>>> On Sat, Aug 24, 2024 at 8:56 PM guo Maxwell 
>>>>>

Re: [DISCUSS] Introduce CREATE TABLE LIKE grammer

[guo Maxwell]

+1 on splitting this task  and adding the ability to copy tables through
different keyspaces in the future.

Štefan Miklošovič  于2024年9月23日周一 22:05写道：

> If we have this table
>
> CREATE TABLE ks.tb2 (
> id int PRIMARY KEY,
> name text
> );
>
> I can either specify name of an index on my own like this:
>
> CREATE INDEX name_index ON ks.tb2 (name) ;
>
> or I can let Cassandra to figure that name on its own:
>
> CREATE INDEX ON ks.tb2 (name) ;
>
> in that case it will name that index "tb2_name_idx".
>
> Hence, I would expect that when we do
>
> ALTER TABLE ks.to_copy LIKE ks.tb2 WITH INDICES;
>
> Then ks.to_copy table will have an index which is called
> "to_copy_name_idx" without me doing anything.
>
> For types, we do not need to do anything when we deal with the same
> keyspace. For simplicity, I mentioned that we might deal with the same
> keyspace scenario only for now and iterate on that in the future.
>
> On Mon, Sep 23, 2024 at 8:53 AM guo Maxwell  wrote:
>
>> Hello everyone,
>>
>> Cep is being written, and I encountered some problems during the process.
>> I would like to discuss them with you. If you read the description of this
>> CASSANDRA-7662 <https://issues.apache.org/jira/browse/CASSANDRA-7662>,
>> we will find that initially the original creator of this jira did not
>> intend to implement structural copying of indexes, views, and triggers
>> only the column and its data type.
>>
>> However, after investigating some db related syntax and function
>> implementation, I found that it may be necessary for us to provide some
>> rich syntax to support the replication of indexes, views, etc.
>>
>> In order to support selective copy of the basic structure of the table
>> (columns and types), table options, table-related indexes, views, triggers,
>> etc. We need some new syntax, it seems that the syntax of pg is relatively
>> comprehensive, it use the keyword INCLUDING/EXCLUDING to flexibly control
>> the removal and retention of indexes, table information, etc. see pg
>> create table like
>> <https://www.postgresql.org/docs/8.1/sql-createtable.html> , the new
>> created index name is different from the original table's index name , 
>> seenewly
>> copied index names are different from original
>> <https://github.com/postgres/postgres/blob/master/doc/src/sgml/ref/create_table.sgml#L749>
>> , the name is based on some rule.
>> Mysql is relatively simple and copies columns and indexes by default. see 
>> mysql
>> create table like
>> <https://dev.mysql.com/doc/refman/8.4/en/create-table-like.html> and the
>> newly created index name is the same with the original table's index name.
>>
>> So for Casandra, I hope it can also support the information copy of index
>> and even view/trigger. And I also hope to be able to flexibly decide which
>> information is copied like pg.
>>
>> Besides, I think the copy can happen between different keyspaces. And UDT
>> needs to be taken into account.
>>
>> But as we know the index/view/trigger name are all under keyspace level,
>> so it seems that the newly created index name (or view name/ trigger name)
>> must be different from the original tables' ,otherwise  names would clash .
>>
>> So regarding the above problem, one idea I have is that for newly created
>> types, indexes and views under different keyspaces and the same keyspace,
>> we first generate random names for them, and then we can add the ability of
>> modifying the names(for types/indexes/views/triggers) so that users can
>> manually change the names.
>>
>>
>> guo Maxwell  于2024年9月20日周五 08:06写道：
>>
>>> No，I think still need some discuss on grammar detail after I finish the
>>> first version
>>>
>>> Patrick McFadin 于2024年9月20日 周五上午2:24写道：
>>>
>>>> Is this CEP ready for a VOTE thread?
>>>>
>>>> On Sat, Aug 24, 2024 at 8:56 PM guo Maxwell 
>>>> wrote:
>>>>
>>>>> Thank you for your replies, I will prepare a CEP later.
>>>>>
>>>>> Patrick McFadin  于2024年8月20日周二 02:11写道：
>>>>>
>>>>>> +1 This is a CEP
>>>>>>
>>>>>> On Mon, Aug 19, 2024 at 10:50 AM Jon Haddad 
>>>>>> wrote:
>>>>>>
>>>>>>> Given the fairly large surface area for this, i think it should be a
>>>>>>> CEP.
>>>>>>>
>>>>>>> —
>>>>>>>

Re: [DISCUSS] Introduce CREATE TABLE LIKE grammer

[guo Maxwell]

Hello everyone,

Cep is being written, and I encountered some problems during the process. I
would like to discuss them with you. If you read the description of this
CASSANDRA-7662 <https://issues.apache.org/jira/browse/CASSANDRA-7662>, we
will find that initially the original creator of this jira did not intend
to implement structural copying of indexes, views, and triggers  only the
column and its data type.

However, after investigating some db related syntax and function
implementation, I found that it may be necessary for us to provide some
rich syntax to support the replication of indexes, views, etc.

In order to support selective copy of the basic structure of the table
(columns and types), table options, table-related indexes, views, triggers,
etc. We need some new syntax, it seems that the syntax of pg is relatively
comprehensive, it use the keyword INCLUDING/EXCLUDING to flexibly control
the removal and retention of indexes, table information, etc. see pg create
table like <https://www.postgresql.org/docs/8.1/sql-createtable.html> , the
new created index name is different from the original table's index name ,
seenewly copied index names are different from original
<https://github.com/postgres/postgres/blob/master/doc/src/sgml/ref/create_table.sgml#L749>
, the name is based on some rule.
Mysql is relatively simple and copies columns and indexes by default. see mysql
create table like
<https://dev.mysql.com/doc/refman/8.4/en/create-table-like.html> and the
newly created index name is the same with the original table's index name.

So for Casandra, I hope it can also support the information copy of index
and even view/trigger. And I also hope to be able to flexibly decide which
information is copied like pg.

Besides, I think the copy can happen between different keyspaces. And UDT
needs to be taken into account.

But as we know the index/view/trigger name are all under keyspace level, so
it seems that the newly created index name (or view name/ trigger name)
must be different from the original tables' ,otherwise  names would clash .

So regarding the above problem, one idea I have is that for newly created
types, indexes and views under different keyspaces and the same keyspace,
we first generate random names for them, and then we can add the ability of
modifying the names(for types/indexes/views/triggers) so that users can
manually change the names.


guo Maxwell  于2024年9月20日周五 08:06写道：

> No，I think still need some discuss on grammar detail after I finish the
> first version
>
> Patrick McFadin 于2024年9月20日 周五上午2:24写道：
>
>> Is this CEP ready for a VOTE thread?
>>
>> On Sat, Aug 24, 2024 at 8:56 PM guo Maxwell  wrote:
>>
>>> Thank you for your replies, I will prepare a CEP later.
>>>
>>> Patrick McFadin  于2024年8月20日周二 02:11写道：
>>>
>>>> +1 This is a CEP
>>>>
>>>> On Mon, Aug 19, 2024 at 10:50 AM Jon Haddad  wrote:
>>>>
>>>>> Given the fairly large surface area for this, i think it should be a
>>>>> CEP.
>>>>>
>>>>> —
>>>>> Jon Haddad
>>>>> Rustyrazorblade Consulting
>>>>> rustyrazorblade.com
>>>>>
>>>>>
>>>>> On Mon, Aug 19, 2024 at 10:44 AM Bernardo Botella <
>>>>> conta...@bernardobotella.com> wrote:
>>>>>
>>>>>> Definitely a nice addition to CQL.
>>>>>>
>>>>>> Looking for inspiration at how Postgres and Mysql do that may also
>>>>>> help with the final design (I like the WITH proposed by Stefan, but I 
>>>>>> would
>>>>>> definitely take a look at the INCLUDING keyword proposed by Postgres).
>>>>>> https://www.postgresql.org/docs/current/sql-createtable.html
>>>>>> https://dev.mysql.com/doc/refman/8.4/en/create-table-like.html
>>>>>>
>>>>>> On top of that, and as part of the interesting questions, I would
>>>>>> like to add the permissions to the mix. Both the question about copying
>>>>>> them over (with a WITH keyword probably), and the need for read 
>>>>>> permissions
>>>>>> on the source table as well.
>>>>>>
>>>>>> Bernardo
>>>>>>
>>>>>> On Aug 19, 2024, at 10:01 AM, Štefan Miklošovič <
>>>>>> smikloso...@apache.org> wrote:
>>>>>>
>>>>>> BTW this would be cool to do as well:
>>>>>>
>>>>>> ALTER TABLE ks.to_copy LIKE ks.tb WITH INDICES;
>>>>>>
>>>>>> This would mean that if we create a copy of a t

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[guo Maxwell]

I discussed this offline with Claude, he is no longer working on this.

It's a pity. I think this is a very valuable thing. Commitlog's archiving
and restore may be able to use the relevant code if it is completed.

Patrick McFadin 于2024年9月20日 周五上午2:01写道：

> Thanks for reviving this one!
>
> On Wed, Sep 18, 2024 at 12:06 AM guo Maxwell  wrote:
>
>> Is there any update on this topic?  It seems that things can make a big
>> progress if  Jake Luciani  can find someone who can make the
>> FileSystemProvider code accessible.
>>
>> Jon Haddad  于2023年12月16日周六 05:29写道：
>>
>>> At a high level I really like the idea of being able to better leverage
>>> cheaper storage especially object stores like S3.
>>>
>>> One important thing though - I feel pretty strongly that there's a big,
>>> deal breaking downside.   Backups, disk failure policies, snapshots and
>>> possibly repairs would get more complicated which haven't been particularly
>>> great in the past, and of course there's the issue of failure recovery
>>> being only partially possible if you're looking at a durable block store
>>> paired with an ephemeral one with some of your data not replicated to the
>>> cold side.  That introduces a failure case that's unacceptable for most
>>> teams, which results in needing to implement potentially 2 different backup
>>> solutions.  This is operationally complex with a lot of surface area for
>>> headaches.  I think a lot of teams would probably have an issue with the
>>> big question mark around durability and I probably would avoid it myself.
>>>
>>> On the other hand, I'm +1 if we approach it something slightly
>>> differently - where _all_ the data is located on the cold storage, with the
>>> local hot storage used as a cache.  This means we can use the cold
>>> directories for the complete dataset, simplifying backups and node
>>> replacements.
>>>
>>> For a little background, we had a ticket several years ago where I
>>> pointed out it was possible to do this *today* at the operating system
>>> level as long as you're using block devices (vs an object store) and LVM
>>> [1].  For example, this works well with GP3 EBS w/ low IOPS provisioning +
>>> local NVMe to get a nice balance of great read performance without going
>>> nuts on the cost for IOPS.  I also wrote about this in a little more detail
>>> in my blog [2].  There's also the new mount point tech in AWS which pretty
>>> much does exactly what I've suggested above [3] that's probably worth
>>> evaluating just to get a feel for it.
>>>
>>> I'm not insisting we require LVM or the AWS S3 fs, since that would rule
>>> out other cloud providers, but I am pretty confident that the entire
>>> dataset should reside in the "cold" side of things for the practical and
>>> technical reasons I listed above.  I don't think it massively changes the
>>> proposal, and should simplify things for everyone.
>>>
>>> Jon
>>>
>>> [1] https://rustyrazorblade.com/post/2018/2018-04-24-intro-to-lvm/
>>> [2] https://issues.apache.org/jira/browse/CASSANDRA-8460
>>> [3]
>>> https://aws.amazon.com/about-aws/whats-new/2023/03/mountpoint-amazon-s3/
>>>
>>>
>>> On Thu, Dec 14, 2023 at 1:56 AM Claude Warren  wrote:
>>>
>>>> Is there still interest in this?  Can we get some points down on
>>>> electrons so that we all understand the issues?
>>>>
>>>> While it is fairly simple to redirect the read/write to something
>>>> other  than the local system for a single node this will not solve the
>>>> problem for tiered storage.
>>>>
>>>> Tiered storage will require that on read/write the primary key be
>>>> assessed and determine if the read/write should be redirected.  My
>>>> reasoning for this statement is that in a cluster with a replication factor
>>>> greater than 1 the node will store data for the keys that would be
>>>> allocated to it in a cluster with a replication factor = 1, as well as some
>>>> keys from nodes earlier in the ring.
>>>>
>>>> Even if we can get the primary keys for all the data we want to write
>>>> to "cold storage" to map to a single node a replication factor > 1 means
>>>> that data will also be placed in "normal storage" on subsequent nodes.
>>>>
>>>> To overcome this, we have to explore ways to route data to different
>>>> storage based on the keys and that different storage may have to be
>>>> available on _all_  the nodes.
>>>>
>>>> Have any of the partial solutions mentioned in this email chain (or
>>>> others) solved this problem?
>>>>
>>>> Claude
>>>>
>>>

Re: [DISCUSS] Introduce CREATE TABLE LIKE grammer

[guo Maxwell]

No，I think still need some discuss on grammar detail after I finish the
first version

Patrick McFadin 于2024年9月20日 周五上午2:24写道：

> Is this CEP ready for a VOTE thread?
>
> On Sat, Aug 24, 2024 at 8:56 PM guo Maxwell  wrote:
>
>> Thank you for your replies, I will prepare a CEP later.
>>
>> Patrick McFadin  于2024年8月20日周二 02:11写道：
>>
>>> +1 This is a CEP
>>>
>>> On Mon, Aug 19, 2024 at 10:50 AM Jon Haddad  wrote:
>>>
>>>> Given the fairly large surface area for this, i think it should be a
>>>> CEP.
>>>>
>>>> —
>>>> Jon Haddad
>>>> Rustyrazorblade Consulting
>>>> rustyrazorblade.com
>>>>
>>>>
>>>> On Mon, Aug 19, 2024 at 10:44 AM Bernardo Botella <
>>>> conta...@bernardobotella.com> wrote:
>>>>
>>>>> Definitely a nice addition to CQL.
>>>>>
>>>>> Looking for inspiration at how Postgres and Mysql do that may also
>>>>> help with the final design (I like the WITH proposed by Stefan, but I 
>>>>> would
>>>>> definitely take a look at the INCLUDING keyword proposed by Postgres).
>>>>> https://www.postgresql.org/docs/current/sql-createtable.html
>>>>> https://dev.mysql.com/doc/refman/8.4/en/create-table-like.html
>>>>>
>>>>> On top of that, and as part of the interesting questions, I would like
>>>>> to add the permissions to the mix. Both the question about copying them
>>>>> over (with a WITH keyword probably), and the need for read permissions on
>>>>> the source table as well.
>>>>>
>>>>> Bernardo
>>>>>
>>>>> On Aug 19, 2024, at 10:01 AM, Štefan Miklošovič <
>>>>> smikloso...@apache.org> wrote:
>>>>>
>>>>> BTW this would be cool to do as well:
>>>>>
>>>>> ALTER TABLE ks.to_copy LIKE ks.tb WITH INDICES;
>>>>>
>>>>> This would mean that if we create a copy of a table, later we can
>>>>> decide that we need indices too, so we might "enrich" that table with
>>>>> indices from the old one without necessarily explicitly re-creating them 
>>>>> on
>>>>> that new table.
>>>>>
>>>>> On Mon, Aug 19, 2024 at 6:55 PM Štefan Miklošovič <
>>>>> smikloso...@apache.org> wrote:
>>>>>
>>>>>> I think this is an interesting idea worth exploring. I definitely
>>>>>> agree with Benjamin who raised important questions which needs to be
>>>>>> answered first. Also, what about triggers?
>>>>>>
>>>>>> It might be rather "easy" to come up with something simple but it
>>>>>> should be a comprehensive solution with predictable behavior we all agree
>>>>>> on.
>>>>>>
>>>>>> If a keyspace of a new table does not exist we would need to create
>>>>>> that one too before. For the simplicity, I would just make it a must to
>>>>>> create it on same keyspace. We might iterate on that in the future.
>>>>>>
>>>>>> UDTs are created per keyspace so there is nothing to re-create. We
>>>>>> just need to reference it from a new table, right?
>>>>>>
>>>>>> Indexes and MVs are interesting but in theory they might be
>>>>>> re-created too.
>>>>>>
>>>>>> Would it be appropriate to use something like this?
>>>>>>
>>>>>> CREATE TABLE ks.tb_copy LIKE ks.tb WITH INDEXES AND VIEWS AND
>>>>>> TRIGGERS 
>>>>>>
>>>>>> Without "WITH" it would just copy a table with nothing else.
>>>>>>
>>>>>> On Mon, Aug 19, 2024 at 6:10 PM guo Maxwell 
>>>>>> wrote:
>>>>>>
>>>>>>> Hello， everyone：
>>>>>>> As  Jira CASSANDRA-7662
>>>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-7662> has
>>>>>>> described , we would like to introduce a new grammer " CREATE TABLE 
>>>>>>> LIKE "
>>>>>>> ,which  simplifies creating new tables duplicating the existing ones .
>>>>>>> The format may be like : CREATE TABLE  LIKE 
>>>>>>>
>>>>>>> Before I implement this function, do you have any suggestions on
>>>>>>> this?
>>>>>>>
>>>>>>> Looking forward to your reply！
>>>>>>>
>>>>>>
>>>>>

Re: 【DISCUSS】The configuration of Commitlog archiving

[guo Maxwell]

ved somewhere else by a script NOT run as the C* user.
>>
> * The commit log archive configuration should not be dynamically
>> updatable, nor should any config affecting directories
>>
>
> This would essentially copy the logic we have for snapshots as Jordan
> mentioned. I do not mind having it like that. It is a good question for
> what exactly we need to have it reconfigurable. Why is it like that? People
> do not want to restart a whole cluster consisting of 100 nodes when the
> destination of the archived commit logs changed? How often is this
> happening that we need to expose ourselves to the problems related to that?
>
>
>>
>> Moving the scell configuration to Java code is a half measure that's only
>> solving a tiny problem in a massive chain of events and security holes.
>>
>> Jon
>>
>>
>>
>> On Tue, Sep 3, 2024 at 4:15 AM Štefan Miklošovič 
>> wrote:
>>
>>> Scott is right that this is also coming from us having a MBean method
>>> which allows commands to be changed in runtime. The solution to that was
>>> that we can prevent it from changing dynamically by having a configuration
>>> property, which is actually by default set to false so FQL archiving is
>>> ever possible only in case an operator explicitly enables that.
>>>
>>> However, even if commands were not modifiable in runtime via JMX and
>>> even an operator has a chance to enable command execution explicitly, that
>>> still does not make it 100% secure because an attacker does not need to
>>> change / modify cassandra.yaml where the script to execute is configure,
>>> just the content of such a script which is executed.
>>>
>>> So, introducing a similar property as it was done for FQL would in this
>>> context mean that it would be used for disabling commitlog archiving /
>>> restoring altogether while for FQL it would still do its thing, it would
>>> just not archive it. Whole commitlog archiving / restoring is now based on
>>> some commands to be executed so disabling commands being executed
>>> practically means we disabled this whole feature as such.
>>>
>>> We could indeed make it flat out impossible to execute anything but
>>> these scripts might contain some custom logic, like uploading to various
>>> cloud storages (AWS, Azure, GCP or something completely custom), people
>>> have their own "storage solutions" like remove the old logs when new come
>>> in etc. so by disabling this altogether we would make it impossible and
>>> users would need to accommodate that which would break their existing
>>> solutions.
>>>
>>> What I find confusing is that commitlog_archiving.properties is used
>>> both for restoration AS WELL AS for archiving. If we're ever going to
>>> change how this works, I think that it should be somehow logically split
>>> into archiving and restoring parts.
>>>
>>> So, we might introduce a property in cassandra.yaml to disable
>>> commitlog_archiving.properties altogether and we might deprecate
>>> commitlog_archiving.properties way of doing this (still keep it there for
>>> legacy reasons), add a new cassandra.yaml configuration section for that
>>> and there make the archiving and the restoration pluggable. By default we
>>> would provide "cp $from $to" implemented by Cassandra itself without any
>>> process invocation. Then we might eventually drop
>>> commitlog_archiving.properties but if the maintenance of that is cheap I
>>> would just keep it, we would just flip the switch so a new way of doing
>>> that would be preferable and the old way of doing it (via properties) would
>>> need to be explicitly enabled.
>>>
>>> On Tue, Sep 3, 2024 at 11:55 AM guo Maxwell 
>>> wrote:
>>>
>>>> Thank you very much for everyone's replies, they are all very valuable
>>>> feedback to me.
>>>>
>>>> I don't really understand what benefit adding restrictions would
>>>>> serve.  Would it be hard coded in C* itself, or configurable?  If it's
>>>>> configurable, then are we just making users enter their commands twice?
>>>>> This is meant to be used by an operator, so who's actually protected by an
>>>>> allow-list?
>>>>>
>>>>
>>>> I agree with you too, so I may prefer to idea 2 with implement
>>>> commitlog archiving in c* (not archiving by user defined shell),
>>>> and deprecate the  

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[guo Maxwell]

Is there any update on this topic?  It seems that things can make a big
progress if  Jake Luciani  can find someone who can make the
FileSystemProvider code accessible.

Jon Haddad  于2023年12月16日周六 05:29写道：

> At a high level I really like the idea of being able to better leverage
> cheaper storage especially object stores like S3.
>
> One important thing though - I feel pretty strongly that there's a big,
> deal breaking downside.   Backups, disk failure policies, snapshots and
> possibly repairs would get more complicated which haven't been particularly
> great in the past, and of course there's the issue of failure recovery
> being only partially possible if you're looking at a durable block store
> paired with an ephemeral one with some of your data not replicated to the
> cold side.  That introduces a failure case that's unacceptable for most
> teams, which results in needing to implement potentially 2 different backup
> solutions.  This is operationally complex with a lot of surface area for
> headaches.  I think a lot of teams would probably have an issue with the
> big question mark around durability and I probably would avoid it myself.
>
> On the other hand, I'm +1 if we approach it something slightly differently
> - where _all_ the data is located on the cold storage, with the local hot
> storage used as a cache.  This means we can use the cold directories for
> the complete dataset, simplifying backups and node replacements.
>
> For a little background, we had a ticket several years ago where I pointed
> out it was possible to do this *today* at the operating system level as
> long as you're using block devices (vs an object store) and LVM [1].  For
> example, this works well with GP3 EBS w/ low IOPS provisioning + local NVMe
> to get a nice balance of great read performance without going nuts on the
> cost for IOPS.  I also wrote about this in a little more detail in my blog
> [2].  There's also the new mount point tech in AWS which pretty much does
> exactly what I've suggested above [3] that's probably worth evaluating just
> to get a feel for it.
>
> I'm not insisting we require LVM or the AWS S3 fs, since that would rule
> out other cloud providers, but I am pretty confident that the entire
> dataset should reside in the "cold" side of things for the practical and
> technical reasons I listed above.  I don't think it massively changes the
> proposal, and should simplify things for everyone.
>
> Jon
>
> [1] https://rustyrazorblade.com/post/2018/2018-04-24-intro-to-lvm/
> [2] https://issues.apache.org/jira/browse/CASSANDRA-8460
> [3]
> https://aws.amazon.com/about-aws/whats-new/2023/03/mountpoint-amazon-s3/
>
>
> On Thu, Dec 14, 2023 at 1:56 AM Claude Warren  wrote:
>
>> Is there still interest in this?  Can we get some points down on
>> electrons so that we all understand the issues?
>>
>> While it is fairly simple to redirect the read/write to something other
>> than the local system for a single node this will not solve the problem for
>> tiered storage.
>>
>> Tiered storage will require that on read/write the primary key be
>> assessed and determine if the read/write should be redirected.  My
>> reasoning for this statement is that in a cluster with a replication factor
>> greater than 1 the node will store data for the keys that would be
>> allocated to it in a cluster with a replication factor = 1, as well as some
>> keys from nodes earlier in the ring.
>>
>> Even if we can get the primary keys for all the data we want to write to
>> "cold storage" to map to a single node a replication factor > 1 means that
>> data will also be placed in "normal storage" on subsequent nodes.
>>
>> To overcome this, we have to explore ways to route data to different
>> storage based on the keys and that different storage may have to be
>> available on _all_  the nodes.
>>
>> Have any of the partial solutions mentioned in this email chain (or
>> others) solved this problem?
>>
>> Claude
>>
>

Re: Welcome Chris Bannister, James Hartig, Jackson Flemming and João Reis, as cassandra-gocql-driver committers

[guo Maxwell]

Congratulations!

James Hartig  于2024年9月13日周五 08:11写道：

> Thanks everyone! Excited to contribute.
>
> On Thu, Sep 12, 2024, at 4:59 PM, Francisco Guerrero wrote:
>
> Congratulations!
>
> On 2024/09/12 11:39:40 Mick Semb Wever wrote:
> > The PMC's members are pleased to announce that Chris Bannister, James
> > Hartig, Jackson Flemming and João Reis have accepted invitations to
> become
> > committers on the Drivers subproject.
> >
> > Thanks a lot for everything you have done with the gocql driver all these
> > years.  We are very excited to see the driver now inside the Apache
> > Cassandra project.
> >
> > Congratulations and welcome!!
> >
> > The Apache Cassandra PMC
> >
>
>

Re: [DISCUSS] CASSANDRA-13704 Safer handling of out of range tokens

[guo Maxwell]

Sorry for sending an interfering email . Please ignore the above one. My
wrong operation cannot be undone.

 For this DISCUSS, I personally +1 with enabling rejection by default. We
did something similar to address this issue.
Although the default behavior may change. But what we solve is the
correctness of the data storage (although it may be a preventive solution),
and I think that should be the most important thing about the database, so
that other things may not be so important.


guo Maxwell  于2024年9月13日周五 10:50写道：

>
> +1，默认在所有分支上启用拒绝。我们做了类似的事情来解决这个问题。虽然默认行为可能会改变。但我们解决的是数据存储的正确性，我认为这应该是数据库最重要的事情，这样其他事情可能就不那么重要了。
>
> Josh McKenzie  于2024年9月13日周五 09:34写道：
>
>> 即使修复只是部分的，那么实际上它更多的是通过过于急切的不可用性更有力地提醒操作员问题……？
>>
>> 有时原则立场可能会使我们远离讨论中的重要细节。
>>
>> 我对此票的理解（没有深入研究代码，只是查看了 JIRA
>> 和此线程）是，这是我们在非确定性、非基于纪元、非事务性元数据系统中可以找到的最有效的解决方案。即
>> Gossip。我不认为这是一个部分修复，但我可能误解了。
>>
>> 我并不主张我们采取僵化的原则立场，拒绝一切细微差别，不讨论任何事物。我主张我们团结一致，除非有其他例外，否则都坚持共同的*默认*
>> 正确立场。我们知道我们是一个多元化的群体，我们都是不同的人，有着不同的历史/价值观/观点/文化，我认为这就是这个社区如此有效的原因。
>>
>> 但我 
>> 认为，根据数据丢失发生的时间长短或项目中某些人观察到某种现象的频率，反复重新争论数据丢失是否可以接受，这对我们*并不*健康。我的直觉告诉我，如果我们都从
>> 0
>> 开始讨论，比如“好吧，数据丢失是不可接受的。除非另有保证，否则我们应该尽一切努力在所有支持的分支上修复此问题，作为我们的默认响应”，那么我们都会处于更好的位置。
>>
>> 2024 年 9 月 12 日星期四晚上 9:02，C. Scott Andreas 写道：
>>
>> 感谢大家对此的讨论。
>>
>>
>> 当我意识到这个问题有多么普遍，以及要证明自己遇到了这个错误有多么困难时，我很难描述那种沉重的感觉。
>>
>>
>> 两年前，我的理解是，这是一个极其罕见且短暂的问题，在我们为 Gossip 投入大量精力之后，不太可能发生。我的观点是，Gossip
>> 基本上已经被解决了，而事务元数据正是通过其纪元设计来解决这个问题的正确方法（这是真的）。
>>
>>
>> Since that time, I’ve received several urgent messages from major users
>> of Apache Cassandra and even customers of Cassandra ecosystem vendors
>> asking about this bug. Some were able to verify the presence of lost data
>> in SSTables on nodes where it didn’t belong, demonstrate empty read
>> responses for data that is known proof-positive to exist (think
>> content-addressable stores), or reproduce this behavior in a local cluster
>> after forcing disagreement.
>>
>>
>> The severity and frequency of this issue combined with the business risk
>> to Apache Cassandra users changed my mind about fixing it in earlier
>> branches despite TCM having been merged to fix it for good on trunk.
>>
>>
>> The guards in this patch are extensive: point reads, range reads,
>> mutations, repair, incoming / outgoing streams, hints, merkle tree
>> requests, and others I’m forgetting. They’re simple guards, and while they
>> touch many subsystems, they’re not invasive changes.
>>
>>
>> There is no reasonable scenario that’s common enough that would justify
>> disabling a guard preventing silent data loss by default. I appreciate that
>> a prop exists to permit or warn in the presence of data loss for anyone who
>> may want that, in the spirit of users being in control of their clusters’
>> behavior.
>>
>>
>> Very large operators may only see indications the guard took effect for a
>> handful of queries per day — but in instances where ownership disagreement
>> is prolonged, the patch is an essential guard against large-scale
>> unrecoverable data loss and incorrect responses to queries. I’ll further
>> take the position that those few queries in transient disagreement
>> scenarios would be justification by themselves.
>>
>>
>> I support merging the patch to all proposed branches and enabling the
>> guard by default.
>>
>>
>> – Scott
>>
>> On Sep 12, 2024, at 3:40 PM, Jeremiah Jordan 
>> wrote:
>>
>> 
>>
>> 1. Rejecting writes does not prevent data loss in this situation.  It
>> only reduces it.  The investigation and remediation of possible mislocated
>> data is still required.
>>
>>
>> All nodes which reject a write prevent mislocated data.  There is still
>> the possibility of some node having the same wrong view of the ring as the
>> coordinator (including if they are the same node) accepting data.  Unless
>> there are multiple nodes with the same wrong view of the ring, data loss is
>> prevented for CL > ONE.
>>
>> 2. Rejecting writes is a louder form of alerting for users unaware of the
>> scenario, those not already monitoring logs or metrics.
>>
>>
>> Without this patch no one is aware of any issues at all.  Maybe you are
>> referring to a situation where the patch is applied, but the default
>> behavior is to still accept the “bad” data?  In that case yes, turning on
>> rejection makes it “louder” in that your queries can fail if too many nodes
>> are wrong.
>>
>> 3. Rejecting writes does not capture all places

Re: [DISCUSS] CASSANDRA-13704 Safer handling of out of range tokens

[guo Maxwell]

+1，默认在所有分支上启用拒绝。我们做了类似的事情来解决这个问题。虽然默认行为可能会改变。但我们解决的是数据存储的正确性，我认为这应该是数据库最重要的事情，这样其他事情可能就不那么重要了。

Josh McKenzie  于2024年9月13日周五 09:34写道：

> 即使修复只是部分的，那么实际上它更多的是通过过于急切的不可用性更有力地提醒操作员问题……？
>
> 有时原则立场可能会使我们远离讨论中的重要细节。
>
> 我对此票的理解（没有深入研究代码，只是查看了 JIRA
> 和此线程）是，这是我们在非确定性、非基于纪元、非事务性元数据系统中可以找到的最有效的解决方案。即
> Gossip。我不认为这是一个部分修复，但我可能误解了。
>
> 我并不主张我们采取僵化的原则立场，拒绝一切细微差别，不讨论任何事物。我主张我们团结一致，除非有其他例外，否则都坚持共同的*默认*
> 正确立场。我们知道我们是一个多元化的群体，我们都是不同的人，有着不同的历史/价值观/观点/文化，我认为这就是这个社区如此有效的原因。
>
> 但我 
> 认为，根据数据丢失发生的时间长短或项目中某些人观察到某种现象的频率，反复重新争论数据丢失是否可以接受，这对我们*并不*健康。我的直觉告诉我，如果我们都从
> 0
> 开始讨论，比如“好吧，数据丢失是不可接受的。除非另有保证，否则我们应该尽一切努力在所有支持的分支上修复此问题，作为我们的默认响应”，那么我们都会处于更好的位置。
>
> 2024 年 9 月 12 日星期四晚上 9:02，C. Scott Andreas 写道：
>
> 感谢大家对此的讨论。
>
>
> 当我意识到这个问题有多么普遍，以及要证明自己遇到了这个错误有多么困难时，我很难描述那种沉重的感觉。
>
>
> 两年前，我的理解是，这是一个极其罕见且短暂的问题，在我们为 Gossip 投入大量精力之后，不太可能发生。我的观点是，Gossip
> 基本上已经被解决了，而事务元数据正是通过其纪元设计来解决这个问题的正确方法（这是真的）。
>
>
> Since that time, I’ve received several urgent messages from major users of
> Apache Cassandra and even customers of Cassandra ecosystem vendors asking
> about this bug. Some were able to verify the presence of lost data in
> SSTables on nodes where it didn’t belong, demonstrate empty read responses
> for data that is known proof-positive to exist (think content-addressable
> stores), or reproduce this behavior in a local cluster after forcing
> disagreement.
>
>
> The severity and frequency of this issue combined with the business risk
> to Apache Cassandra users changed my mind about fixing it in earlier
> branches despite TCM having been merged to fix it for good on trunk.
>
>
> The guards in this patch are extensive: point reads, range reads,
> mutations, repair, incoming / outgoing streams, hints, merkle tree
> requests, and others I’m forgetting. They’re simple guards, and while they
> touch many subsystems, they’re not invasive changes.
>
>
> There is no reasonable scenario that’s common enough that would justify
> disabling a guard preventing silent data loss by default. I appreciate that
> a prop exists to permit or warn in the presence of data loss for anyone who
> may want that, in the spirit of users being in control of their clusters’
> behavior.
>
>
> Very large operators may only see indications the guard took effect for a
> handful of queries per day — but in instances where ownership disagreement
> is prolonged, the patch is an essential guard against large-scale
> unrecoverable data loss and incorrect responses to queries. I’ll further
> take the position that those few queries in transient disagreement
> scenarios would be justification by themselves.
>
>
> I support merging the patch to all proposed branches and enabling the
> guard by default.
>
>
> – Scott
>
> On Sep 12, 2024, at 3:40 PM, Jeremiah Jordan 
> wrote:
>
> 
>
> 1. Rejecting writes does not prevent data loss in this situation.  It only
> reduces it.  The investigation and remediation of possible mislocated data
> is still required.
>
>
> All nodes which reject a write prevent mislocated data.  There is still
> the possibility of some node having the same wrong view of the ring as the
> coordinator (including if they are the same node) accepting data.  Unless
> there are multiple nodes with the same wrong view of the ring, data loss is
> prevented for CL > ONE.
>
> 2. Rejecting writes is a louder form of alerting for users unaware of the
> scenario, those not already monitoring logs or metrics.
>
>
> Without this patch no one is aware of any issues at all.  Maybe you are
> referring to a situation where the patch is applied, but the default
> behavior is to still accept the “bad” data?  In that case yes, turning on
> rejection makes it “louder” in that your queries can fail if too many nodes
> are wrong.
>
> 3. Rejecting writes does not capture all places where the problem is
> occurring.  Only logging/metrics fully captures everywhere the problem is
> occurring.
>
>
> Not sure what you are saying here.
>
> nodes can be rejecting writes when they are in fact correct hence causing 
> “over-eager
> unavailability”.
>
>
> When would this occur?  I guess when the node with the bad ring
> information is a replica sent data from a coordinator with the correct ring
> state?  There would be no “unavailability” here unless there were multiple
> nodes in such a state.  I also again would not call this over eager,
> because the node with the bad ring state is f’ed up and needs to be fixed.
> So if being considered unavailable doesn’t seem over-eager to me.
>
> Given the fact that a user can read NEWS.txt and turn off this rejection
> of writes, I see no reason not to err on the side of “the setting which
> gives better protection even if it is not perfect”.  We should not let the
> want to solve everything prevent incremental improvements, especially when
> we actually do have the solution coming in TCM.
>
> -Jeremiah
>
> On Sep 12, 2024 at 5:25:25 PM, Mick Semb Wever  wrote:
>
>
> I'm less concerned with what the defaults are in each branch, and more the
> accuracy of

Re: 【DISCUSS】The configuration of Commitlog archiving

[guo Maxwell]

Yes, commitlog_archiving.properties is used  both for archiving and
restoration for commitlog, but we don't have a tool or interface to do
commitlog restore online which I have created the commitlog tool issue
CASSANDRA-15156 <https://issues.apache.org/jira/browse/CASSANDRA-15156> .
>
> By default we would provide "cp $from $to" implemented by Cassandra itself
> without any process invocation.
>

What about providing a hard link for commitlog archive by default, if we
need other features, such as log compression, we may need to specify the
copy to a specific location (with sufficient capacity).

Štefan Miklošovič  于2024年9月3日周二 19:15写道：

> Scott is right that this is also coming from us having a MBean method
> which allows commands to be changed in runtime. The solution to that was
> that we can prevent it from changing dynamically by having a configuration
> property, which is actually by default set to false so FQL archiving is
> ever possible only in case an operator explicitly enables that.
>
> However, even if commands were not modifiable in runtime via JMX and even
> an operator has a chance to enable command execution explicitly, that still
> does not make it 100% secure because an attacker does not need to change /
> modify cassandra.yaml where the script to execute is configure, just the
> content of such a script which is executed.
>
> So, introducing a similar property as it was done for FQL would in this
> context mean that it would be used for disabling commitlog archiving /
> restoring altogether while for FQL it would still do its thing, it would
> just not archive it. Whole commitlog archiving / restoring is now based on
> some commands to be executed so disabling commands being executed
> practically means we disabled this whole feature as such.
>
> We could indeed make it flat out impossible to execute anything but these
> scripts might contain some custom logic, like uploading to various cloud
> storages (AWS, Azure, GCP or something completely custom), people have
> their own "storage solutions" like remove the old logs when new come in
> etc. so by disabling this altogether we would make it impossible and users
> would need to accommodate that which would break their existing solutions.
>
> What I find confusing is that commitlog_archiving.properties is used both
> for restoration AS WELL AS for archiving. If we're ever going to change how
> this works, I think that it should be somehow logically split into
> archiving and restoring parts.
>
> So, we might introduce a property in cassandra.yaml to disable
> commitlog_archiving.properties altogether and we might deprecate
> commitlog_archiving.properties way of doing this (still keep it there for
> legacy reasons), add a new cassandra.yaml configuration section for that
> and there make the archiving and the restoration pluggable. By default we
> would provide "cp $from $to" implemented by Cassandra itself without any
> process invocation. Then we might eventually drop
> commitlog_archiving.properties but if the maintenance of that is cheap I
> would just keep it, we would just flip the switch so a new way of doing
> that would be preferable and the old way of doing it (via properties) would
> need to be explicitly enabled.
>
> On Tue, Sep 3, 2024 at 11:55 AM guo Maxwell  wrote:
>
>> Thank you very much for everyone's replies, they are all very valuable
>> feedback to me.
>>
>> I don't really understand what benefit adding restrictions would serve.
>>> Would it be hard coded in C* itself, or configurable?  If it's
>>> configurable, then are we just making users enter their commands twice?
>>> This is meant to be used by an operator, so who's actually protected by an
>>> allow-list?
>>>
>>
>> I agree with you too, so I may prefer to idea 2 with implement commitlog
>> archiving in c* (not archiving by user defined shell), and deprecate the
>> commitlog_archiving.properties
>> <https://github.com/apache/cassandra/blob/trunk/conf/commitlog_archiving.properties#L28>
>>  configuration
>> through which  we  can set the properties of commitlog archiving. This view
>> may be similar to that of Scott.
>>
>> If I want to use rclone or aws-cli to archive my commit logs that's my
>>> prerogative.
>>>
>>
>> Yes, it may be very flexible if we set aws-cli in shell. But as I know
>> cassandra-medusa can also do this , and for me letting other tools to do
>> this work may be better , for example we can upload more than one log (if
>> log size is not big ) in a rpc to improve write throughput.
>>
>> I think we can divide this big task into several s

Re: 【DISCUSS】The configuration of Commitlog archiving

[guo Maxwell]

Thank you very much for everyone's replies, they are all very valuable
feedback to me.

I don't really understand what benefit adding restrictions would serve.
> Would it be hard coded in C* itself, or configurable?  If it's
> configurable, then are we just making users enter their commands twice?
> This is meant to be used by an operator, so who's actually protected by an
> allow-list?
>

I agree with you too, so I may prefer to idea 2 with implement commitlog
archiving in c* (not archiving by user defined shell), and deprecate the
commitlog_archiving.properties
<https://github.com/apache/cassandra/blob/trunk/conf/commitlog_archiving.properties#L28>
configuration
through which  we  can set the properties of commitlog archiving. This view
may be similar to that of Scott.

If I want to use rclone or aws-cli to archive my commit logs that's my
> prerogative.
>

Yes, it may be very flexible if we set aws-cli in shell. But as I know
cassandra-medusa can also do this , and for me letting other tools to do
this work may be better , for example we can upload more than one log (if
log size is not big ) in a rpc to improve write throughput.

I think we can divide this big task into several subtasks:

   - Add this feature that Stefan mentioned before for commitlog archive
   CASSANDRA-18550 <https://issues.apache.org/jira/browse/CASSANDRA-18550> in
   5.x  and may the original commitlog_archiving.properties  deprecate.
   - Add the feature of archiving for cassandra (commitlog/query log/or
   event sstable) in the long run such as 6.0.

I can prepare a cep if necessary. Looking forward to your feedback.


We can divide this task into several subtasks and complete them step by step



Jordan West  于2024年9月3日周二 00:55写道：

> +1 to Scott’s comments. Once you expose those YAML config params outside
> of a single node which many of us do, this becomes an RCE attack vector.
> Something more structured as Scott proposes, similar to snapshots, would be
> preferred. Would recommend a CEP.
>
> Jordan
>
> On Fri, Aug 30, 2024 at 20:58 C. Scott Andreas  wrote:
>
>> I appreciate this report and would love to work toward the direction it
>> recommends.
>>
>> I’m also familiar with past concerns raised by others with our FQL
>> configuration parameters that allow passing shell commands for FQL segment
>> archival.
>>
>> We bias toward ensuring an MBean exists for dynamic modification of yaml
>> parameters. When we couple dynamic configuration updates and arbitrary
>> shell command execution, we introduce vectors for arbitrary code execution,
>> data exfiltration, and data compromise that have a lower bar to achieve
>> than local file write.
>>
>> I agree that we should work toward removing operator-provided shell
>> commands in yaml.
>>
>> For concerns like archival, these seem like areas that Cassandra could
>> easily accomplish itself without shelling out to gzip/zstd/lz4-compress a
>> file. Introducing a new config structure that declares an archival format,
>> accompanying implementations for compression/decompression, and deprecation
>> of the prior approach sounds both reasonable and desirable to me.
>>
>> – Scott
>>
>> —
>> Mobile
>>
>> On Aug 30, 2024, at 10:25 PM, Bowen Song via dev <
>> dev@cassandra.apache.org> wrote:
>>
>> 
>>
>> I'm not sure what is the concern here. Is it a malicious user exploiting
>> this? Or human error with unintended consequences?
>>
>> For malicious user, in order to exploit this, an attacker needs to be
>> able to write to the config file. The config file on Linux by default is
>> owned by the root user and has the -rw-r--r-- permission, that means the
>> attacker must either gain root access to the system or has the ability to
>> write arbitrary file on the filesystem. With either of these permission,
>> they can already do almost anything they want (e.g. modify a SUID
>> executable file). They wouldn't even need to exploit this to run a script
>> or dangerous command. So this sounds like a non-issue to me, at least on
>> Linux-based OSes.
>>
>> For human error, if the operator puts "rm -rf" in it, the software should
>> treat it as the operator actually wants to do that. I personally don't like
>> software attempting to outsmart human, which often ends up interfering with
>> legitimate use cases. The best thing a software can do is log it, so
>> there's some traceability if and when things go wrong.
>>
>> So, IMO, there's nothing wrong with the implementation in Cassandra.
>>
>>
>> On 30/08/2024 17:13, guo Maxwell wrote:
>>
>>   

Re: Welcome Jordan West and Stefan Miklosovic as Cassandra PMC members!

[guo Maxwell]

Congrats Stefan and Jordan!!!

Jacek Lewandowski 于2024年9月1日 周日下午4:39写道：

> Congrats Stefan and Jordan!!! This is great!
>
> sob., 31 sie 2024, 22:21 użytkownik Jordan West 
> napisał:
>
>> Thanks all!!!
>>
>> On Sat, Aug 31, 2024 at 07:55 J. D. Jordan 
>> wrote:
>>
>>> Two great additions to the PMC. Congratulations to you both!
>>>
>>> -Jeremiah Jordan
>>>
>>> > On Aug 30, 2024, at 3:21 PM, Jon Haddad  wrote:
>>> >
>>> > 
>>> > The PMC's members are pleased to announce that Jordan West and Stefan
>>> Miklosovic have accepted invitations to become PMC members.
>>> >
>>> > Thanks a lot, Jordan and Stefan, for everything you have done for the
>>> project all these years.
>>> >
>>> > Congratulations and welcome!!
>>> >
>>> > The Apache Cassandra PMC
>>>
>>

Re: [DISCUSS] Introduce CREATE TABLE LIKE grammer

[guo Maxwell]

Thank you for your replies, I will prepare a CEP later.

Patrick McFadin  于2024年8月20日周二 02:11写道：

> +1 This is a CEP
>
> On Mon, Aug 19, 2024 at 10:50 AM Jon Haddad  wrote:
>
>> Given the fairly large surface area for this, i think it should be a CEP.
>>
>> —
>> Jon Haddad
>> Rustyrazorblade Consulting
>> rustyrazorblade.com
>>
>>
>> On Mon, Aug 19, 2024 at 10:44 AM Bernardo Botella <
>> conta...@bernardobotella.com> wrote:
>>
>>> Definitely a nice addition to CQL.
>>>
>>> Looking for inspiration at how Postgres and Mysql do that may also help
>>> with the final design (I like the WITH proposed by Stefan, but I would
>>> definitely take a look at the INCLUDING keyword proposed by Postgres).
>>> https://www.postgresql.org/docs/current/sql-createtable.html
>>> https://dev.mysql.com/doc/refman/8.4/en/create-table-like.html
>>>
>>> On top of that, and as part of the interesting questions, I would like
>>> to add the permissions to the mix. Both the question about copying them
>>> over (with a WITH keyword probably), and the need for read permissions on
>>> the source table as well.
>>>
>>> Bernardo
>>>
>>> On Aug 19, 2024, at 10:01 AM, Štefan Miklošovič 
>>> wrote:
>>>
>>> BTW this would be cool to do as well:
>>>
>>> ALTER TABLE ks.to_copy LIKE ks.tb WITH INDICES;
>>>
>>> This would mean that if we create a copy of a table, later we can decide
>>> that we need indices too, so we might "enrich" that table with indices from
>>> the old one without necessarily explicitly re-creating them on that new
>>> table.
>>>
>>> On Mon, Aug 19, 2024 at 6:55 PM Štefan Miklošovič <
>>> smikloso...@apache.org> wrote:
>>>
>>>> I think this is an interesting idea worth exploring. I definitely agree
>>>> with Benjamin who raised important questions which needs to be answered
>>>> first. Also, what about triggers?
>>>>
>>>> It might be rather "easy" to come up with something simple but it
>>>> should be a comprehensive solution with predictable behavior we all agree
>>>> on.
>>>>
>>>> If a keyspace of a new table does not exist we would need to create
>>>> that one too before. For the simplicity, I would just make it a must to
>>>> create it on same keyspace. We might iterate on that in the future.
>>>>
>>>> UDTs are created per keyspace so there is nothing to re-create. We just
>>>> need to reference it from a new table, right?
>>>>
>>>> Indexes and MVs are interesting but in theory they might be re-created
>>>> too.
>>>>
>>>> Would it be appropriate to use something like this?
>>>>
>>>> CREATE TABLE ks.tb_copy LIKE ks.tb WITH INDEXES AND VIEWS AND TRIGGERS
>>>> 
>>>>
>>>> Without "WITH" it would just copy a table with nothing else.
>>>>
>>>> On Mon, Aug 19, 2024 at 6:10 PM guo Maxwell 
>>>> wrote:
>>>>
>>>>> Hello， everyone：
>>>>> As  Jira CASSANDRA-7662
>>>>> <https://issues.apache.org/jira/browse/CASSANDRA-7662> has described
>>>>> , we would like to introduce a new grammer " CREATE TABLE LIKE "
>>>>> ,which  simplifies creating new tables duplicating the existing ones .
>>>>> The format may be like : CREATE TABLE  LIKE 
>>>>>
>>>>> Before I implement this function, do you have any suggestions on this?
>>>>>
>>>>> Looking forward to your reply！
>>>>>
>>>>
>>>

[DISCUSS] Introduce CREATE TABLE LIKE grammer

[guo Maxwell]

Hello， everyone：
As  Jira CASSANDRA-7662
 has described , we
would like to introduce a new grammer " CREATE TABLE LIKE "
,which  simplifies creating new tables duplicating the existing ones .
The format may be like : CREATE TABLE  LIKE 

Before I implement this function, do you have any suggestions on this?

Looking forward to your reply！

Re: [DISCUSS] New CQL command/option for listing roles with superuser privileges

[guo Maxwell]

Hi ,
 1. can this cql "SELECT role from system_auth.roles where is_superuser =
True ALLOW FILTERING ;"  meet your needs if the user to execute the cql
have the right to do so  ？
 2. I think may be we can also add the ability to filter on list role/user
grammar, for example : list user where super = True;



Shailaja Koppu  于2024年2月28日周三 20:40写道：

> Hi Team,
>
> Currently LIST ROLES command doesn’t indicate if a role has superuser
> privilege, if acquired through a grant in roles hierarchy (LIST ROLES has
> super column true only if the role is created with SUPERUSER=true). For
> example, in the below example, super1 is a superuser, role1 acquired
> superuser status through grant of super1 and role11 acquired superuser
> status through grant of role1. LIST ROLES output has super column true only
> for super1.
>
>
> cassandra@cqlsh> create role super1 WITH SUPERUSER = true;
> cassandra@cqlsh> create role role1;
> cassandra@cqlsh> create role role11;
> cassandra@cqlsh> create role role2;
> cassandra@cqlsh> grant super1 to role1;
> cassandra@cqlsh> grant role1 to role11;
> cassandra@cqlsh> list roles;
>
>  role  | super | login | options | datacenters
> ---+---+---+-+-
>  role1 | False | False |{} | ALL
> role11 | False | False |{} | ALL
>  role2 | False | False |{} | ALL
> super1 |  True | False |{} | ALL
>
>
> One way to check has a role acquired superuser status is by running LIST
> ROLES of  and looking for at least one row with super column
> true. This works fine to check superuser status of a given role.
>
> cassandra@cqlsh> list roles of role11;
>
>  role   | super | login | options | datacenters
> +---+---+-+-
>   role1 | False | False |{} | ALL
>  role11 | False | False |{} | ALL
>  super1 |  True | False |{} | ALL
>
>
> But if we need to get list of all roles having superuser status (acquired
> through grant as well), there is no easy way to retrieve this from C*. This
> can be a requirement for an external service interacting with C* and
> performing their own checks (for example, Sidecar). So I am proposing a new
> CQL command LIST SUPERUSERS, which lists all roles having superuser status
> (acquired as well). We will ensure that the user running this command has
> DESCRIBE permission on root roles resource, i.e, to run this command user
> must be either a superuser or granted DESCRIBE permission on ALL ROLES.
> Here is the Jira  and
> sample output for the above example.
>
> cassandra@cqlsh> list superusers;
>
>  role
> ---
>  role1
> role11
> super1
>
> Alternatives thought of so far,
> - LIST ROLES SUPERUSERSONLY
> - LIST ROLES superuseronly=true
> - LIST USERS superuseronly=true  command : I have a question here, is
> LIST USERS command deprecated? I see this link saying that
> https://docs.datastax.com/en/cql-oss/3.3/cql/cql_reference/cqlListUsers.html.
> if LIST USERS and LISR ROLES commands are same, why don’t we just pick one
> so we don’t have to maintain two different commands ?
> - LIST ROLES command default i.e, without NORECURSIVE clause : to print
> super column true for acquired superusers as well, but this may break
> existing tools/scripts of customers as we are changing the default
> behavior
>
>
> I prefer LIST SUPERUSERS command because - This command looks neat and
> simple and we don’t have to worry about handling/breaking other
> options/columns supported by these existing commands. For example we don’t
> have to worry about handling/breaking OF clause of LIST ROLES command. And
> any new options we add to these commands in the future, don’t have to worry
> about handling/breaking of SUPERUSERS option. Please let me know your
> thoughts on this.
>
>
> Thanks,
> Shailaja
>
>
>
>

Re: Welcome Maxim Muzafarov as Cassandra Committer

[guo Maxwell]

Congratulations, Maxim!

Francisco Guerrero  于2024年1月9日周二 09:00写道：

> Congratulations, Maxim! Well deserved!
>
> On 2024/01/08 18:19:04 Josh McKenzie wrote:
> > The Apache Cassandra PMC is pleased to announce that Maxim Muzafarov has
> accepted
> > the invitation to become a committer.
> >
> > Thanks for all the hard work and collaboration on the project thus far,
> and we're all looking forward to working more with you in the future.
> Congratulations and welcome!
> >
> > The Apache Cassandra PMC members
> >
> >
>

Re: [DISCUSS] Replace Sigar with OSHI (CASSANDRA-16565)

[guo Maxwell]

+1 too

Mick Semb Wever  于2023年12月15日周五 10:01写道：

>
>
>
>>
>> Are there objections to making this switch and adding a new dependency?
>>
>> [1] https://github.com/apache/cassandra/pull/2842/files
>> [2] https://issues.apache.org/jira/browse/CASSANDRA-16565
>>
>
>
>
> +1 to removing sigar and to add oshi-core
>
>

Re: [DISCUSS] CEP-39: Cost Based Optimizer

[guo Maxwell]

calcite has it‘s own way of sql parser which used javacc that is different
from antlr. though I think calcite is a very good sql framework in my
opinion.
We have tried to use calcite to develop our own sql which can both support
cql ，and some normal sql like presto sql ，also we want to take advantage of
it‘s complex sql feature like join window function. we tried but failed，it
is not a good way for us

Maxim Muzafarov 于2023年12月13日 周三下午10:43写道：

> Hello Benjamin,
>
> Can you share the reasons why Apache Calcite is not suitable for this
> case and why it was rejected? It has custom syntax support, CBO, so I
> am interested to see some technical details in the "Rejected
> Alternatives" section, I'm pretty sure they exist, but they weren't
> mentioned there, and don't take this as an ad, please :-)
>
> In Apache Ignite, I had experience in improving the query execution
> engine and one of the reasons for moving from one query engine to
> another (to Calcite, to be precise), was that we had a problem with
> calculating memory quotas for queries and aborting a query when those
> quotas were exceeded the limit. An engine can load and hold rows in
> memory, preventing the GC from collecting them, or objects that are
> too large, so the JVM can easily run out of memory, and it is
> important to have full control over a query execution path.
>
> btw, here is a Calcite adapter for Cassandra:
> https://calcite.apache.org/docs/cassandra_adapter.html
>
> On Wed, 13 Dec 2023 at 09:55, Benedict  wrote:
> >
> > A CBO can only make worse decisions than the status quo for what I
> presume are the majority of queries - i.e. those that touch only primary
> indexes. In general, there are plenty of use cases that prefer determinism.
> So I agree that there should at least be a CBO implementation that makes
> the same decisions as the status quo, deterministically.
> >
> >
> > I do support the proposal, but would like to see some elements discussed
> in more detail. The maintenance and distribution of summary statistics in
> particular is worthy of its own CEP, and it might be preferable to split it
> out. The proposal also seems to imply we are aiming for coordinators to all
> make the same decision for a query, which I think is challenging, and it
> would be worth fleshing out the design here a little (perhaps just in Jira).
> >
> >
> > While I’m not a fan of ALLOW FILTERING, I’m not convinced that this CEP
> deprecates it. It is a concrete qualitative guard rail, that I expect some
> users will prefer to a cost-based guard rail. Perhaps this could be left to
> the CBO to decide how to treat.
> >
> >
> > There’s also not much discussion of the execution model: I think it
> would make most sense for this to be independent of any cost and optimiser
> models (though they might want to operate on them), so that EXPLAIN and
> hints can work across optimisers (a suitable hint might essentially bypass
> the optimiser, if the optimiser permits it, by providing a standard
> execution model)
> >
> >
> > I think it would be worth considering providing the execution plan to
> the client as part of query preparation, as an opaque payload to supply to
> coordinators on first contact, as this might simplify the problem of
> ensuring queries behave the same without adopting a lot of complexity for
> synchronising statistics (which will never provide strong guarantees). Of
> course, re-preparing a query might lead to a new plan, though any
> coordinators with the query in their cache should be able to retrieve it
> cheaply. If the execution model is efficiently serialised this might have
> the ancillary benefit of improving the occupancy of our prepared query
> cache.
> >
> >
> > On 13 Dec 2023, at 00:44, Jon Haddad  wrote:
> >
> > 
> > I think it makes sense to see what the actual overhead is of CBO before
> making the assumption it'll be so high that we need to have two code
> paths.  I'm happy to provide thorough benchmarking and analysis when it
> reaches a testing phase.
> >
> > I'm excited to see where this goes.  I think it sounds very forward
> looking and opens up a lot of possibilities.
> >
> > Jon
> >
> > On Tue, Dec 12, 2023 at 4:25 PM guo Maxwell 
> wrote:
> >>
> >> Nothing expresses my thoughts better than +1
> >> ，It feels like it means a lot to Cassandra.
> >>
> >> I have a question. Is it easy to turn off cbo's optimizer or by pass in
> some way? Because some simple read and write requests will have better
> performance without cbo, which is also the advantage of Cassandra compared
> to some rdbms.
> >>
> >&

Re: [DISCUSS] CEP-39: Cost Based Optimizer

[guo Maxwell]

Nothing expresses my thoughts better than +1
，It feels like it means a lot to Cassandra.

I have a question. Is it easy to turn off cbo's optimizer or by pass in
some way? Because some simple read and write requests will have better
performance without cbo, which is also the advantage of Cassandra compared
to some rdbms.


David Capwell 于2023年12月13日 周三上午3:37写道：

> Overall LGTM.
>
>
> On Dec 12, 2023, at 5:29 AM, Benjamin Lerer  wrote:
>
> Hi everybody,
>
> I would like to open the discussion on the introduction of a cost based
> optimizer to allow Cassandra to pick the best execution plan based on the
> data distribution.Therefore, improving the overall query performance.
>
> This CEP should also lay the groundwork for the future addition of
> features like joins, subqueries, OR/NOT and index ordering.
>
> The proposal is here:
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-39%3A+Cost+Based+Optimizer
>
> Thank you in advance for your feedback.
>
>
>

Re: Welcome Mike Adamson as Cassandra committer

[guo Maxwell]

Congrats Mike!

Jasonstack Zhao Yang  于2023年12月9日周六 20:59写道：

> Congrats Mike!
>
> On Sat, 9 Dec 2023 at 8:47 PM, Piotr Kołaczkowski 
> wrote:
>
>> Congratulations, Mike! Well deserved, working with you has always been a
>> pleasure!
>>
>>
>> Wiadomość napisana przez Melissa Logan  w dniu
>> 09.12.2023, o godz. 02:35:
>>
>> 
>>
>> Congratulations, Mike!
>>
>> On Fri, Dec 8, 2023 at 11:13 AM David Capwell  wrote:
>>
>>> Congrats!
>>>
>>> On Dec 8, 2023, at 11:00 AM, Lorina Poland  wrote:
>>>
>>> Congratulations, Mike!
>>>
>>>
>>>

Re: Downgradability

[guo Maxwell]

Hello everyone :
What is the final conclusion of this discuss ?

As https://issues.apache.org/jira/browse/CASSANDRA-18934 has been created ,
and we know that for system table of different c* version , the schema info
may be different as we may
add or delete column or modify the table properties in different versions.
For 18934 , because I have introduced a new column " compaction_properties"
for compaction_history table in 5.0. So the downgrade test to 4.1 failed at
the 4.1 node startup stage  due to  4.1’s code encountering an incompatible
persisted column "compaction_properties".

So now for me, I might want to know, which solution did everyone finally
decide to use for the sstable ?

1. Are our current downgrades intended to only support specific version
downgrades, like 5.0 -> 4.1 ?
2.Can we modify the low version code now ? that is say , if we add some
feature  for example in 4.1.0 and that means we may only support downgrade
to 4.1.1
3.If we choose to rewrite sstable, please help consider the differences in
system tables between different versions.

Jacek Lewandowski  于2023年3月6日周一 16:58写道：

> A bit of organization - I've created
> https://issues.apache.org/jira/browse/CASSANDRA-18300 epic to track
> tickets related to the downgradability. Please add the tickets you are
> aware of.
>
> thanks
> - - -- --- -  -
> Jacek Lewandowski
>
>
> czw., 23 lut 2023 o 17:47 Benedict  napisał(a):
>
>> Either way, it feels like this has become much more of a big deal than it
>> needed to.
>>
>> I would prefer the pending patches to avoid breaking compatibility, as I
>> think they can do it easily. But, if we agree to block release until we can
>> double back to fix it with versioned writing (which I agree with Jacek are
>> LHF - I think we literally just need a method that chooses the descriptor
>> version) then let’s not further agonise over this.
>>
>> Alternatively I’d be happy to work with the authors to get this done
>> alongside their work, as I don’t think it would hold anything up. We just
>> need something to pick a descriptor besides latest on write, everything
>> else is basically there for these patches.
>>
>>
>> On 23 Feb 2023, at 16:37, Henrik Ingo  wrote:
>>
>> 
>> Right. So I'm speculating everyone else who worked on a patch that breaks
>> compatibility has been working under the mindset "I'll just put this behind
>> the same switch". Or something more vague / even less correct, such as
>> assuming that tries would become the default immediately.
>>
>> At least in my mind when we talk about the "switch to enable tries" I do
>> also consider things like "don't break streaming". So I guess whether one
>> considers "a switch" to exist already or not, might be subjective in this
>> case, because people have different assumptions on the definition of done
>> of such a switch.
>>
>> henrik
>>
>> On Thu, Feb 23, 2023 at 2:53 PM Benedict  wrote:
>>
>>> I don’t think there’s anything about a new format that requires a
>>> version bump, but I could be missing something.
>>>
>>> We have to have a switch to enable tries already don’t we? I’m pretty
>>> sure we haven’t talked about switching the default format?
>>>
>>> On 23 Feb 2023, at 12:12, Henrik Ingo  wrote:
>>>
>>> 
>>> On Thu, Feb 23, 2023 at 11:57 AM Benedict  wrote:
>>>
 Can somebody explain to me why this is being fought tooth and nail,
 when the work involved is absolutely minimal?


>>> I don't know how each individual has been thinking about this, but it
>>> seems to me just looking at all the tasks that at least the introduction of
>>> tries is a major format change anyway - since it's the whole point - and
>>> therefore people working on other tasks may have assumed the format is
>>> changing anyway and therefore something like a switch (is this what is
>>> referred to as the C-8110 solution?) will take care of it for everyone.
>>>
>>> I'm not sure there's consensus that such a switch is a sufficient
>>> resolution to this discussion, but if there were such a consensus, the next
>>> question would be whether the patches that are otherwise ready now can
>>> merge, or whether they will all be blocked waiting for the compatibility
>>> solution first. And possibly better testing, etc. Letting them merge would
>>> be justified by the desire to have more frequent and smaller increments of
>>> work merged into trunk... well, I'm not going to repeat everything from
>>> that discussion but the same pro's and con's apply.
>>>
>>> henrik
>>> --
>>>
>>> Henrik Ingo
>>>
>>> c. +358 40 569 7354
>>>
>>> w. www.datastax.com
>>>
>>>
>>> 
>>> 
>>> 
>>> 

Re: Welcome Francisco Guerrero Hernandez as Cassandra Committer

[guo Maxwell]

Congrats!

Jacek Lewandowski  于2023年11月29日周三 06:16写道：

> Congrats!!!
>
> wt., 28 lis 2023, 23:08 użytkownik Abe Ratnofsky  napisał:
>
>> Congrats Francisco!
>>
>> > On Nov 28, 2023, at 1:56 PM, C. Scott Andreas 
>> wrote:
>> >
>> > Congratulations, Francisco!
>> >
>> > - Scott
>> >
>> >> On Nov 28, 2023, at 10:53 AM, Dinesh Joshi  wrote:
>> >>
>> >> The PMC members are pleased to announce that Francisco Guerrero
>> Hernandez has accepted
>> >> the invitation to become committer today.
>> >>
>> >> Congratulations and welcome!
>> >>
>> >> The Apache Cassandra PMC members
>>
>>

Re: Include CASSANDRA-18464 in 5.0-beta1 (direct I/O support for commitlog write)

[guo Maxwell]

+1 😁

Brandon Williams  于2023年11月27日周一 22:25写道：

> As long as it's disabled by default that's an easy +1 from me.
>
> Kind Regards,
> Brandon
>
> On Mon, Nov 27, 2023 at 7:03 AM Jacek Lewandowski
>  wrote:
> >
> > Hey,
> >
> > I'd like to ask if we can include
> https://issues.apache.org/jira/browse/CASSANDRA-18464 in the 5.0-beta1
> release. This introduces the ability to write to the commitlog using direct
> I/O and bringing some noticeable performance improvements when enabled
> (disabled by default).
> >
> > Since it introduces a change in the yaml config, it probably cannot be
> delivered in RC or 5.0.x - hence my question.
> >
> > The ticket has been reviewed and tested. It is basically in the
> read-to-commit state.
> >
> > thanks,
> > Jacek
> >
>

Re: Road to 5.0-GA (was: [VOTE] Release Apache Cassandra 5.0-alpha2)

[guo Maxwell]

Do I need permission to view this link? When I open it, an error appears,
saying “It may have been deleted or you don't have permission to view it.”

Benjamin Lerer  于2023年11月6日周一 18:34写道：

> I created a Dashboard to track the progress and remaining tasks for 5.0:
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=593
> Everybody logged in should have access. Ping me if it is not the case.
>
> Le sam. 4 nov. 2023 à 19:54, Mick Semb Wever  a écrit :
>
>>
>> Please mark such bugs with fixVersion 5.0-beta
>>
>> If there are no more tickets that need API changes (i.e. those that
>> should be marked fixVersion 5.0-alpha) this then indicates we do not need a
>> 5.0-alpha3 release and can focus towards 5.0-beta1 (regardless of having
>> blockers open to it).
>>
>> Appreciate the attention 18993 is getting – we do have a shortlist of
>> beta blockers that we gotta prioritise !
>>
>>
>> On Sat, 4 Nov 2023 at 18:33, Benedict  wrote:
>>
>>> Yep, data loss bugs are not any old bug. I’m concretely -1 (binding)
>>> releasing a beta with one that’s either under investigation or confirmed.
>>>
>>> As Scott says, hopefully it won’t come to that - the joy of
>>> deterministic testing is this should be straightforward to triage.
>>>
>>> On 4 Nov 2023, at 17:30, C. Scott Andreas  wrote:
>>>
>>> I’d happily be the first to vote -1(nb) on a release containing a known
>>> and reproducible bug that can result in data loss or an incorrect response
>>> to a query. And I certainly wouldn’t run it.
>>>
>>> Since we have a programmatic repro within just a few seconds, this
>>> should not take long to root-cause.
>>>
>>> On Friday, Alex worked to get this reproducing on a Cassandra branch
>>> rather than via unstaged changes. We should have a published / shareable
>>> example with details near the beginning of the week.
>>>
>>> – Scott
>>>
>>> On Nov 4, 2023, at 10:17 AM, Josh McKenzie  wrote:
>>>
>>> 
>>>
>>> I think before we cut a beta we need to have diagnosed and fixed 18993
>>> (assuming it is a bug).
>>>
>>> Before a beta? I could see that for rc or GA definitely, but having a
>>> known (especially non-regressive) data loss bug in a beta seems like it's
>>> compatible with the guarantees we're providing for it:
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>>>
>>> This release is recommended for test/QA clusters where short(order of
>>> minutes) downtime during upgrades is not an issue
>>>
>>>
>>> On Sat, Nov 4, 2023, at 12:56 PM, Ekaterina Dimitrova wrote:
>>>
>>> Totally agree with the others. Such an issue on its own should be a
>>> priority in any release. Looking forward to the reproduction test mentioned
>>> on the ticket.
>>>
>>> Thanks to Alex for his work on harry!
>>>
>>> On Sat, 4 Nov 2023 at 12:47, Benedict  wrote:
>>>
>>> Alex can confirm but I think it actually turns out to be a new bug in
>>> 5.0, but either way we should not cut a release with such a serious
>>> potential known issue.
>>>
>>> > On 4 Nov 2023, at 16:18, J. D. Jordan 
>>> wrote:
>>> >
>>> > Sounds like 18993 is not a regression in 5.0? But present in 4.1 as
>>> well?  So I would say we should fix it with the highest priority and get a
>>> new 4.1.x released. Blocking 5.0 beta voting is a secondary issue to me if
>>> we have a “data not being returned” issue in an existing release?
>>> >
>>> >> On Nov 4, 2023, at 11:09 AM, Benedict  wrote:
>>> >>
>>> >> I think before we cut a beta we need to have diagnosed and fixed
>>> 18993 (assuming it is a bug).
>>> >>
>>>  On 4 Nov 2023, at 16:04, Mick Semb Wever  wrote:
>>> >>>
>>> >>> 
>>> 
>>>  With the publication of this release I would like to switch the
>>>  default 'latest' docs on the website from 4.1 to 5.0.  Are there any
>>>  objections to this ?
>>> >>>
>>> >>>
>>> >>> I would also like to propose the next 5.0 release to be 5.0-beta1
>>> >>>
>>> >>> With the aim of reaching GA for the Summit, I would like to suggest
>>> we
>>> >>> work towards the best-case scenario of 5.0-beta1 in two weeks and
>>> >>> 5.0-rc1 first week Dec.
>>> >>>
>>> >>> I know this is a huge ask with lots of unknowns we can't actually
>>> >>> commit to.  But I believe it is a worthy goal, and possible if
>>> nothing
>>> >>> sideswipes us – but we'll need all the help we can get this month to
>>> >>> make it happen.
>>> >>
>>>
>>>
>>>

Re: [EXTERNAL] Re: [VOTE] Release Apache Cassandra 5.0-alpha2

[guo Maxwell]

+1

German Eichberger via dev  于2023年11月1日周三 04:58写道：

> +1
>
> Heck, yeah, we already tested the branch (build ourselves) and it works
> great so far.
> --
> *From:* Mick Semb Wever 
> *Sent:* Tuesday, October 31, 2023 1:43 PM
> *Cc:* dev 
> *Subject:* [EXTERNAL] Re: [VOTE] Release Apache Cassandra 5.0-alpha2
>
> > The vote will be open for 72 hours (longer if needed). Everyone who
> > has tested the build is invited to vote. Votes by PMC members are
> > considered binding. A vote passes if there are at least three binding
> > +1s and no -1's.
>
>
> +1
>
> Checked
> - signing correct
> - checksums are correct
> - source artefact builds (JDK 11+17)
> - binary artefact runs (JDK 11+17)
> - debian package runs (JDK 11+17)
> - debian repo runs (JDK 11+17)
> - redhat* package runs (JDK11+17)
> - redhat* repo runs (JDK 11+17)
>

Re: [DISCUSS] CommitLog default disk access mode

[guo Maxwell]

Thanks  for your contribution 😀

Pawar, Amit 于2023年10月26日 周四下午11:41写道：

> [Public]
>
> Default behavior is not changed. Thank you, Josh for your appreciation.
> This is my first patch, and it means lot to me.
>
>
>
> Thanks again,
>
> Amit
>
>
>
> +1 to adding the feature, clear and easy configurability, and if after a
> major cycle we can say with confidence it's beating the status quo in the
> vast majority of general cases, flip default. I mean, logically it
> *should* be, but infra software at the scale we do requires great care. :)
>
>
>
> This is great work Amit - well done.
>
>
>
> On Mon, Oct 16, 2023, at 4:28 PM, Dinesh Joshi wrote:
>
> I haven't looked at the patch yet so take whatever I say here with a pinch
> of salt.
>
>
>
> Philosophically, defaults should not change unless there is a clear
> demonstrable benefit in majority cases for our users. In this case DirectIO
> should have clear benefits. That said, this is a new feature and I would
> personally default it to off. We should document it and allow for our users
> to enable it. This derisks the project in case there is an inadvertent
> change in behavior.
>
>
>
> Dinesh
>
>
>
> On Oct 15, 2023, at 11:34 PM, Pawar, Amit  wrote:
>
>
>
> [Public]
>
>
>
> Hi,
>
>
>
> CommitLog uses mmap (memory mapped ) segments by default. Direct-IO
> feature is proposed through new PR[1] to improve the CommitLog IO speed.
> Enabling this by default could be useful feature to address IO bottleneck
> seen during peak load.
>
>
>
> Need your input regarding changing this default. Please suggest.
>
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-18464
>
>
>
> thanks,
>
> Amit Pawar
>
>
>
> [1] - https://github.com/apache/cassandra/pull/2777
>
>
>

Re: CASSANDRA-18941 produce size bounded SSTables from CQLSSTableWriter

[guo Maxwell]

😄

Chris Lohfink  于2023年10月25日周三 05:02写道：

> +1
>
> On Tue, Oct 24, 2023 at 11:24 AM Brandon Williams 
> wrote:
>
>> +1
>>
>> Kind Regards,
>> Brandon
>>
>> On Mon, Oct 23, 2023 at 6:22 PM Yifan Cai  wrote:
>> >
>> > Hi,
>> >
>> > I want to propose merging the patch in CASSANDRA-18941 to 4.0 and up to
>> trunk and hope we are all OK with it.
>> >
>> > In CASSANDRA-18941, I am adding the capability to produce size-bounded
>> SSTables in CQLSSTableWriter for sorted data. It can greatly benefit
>> Cassandra Analytics (https://github.com/apache/cassandra-analytics) for
>> bulk writing SSTables, since it avoids buffering and sorting on flush,
>> given the data source is sorted already in the bulk write process.
>> Cassandra Analytics supports Cassandra 4.0 and depends on the cassandra-all
>> 4.0.x library. Therefore, we are mostly interested in using the new
>> capability in 4.0.
>> >
>> > CQLSSTableWriter is only used in offline tools and never in the code
>> path of Cassandra server.
>> >
>> > Any objections to merging the patch to 4.0 and up to trunk?
>> >
>> > - Yifan
>>
>

Re: CASSANDRA-18941 produce size bounded SSTables from CQLSSTableWriter

[guo Maxwell]

+1, but I want to know why only trunk and 4.0 ? not all the versions
involved, like 4.1 ,5.0 。

Francisco Guerrero  于2023年10月24日周二 07:47写道：

> +1 (nb). I think this is a great addition to offline tools that use
> SSTable writer in general.
>
> On 2023/10/23 23:21:13 Yifan Cai wrote:
> > Hi,
> >
> > I want to propose merging the patch in CASSANDRA-18941 to 4.0 and up to
> > trunk and hope we are all OK with it.
> >
> > In CASSANDRA-18941, I am adding the capability to produce size-bounded
> > SSTables in CQLSSTableWriter for sorted data. It can greatly benefit
> > Cassandra Analytics (https://github.com/apache/cassandra-analytics) for
> > bulk writing SSTables, since it avoids buffering and sorting on flush,
> > given the data source is sorted already in the bulk write process.
> > Cassandra Analytics supports Cassandra 4.0 and depends on the
> cassandra-all
> > 4.0.x library. Therefore, we are mostly interested in using the new
> > capability in 4.0.
> >
> > CQLSSTableWriter is only used in offline tools and never in the code path
> > of Cassandra server.
> >
> > Any objections to merging the patch to 4.0 and up to trunk?
> >
> > - Yifan
> >
>

Re: [DISCUSS] Change the useage of nodetool tablehistograms

[guo Maxwell]

According to the conclusion, I think we can fix the bug when the argument
number is bigger than  2 first.

Thanks everyone.

Bowen Song via dev  于2023年3月23日周四 22:17写道：

> In that case, I would recommend fix the bug that prints everything when an
> arbitrary number of arguments is given.
> On 23/03/2023 13:40, guo Maxwell wrote:
>
> firstly I think anything existing must be reasonable，so ignore option for
> tablestats must be a need for the user to use. at least I used it some time
> ；
> secondly in order  to keep this as simple as possible ，I think left the
> option unchanged is enough ，because the original usage is simple enough.
> user can just print the specified table after set nodetool tablehistorgrams
> ks table ，and if there is ten tables in kesypace  ，it is simple for him to
> type ten times with different table names which I think at first Only set
> with argument ks keyspace name is enough.
> When we just want to see eight tables in the ks ，the user should just type
> eight table name which ignore two table may be enough.
>
>
>
>
> Bowen Song via dev 于2023年3月23日 周四下午8:07写道：
>
>> I don't think the nodetool tablestats command's parameters should be
>> used as a reference implementation for the nodetool tablehistograms
>> command. Because:
>>
>>- the tablehistograms command can take the keyspace and table as two
>>separate parameters, but the tablestats command can't.
>>- the tablestats command can take keyspace (without table) as a
>>parameter, but the tablehistograms command can't.
>>
>> The introduction of the -ks and -tbs options are unnecessary for the
>> tablestats command, because it's parameters are:
>>
>> nodetool tablestats [| [> table>| [...]]]
>>
>> Which means any positional parameter without a dot is treated as a
>> keyspace name, otherwise it's treated as dot-separated keyspace and table
>> name. That, however, does not apply to the nodetool tablehistograms
>> command, which led to your workaround - the addition of the -ks and -tbs
>> options.
>>
>> But if you could just forget about the nodetool tablestats command for a
>> moment, and look at the nodetool tablehistograms command alone, you will
>> see that it's unnecessary to introduce the -ks and -tbs options, because
>> the command already takes keyspace name and table name, just in a different
>> format.
>>
>> In addition to that, I would be interested to know how often do people
>> use the -i option in the nodetool tablestats command. My best guess is,
>> very very rarely.
>>
>> If my guess is correct, we should keep the nodetool tablehistograms
>> command as simple as:
>>
>> nodetool tablehistograms [  [ [...]] |
>>  [ [...]]]
>>
>> It's good enough if the above can cover the majority of use cases. The
>> remaining use cases can be dealt with individually, by multiple invocations
>> of the same command or providing it with a script-generated list of tables
>> in the  format.
>>
>> TL;DR: The KISS principle <https://en.wikipedia.org/wiki/KISS_principle>
>> should apply here - keep it simple.
>>
>>
>> On 23/03/2023 03:05, guo Maxwell wrote:
>>
>> Maybe I didn't describe the usage of option "-i" clearly, The reason why
>> I think the command argument should be like this :
>>
>>
>> 1. nodetool tablehistograms ks.tb1 or ks tb1  ... //this is *one of the
>>> old way *of using tablehistogram. will print out the histograms of
>>> tabke ks.tb1 , we keep the old format to print out the table
>>> histograms,besides if more than two arguments is provied, suchu as nodetool
>>> tablehistograms system.local system_schema.columns system_schema.tables
>>> then all tables's  histograms will be printed out (I think this is a
>>> bug that not as excepted in the document's decription, we should remind the
>>> user that this is an incorrenct usage)
>>>
>>> 2. nodetool tablehistograms -tbs ks.tb1 ks.tb2  //print out list of
>>> tables' histograms with format keyspace.table
>>> 3.nodetool tablehistograms -ks ks1 ks2 ks3 ... //print out list of
>>> keyspaces histograms
>>> 4.nodetool tablehistograms -i -ks ks1 ks2  //print out list of table
>>> histograms except for the keyspaces list behind the option -i
>>> 5.nodetool tablehistograns -i -tbs ks.tb1 ks.tb2 // print out list
>>> tables' histograms except for table in ks.tb1 ks.tb2
>>> 6.nodetool tablehistograns -i -tbs ks.tb1 ks.tb2 -ks ks1 // print out
>>

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[guo Maxwell]

If it is ok for Henrik to rebase the Astra implementation of this
functionality (FileSystemProvider) onto Cassandra trunk.

Then we can create a jira to move this forward for a small step.

Claude Warren, Jr  于2023年10月18日周三 15:05写道：

> Henrik and Guo,
>
> Have you moved forward on this topic?  I have not seen anything recently.
> I have posted a solution that intercepts calls for directories and injects
> directories from different FileSystems.  This means that a node can have
> keyspaces both on the local file system and one or more other FileSystem
> implementations.
>
> I look forward to hearing from you,
> Claude
>
>
> On Wed, Oct 18, 2023 at 9:00 AM Claude Warren, Jr 
> wrote:
>
>> After a bit more analysis and some testing I have a new branch that I
>> think solves the problem. [1]  I have also created a pull request internal
>> to my clone so that it is easy to see the changes. [2]
>>
>> The strategy change is to move the insertion of the proxy from the
>> Cassandra File class to the Directories class.  This means that all action
>> with the table is captured (this solves a problem encountered in the
>> earlier strategy).
>> The strategy is to create a path on a different FileSystem and return
>> that.  The example code only moves the data for the table to another
>> directory on the same FileSystem but using a different FileSystem
>> implementation should be a trivial change.
>>
>> The current code works on an entire keyspace.  I, while code exists to
>> limit the redirect to a table I have not tested that branch yet and am not
>> certain that it will work.  There is also some code (i.e. the PathParser)
>> that may no longer be needed but has not been removed yet.
>>
>> Please take a look and let me know if you see any issues with this
>> solution.
>>
>> Claude
>>
>> [1] https://github.com/Claudenw/cassandra/tree/FileSystemProxy
>> [2] https://github.com/Claudenw/cassandra/pull/5/files
>>
>>
>>
>> On Tue, Oct 10, 2023 at 10:28 AM Claude Warren, Jr <
>> claude.war...@aiven.io> wrote:
>>
>>> I have been exploring adding a second Path to the Cassandra File
>>> object.  The original path being the path within the standard Cassandra
>>> directory tree and the second being a translated path when there is what
>>> was called a ChannelProxy in place.
>>>
>>> A problem arises when the Directories.getLocationForDisk() is called.
>>> It seems to be looking for locations that start with the data directory
>>> absolute path.   I can change it to make it look for the original path not
>>> the translated path.  But in other cases the translated path is the one
>>> that is needed.
>>>
>>> I notice that there is a concept of multiple file locations in the code
>>> base, particularly in the Directories.DataDirectories class where there are
>>> "locationsForNonSystemKeyspaces" and "locationsForSystemKeyspace" in the
>>> constructor, and in the
>>> DatabaseDescriptor.getNonLocalSystemKeyspacesDataFileLocations() method
>>> which returns an array of String and is populated from the cassandra.yaml
>>> file.
>>>
>>> The DatabaseDescriptor.getNonLocalSystemKeyspacesDataFileLocations()
>>> only ever seems to return an array of one item.
>>>
>>> Why does
>>> DatabaseDescriptor.getNonLocalSystemKeyspacesDataFileLocations()  return an
>>> array?
>>>
>>> Should the system set the path to the root of the ColumnFamilyStore in
>>> the ColumnFamilyStore directories instance?
>>> Should the Directories.getLocationForDisk() do the proxy to the other
>>> file system?
>>>
>>> Where is the proper location to change from the standard internal
>>> representation to the remote location?
>>>
>>>
>>> On Fri, Sep 29, 2023 at 8:07 AM Claude Warren, Jr <
>>> claude.war...@aiven.io> wrote:
>>>
>>>> Sorry I was out sick and did not respond yesterday.
>>>>
>>>> Henrik,  How does your system work?  What is the design strategy?  Also
>>>> is your code available somewhere?
>>>>
>>>> After looking at the code some more I think that the best solution is
>>>> not a FileChannelProxy but to modify the Cassandra File class to get a
>>>> FileSystem object for a Factory to build the Path that is used within that
>>>> object.  I think that this makes if very small change that will pick up
>>>> 90+% of the cases.  We then just need to find the ed

Re: [DISCUSS] CommitLog default disk access mode

[guo Maxwell]

-1

I still think we should keep it as it is until the  direct io
for commitlog (read and write) is ready and relatively stable. And then we
may change the default value to direct io from mmap in a future version,
such as 5.2, or 6.0.

Pawar, Amit  于2023年10月17日周二 19:03写道：

> [AMD Official Use Only - General]
>
> Thank you all for your input. Received total 6 replies and below is the
> summary.
>
>
>
> 1. Mmap   : 2/6
>
> 2. Direct-I/O : 4/6
>
>
>
> Default should be changed to Direct-IO then ? please confirm.
>
>
>
> Thanks,
>
> Amit
>
>
>
> Strongly agree with this point of view that direct IO  can bring great
> benefits.
>
>
>
> I have reviewed part of the code, and my preliminary judgment is that it
> is not very common and limited in some situations, for example, it  works
> for commitlog's write path only for this patch.So I suggest that the
> default value should not be modified until the entire function is
> comprehensive and stable, and then modified in a future version.
>
>
>
> Sam  于2023年10月17日周二 05:39写道：
>
> *Glad you brought up compaction here - I think there would be a
> significant benefit to moving compaction to direct i/o.*
>
>
>
> +1. Would love to see this get traction.
>
>
>
> On Mon, 16 Oct 2023 at 19:38, Jon Haddad 
> wrote:
>
> Glad you brought up compaction here - I think there would be a significant
> benefit to moving compaction to direct i/o.
>
>
> On 2023/10/16 16:14:28 Benedict wrote:
> > I have some plans to (eventually) use the commit log as memtable payload
> storage (ie memtables would reference the commit log entries directly,
> storing only indexing info), and to back first level of sstables by
> reference to commit log entries. This will permit us to deliver not only
> much bigger memtables (cutting compaction throughput requirements by the
> ratio of size increase - so pretty dramatically), and faster flushing (so
> better behaviour ling write bursts), but also a fairly cheap and simple way
> to support MVCC - which will be helpful for transaction throughput.
> >
> > There is also a new commit log (“journal”) coming with Accord, that the
> rest of C* may or may not transition to.
> >
> > I only say this because this makes the utility of direct IO for commit
> log suspect, as we will be reading from the files as a matter of course
> should this go ahead; and we may end up relying on a different commit log
> implementation before long anyway.
> >
> > This is obviously a big suggestion and is not guaranteed to transpire,
> and probably won’t within the next year or so, but it should perhaps form
> some minimal part of any calculus. If the patch is otherwise simple and
> beneficial I don’t have anything against it, and the use of direct IO could
> well be of benefit eg in compaction - and also in future if we manage to
> bring a page management in process. So laying foundations there could be of
> benefit, even if the commit log eventually does not use it.
> >
> > > On 16 Oct 2023, at 17:00, Jon Haddad 
> wrote:
> > >
> > > I haven't looked at the patch, but at a high level, defaulting to
> direct I/O for commit logs makes a lot of sense to me.
> > >
> > >> On 2023/10/16 06:34:05 "Pawar, Amit" wrote:
> > >> [Public]
> > >>
> > >> Hi,
> > >>
> > >> CommitLog uses mmap (memory mapped ) segments by default. Direct-IO
> feature is proposed through new PR[1] to improve the CommitLog IO speed.
> Enabling this by default could be useful feature to address IO bottleneck
> seen during peak load.
> > >>
> > >> Need your input regarding changing this default. Please suggest.
> > >>
> > >> https://issues.apache.org/jira/browse/CASSANDRA-18464
> > >>
> > >> thanks,
> > >> Amit Pawar
> > >>
> > >> [1] - https://github.com/apache/cassandra/pull/2777
> > >>
> >
>
>
>
>
> --
>
> you are the apple of my eye !
>


-- 
you are the apple of my eye !

Re: [DISCUSS] CommitLog default disk access mode

[guo Maxwell]

Strongly agree with this point of view that direct IO  can bring great
benefits.

I have reviewed part of the code, and my preliminary judgment is that it is
not very common and limited in some situations, for example, it  works for
commitlog's write path only for this patch.So I suggest that the default
value should not be modified until the entire function is comprehensive and
stable, and then modified in a future version.

Sam  于2023年10月17日周二 05:39写道：

>
> *Glad you brought up compaction here - I think there would be a
> significant benefit to moving compaction to direct i/o.*
>
> +1. Would love to see this get traction.
>
> On Mon, 16 Oct 2023 at 19:38, Jon Haddad 
> wrote:
>
>> Glad you brought up compaction here - I think there would be a
>> significant benefit to moving compaction to direct i/o.
>>
>>
>> On 2023/10/16 16:14:28 Benedict wrote:
>> > I have some plans to (eventually) use the commit log as memtable
>> payload storage (ie memtables would reference the commit log entries
>> directly, storing only indexing info), and to back first level of sstables
>> by reference to commit log entries. This will permit us to deliver not only
>> much bigger memtables (cutting compaction throughput requirements by the
>> ratio of size increase - so pretty dramatically), and faster flushing (so
>> better behaviour ling write bursts), but also a fairly cheap and simple way
>> to support MVCC - which will be helpful for transaction throughput.
>> >
>> > There is also a new commit log (“journal”) coming with Accord, that the
>> rest of C* may or may not transition to.
>> >
>> > I only say this because this makes the utility of direct IO for commit
>> log suspect, as we will be reading from the files as a matter of course
>> should this go ahead; and we may end up relying on a different commit log
>> implementation before long anyway.
>> >
>> > This is obviously a big suggestion and is not guaranteed to transpire,
>> and probably won’t within the next year or so, but it should perhaps form
>> some minimal part of any calculus. If the patch is otherwise simple and
>> beneficial I don’t have anything against it, and the use of direct IO could
>> well be of benefit eg in compaction - and also in future if we manage to
>> bring a page management in process. So laying foundations there could be of
>> benefit, even if the commit log eventually does not use it.
>> >
>> > > On 16 Oct 2023, at 17:00, Jon Haddad 
>> wrote:
>> > >
>> > > I haven't looked at the patch, but at a high level, defaulting to
>> direct I/O for commit logs makes a lot of sense to me.
>> > >
>> > >> On 2023/10/16 06:34:05 "Pawar, Amit" wrote:
>> > >> [Public]
>> > >>
>> > >> Hi,
>> > >>
>> > >> CommitLog uses mmap (memory mapped ) segments by default. Direct-IO
>> feature is proposed through new PR[1] to improve the CommitLog IO speed.
>> Enabling this by default could be useful feature to address IO bottleneck
>> seen during peak load.
>> > >>
>> > >> Need your input regarding changing this default. Please suggest.
>> > >>
>> > >> https://issues.apache.org/jira/browse/CASSANDRA-18464
>> > >>
>> > >> thanks,
>> > >> Amit Pawar
>> > >>
>> > >> [1] - https://github.com/apache/cassandra/pull/2777
>> > >>
>> >
>>
>

-- 
you are the apple of my eye !

Re: [DISCUSS] CommitLog default disk access mode

[guo Maxwell]

I think what we should do is  provide options, and let users make their own
decisions.

The default behavior should not be modified until some future released
version.


Pawar, Amit  于2023年10月16日周一 15:51写道：

> [Public]
>
> Hi,
>
>
>
> CommitLog uses mmap (memory mapped ) segments by default. Direct-IO
> feature is proposed through new PR[1] to improve the CommitLog IO speed.
> Enabling this by default could be useful feature to address IO bottleneck
> seen during peak load.
>
>
>
> Need your input regarding changing this default. Please suggest.
>
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-18464
>
>
>
> thanks,
>
> Amit Pawar
>
>
>
> [1] - https://github.com/apache/cassandra/pull/2777
>


-- 
you are the apple of my eye !

Re: [VOTE] Accept java-driver

[guo Maxwell]

+1

Benedict  于2023年10月7日周六 18:27写道：

> +1
>
> On 7 Oct 2023, at 10:03, Mick Semb Wever  wrote:
>
> 
>
> LEGAL-658
>
>
> On Fri, 6 Oct 2023 at 17:43, J. D. Jordan 
> wrote:
>
>> The software grant agreement covers all donated code.  The ASF does not
>> need any historical agreements. The agreement giving the ASF copyright etc
>> is the Software Grant Agreement. Yes, any future work done after donation
>> needs to be covered by ASF CLAs.
>>
>> But happy to see someone ask legal@ to confirm this so we can move
>> forward.
>>
>> On Oct 6, 2023, at 3:33 AM, Benedict  wrote:
>>
>> 
>> Are we certain about that? It’s unclear to me from the published
>> guidance; would be nice to get legal to weigh in to confirm to make sure we
>> aren’t skipping any steps, as we haven’t been involved until now so haven’t
>> the visibility. At the very least it reads to me that anyone expected to be
>> maintaining the software going forwards should have a CLA on file with ASF,
>> but I’d have expected the ASF to also want a record of the historic CLAs.
>>
>> On 6 Oct 2023, at 09:28, Mick Semb Wever  wrote:
>>
>> 
>>
>>
>> On Thu, 5 Oct 2023 at 17:50, Jeremiah Jordan 
>> wrote:
>>
>>> I think this is covered by the grant agreement?
>>>
>>> https://www.apache.org/licenses/software-grant-template.pdf
>>>
>>> 2. Licensor represents that, to Licensor's knowledge, Licensor is
>>> legally entitled to grant the above license. Licensor agrees to notify
>>> the Foundation of any facts or circumstances of which Licensor becomes
>>> aware and which makes or would make Licensor's representations in this
>>> License Agreement inaccurate in any respect.
>>>
>>>
>>>
>>> On Oct 5, 2023 at 4:35:08 AM, Benedict  wrote:
>>>
 Surely it needs to be shared with the foundation and the PMC so we can
 verify? Or at least have ASF legal confirm they have received and are
 satisfied with the tarball? It certainly can’t be kept private to DS,
 AFAICT.

 Of course it shouldn’t be shared publicly but not sure how PMC can
 fulfil its verification function here without it

>>>
>>
>> Correct, thanks JD.
>>
>> These are CLAs that were submitted to DS, not to ASF.
>>
>> It is DS's legal responsibility to ensure what they are donating they
>> have the right to (i.e. have the copyright), when submitting the SGA.  It's
>> not on the ASF or the PMC to verify this.  Here we're simply demonstrating
>> that we (DS) have done that due diligence, and are keeping record of it.
>>
>>
>>

-- 
you are the apple of my eye !

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[guo Maxwell]

Thanks , So I think a jira can be created now. And I'd be happy to provide
some help with this as well if needed.

Henrik Ingo  于2023年9月28日周四 00:21写道：

> It seems I was volunteered to rebase the Astra implementation of this
> functionality (FileSystemProvider) onto Cassandra trunk. (And publish it,
> of course) I'll try to get going today or tomorrow, so that this
> discussion can then benefit from having that code available for inspection.
> And potentially using it as a soluttion to this use case.
>
> On Tue, Sep 26, 2023 at 8:04 PM Jake Luciani  wrote:
>
>> We (DataStax) have a FileSystemProvider for Astra we can provide.
>> Works with S3/GCS/Azure.
>>
>> I'll ask someone on our end to make it accessible.
>>
>> This would work by having a bucket prefix per node. But there are lots
>> of details needed to support things like out of bound compaction
>> (mentioned in CEP).
>>
>> Jake
>>
>> On Tue, Sep 26, 2023 at 12:56 PM Benedict  wrote:
>> >
>> > I agree with Ariel, the more suitable insertion point is probably the
>> JDK level FileSystemProvider and FileSystem abstraction.
>> >
>> > It might also be that we can reuse existing work here in some cases?
>> >
>> > On 26 Sep 2023, at 17:49, Ariel Weisberg  wrote:
>> >
>> > 
>> > Hi,
>> >
>> > Support for multiple storage backends including remote storage backends
>> is a pretty high value piece of functionality. I am happy to see there is
>> interest in that.
>> >
>> > I think that `ChannelProxyFactory` as an integration point is going to
>> quickly turn into a dead end as we get into really using multiple storage
>> backends. We need to be able to list files and really the full range of
>> filesystem interactions that Java supports should work with any backend to
>> make development, testing, and using existing code straightforward.
>> >
>> > It's a little more work to get C* to creates paths for alternate
>> backends where appropriate, but that works is probably necessary even with
>> `ChanelProxyFactory` and munging UNIX paths (vs supporting multiple
>> Fileystems). There will probably also be backend specific behaviors that
>> show up above the `ChannelProxy` layer that will depend on the backend.
>> >
>> > Ideally there would be some config to specify several backend
>> filesystems and their individual configuration that can be used, as well as
>> configuration and support for a "backend file router" for file creation
>> (and opening) that can be used to route files to the backend most
>> appropriate.
>> >
>> > Regards,
>> > Ariel
>> >
>> > On Mon, Sep 25, 2023, at 2:48 AM, Claude Warren, Jr via dev wrote:
>> >
>> > I have just filed CEP-36 [1] to allow for keyspace/table storage
>> outside of the standard storage space.
>> >
>> > There are two desires  driving this change:
>> >
>> > The ability to temporarily move some keyspaces/tables to storage
>> outside the normal directory tree to other disk so that compaction can
>> occur in situations where there is not enough disk space for compaction and
>> the processing to the moved data can not be suspended.
>> > The ability to store infrequently used data on slower cheaper storage
>> layers.
>> >
>> > I have a working POC implementation [2] though there are some issues
>> still to be solved and much logging to be reduced.
>> >
>> > I look forward to productive discussions,
>> > Claude
>> >
>> > [1]
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-36%3A+A+Configurable+ChannelProxy+to+alias+external+storage+locations
>> > [2] https://github.com/Claudenw/cassandra/tree/channel_proxy_factory
>> >
>> >
>> >
>>
>>
>> --
>> http://twitter.com/tjake
>>
>
>
> --
>
> Henrik Ingo
>
> c. +358 40 569 7354
>
> w. www.datastax.com
>
>   
> 
> 
>
>

-- 
you are the apple of my eye !

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[guo Maxwell]

Yeah, there is so much things to do as cassandra (share-nothing) is
different from some other system like hbase , So I think we can break the
final goal into multiple steps. first is what Claude proposed. But I
suggest that this design can make the interface more scalable and we can
consider the implementation of cloud storage. so that someone can extend
the interface in the future.

Josh McKenzie  于2023年9月26日周二 18:40写道：

> it may be better to support most cloud storage
> It simply only supports S3, which feels a bit customized for a certain
> user and is not universal enough.Am I right ?
>
> I agree w/the eventual goal (and constraint on design now) of supporting
> most popular cloud storage vendors, but if we have someone with an itch to
> scratch and at the end of that we end up with first steps in a compatible
> direction to ultimately supporting decoupled / abstracted storage systems,
> that's fantastic.
>
> To Jeff's point - so long as we can think about and chart a general path
> of where we want to go, if Claude has the time and inclination to handle
> abstracting out the API in that direction and one implementation, that's
> fantastic IMO.
>
> I know there's some other folks out there who've done some interception /
> refactoring of the FileChannel stuff to support disaggregated storage;
> curious what their experiences were like.
>
>
> On Tue, Sep 26, 2023, at 4:20 AM, Claude Warren, Jr via dev wrote:
>
> The intention of the CEP is to lay the groundwork to allow development of
> ChannelProxyFactories that are pluggable in Cassandra.  In this way any
> storage system can be a candidate for Cassandra storage provided
> FileChannels can be created for the system.
>
> As I stated before I think that there may be a need for a
> java.nio.FileSystem implementation for  the proxies but I have not had the
> time to dig into it yet.
>
> Claude
>
>
> On Tue, Sep 26, 2023 at 9:01 AM guo Maxwell  wrote:
>
> In my mind , it may be better to support most cloud storage : aws,
> azure,gcp,aliyun and so on . We may make it a plugable. But in that way, it
> seems there may need a filesystem interface layer for object storage. And
> should we support ,distributed system like hdfs ,or something else. We
> should first discuss what should be done and what should not be done. It
> simply only supports S3, which feels a bit customized for a certain user
> and is not universal enough.Am I right ?
>
> Claude Warren, Jr  于2023年9月26日周二 14:36写道：
>
> My intention is to develop an S3 storage system using
> https://github.com/carlspring/s3fs-nio
>
> There are several issues yet to be solved:
>
>1. There are some internal calls that create files in the table
>directory that do not use the channel proxy.  I believe that these are
>making calls on File objects.  I think those File objects are Cassandra
>File objects not Java I/O File objects, but am unsure.
>2. Determine if the carlspring s3fs-nio library will be performant
>enough to work in the long run.  There may be issues with it:
>1. Downloading entire files before using them rather than using views
>   into larger remotely stored files.
>   2. Requiring a complete file to upload rather than using the
>   partial upload capability of the S3 interface.
>
>
>
> On Tue, Sep 26, 2023 at 4:11 AM guo Maxwell  wrote:
>
> "Rather than building this piece by piece, I think it'd be awesome if
> someone drew up an end-to-end plan to implement tiered storage, so we can
> make sure we're discussing the whole final state, and not an implementation
> detail of one part of the final state?"
>
> Do agree with jeff for this ~~~ If these feature can be supported in oss
> cassandra , I think it will be very popular, whether in  a private
> deployment environment or a public cloud service (our experience can prove
> it). In addition, it is also a cost-cutting option for users too
>
> Jeff Jirsa  于2023年9月26日周二 00:11写道：
>
>
> - I think this is a great step forward.
> - Being able to move sstables around between tiers of storage is a feature
> Cassandra desperately needs, especially if one of those tiers is some sort
> of object storage
> - This looks like it's a foundational piece that enables that. Perhaps by
> a team that's already implemented this end to end?
> - Rather than building this piece by piece, I think it'd be awesome if
> someone drew up an end-to-end plan to implement tiered storage, so we can
> make sure we're discussing the whole final state, and not an implementation
> detail of one part of the final state?
>
>
>
>
>
>
> On Sun, Sep 24, 2023 at 11:49 PM Claude Warren, Jr via dev <
&

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[guo Maxwell]

In my mind , it may be better to support most cloud storage : aws,
azure,gcp,aliyun and so on . We may make it a plugable. But in that way, it
seems there may need a filesystem interface layer for object storage. And
should we support ,distributed system like hdfs ,or something else. We
should first discuss what should be done and what should not be done. It
simply only supports S3, which feels a bit customized for a certain user
and is not universal enough.Am I right ?

Claude Warren, Jr  于2023年9月26日周二 14:36写道：

> My intention is to develop an S3 storage system using
> https://github.com/carlspring/s3fs-nio
>
> There are several issues yet to be solved:
>
>1. There are some internal calls that create files in the table
>directory that do not use the channel proxy.  I believe that these are
>making calls on File objects.  I think those File objects are Cassandra
>File objects not Java I/O File objects, but am unsure.
>2. Determine if the carlspring s3fs-nio library will be performant
>enough to work in the long run.  There may be issues with it:
>   1. Downloading entire files before using them rather than using
>   views into larger remotely stored files.
>   2. Requiring a complete file to upload rather than using the
>   partial upload capability of the S3 interface.
>
>
>
> On Tue, Sep 26, 2023 at 4:11 AM guo Maxwell  wrote:
>
>> "Rather than building this piece by piece, I think it'd be awesome if
>> someone drew up an end-to-end plan to implement tiered storage, so we can
>> make sure we're discussing the whole final state, and not an implementation
>> detail of one part of the final state?"
>>
>> Do agree with jeff for this ~~~ If these feature can be supported in oss
>> cassandra , I think it will be very popular, whether in  a private
>> deployment environment or a public cloud service (our experience can prove
>> it). In addition, it is also a cost-cutting option for users too
>>
>> Jeff Jirsa  于2023年9月26日周二 00:11写道：
>>
>>>
>>> - I think this is a great step forward.
>>> - Being able to move sstables around between tiers of storage is a
>>> feature Cassandra desperately needs, especially if one of those tiers is
>>> some sort of object storage
>>> - This looks like it's a foundational piece that enables that. Perhaps
>>> by a team that's already implemented this end to end?
>>> - Rather than building this piece by piece, I think it'd be awesome if
>>> someone drew up an end-to-end plan to implement tiered storage, so we can
>>> make sure we're discussing the whole final state, and not an implementation
>>> detail of one part of the final state?
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sun, Sep 24, 2023 at 11:49 PM Claude Warren, Jr via dev <
>>> dev@cassandra.apache.org> wrote:
>>>
>>>> I have just filed CEP-36 [1] to allow for keyspace/table storage
>>>> outside of the standard storage space.
>>>>
>>>> There are two desires  driving this change:
>>>>
>>>>1. The ability to temporarily move some keyspaces/tables to storage
>>>>outside the normal directory tree to other disk so that compaction can
>>>>occur in situations where there is not enough disk space for compaction 
>>>> and
>>>>the processing to the moved data can not be suspended.
>>>>2. The ability to store infrequently used data on slower cheaper
>>>>storage layers.
>>>>
>>>> I have a working POC implementation [2] though there are some issues
>>>> still to be solved and much logging to be reduced.
>>>>
>>>> I look forward to productive discussions,
>>>> Claude
>>>>
>>>> [1]
>>>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-36%3A+A+Configurable+ChannelProxy+to+alias+external+storage+locations
>>>> [2] https://github.com/Claudenw/cassandra/tree/channel_proxy_factory
>>>>
>>>>
>>>>
>>
>> --
>> you are the apple of my eye !
>>
>

-- 
you are the apple of my eye !

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[guo Maxwell]

"Rather than building this piece by piece, I think it'd be awesome if
someone drew up an end-to-end plan to implement tiered storage, so we can
make sure we're discussing the whole final state, and not an implementation
detail of one part of the final state?"

Do agree with jeff for this ~~~ If these feature can be supported in oss
cassandra , I think it will be very popular, whether in  a private
deployment environment or a public cloud service (our experience can prove
it). In addition, it is also a cost-cutting option for users too

Jeff Jirsa  于2023年9月26日周二 00:11写道：

>
> - I think this is a great step forward.
> - Being able to move sstables around between tiers of storage is a feature
> Cassandra desperately needs, especially if one of those tiers is some sort
> of object storage
> - This looks like it's a foundational piece that enables that. Perhaps by
> a team that's already implemented this end to end?
> - Rather than building this piece by piece, I think it'd be awesome if
> someone drew up an end-to-end plan to implement tiered storage, so we can
> make sure we're discussing the whole final state, and not an implementation
> detail of one part of the final state?
>
>
>
>
>
>
> On Sun, Sep 24, 2023 at 11:49 PM Claude Warren, Jr via dev <
> dev@cassandra.apache.org> wrote:
>
>> I have just filed CEP-36 [1] to allow for keyspace/table storage outside
>> of the standard storage space.
>>
>> There are two desires  driving this change:
>>
>>1. The ability to temporarily move some keyspaces/tables to storage
>>outside the normal directory tree to other disk so that compaction can
>>occur in situations where there is not enough disk space for compaction 
>> and
>>the processing to the moved data can not be suspended.
>>2. The ability to store infrequently used data on slower cheaper
>>storage layers.
>>
>> I have a working POC implementation [2] though there are some issues
>> still to be solved and much logging to be reduced.
>>
>> I look forward to productive discussions,
>> Claude
>>
>> [1]
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-36%3A+A+Configurable+ChannelProxy+to+alias+external+storage+locations
>> [2] https://github.com/Claudenw/cassandra/tree/channel_proxy_factory
>>
>>
>>

-- 
you are the apple of my eye !

Re: [DISCUSS] CEP-36: A Configurable ChannelProxy to alias external storage locations

[guo Maxwell]

Great suggestion,  Can external storage only be local storage media? Or can
it be stored in any storage medium, such as object storage s3 ?
We have previously implemented a tiered storage capability, that is, there
are multiple storage media on one node, SSD, HDD, and data placement based
on requests. After briefly browsing the proposals, it seems that there are
some differences. Can you help to do some explain ？ Thanks 。


Claude Warren, Jr via dev  于2023年9月25日周一 14:49写道：

> I have just filed CEP-36 [1] to allow for keyspace/table storage outside
> of the standard storage space.
>
> There are two desires  driving this change:
>
>1. The ability to temporarily move some keyspaces/tables to storage
>outside the normal directory tree to other disk so that compaction can
>occur in situations where there is not enough disk space for compaction and
>the processing to the moved data can not be suspended.
>2. The ability to store infrequently used data on slower cheaper
>storage layers.
>
> I have a working POC implementation [2] though there are some issues still
> to be solved and much logging to be reduced.
>
> I look forward to productive discussions,
> Claude
>
> [1]
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-36%3A+A+Configurable+ChannelProxy+to+alias+external+storage+locations
> [2] https://github.com/Claudenw/cassandra/tree/channel_proxy_factory
>
>
>

-- 
you are the apple of my eye !

Re: [VOTE] Release Apache Cassandra 5.0-alpha1 (take3)

[guo Maxwell]

+1

Berenguer Blasi  于2023年9月7日周四 15:27写道：

> +1
> On 4/9/23 22:28, Mick Semb Wever wrote:
>
>
> The vote will be open for 72 hours (longer if needed). Everyone who has
>> tested the build is invited to vote. Votes by PMC members are considered
>> binding. A vote passes if there are at least three binding +1s and no -1's.
>>
>
>
> +1
>
> Checked
> - signing correct
> - checksums are correct
> - source artefact builds (JDK 11+17)
> - binary artefact runs (JDK 11+17)
> - debian package runs (JDK 11+17)
> - debian repo runs (JDK 11+17)
> - redhat* package runs (JDK11+17)
> - redhat* repo runs (JDK 11+17)
>
>
>

-- 
you are the apple of my eye !

Re: [DISCUSS] Creating a 5.0 landing page

[guo Maxwell]

Can we have some pages for 5.0 in Chinese？If we can，I can do some help to
doing some translation ！
Ekaterina Dimitrova 于2023年8月4日 周五上午10:55写道：

> I honestly didn’t realize events page was added already. Thank you, Josh!
>
> So yes, in that case I guess that is what I am asking - shall we add there
> town halle, etc?
>
> On Thu, 3 Aug 2023 at 22:37, Josh McKenzie  wrote:
>
>> We actually already have an events page:
>> https://cassandra.apache.org/_/events.html; not sure if you were saying
>> we should add one Ekaterina or saying we should add this content there. +1
>> to the content there and having a landing page that points there +
>> integrating meetups, town halls, etc.
>>
>> Community -> Events on the menu up top in case someone missed it.
>>
>> On Thu, Aug 3, 2023, at 4:21 PM, Ekaterina Dimitrova wrote:
>>
>>
>> Hi Hugh,
>>
>> Thank you for reaching out. I think this is a great idea. Also, great
>> timing, considering the community is discussing a potential 5.0 alpha
>> release soon.
>>
>> It seems to me you actually suggest more than one page?
>> 1) 5.0 and new features - could this be an update of the What’s new page?
>> -
>> https://cassandra.apache.org/doc/trunk/cassandra/new/index.html
>> Adding also links to some of the talks sounds great to me.
>> 2) Dedicated events page? We were using the Blogs page before but I don’t
>> think it is a bad idea to split Blog posts from Events page.
>>
>> Thank you
>> Ekaterina
>>
>>
>> On Wed, 2 Aug 2023 at 21:03, Hugh Lashbrooke  wrote:
>>
>> With the upcoming release of Apache Cassandra 5.0, I’d like to create a
>> landing page for the release and what that could look like.
>>
>> The landing page would be intended to educate users about what is coming
>> up in this important release, highlighting why upgrading will be valuable
>> to them, as well as guiding them into more community activities, such as
>> Town Halls, where they can learn more and become further involved.
>>
>> The 5.0 landing page could include:
>>
>>- An overview of the release with a brief summary of the major
>>features
>>- A page for each CEP that is likely to be included–with key
>>features, implementation information, and other technical details. These
>>pages can also include recordings of relevant Contributor Meetings. Here
>>is an example for CEP-28 - Spark Bulk Analytics Library
>>
>> 
>>.
>>- CTAs to community platforms and activities - Slack, Meetups, Town
>>Halls, Contributor Meetings, etc.
>>
>> Let’s discuss! Does this sound valuable? If so, I will create a Jira
>> ticket and am happy to get started. What other things do you think should
>> be included in a page like this?
>>
>>
>> --
you are the apple of my eye !

Re: Who wants a free Cassandra t-shirt?

[guo Maxwell]

It seems I‘ve never had one… 😸

Patrick McFadin 于2023年7月22日 周六上午8:48写道：

> We have about another week left on the user survey I posted last week. The
> response has been slow, so it's time to get things in gear.
>
> I found a box of Cassandra t-shirts that will make an excellent thank you
> for anyone filling out the survey. Once the survey window closes, I'll pick
> a random group of emails to receive a shirt. Given the tepid response so
> far, your chances are decent to receive a shirt!
>
> 5-10 minutes. That's all it takes. Promote to your networks and let's get
> some opinions known!
>
> https://forms.gle/KVNd7UmUfcBuoNvF7
>
> Thanks again,
>
> Patrick
>
-- 
you are the apple of my eye !

Re: CASSANDRA-18654 - start publishing CQLSH to PyPI as part of the release process

[guo Maxwell]

Hi ：
First of all, thank you very much for your work. I have a question: what is
your long-term evolution plan for this project? How to achieve long-term
continuous maintenance of this project? I have encountered some situations
where some people's work is related to a certain project, and then they may
have time to maintain, but once they change jobs, they may not have enough
time to do this.  Besides, can you share more about the code management
mechanism?

Jeff Widman  于2023年7月7日周五 08:56写道：

> Myself and Brad Schoening currently maintain
> https://pypi.org/project/cqlsh/ which repackages CQLSH that ships with
> every Cassandra release.
>
> This way:
>
>- anyone who wants a lightweight client to talk to a remote cassandra
>can simply `pip install cqlsh` without having to download the full
>cassandra source, unzip it, etc.
>- it's very easy for folks to use it as scaffolding in their python
>scripts/tooling since they can simply include it in the list of their
>required dependencies.
>
> We currently handle the packaging by waiting for a release, then manually
> copy/pasting the code out of the cassandra source tree into
> https://github.com/jeffwidman/cqlsh which has some additional
> build/python package configuration files, then using standard
> python tooling to publish to PyPI.
>
> Given that our project is simply a build/packaging project, I wanted to
> start a conversation about upstreaming this into core Cassandra. I realize
> that Cassandra has no interest in maintaining lots of build targets... but
> given that cqlsh is written in Python and publishing to PyPI enables DBA's
> to share more complicated tooling built on top of it this seems like a
> natural fit for core cassandra rather than a standalone project.
>
> Goal:
> When a Cassandra release happens, the build/release process automatically
> publishes cqlsh to https://pypi.org/project/cqlsh/.
>
> Non-Goal: This is _not_ about having cassandra itself rely on PyPI. There
> was some initial chatter about that in
> https://issues.apache.org/jira/browse/CASSANDRA-18654, but that adds a
> lot of complexity, and I'm honestly not sure it's a great idea. Even if
> folks later want to go that route, the first hurdle is publishing to PyPI,
> so for now let's keep the scope of the discussion limited to treating PyPI
> purely as a release target, and not as an ingredient to a release.
>
> From an implementation perspective, this should be very straightforward.
> We don't have any differences from the CQLSH source that's in cassandra,
> instead we point folks to make changes to cqlsh in the Cassandra source. In
> fact we've made multiple contributions back to `cqlsh` ourselves and have
> drastically cleaned up the code:
> https://github.com/search?q=repo%3Aapache%2Fcassandra%20is%3Apr%20author%3Ajeffwidman%20author%3Abschoening&type=pullrequests.
> So the only real change is adding the package config files and the build /
> release pipeline.
>
> We realize the Cassandra team isn't python/PyPI experts, so we'd be more
> than happy to help wire this up and maintain it. I am also a maintainer of
> kazoo and kafka-python which are both popular python clients for other
> distributed databases. So I'm very familiar with open source, python, and
> distributed databases.
>
> My one hesitation around this discussion is that I'm a little concerned
> that we might lose the nimbleness we've currently got from having a
> separate project. Ie, if something is screwed up on PyPI / the build
> process, we can quickly get it fixed and get a new release out so that
> users aren't blocked. Would it be possible as part of this process to
> continue that myself/Brad had commit rights to the build process for PyPI?
> To be clear, I'm not asking for commit rights to the Java code or anything
> outside of Python, I just want to be sure that if we go to the trouble of
> working with you to upstream this that there's a commitment from Cassandra
> to keeping this build working, or to letting us be able to fix the build.
> Otherwise there's no point in upstreaming it only for it to go unmaintained
> leaving us looking on helplessly from the sidelines. I'm very flexible here
> on the solution.
>
> Thoughts?
>
> --
>
> *Jeff Widman*
> jeffwidman.com  | 740-WIDMAN-J (943-6265)
> <><
>


-- 
you are the apple of my eye !

Re: [VOTE] CEP 33 - CIDR filtering authorizer

[guo Maxwell]

+1

Nate McCall 于2023年6月29日 周四上午9:25写道：

> +1
>
> On Wed, Jun 28, 2023 at 5:17 AM Shailaja Koppu  wrote:
>
>> Hi Team,
>>
>> (Starting a new thread for VOTE instead of reusing the DISCUSS thread, to
>> follow usual procedure).
>>
>> Please vote on CEP 33 - CIDR filtering authorizer
>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-33%3A+CIDR+filtering+authorizer
>> 
>> .
>>
>> Thanks,
>> Shailaja
>>
> --
you are the apple of my eye !

Re: [VOTE] CEP-8 Datastax Drivers Donation

[guo Maxwell]

+1

Jake Luciani  于2023年6月14日周三 09:21写道：

> +1
>
> On Tue, Jun 13, 2023 at 7:14 PM Nate McCall  wrote:
>
>> +1
>>
>> On Wed, Jun 14, 2023 at 2:15 AM Jeremy Hanna 
>> wrote:
>>
>>> Calling for a vote on CEP-8 [1].
>>>
>>> To clarify the intent, as Benjamin said in the discussion thread [2],
>>> the goal of this vote is simply to ensure that the community is in
>>> favor of the donation. Nothing more.
>>> The plan is to introduce the drivers, one by one. Each driver donation
>>> will need to be accepted first by the PMC members, as it is the case for
>>> any donation. Therefore the PMC should have full control on the pace at
>>> which new drivers are accepted.
>>>
>>> If this vote passes, we can start this process for the Java driver under
>>> the direction of the PMC.
>>>
>>> Jeremy
>>>
>>> 1.
>>> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-8%3A+Datastax+Drivers+Donation
>>> 2. https://lists.apache.org/thread/opt630do09phh7hlt28odztxdv6g58dp
>>>
>>
>
> --
> http://twitter.com/tjake
>


-- 
you are the apple of my eye !

Re: [VOTE] CEP-30 ANN Vector Search

[guo Maxwell]

+1

Dinesh Joshi 于2023年5月26日 周五上午11:08写道：

> +1
>
>
> On May 25, 2023, at 8:45 AM, Jonathan Ellis  wrote:
>
> 
>
> Let's make this official.
>
> CEP:
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-30%3A+Approximate+Nearest+Neighbor%28ANN%29+Vector+Search+via+Storage-Attached+Indexes
>
> POC that demonstrates all the big rocks, including distributed queries:
> https://github.com/datastax/cassandra/tree/cep-vsearch
>
> --
> Jonathan Ellis
> co-founder, http://www.datastax.com
> @spyced
>
> --
you are the apple of my eye !

Re: [DISCUSS] The future of CREATE INDEX

[guo Maxwell]

>
> [POLL] Centralize existing syntax or create new syntax?


1.) CREATE INDEX ... USING  WITH OPTIONS...

and I  think we should keep CREATE CUSTOM INDEX

[POLL] Should there be a default? (YES/NO)


of course  YES

[POLL] What do do with the default?


4.) YAML config/guardrail to require index type selection (not required by
default)



Jonathan Ellis  于2023年5月16日周二 07:18写道：

> On Fri, May 12, 2023 at 1:39 PM Caleb Rackliffe 
> wrote:
>
>> [POLL] Centralize existing syntax or create new syntax?
>>
>
> 1 (Existing)
>
> [POLL] Should there be a default? (YES/NO)
>>
>
> YES
>
>
>> [POLL] What do do with the default?
>>
>
> 1 (Default SAI)
>
>


-- 
you are the apple of my eye !

Re: [DISCUSS] The future of CREATE INDEX

[guo Maxwell]

+1 , as we must Improve the image of your own default indexing ability.

and As for *CREATE CUSTOM INDEX *, should we just left as it is and we can
disable the ability for create SAI through  *CREATE CUSTOM INDEX*  in some
version after 5.0?

for as I know there may be users using this as a plugin-index interface,
like https://github.com/Stratio/cassandra-lucene-index (though these
project may be inactive， But if someone wants to do something similar in
the future, we don't have to stop).



Jonathan Ellis  于2023年5月10日周三 10:01写道：

> +1 for this, especially in the long term.  CREATE INDEX should do the
> right thing for most people without requiring extra ceremony.
>
> On Tue, May 9, 2023 at 5:20 PM Jeremiah D Jordan <
> jeremiah.jor...@gmail.com> wrote:
>
>> If the consensus is that SAI is the right default index, then we should
>> just change CREATE INDEX to be SAI, and legacy 2i to be a CUSTOM INDEX.
>>
>>
>> On May 9, 2023, at 4:44 PM, Caleb Rackliffe 
>> wrote:
>>
>> Earlier today, Mick started a thread on the future of our index creation
>> DDL on Slack:
>>
>> https://the-asf.slack.com/archives/C018YGVCHMZ/p1683527794220019
>>
>> At the moment, there are two ways to create a secondary index.
>>
>> *1.) CREATE INDEX [IF NOT EXISTS] [name] ON  ()*
>>
>> This creates an optionally named legacy 2i on the provided table and
>> column.
>>
>> ex. CREATE INDEX my_index ON kd.tbl(my_text_col)
>>
>> *2.) CREATE CUSTOM INDEX [IF NOT EXISTS] [name] ON  ()
>> USING  [WITH OPTIONS = ]*
>>
>> This creates a secondary index on the provided table and column using the
>> specified 2i implementation class and (optional) parameters.
>>
>> ex. CREATE CUSTOM INDEX my_index ON ks.tbl(my_text_col) USING
>> 'StorageAttachedIndex'
>>
>> (Note that the work on SAI added aliasing, so `StorageAttachedIndex` is
>> shorthand for the fully-qualified class name, which is also valid.)
>>
>> So what is there to discuss?
>>
>> The concern Mick raised is...
>>
>> "...just folk continuing to use CREATE INDEX  because they think CREATE
>> CUSTOM INDEX is advanced (or just don't know of it), and we leave users
>> doing 2i (when they think they are, and/or we definitely want them to be,
>> using SAI)"
>>
>> To paraphrase, we want people to use SAI once it's available where
>> possible, and the default behavior of CREATE INDEX could be at odds w/
>> that.
>>
>> The proposal we seem to have landed on is something like the following:
>>
>> For 5.0:
>>
>> 1.) Disable by default the creation of new legacy 2i via CREATE INDEX.
>> 2.) Leave CREATE CUSTOM INDEX...USING... available by default.
>>
>> (Note: How this would interact w/ the existing secondary_indexes_enabled
>> YAML options isn't clear yet.)
>>
>> Post-5.0:
>>
>> 1.) Deprecate and eventually remove SASI when SAI hits full feature
>> parity w/ it.
>> 2.) Replace both CREATE INDEX and CREATE CUSTOM INDEX w/ something of a
>> hybrid between the two. For example, CREATE INDEX...USING...WITH. This
>> would both be flexible enough to accommodate index implementation selection
>> and prescriptive enough to force the user to make a decision (and wouldn't
>> change the legacy behavior of the existing CREATE INDEX). In this world,
>> creating a legacy 2i might look something like CREATE INDEX...USING
>> `legacy`.
>> 3.) Eventually deprecate CREATE CUSTOM INDEX...USING.
>>
>> Eventually we would have a single enabled DDL statement for index
>> creation that would be minimal but also explicit/able to handle some
>> evolution.
>>
>> What does everyone think?
>>
>>
>>
>
> --
> Jonathan Ellis
> co-founder, http://www.datastax.com
> @spyced
>


-- 
you are the apple of my eye !

Re: [DISCUSS] CEP-28: Reading and Writing Cassandra Data with Spark Bulk Analytics

[guo Maxwell]

Thanks Dinesh ，
That will be great.👍

Dinesh Joshi 于2023年5月4日 周四下午11:06写道：

> Hi Guo,
>
> I would expect that there would be release artifacts for the sidecar as
> well as the library once this functionality is available.
>
> Dinesh
>
> On May 4, 2023, at 12:03 AM, guo Maxwell  wrote:
>
> This is a very meaningful work, thanks , but I would like to ask a
> question that is not particularly related to the cep project's code design 
> itself
> but the project engineering management : what is the future development and
> release plan of this project?
> As far as I know, project Cassandra Sidecar does not actually have an
> finnally release version. I think everyone will definitely not want the
> project code to be merged, but it has been unable to release for a long
> time as this project relies on Cassandra sidecar.
>
> Dinesh Joshi  于2023年5月4日周四 02:35写道：
>
>> If there aren't additional questions / comments I will start the VOTE
>> thread on this CEP tonight.
>>
>> On 2023/05/01 19:50:12 Dinesh Joshi wrote:
>> > Does anybody have any questions that we could answer about this
>> proposal?
>>
>
>
> --
> you are the apple of my eye !
>
>
> --
you are the apple of my eye !

Re: [DISCUSS] CEP-28: Reading and Writing Cassandra Data with Spark Bulk Analytics

[guo Maxwell]

This is a very meaningful work, thanks , but I would like to ask a question
that is not particularly related to the cep project's code design itself
but the project engineering management : what is the future development and
release plan of this project?
As far as I know, project Cassandra Sidecar does not actually have an
finnally release version. I think everyone will definitely not want the
project code to be merged, but it has been unable to release for a long
time as this project relies on Cassandra sidecar.

Dinesh Joshi  于2023年5月4日周四 02:35写道：

> If there aren't additional questions / comments I will start the VOTE
> thread on this CEP tonight.
>
> On 2023/05/01 19:50:12 Dinesh Joshi wrote:
> > Does anybody have any questions that we could answer about this proposal?
>


-- 
you are the apple of my eye !

Re: [DISCUSS] Introduce DATABASE as an alternative to KEYSPACE

[guo Maxwell]

>
> So lets rename Keyspace (Java class) to Database then. If we are concerned
> that looking into logs would be full of "keyspaces" but a user created
> "database" and it is a source of inconsistencies, should not it be somehow
> resolved and unified?
>
> I think that it is just too late to rename keyspace to something else.
> That term is so entrenched over the years in Cassandra-verse that it just
> does not make sense to try to get rid of that.
>
> Also, a "beginner" might not look into the logs at all. I think that they
> will be all over CQL trying to load there some data etc rather than looking
> into the logs  not important. Who is looking into the actual logs while
> logging into the console, whatever DB they are using? These are not
> beginners imho.
>
> BTW keep in mind that all nodetool commands which are using "keyspace"
> terminology would have to be probably accommodated to "database" term as
> well.
>

+1

Miklosovic, Stefan  于2023年4月6日周四 16:01写道：

> So lets rename Keyspace (Java class) to Database then. If we are concerned
> that looking into logs would be full of "keyspaces" but a user created
> "database" and it is a source of inconsistencies, should not it be somehow
> resolved and unified?
>
> I think that it is just too late to rename keyspace to something else.
> That term is so entrenched over the years in Cassandra-verse that it just
> does not make sense to try to get rid of that.
>
> Also, a "beginner" might not look into the logs at all. I think that they
> will be all over CQL trying to load there some data etc rather than looking
> into the logs  not important. Who is looking into the actual logs while
> logging into the console, whatever DB they are using? These are not
> beginners imho.
>
> BTW keep in mind that all nodetool commands which are using "keyspace"
> terminology would have to be probably accommodated to "database" term as
> well.
>
> 
> From: Berenguer Blasi 
> Sent: Thursday, April 6, 2023 9:47
> To: dev@cassandra.apache.org
> Subject: Re: [DISCUSS] Introduce DATABASE as an alternative to KEYSPACE
>
> NetApp Security WARNING: This is an external email. Do not click links or
> open attachments unless you recognize the sender and know the content is
> safe.
>
>
>
>
> One aspect to take into account is that we might not go even get as far
> as having a chance to educate the user. They start the thing up, see a
> wall of logs and then start seeing 'keyspace' (what is that?), etc.
> Everything seems so foreign and out of band to their 'normal' experience
> they just move on to the next option they had in mind.
>
> My 2cts.
>
> On 6/4/23 9:30, Miklosovic, Stefan wrote:
> > I am against simplifying that so much, up to the point that there is
> some implicit replication strategy. While I understand the preferences
> towards having it all "easier", what is wrong with knowing that there are
> some replication strategies and my data will be replicated just once? There
> is also an educational aspect to that.
> >
> > Also, having 4 ways how to create "keyspace" (keyspace, schema,
> database, namespace) feels pretty confusing to me. Are they equal? Why
> four? Is not it just better to have one way of doing that? Having 4 ways to
> do that feels like we do not know how to name it.
> >
> > Somebody already mentioned in this thread that Postgres is quite complex
> in this. Maybe adding "DATABASE" would be OK but anything beyond that
> (NAMESPACE etc) is just too much imo.
> >
> > 
> > From: Jacek Lewandowski 
> > Sent: Thursday, April 6, 2023 8:36
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSS] Introduce DATABASE as an alternative to KEYSPACE
> >
> > NetApp Security WARNING: This is an external email. Do not click links
> or open attachments unless you recognize the sender and know the content is
> safe.
> >
> >
> >
> > Haha... we have opinions against each name :)
> >
> > According to what Caleb said, I don't think all new users start learning
> Cassandra from understanding the replication.
> > There are probably many small projects where Cassandra is used on a
> single node, or bigger projects where people
> > try different things to make some PoC. Understanding the internals,
> architecture of Cassandra is not crucial - they
> >   want to start writing queries as soon as possible and the less prior
> knowledge is required to do that the better.
> >
> > That

Re: [DISCUSS] Introduce DATABASE as an alternative to KEYSPACE

[guo Maxwell]

I think replication_factor or replication is important😄. This concepts
will correspondingly lead to the concept of read and write consistency(ie :
ONE/QUORUM/ALL and so on) that users need to care about.
And the consistency level is very important to cassandra in my opinion.

Our experience is that there are many users who may not initially care
about replication factor and consistency level，then latter a lot of
explanation costs are introduced, and users will also feel that your
database is not easy to use.

So, why not educate users well from the beginning, and there are not many
of these concepts. Just like some data tell users from the beginning that
we are cloud-native, and we separate storage and computing.I think the
replication factor  should be easier to understand than these.


Jacek Lewandowski  于2023年4月6日周四 14:37写道：

> Haha... we have opinions against each name :)
>
> According to what Caleb said, I don't think all new users start learning
> Cassandra from understanding the replication.
> There are probably many small projects where Cassandra is used on a single
> node, or bigger projects where people
> try different things to make some PoC. Understanding the internals,
> architecture of Cassandra is not crucial - they
>  want to start writing queries as soon as possible and the less prior
> knowledge is required to do that the better.
>
> That being said, we should maybe even go further and assume some default
> replication config, like simple 1, so that
> creating a names boils down to a simply CREATE
> KEYSPACE|SCHEMA|DATABASE|NAMESPACE ;
>
> thanks,
> - - -- --- -  ---------
> Jacek Lewandowski
>
>
> czw., 6 kwi 2023 o 04:09 guo Maxwell  napisał(a):
>
>> either KEYSPACE or DATABASE or SCHEMA is better than NAMESPACE
>> NAMESPACE is always used in hbase which is a table store in my mind.
>> For existing users, NAMESPACE may take some time to be accepted. For
>> hbase and cassandra users, it may be necessary to mix the corresponding
>> terms.
>> From the terminology standard of the database， DATABASE or SCHAME may be
>> better , for terminology standard of the nosql database (cassandra),
>> KESYACEP is better.
>>
>>
>> Caleb Rackliffe  于2023年4月6日周四 07:09写道：
>>
>>> KEYSPACE isn’t a terrible name for a namespace that also configures how
>>> keys are replicated. NAMESPACE is accurate but not comprehensive. DATABASE
>>> doesn’t seem to have the advantages of either.
>>>
>>> I’m neutral on NAMESPACE and slightly -1 on DATABASE. It’s hard for me
>>> to believe KEYSPACE is really a stumbling block for new users, especially
>>> when it connotes something those users should understand about them (the
>>> replication configuration).
>>>
>>> On Apr 5, 2023, at 4:16 AM, Aleksey Yeshchenko 
>>> wrote:
>>>
>>> FYI we support SCHEMA as an alias to KEYSPACE today (have since
>>> always). Can use CREATE SCHEMA in place of CREATE KEYSPACE, etc.
>>>
>>> On 4 Apr 2023, at 19:23, Henrik Ingo  wrote:
>>>
>>> I find the Postgres terminology overly complex. Where most SQL databases
>>> will have several *databases*, each containing several *tables*, in
>>> Postgres we have namespaces, databases, schemas and tables...
>>>
>>> Oracle seems to also use the words database, schema and tables. I don't
>>> know if it has namespaces.
>>>
>>> Ah, ok, so SQL Server actually is like Oracle too!
>>>
>>>
>>> So in MySQL, referring unambiguously (aka full path) to a table would be:
>>>
>>> SELECT * FROM mydb.mytable;
>>>
>>> Whereas in Postgresql and Oracle and SQL Server you'd have to:
>>>
>>> SELECT * FROM mydb.myschema.mytable;   /* And I don't even know what
>>> to do with the namespace! */
>>>
>>>
>>> https://www.postgresql.org/docs/current/catalog-pg-namespace.html
>>> https://www.postgresql.org/docs/current/ddl-schemas.html
>>>
>>> https://docs.oracle.com/database/121/ADMQS/GUID-6E0CE8C9-7DC4-450C-BAE0-2E1CDD882993.htm#ADMQS0821
>>>
>>> https://docs.oracle.com/database/121/ADMQS/GUID-8AC1A325-3542-48A0-9B0E-180D633A5BD1.htm#ADMQS081
>>>
>>> https://learn.microsoft.com/en-us/sql/t-sql/statements/create-schema-transact-sql?view=sql-server-ver16
>>>
>>> https://learn.microsoft.com/en-us/sql/t-sql/statements/create-database-transact-sql?view=sql-server-ver16&tabs=sqlpool
>>>
>>> The Microsoft docs perhaps best explain the role of each: The Database
>>> contains the configuration of physi

Re: [DISCUSS] Introduce DATABASE as an alternative to KEYSPACE

[guo Maxwell]

either KEYSPACE or DATABASE or SCHEMA is better than NAMESPACE
NAMESPACE is always used in hbase which is a table store in my mind.
For existing users, NAMESPACE may take some time to be accepted. For hbase
and cassandra users, it may be necessary to mix the corresponding terms.
>From the terminology standard of the database， DATABASE or SCHAME may be
better , for terminology standard of the nosql database (cassandra),
KESYACEP is better.


Caleb Rackliffe  于2023年4月6日周四 07:09写道：

> KEYSPACE isn’t a terrible name for a namespace that also configures how
> keys are replicated. NAMESPACE is accurate but not comprehensive. DATABASE
> doesn’t seem to have the advantages of either.
>
> I’m neutral on NAMESPACE and slightly -1 on DATABASE. It’s hard for me to
> believe KEYSPACE is really a stumbling block for new users, especially when
> it connotes something those users should understand about them (the
> replication configuration).
>
> On Apr 5, 2023, at 4:16 AM, Aleksey Yeshchenko  wrote:
>
> FYI we support SCHEMA as an alias to KEYSPACE today (have since always).
> Can use CREATE SCHEMA in place of CREATE KEYSPACE, etc.
>
> On 4 Apr 2023, at 19:23, Henrik Ingo  wrote:
>
> I find the Postgres terminology overly complex. Where most SQL databases
> will have several *databases*, each containing several *tables*, in
> Postgres we have namespaces, databases, schemas and tables...
>
> Oracle seems to also use the words database, schema and tables. I don't
> know if it has namespaces.
>
> Ah, ok, so SQL Server actually is like Oracle too!
>
>
> So in MySQL, referring unambiguously (aka full path) to a table would be:
>
> SELECT * FROM mydb.mytable;
>
> Whereas in Postgresql and Oracle and SQL Server you'd have to:
>
> SELECT * FROM mydb.myschema.mytable;   /* And I don't even know what
> to do with the namespace! */
>
>
> https://www.postgresql.org/docs/current/catalog-pg-namespace.html
> https://www.postgresql.org/docs/current/ddl-schemas.html
>
> https://docs.oracle.com/database/121/ADMQS/GUID-6E0CE8C9-7DC4-450C-BAE0-2E1CDD882993.htm#ADMQS0821
>
> https://docs.oracle.com/database/121/ADMQS/GUID-8AC1A325-3542-48A0-9B0E-180D633A5BD1.htm#ADMQS081
>
> https://learn.microsoft.com/en-us/sql/t-sql/statements/create-schema-transact-sql?view=sql-server-ver16
>
> https://learn.microsoft.com/en-us/sql/t-sql/statements/create-database-transact-sql?view=sql-server-ver16&tabs=sqlpool
>
> The Microsoft docs perhaps best explain the role of each: The Database
> contains the configuration of physical things like where on disk is the
> database stored. The Schema on the other hand contains "logical" objects
> like tables, views andprocedures.
>
> MongoDB has databases and collections. As an easter egg / inside joke, it
> also supports the command `SHOW TABLES` as a synonym for collections.
>
> A TABLESPACE btw is something else completely:
> https://docs.oracle.com/database/121/ADMQS/GUID-F05EE514-FFC6-4E86-A592-802BA5A49254.htm#ADMQS12053
>
>
>
> Personally I would be in favor of introducing `DATABASE` as a synonym for
> KEYSPACE. The latter could remain the "official" usage.
>
> henrik
>
>
> On Tue, Apr 4, 2023 at 8:32 PM Jacek Lewandowski <
> lewandowski.ja...@gmail.com> wrote:
>
>> While for someone who already knows Cassandra keyspace is something
>> natural, for newcomers it is yet another concept to understand.
>>
>> If namespace is used in PostgreSQL, it sounds even better to me.
>>
>> Thanks,
>> - - -- --- -  -
>> Jacek Lewandowski
>>
>>
>> wt., 4 kwi 2023 o 19:07 Rahul Xavier Singh 
>> napisał(a):
>>
>>> My 2 cents:
>>>
>>> Keeping it keyspace works for me, namespace could be cool also since we
>>> decide where that namespace exists in relation to Datacenters, etc.  In our
>>> case, a Keyspace is more similar to a namespace than it is to a database
>>> since we expect all the UDTs,/UDFs, indexes to refer to only the tables in
>>> that keyspace/namespace.
>>>
>>> Alternatively interesting to observe and throw some fuel into the
>>> discussion , looking at the Postgres (only because there are many
>>> distributed databases that are now PG compliant) :
>>> From the interwebs:
>>> *In PostgreSQL, a schema is a namespace that contains named database
>>> objects such as tables, views, indexes, data types, functions, stored
>>> procedures and operators. A database can contain one or multiple schemas
>>> and each schema belongs to only one database.*
>>> I used to gripe about this but as a platform gets more complex it is
>>> useful to organize PG DBs into schemas. In C* world, I found myself doing
>>> similar things by having a prefix : e.g. appprefix_system1
>>> appprefix_system2 ...
>>>
>>>
>>> Rahul Singh
>>> Chief Executive Officer | Business Platform Architect m: 202.905.2818
>>> e: rahul.si...@anant.us li: http://linkedin.com/in/xingh
>>> 

Re: [DISCUSS] Change the useage of nodetool tablehistograms

[guo Maxwell]

firstly I think anything existing must be reasonable，so ignore option for
tablestats must be a need for the user to use. at least I used it some time
；
secondly in order  to keep this as simple as possible ，I think left the
option unchanged is enough ，because the original usage is simple enough.
user can just print the specified table after set nodetool tablehistorgrams
ks table ，and if there is ten tables in kesypace  ，it is simple for him to
type ten times with different table names which I think at first Only set
with argument ks keyspace name is enough.
When we just want to see eight tables in the ks ，the user should just type
eight table name which ignore two table may be enough.




Bowen Song via dev 于2023年3月23日 周四下午8:07写道：

> I don't think the nodetool tablestats command's parameters should be used
> as a reference implementation for the nodetool tablehistograms command.
> Because:
>
>- the tablehistograms command can take the keyspace and table as two
>separate parameters, but the tablestats command can't.
>- the tablestats command can take keyspace (without table) as a
>parameter, but the tablehistograms command can't.
>
> The introduction of the -ks and -tbs options are unnecessary for the
> tablestats command, because it's parameters are:
>
> nodetool tablestats [| [ table>| [...]]]
>
> Which means any positional parameter without a dot is treated as a
> keyspace name, otherwise it's treated as dot-separated keyspace and table
> name. That, however, does not apply to the nodetool tablehistograms
> command, which led to your workaround - the addition of the -ks and -tbs
> options.
>
> But if you could just forget about the nodetool tablestats command for a
> moment, and look at the nodetool tablehistograms command alone, you will
> see that it's unnecessary to introduce the -ks and -tbs options, because
> the command already takes keyspace name and table name, just in a different
> format.
>
> In addition to that, I would be interested to know how often do people use
> the -i option in the nodetool tablestats command. My best guess is, very
> very rarely.
>
> If my guess is correct, we should keep the nodetool tablehistograms
> command as simple as:
>
> nodetool tablehistograms [  [ [...]] |
>  [ [...]]]
>
> It's good enough if the above can cover the majority of use cases. The
> remaining use cases can be dealt with individually, by multiple invocations
> of the same command or providing it with a script-generated list of tables
> in the  format.
>
> TL;DR: The KISS principle <https://en.wikipedia.org/wiki/KISS_principle>
> should apply here - keep it simple.
>
>
> On 23/03/2023 03:05, guo Maxwell wrote:
>
> Maybe I didn't describe the usage of option "-i" clearly, The reason why
> I think the command argument should be like this :
>
>
> 1. nodetool tablehistograms ks.tb1 or ks tb1  ... //this is *one of the
>> old way *of using tablehistogram. will print out the histograms of tabke
>> ks.tb1 , we keep the old format to print out the table histograms,besides
>> if more than two arguments is provied, suchu as nodetool tablehistograms
>> system.local system_schema.columns system_schema.tables then all tables's
>>   histograms will be printed out (I think this is a bug that not as
>> excepted in the document's decription, we should remind the user that this
>> is an incorrenct usage)
>>
>> 2. nodetool tablehistograms -tbs ks.tb1 ks.tb2  //print out list of
>> tables' histograms with format keyspace.table
>> 3.nodetool tablehistograms -ks ks1 ks2 ks3 ... //print out list of
>> keyspaces histograms
>> 4.nodetool tablehistograms -i -ks ks1 ks2  //print out list of table
>> histograms except for the keyspaces list behind the option -i
>> 5.nodetool tablehistograns -i -tbs ks.tb1 ks.tb2 // print out list
>> tables' histograms except for table in ks.tb1 ks.tb2
>> 6.nodetool tablehistograns -i -tbs ks.tb1 ks.tb2 -ks ks1 // print out
>> list tables' histograms except for table in ks.tb1 ks.tb2 and all tables
>> in ks1
>> 6.none option specified ,then all tables histograms will be print out.// this
>> is *another one of the old way* of using tablehistogram.
>>
>
>  is to make the command format  to be consistent with the format of
> nodetool tablestats, so for users, there will be a unified awareness of
> using these two commands, rather than different commands requiring
> different usage awareness , we can see the description of the tablestats
> doc for option "-i "
>
> Ignore the list of tables and display the remaining tables
>>
>
> that is to say  if -i app

Re: Welcome our next PMC Chair Josh McKenzie

[guo Maxwell]

Congratulations  to Josh McKenzie !!!

Mick Semb Wever  于2023年3月23日周四 16:23写道：

> It is time to pass the baton on, and on behalf of the Apache Cassandra
> Project Management Committee (PMC) I would like to welcome and congratulate
> our next PMC Chair Josh McKenzie (jmckenzie).
>
> Most of you already know Josh, especially through his regular and valuable
> project oversight and status emails, always presenting a balance and
> understanding to the various views and concerns incoming.
>
> Repeating Paulo's words from last year: The chair is an administrative
> position that interfaces with the Apache Software Foundation Board, by
> submitting regular reports about project status and health. Read more about
> the PMC chair role on Apache projects:
> - https://www.apache.org/foundation/how-it-works.html#pmc
> - https://www.apache.org/foundation/how-it-works.html#pmc-chair
> - https://www.apache.org/foundation/faq.html#why-are-PMC-chairs-officers
>
> The PMC as a whole is the entity that oversees and leads the project and
> any PMC member can be approached as a representative of the committee. A
> list of Apache Cassandra PMC members can be found on:
> https://cassandra.apache.org/_/community.html
>


-- 
you are the apple of my eye !

Re: [DISCUSS] Change the useage of nodetool tablehistograms

[guo Maxwell]

and a syntax error.
>
>
> Beyond that, I don't see why the user can't make multiple invocations of
> the nodetool tablehistograms command if they have more complex or
> specific need.
>
> For example, in this case:
>
> *> 6.nodetool tablehistograns -i -tbs ks.tb1 ks.tb2 -ks ks1 // print out
> list tables' histograms except for table in ks.tb1 ks.tb2 and all tables in
> ks1*
>
> The same result can be achieved by concatenating the outputs of the
> following two commands:
>
> nodetool tablehistograms -i ks -i ks1
>
> nodetool tablehistograms ks -i ks.tb1 -i ks.tb2
>
>
> On 22/03/2023 05:12, guo Maxwell wrote:
>
> Thanks everyone , So It seems that it is better to add new parameter
> options to meet our needs, while keeping the original parameter functions
> unaffected to achieve backward compatibility.
> So the new options are :
> 1. nodetool tablehistograms ks.tb1 or ks tb1  ... //this is *one of the
> old way *of using tablehistogram. will print out the histograms of tabke
> ks.tb1 , we keep the old format to print out the table histograms,besides
> if more than two arguments is provied, suchu as nodetool tablehistograms
> system.local system_schema.columns system_schema.tables then all tables's
>   histograms will be printed out (I think this is a bug that not as
> excepted in the document's decription, we should remind the user that this
> is an incorrenct usage)
>
> 2. nodetool tablehistograms -tbs ks.tb1 ks.tb2  //print out list of
> tables' histograms with format keyspace.table
> 3.nodetool tablehistograms -ks ks1 ks2 ks3 ... //print out list of
> keyspaces histograms
> 4.nodetool tablehistograms -i -ks ks1 ks2  //print out list of table
> histograms except for the keyspaces list behind the option -i
> 5.nodetool tablehistograns -i -tbs ks.tb1 ks.tb2 // print out list tables'
> histograms except for table in ks.tb1 ks.tb2
> 6.nodetool tablehistograns -i -tbs ks.tb1 ks.tb2 -ks ks1 // print out list
> tables' histograms except for table in ks.tb1 ks.tb2 and all tables in ks1
> 6.none option specified ,then all tables histograms will be print out.// this
> is *another one of the old way* of using tablehistogram.
>
> So we add some more options like "-i", "-ks", "-tbs" , we can combine
> these options  and we can also use any of them individually, besides, we
> can also use the tool through old way if a table with format ks.tb is
> provied.
>
>
> Jeremiah D Jordan  于2023年3月16日周四 23:14写道：
>
> -1 on any change which breaks the previously documented usage.
> +1 any additions to what the tool can do without breaking previously
> documented behavior.
>
> On Mar 16, 2023, at 7:42 AM, Josh McKenzie  wrote:
>
> We could also consider augmenting the tool with new named arguments with
> the functionality you described and leave the positional usage intact.
>
> On Thu, Mar 16, 2023, at 6:43 AM, Bowen Song via dev wrote:
>
> The documented command options are:
>
> nodetool tablehistograms [  | ]
>
>
>
> That means one parameter will be treated as dot separated keyspace and
> table. Alternatively, two parameters will be treated as the keyspace and
> table respectively.
>
> To remain compatible with the documented behaviour, my suggestion is to
> change the command options to:
>
> nodetool tablehistograms [  [ [...]] |
>  [ [...]]]
>
> Feel free to add the "all except ..." feature to the above.
>
> This doesn't break backward compatibility in documented ways. It only
> changes the undocumented behaviour. If someone is using the undocumented
> behaviour, they must know things may break when the software is upgraded.
> We can just add a line to the NEWS.txt and let them update their scripts.
>
>
> On 16/03/2023 08:53, guo Maxwell wrote:
>
> Hello everyone :
> The nodetool tablehistograms have one argument which you can fill with
> only one table name with the format "keyspace_name.table_name
> /keyspace_name table_name", so that you can get the table histograms of the
> specied table.
>
> And  if none arguments is set, all the tables' histograms will be print
> out.And if more than 2 arguments (nomatter the format is right or wrong) are
> set , all the tables' histograms will also be print out too(Which is a bug
> In my mind).
>
> So the usage of nodetool tablehistograms has some usage restrictions, That
> is either output one , or all informations.
>
> As CASSANDRA-18296 <https://issues.apache.org/jira/browse/CASSANDRA-18296> 
> described
> , I will change the usage of nodetool tablehistograms, which support the
> feature below:
> 1. nodetool tablehistograms ks.tb1 ks.tb2  //print out list of tables'
> histograms with format keyspace.table
> 2.nodetool tablehistograms ks1 ks2 ks3 ... //print out list of keyspaces
> histograms
> 3.nodetool tablehistograms -i ks1 ks2  //print out list of table
> histograms except for the keyspaces list behind the option -i
> 4.nodetool tablehistograns -i ks ks.tb // print out list tables'
> histograms except for table in keyspace ks and ks.tb table.
> 5.none option specified ,then all tables histograms will be print out.
>
> The usage will breaks compatibility with how it was done previously, and
> as this is a user facing tool.
>
> So, What do you think?
>
> Thanks~~~
>
>
>
>
> --
> you are the apple of my eye !
>
>
>

-- 
you are the apple of my eye !

Re: [DISCUSS] Change the useage of nodetool tablehistograms

[guo Maxwell]

Thanks everyone , So It seems that it is better to add new parameter
options to meet our needs, while keeping the original parameter functions
unaffected to achieve backward compatibility.
So the new options are :
1. nodetool tablehistograms ks.tb1 or ks tb1  ... //this is *one of the old
way *of using tablehistogram. will print out the histograms of tabke ks.tb1
, we keep the old format to print out the table histograms,besides if more
than two arguments is provied, suchu as nodetool tablehistograms
system.local system_schema.columns system_schema.tables then all
tables's  histograms
will be printed out (I think this is a bug that not as excepted in the
document's decription, we should remind the user that this is an incorrenct
usage)

2. nodetool tablehistograms -tbs ks.tb1 ks.tb2  //print out list of
tables' histograms with format keyspace.table
3.nodetool tablehistograms -ks ks1 ks2 ks3 ... //print out list of
keyspaces histograms
4.nodetool tablehistograms -i -ks ks1 ks2  //print out list of table
histograms except for the keyspaces list behind the option -i
5.nodetool tablehistograns -i -tbs ks.tb1 ks.tb2 // print out list tables'
histograms except for table in ks.tb1 ks.tb2
6.nodetool tablehistograns -i -tbs ks.tb1 ks.tb2 -ks ks1 // print out list
tables' histograms except for table in ks.tb1 ks.tb2 and all tables in ks1
6.none option specified ,then all tables histograms will be print out.// this
is *another one of the old way* of using tablehistogram.

So we add some more options like "-i", "-ks", "-tbs" , we can combine these
options  and we can also use any of them individually, besides, we can also
use the tool through old way if a table with format ks.tb is provied.


Jeremiah D Jordan  于2023年3月16日周四 23:14写道：

> -1 on any change which breaks the previously documented usage.
> +1 any additions to what the tool can do without breaking previously
> documented behavior.
>
> On Mar 16, 2023, at 7:42 AM, Josh McKenzie  wrote:
>
> We could also consider augmenting the tool with new named arguments with
> the functionality you described and leave the positional usage intact.
>
> On Thu, Mar 16, 2023, at 6:43 AM, Bowen Song via dev wrote:
>
> The documented command options are:
>
> nodetool tablehistograms [  | ]
>
>
>
> That means one parameter will be treated as dot separated keyspace and
> table. Alternatively, two parameters will be treated as the keyspace and
> table respectively.
>
> To remain compatible with the documented behaviour, my suggestion is to
> change the command options to:
>
> nodetool tablehistograms [  [ [...]] |
>  [ [...]]]
>
> Feel free to add the "all except ..." feature to the above.
>
> This doesn't break backward compatibility in documented ways. It only
> changes the undocumented behaviour. If someone is using the undocumented
> behaviour, they must know things may break when the software is upgraded.
> We can just add a line to the NEWS.txt and let them update their scripts.
>
>
> On 16/03/2023 08:53, guo Maxwell wrote:
>
> Hello everyone :
> The nodetool tablehistograms have one argument which you can fill with
> only one table name with the format "keyspace_name.table_name
> /keyspace_name table_name", so that you can get the table histograms of the
> specied table.
>
> And  if none arguments is set, all the tables' histograms will be print
> out.And if more than 2 arguments (nomatter the format is right or wrong) are
> set , all the tables' histograms will also be print out too(Which is a bug
> In my mind).
>
> So the usage of nodetool tablehistograms has some usage restrictions, That
> is either output one , or all informations.
>
> As CASSANDRA-18296 <https://issues.apache.org/jira/browse/CASSANDRA-18296> 
> described
> , I will change the usage of nodetool tablehistograms, which support the
> feature below:
> 1. nodetool tablehistograms ks.tb1 ks.tb2  //print out list of tables'
> histograms with format keyspace.table
> 2.nodetool tablehistograms ks1 ks2 ks3 ... //print out list of keyspaces
> histograms
> 3.nodetool tablehistograms -i ks1 ks2  //print out list of table
> histograms except for the keyspaces list behind the option -i
> 4.nodetool tablehistograns -i ks ks.tb // print out list tables'
> histograms except for table in keyspace ks and ks.tb table.
> 5.none option specified ,then all tables histograms will be print out.
>
> The usage will breaks compatibility with how it was done previously, and
> as this is a user facing tool.
>
> So, What do you think?
>
> Thanks~~~
>
>
>

-- 
you are the apple of my eye !

Re: [EXTERNAL] Re: Cassandra CI Status 2023-01-07

[guo Maxwell]

Hi all :
Here comes Cassandra CI status for  2023-3-13 - 2023-23-179 :

*** CASSANDRA-18338 <https://issues.apache.org/jira/browse/CASSANDRA-18338>
 -  dtest.bootstrap_test.TestBootstrap.test_cleanup trunk
***  CASSANDRA-18338 <https://issues.apache.org/jira/browse/CASSANDRA-18338>
 - test:
org.apache.cassandra.distributed.test.ByteBuddyExamplesTest.countTest ,
this failed twice with jdk 8 and jdk 11, on trunk and  4.1
others are some timeout exception.


Mick Semb Wever  于2023年2月28日周二 05:37写道：

> New failures from Build Lead week 8
>
> *** CASSANDRA-18290 – SecondaryIndexTest.testUpdatesToMemtableData
> 4.1, row did not delete
>
> *** CASSANDRA-18289 –
>
> sslnodetonode_test.TestNodeToNodeSSLEncryption.test_ssl_client_auth_required_fail
> trunk, error not found in logs
>
> *** CASSANDRA-18287 –
> InsertUpdateIfConditionTest.testMultiExistConditionOnSameRowClustering
> trunk,
>
> *** CASSANDRA-18288 – TopPartitionsTest.basicRegularTombstonesTest
> trunk, missing tombstones
>
> ***CASSANDRA-18286 – TTLTest.testCapWarnExpirationOverflowPolicy
> negative deletion and expiry value
> 3.0, MarshalException: A local expiration time should not be negative
>
>
>
> On Tue, 21 Feb 2023 at 03:41, guo Maxwell  wrote:
> >
> > Hi all :
> > Here comes Cassandra CI status for  2023-2-13 - 2023-2-17 :
> >
> > *** CASSANDRA-18274 - Test
> Failures:org.apache.cassandra.utils.binlog.BinLogTest.testTruncationReleasesLogSpace-compression
> -linked in 4.1
> > Other tests below are time out exceptions, and we can ignore them as
> it's considered test-infrastructure failures. Which we are working on
> separately (CASSANDRA-18137), and I have already modify this notification
> in Build Lead page.
> > *** CASSANDRA-18273: Timeout occurred. Please note the time in the
> report does not reflect the time until the timeout. - linked in trunk, 4.0
> > *** CASSANDRA-11493:dtest failure in
> consistency_test.TestAccuracy.test_simple_strategy_users - it is timeout
> exception for CASSANDRA-11493 so I do not reopen it .
> >
> >
> >
> > German Eichberger via dev  于2023年2月14日周二
> 00:29写道：
> >>
> >> First, one of my learnings was that a ticket assigned to an issue in
> one branch of butler doesn't carry to another. So always search.
> >>
> >> New failures from build lead week 7:
> >>
> >> I created a Jira filter for finding the tickets I created:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20issuetype%20%3D%20Bug%20AND%20component%20in%20(%22Test%2Fdtest%2Fjava%22%2C%20%22Test%2Fdtest%2Fpython%22%2C%20%22Test%2Ffuzz%22%2C%20%22Test%2Funit%22)%20AND%20created%20%3E%3D%20-7d%20AND%20reporter%20in%20(xgerman42)
> >>
> >> *** CASSANDRA-18257 - Test Failures: 
> >> org.apache.cassandra.net.ProxyHandlerConnectionsTest.testExpireSome
> - linked in 4.0, 4.1, trunk
> >> *** CASSANDRA-18253 - Test Failures: dtest
> repair_tests.repair_test.TestRepair.test_simple_sequential_repair - linked
> in 4.0, trunk
> >> *** CASSANDRA-18246 - Test Failures:
> org.apache.cassandra.cql3.validation.operations.TTLTest.testCapNoWarnExpirationOverflowPolicy
> - linked in 3.11
> >> *** CASSANDRA-18245 - Test Failures:
> org.apache.cassandra.db.compaction.CompactionsTest.testDontPurgeAccidentally
> - linked in 3.11
> >> -
> >>
> >> 
> >> From: Dan Jatnieks 
> >> Sent: Friday, February 10, 2023 2:42 PM
> >> To: dev@cassandra.apache.org ; Claude
> Warren, Jr 
> >> Subject: [EXTERNAL] Re: Cassandra CI Status 2023-01-07
> >>
> >> You don't often get email from d...@datastax.com. Learn why this is
> important
> >> New Failures from Build Lead Week 6:
> >>
> >> *** CASSANDRA-18021 - Flaky
> org.apache.cassandra.distributed.test.ReprepareTestOldBehaviour#testReprepareMixedVersionWithoutReset
> >> - This existing ticket has been linked in butler to new failures on 3.11
> >>
> >> *** CASSANDRA-17608 - Fix testMetricsWithRebuildAndStreamingToTwoNodes
> >> - Re-opened as intermittent failure occurred in build 1445 on trunk
> >>
> >> Several new failures had only a single occurrence; no new tickets were
> opened during this time.
> >>
> >>
> >>
> >> On Fri, Feb 10, 2023 at 12:44 AM Claude Warren, Jr via dev <
> dev@cassandra.apache.org> wrote:
> >>
> >> New Failures from Build Lead Week 5
> >>
> >> *** CASSANDRA-18198 - "AttributeError: module 'py' has no attribute
> 'io'" reported in multiple tests

[DISCUSS] Change the useage of nodetool tablehistograms

[guo Maxwell]

Hello everyone :
The nodetool tablehistograms have one argument which you can fill with only
one table name with the format "keyspace_name.table_name /keyspace_name
table_name", so that you can get the table histograms of the specied table.

And  if none arguments is set, all the tables' histograms will be print
out.And if more than 2 arguments (nomatter the format is right or wrong) are
set , all the tables' histograms will also be print out too(Which is a bug
In my mind).

So the usage of nodetool tablehistograms has some usage restrictions, That
is either output one , or all informations.

As CASSANDRA-18296
 described
, I will change the usage of nodetool tablehistograms, which support the
feature below:
1. nodetool tablehistograms ks.tb1 ks.tb2  //print out list of tables'
histograms with format keyspace.table
2.nodetool tablehistograms ks1 ks2 ks3 ... //print out list of keyspaces
histograms
3.nodetool tablehistograms -i ks1 ks2  //print out list of table
histograms except for the keyspaces list behind the option -i
4.nodetool tablehistograns -i ks ks.tb // print out list tables'
histograms except
for table in keyspace ks and ks.tb table.
5.none option specified ,then all tables histograms will be print out.

The usage will breaks compatibility with how it was done previously, and as
this is a user facing tool.

So, What do you think?

Thanks~~~

Re: [jira] [Updated] (CASSANDRA-18305) Enhance nodetool compactionstats with existing MBean metrics

[guo Maxwell]

Had a similar need early and have been trying to solve it. Looking forward
to this patch.

Brad Schoening (Jira) 于2023年3月8日 周三下午12:15写道：

>
>  [
> https://issues.apache.org/jira/browse/CASSANDRA-18305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
> ]
>
> Brad Schoening updated CASSANDRA-18305:
> ---
> Description:
> Nodetool compactionstats reports only on active compactions, if nothing is
> active, you see only:
> {quote}$nodetool compactionstats
>
> pending tasks: 0
> {quote}
> but in the MBean Compaction/TotalCompactionsCompleted there are recent
> statistic in events/second for:
>  * Count
>  * FifteenMinueRate
>  * FiveMinueRate
>  * MeanRate
>  * OneMinuteRate
>
> It would be useful to see in addition:
> {quote}pending tasks: 0
>
> compactions completed: 20
>
> 1 minute rate:0/second
>
>5 minute rate:2.3/second
>
>   15 minute rate:   4.6/second
> {quote}
> Also, since compaction_throughput_mb_per_sec is a throttling parameter in
> cassandra.yaml, it would be nice to show the actual compaction throughput
> and be able to observe if you're close to the limit.  I.e.,
> {quote}compaction throughput 13.2 MBps / 16 MBps (82.5%)
> {quote}
>
>   was:
> Nodetool compactionstats reports only on active compactions, if nothing is
> active, you see only:
> {quote}$nodetool compactionstats
>
> pending tasks: 0
> {quote}
> but in the MBean Compaction/TotalCompactionsCompleted there are recent
> statistic in events/second for:
>  * Count
>  * FifteenMinueRate
>  * FiveMinueRate
>  * MeanRate
>  * OneMinuteRate
>
> It would be useful to see in addition:
> {quote}pending tasks: 0
>
> compactions completed: 20
>
> 1 minute rate:0/second
>
>5 minute rate:2.3/second
>
>   15 minute rate:   4.6/second
> {quote}
> Also, since compaction_throughput_mb_per_sec is a throttling parameter in
> cassandra.yaml, it would be nice to report the actual compaction throughput
> and be able to observe if you're close to the limit.  I.e.,
>
>   compaction throughput 13.2 MBps / 16 MBps (82.5%)
>
>
> > Enhance nodetool compactionstats with existing MBean metrics
> > 
> >
> > Key: CASSANDRA-18305
> > URL:
> https://issues.apache.org/jira/browse/CASSANDRA-18305
> > Project: Cassandra
> >  Issue Type: Improvement
> >  Components: Tool/nodetool
> >Reporter: Brad Schoening
> >Assignee: Stefan Miklosovic
> >Priority: Normal
> >
> > Nodetool compactionstats reports only on active compactions, if nothing
> is active, you see only:
> > {quote}$nodetool compactionstats
> > pending tasks: 0
> > {quote}
> > but in the MBean Compaction/TotalCompactionsCompleted there are recent
> statistic in events/second for:
> >  * Count
> >  * FifteenMinueRate
> >  * FiveMinueRate
> >  * MeanRate
> >  * OneMinuteRate
> > It would be useful to see in addition:
> > {quote}pending tasks: 0
> > compactions completed: 20
> > 1 minute rate:0/second
> >5 minute rate:2.3/second
> >   15 minute rate:   4.6/second
> > {quote}
> > Also, since compaction_throughput_mb_per_sec is a throttling parameter
> in cassandra.yaml, it would be nice to show the actual compaction
> throughput and be able to observe if you're close to the limit.  I.e.,
> > {quote}compaction throughput 13.2 MBps / 16 MBps (82.5%)
> > {quote}
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.20.10#820010)
>
> -
> To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: commits-h...@cassandra.apache.org
>
> --
you are the apple of my eye !

Re: [EXTERNAL] Re: Cassandra CI Status 2023-01-07

[guo Maxwell]

Hi all :
Here comes Cassandra CI status for  2023-2-13 - 2023-2-17 :

*** CASSANDRA-18274 
 - Test
Failures:org.apache.cassandra.utils.binlog.BinLogTest.testTruncationReleasesLogSpace-compression
-linked in 4.1
Other tests below are time out exceptions, and we can ignore them as it's
considered test-infrastructure failures. Which we are working on separately
(CASSANDRA-18137 ),
and I have already modify this notification in Build Lead page.
*** CASSANDRA-18273 
: Timeout occurred. Please note the time in the report does not reflect the
time until the timeout. - linked in trunk, 4.0
*** CASSANDRA-11493
:dtest
failure in consistency_test.TestAccuracy.test_simple_strategy_users - it is
timeout exception for CASSANDRA-11493 so I do not reopen it .



German Eichberger via dev  于2023年2月14日周二 00:29写道：

> First, one of my learnings was that a ticket assigned to an issue in one
> branch of butler doesn't carry to another. So always search.
>
> New failures from build lead week 7:
>
> I created a Jira filter for finding the tickets I created:
> https://issues.apache.org/jira/issues/?jql=project%20%3D%20CASSANDRA%20AND%20issuetype%20%3D%20Bug%20AND%20component%20in%20(%22Test%2Fdtest%2Fjava%22%2C%20%22Test%2Fdtest%2Fpython%22%2C%20%22Test%2Ffuzz%22%2C%20%22Test%2Funit%22)%20AND%20created%20%3E%3D%20-7d%20AND%20reporter%20in%20(xgerman42)
>
> *** CASSANDRA-18257
>  - Test Failures:
> org.apache.cassandra.net.ProxyHandlerConnectionsTest.testExpireSome -
> linked in 4.0, 4.1, trunk
> *** CASSANDRA-18253
>  - Test Failures:
> dtest repair_tests.repair_test.TestRepair.test_simple_sequential_repair -
> linked in 4.0, trunk
> *** CASSANDRA-18246
>  - Test Failures:
> org.apache.cassandra.cql3.validation.operations.TTLTest.testCapNoWarnExpirationOverflowPolicy
> - linked in 3.11
> *** CASSANDRA-18245
>  - Test Failures:
> org.apache.cassandra.db.compaction.CompactionsTest.testDontPurgeAccidentally
> - linked in 3.11
> -
>
> --
> *From:* Dan Jatnieks 
> *Sent:* Friday, February 10, 2023 2:42 PM
> *To:* dev@cassandra.apache.org ; Claude Warren,
> Jr 
> *Subject:* [EXTERNAL] Re: Cassandra CI Status 2023-01-07
>
> You don't often get email from d...@datastax.com. Learn why this is
> important 
> New Failures from Build Lead Week 6:
>
> *** CASSANDRA-18021 - Flaky
> org.apache.cassandra.distributed.test.ReprepareTestOldBehaviour#testReprepareMixedVersionWithoutReset
> - This existing ticket has been linked in butler to new failures on 3.11
>
> *** CASSANDRA-17608 - Fix testMetricsWithRebuildAndStreamingToTwoNodes
> - Re-opened as intermittent failure occurred in build 1445 on trunk
>
> Several new failures had only a single occurrence; no new tickets were
> opened during this time.
>
>
>
> On Fri, Feb 10, 2023 at 12:44 AM Claude Warren, Jr via dev <
> dev@cassandra.apache.org> wrote:
>
> New Failures from Build Lead Week 5
>
> *** CASSANDRA-18198 - "AttributeError: module 'py' has no attribute 'io'"
> reported in multiple tests
> - reported in 4.1, 3.11, and 3.0
> - identified as a possible class loader issue associated with
> CASSANDRA-18150
>
> *** CASSANDRA-18191 - Native Transport SSL tests failing
> - TestNativeTransportSSL.test_connect_to_ssl and
> TestNativeTransportSSL.test_connect_to_ssl (novnode)
> - TestNativeTransportSSL.test_connect_to_ssl_optional and
> TestNativeTransportSSL.test_connect_to_ssl_optional (nvnode)
>
>
> On Mon, Jan 23, 2023 at 10:10 PM Caleb Rackliffe 
> wrote:
>
> New failures from Build Lead Week 4:
>
> *** CASSANDRA-18188 - Test failure in
> upgrade_tests.cql_tests.cls.test_limit_ranges
> - trunk
> - AttributeError: module 'py' has no attribute 'io'
>
> *** CASSANDRA-18189 - Test failure in
> cqlsh_tests.test_cqlsh_copy.TestCqlshCopy.test_bulk_round_trip_with_timeouts
> - 4.0
> - assert 10 == 94764
> - other failures currently open in this test class, but at least
> superficially, different errors (see CASSANDRA-17322, CASSANDRA-18162)
>
> Timeouts continue to manifest in many places.
>
> On Sun, Jan 15, 2023 at 6:02 AM Mick Semb Wever  wrote:
>
> *** The Butler (Build Lead)
>
> The introduction of Butler and the Build Lead was a wonderful
> improvement to our CI efforts.  It has brought a lot of hygiene in
> listing out flakies as they happened.  Noted that this has in-turn
> increased the burden in getting our major releases out, but that's to
> be seen as a one-off cost.
>
>
>
> New Failures from Build Lead Week 3.
>
>
> *** CASSANDRA-18156
> – 
> repair_tests.deprecated_repair_test.T

Re: Downgradability

[guo Maxwell]

It seems “An alternative solution is to implement/complete CASSANDRA-8110
” can give us more
options if it is finished😉

Branimir Lambov 于2023年2月20日 周一下午11:03写道：

> Hi everyone,
>
> There has been a discussion lately about changes to the sstable format in
> the context of being able to abort a cluster upgrade, and the fact that
> changes to sstables can prevent downgraded nodes from reading any data
> written during their temporary operation with the new version.
>
> Most of the discussion is in CASSANDRA-18134
> , and is spreading
> into CASSANDRA-14277
>  and
> CASSANDRA-17698 ,
> none of which is a good place to discuss the topic seriously.
>
> Downgradability is a worthy goal and is listed in the current roadmap. I
> would like to open a discussion here on how it would be achieved.
>
> My understanding of what has been suggested so far translates to:
> - avoid changes to sstable formats;
> - if there are changes, implement them in a way that is
> backwards-compatible, e.g. by duplicating data, so that a new version is
> presented in a component or portion of a component that legacy nodes will
> not try to read;
> - if the latter is not feasible, make sure the changes are only applied if
> a feature flag has been enabled.
>
> To me this approach introduces several risks:
> - it bloats file and parsing complexity;
> - it discourages improvement (e.g. CASSANDRA-17698 is no longer a LHF
> ticket once this requirement is in place);
> - it needs care to avoid risky solutions to address technical issues with
> the format versioning (e.g. staying on n-versions for 5.0 and needing a
> bump for a 4.1 bugfix might require porting over support for new features);
> - it requires separate and uncoordinated solutions to the problem and
> switching mechanisms for each individual change.
>
> An alternative solution is to implement/complete CASSANDRA-8110
> , which provides a
> method of writing sstables for a target version. During upgrades, a node
> could be set to produce sstables corresponding to the older version, and
> there is a very straightforward way to implement modifications to formats
> like the tickets above to conform to its requirements.
>
> What do people think should be the way forward?
>
> Regards,
> Branimir
>
>
> --
you are the apple of my eye !

Re: [jira] [Commented] (CASSANDRA-17698) sstabledump errors when dumping data from index

[guo Maxwell]

for old version‘s index data using this version’s dump code ，I think there
will not be much difference（may be only exception message‘s difference），It
is backwards compactibility I think .But I change the content of
sstable（adding a new base type for partitionordertype）.😅

Benedict Elliott Smith (Jira) 于2023年2月20日 周一下午5:07写道：

>
> [
> https://issues.apache.org/jira/browse/CASSANDRA-17698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691078#comment-17691078
> ]
>
> Benedict Elliott Smith commented on CASSANDRA-17698:
> 
>
> Could somebody ELI5 why this needs to break backwards compatibility?
>
> > sstabledump errors when dumping data from index
> > ---
> >
> > Key: CASSANDRA-17698
> > URL:
> https://issues.apache.org/jira/browse/CASSANDRA-17698
> > Project: Cassandra
> >  Issue Type: Bug
> >  Components: Tool/sstable
> >Reporter: Stefan Miklosovic
> >Assignee: maxwellguo
> >Priority: Normal
> > Fix For: 5.x
> >
> >  Time Spent: 12h 40m
> >  Remaining Estimate: 0h
> >
> > {code:java}
> > cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class':
> 'SimpleStrategy', 'replication_factor': 1};
> > cqlsh> CREATE TABLE ks1.tb1 ( id text, name text, primary key (id));
> > cqlsh> CREATE INDEX IF NOT EXISTS ON ks1.tb1(name);
> > cqlsh> INSERT INTO ks1.tb1 (id, name ) VALUES ( '1', 'Joe');
> > cqlsh> exit
> > ./bin/nodetool flush
> > ./tools/bin/sstabledump
> data/data/ks1/tb1-1c3c5f10ee4711ecab82eda2f44200b3/.tb1_name_idx/nb-1-big-Data.db
> > [
> >   {
> > "partition" : {
> >   "key" : [ "Joe" ],
> >   "position" : 0
> > },
> > "rows" : [
> >   {
> > "type" : "row",
> > "position" : 17,
> > "clustering" : [ ] } ] } ]Exception in thread "main"
> java.lang.UnsupportedOperationException
> > at
> org.apache.cassandra.db.marshal.PartitionerDefinedOrder.toJSONString(PartitionerDefinedOrder.java:87)
> > at
> org.apache.cassandra.db.marshal.AbstractType.toJSONString(AbstractType.java:187)
> > at
> org.apache.cassandra.tools.JsonTransformer.serializeClustering(JsonTransformer.java:372)
> > at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:269)
> > at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:235)
> > at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> > at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> > at java.util.Iterator.forEachRemaining(Iterator.java:116)
> > at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> > at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> > at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> > at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> > at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> > at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> > at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> > at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:113)
> > at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:214) {code}
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.20.10#820010)
>
> -
> To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: commits-h...@cassandra.apache.org
>
> --
you are the apple of my eye !

Re: [VOTE] CEP-21 Transactional Cluster Metadata

[guo Maxwell]

+1

Yifan Cai 于2023年2月8日 周三下午4:11写道：

> +1
> --
> *From:* Jon Haddad 
> *Sent:* Tuesday, February 7, 2023 4:55:51 PM
> *To:* dev@cassandra.apache.org 
> *Subject:* Re: [VOTE] CEP-21 Transactional Cluster Metadata
>
> +1
>
> On 2023/02/06 16:15:19 Sam Tunnicliffe wrote:
> > Hi everyone,
> >
> > I would like to start a vote on this CEP.
> >
> > Proposal:
> >
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-21%3A+Transactional+Cluster+Metadata
> >
> > Discussion:
> > https://lists.apache.org/thread/h25skwkbdztz9hj2pxtgh39rnjfzckk7
> >
> > The vote will be open for 72 hours.
> > A vote passes if there are at least three binding +1s and no binding
> vetoes.
> >
> > Thanks,
> > Sam
>
-- 
you are the apple of my eye !

Re: Should we change 4.1 to G1 and offheap_objects ?

[guo Maxwell]

same with David Capwell，+1 on updating NEWS in 4.1.x and really change in
4.x /5.0

David Capwell 于2023年1月13日 周五上午3:11写道：

> I am cool with updating NEWS in 4.1.1 to recommend the change and change
> it in 4.2/5.0
>
>
> On Jan 12, 2023, at 10:56 AM, Josh McKenzie  wrote:
>
> Potential compromise: We change it in trunk, and we NEWS.txt in the minor
> about that change in trunk, why, and recommend users consider qualifying
> the same change on their 4.1 release.
>
> In case it's not clear from me:
> +1 to changing on trunk for 5.0 here
> -1 to changing on minor release given how little (i.e. nonexistent) perf
> testing we have on the OSS project right now.
>
> On Thu, Jan 12, 2023, at 11:47 AM, Paulo Motta wrote:
>
> I tend to agree with Aleksey's sentiment. Why do we need to change the
> default in a minor release if we already provide G1 options for users that
> want to opt-in?
>
> On Thu, Jan 12, 2023 at 9:46 AM Aleksey Yeshchenko 
> wrote:
>
> Switching a major default in a minor release is way worse than doing it in
> a GA - less notice and visibility, many folks don’t even read minor version
> NEWS.txt before upgrading.
>
> Trunk is fine by me though.
>
> > On 12 Jan 2023, at 13:14, Mick Semb Wever  wrote:
> >
> >> Ok, wrt G1 default, this is won't go ahead for 4.1-rc1
> >>
> >> We can revisit it for 4.1.x
> >>
> >> We have a lot of voices here adamantly positive for it, and those of us
> that have done the performance testing over the years know why. But being
> called to prove it is totally valid, if you have data to any such tests
> please add them to the ticket 18027
> >
> >
> > Revisiting. Are there any vetoes to making G1 the default (and
> > updating the G1 settings, see the patch on
> > https://issues.apache.org/jira/browse/CASSANDRA-18027 ) for 4.1.1 ?
> >
> > IIUC , the summary of this thread till now was: there were no vetoes
> > to the change in trunk, but there were vetoes to 4.1.0 (because we
> > were inside the beta to GA window), and there was a desire to have
> > benchmarking data presented.
> >
> > WRT benchmarking, we have a separate thread for performance testing in
> > the project.  The ticket admittedly does not do its due diligence on
> > data presentation and analysis of smaller heaps: a precedent we should
> > be creating; but instead relies upon experience from many. Are we ok
> > with this this time around, or shall the patch only be applied to
> > trunk (where we have no choice w/ jdk17 landing)?
>
>
> --
you are the apple of my eye !

Re: Event Report -- Cassandra Day China 2022 was a big success

[guo Maxwell]

thank you tom，and hoping there will be more and more Cassandra users
&developers in China.

Mick Semb Wever 于2022年12月28日 周三上午5:04写道：

> > Thank you all again for introducing and approving of this Cassandra Day
> China event, we also thank the people at China Golden Bridge for their
> dedication throughout the weeks of preparation, without these, we wouldn’t
> be able to pull this off in such a short amount of time.
> >
> > We look forward to future cooperation in the Cassandra community in
> China,
>
>
> Thank you Tom for the retro written up! I am very happy that you put
> the effort into organising this and that it was a huge success. What a
> great contribution and initiative to our community. We would all love
> to hear more from these Cassandra production users. With the language
> challenges that present I am interested to hear any and all
> suggestions, particularly any successful approaches other Apache
> projects have taken.
>
> For those that missed this: the turn around on it happened quickly;
> the blog post for it is here:
> https://cassandra.apache.org/_/events/20221222-cday-china.html
>
-- 
you are the apple of my eye !

Re: [VOTE] Release Apache Cassandra 4.1.0 (take2)

[guo Maxwell]

+1

Jeremiah D Jordan 于2022年12月10日 周六上午5:59写道：

> +1 nb
>
>
> On Dec 7, 2022, at 3:40 PM, Mick Semb Wever  wrote:
>
>
> Proposing the (second) test build of Cassandra 4.1.0 for release.
>
> sha1: f9e033f519c14596da4dc954875756a69aea4e78
> Git:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.1.0-tentative
> Maven Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1282/org/apache/cassandra/cassandra-all/4.1.0/
>
> The Source and Build Artifacts, and the Debian and RPM packages and
> repositories, are available here:
> https://dist.apache.org/repos/dist/dev/cassandra/4.1.0/
>
> The vote will be open for 96 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>
> [1]: CHANGES.txt:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/4.1.0-tentative
> [2]: NEWS.txt:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/4.1.0-tentative
>
>
> --
you are the apple of my eye !

Re: [VOTE] Release Apache Cassandra 4.1.0 GA

[guo Maxwell]

+1

Mick Semb Wever 于2022年12月5日 周一下午5:33写道：

>
> Proposing the test build of Cassandra 4.1.0 GA for release.
>
> sha1: b807f97b37933fac251020dbd949ee8ef245b158
> Git:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.1.0-tentative
> Maven Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1281/org/apache/cassandra/cassandra-all/4.1.0/
>
> The Source and Build Artifacts, and the Debian and RPM packages and
> repositories, are available here:
> https://dist.apache.org/repos/dist/dev/cassandra/4.1.0/
>
> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s and no -1's.
>
> [1]: CHANGES.txt:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/4.1.0-tentative
> [2]: NEWS.txt:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/4.1.0-tentative
>
-- 
you are the apple of my eye !

Re: [jira] [Commented] (CASSANDRA-17380) Add support for EXPLAIN statements

[guo Maxwell]

i would like to work on this issue if no one is working on this

Paulo Motta (Jira) 于2022年5月23日 周一下午10:07写道：

>
> [
> https://issues.apache.org/jira/browse/CASSANDRA-17380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17540970#comment-17540970
> ]
>
> Paulo Motta commented on CASSANDRA-17380:
> -
>
> No update, this ticket is looking for an assignee.
>
> > Add support for EXPLAIN statements
> > --
> >
> > Key: CASSANDRA-17380
> > URL:
> https://issues.apache.org/jira/browse/CASSANDRA-17380
> > Project: Cassandra
> >  Issue Type: Improvement
> >Reporter: Benjamin Lerer
> >Priority: Normal
> >  Labels: gsoc, gsoc2022
> >
> > We should provide users a way to understand how their query will be
> executed and some information on the amount of work that will be performed.
> > Explain statements are the most common way to do that.
> > A CEP Draft has been open for that: [(DRAFT) CEP-4: Explain|
> https://docs.google.com/document/d/1s_gc4TDYdDbHnYHHVxxjqVVUn3MONUqG6W2JehnC11g/edit].
> This draft propose to add support for {{EXPLAIN}} and {{EXPLAIN ANALYZE}}
> but I believe that we should split the work in 2 parts because a simple
> {{EXPLAIN}} would already provide relevant information.
> > To complete this work I believe that the following steps will be
> required:
> > * Rework and submit the CEP
> > * Add missing statistics
> > * Implements the logic behind the EXPLAIN statements
>
>
>
> --
> This message was sent by Atlassian Jira
> (v8.20.7#820007)
>
> -
> To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: commits-h...@cassandra.apache.org
>
> --
you are the apple of my eye !

Re: [VOTE] Release Apache Cassandra 4.0.0 (take2)

[guo Maxwell]

+1

Paulo Motta  于2021年7月14日周三 下午7:03写道：

> +1
>
> Em qua., 14 de jul. de 2021 às 05:10, Benjamin Lerer 
> escreveu:
>
> > +1
> >
> > Le mer. 14 juil. 2021 à 09:48, Mick Semb Wever  a écrit
> :
> >
> > > >
> > > > The vote will be open for 72 hours (longer if needed). Everyone who
> > > > has tested the build is invited to vote. Votes by PMC members are
> > > > considered binding. A vote passes if there are at least three binding
> > > > +1s and no -1's.
> > > >
> > >
> > >
> > > +1
> > >
> > >
> > > Tested:
> > > sha1, md5, sha256, sha512 match.
> > > gpg signatures match.
> > > Source artefact builds. (and doesn't contain compiled dependencies)
> > > Tarball Binary artefact runs.
> > > Debian binary package runs.
> > > Redhat binary package runs.
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> > >
> >
>


-- 
you are the apple of my eye !

Re: [VOTE] Release Apache Cassandra 4.0.0

[guo Maxwell]

+1

Benjamin Lerer  于2021年7月13日周二 下午4:12写道：

> +1
>
> Le mar. 13 juil. 2021 à 08:19, Mick Semb Wever  a écrit :
>
> > Proposing the test build of Cassandra 4.0.0 for release.
> >
> > sha1: 69b8cc5cecc196fddb549f2edf01d962e7ef2dae
> > Git:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.0.0-tentative
> > Maven Artifacts:
> >
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1241/org/apache/cassandra/cassandra-all/4.0.0/
> >
> > The Source and Build Artifacts, and the Debian and RPM packages and
> > repositories, are available here:
> > https://dist.apache.org/repos/dist/dev/cassandra/4.0.0/
> >
> > The vote will be open for 72 hours (longer if needed). Everyone who has
> > tested the build is invited to vote. Votes by PMC members are considered
> > binding. A vote passes if there are at least three binding +1s and no
> -1's.
> >
> > [1]: CHANGES.txt:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/4.0.0-tentative
> > [2]: NEWS.txt:
> >
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/4.0.0-tentative
> >
>


-- 
you are the apple of my eye !

Re: [DISCUSSION] Next release roadmap

[guo Maxwell]

+1

Brandon Williams  于2021年4月15日周四 上午4:48写道：

> Agreed.  Everyone just please keep in mind this thread is for roadmap
> contributions you plan to make, not contributions you would like to
> see.
>
> On Wed, Apr 14, 2021 at 3:45 PM Nate McCall  wrote:
> >
> > Agree with Stefan 100% on this. We need to move towards pluggability. Our
> > users are asking for it, it makes sense architecturally, and people are
> > doing it anyway.
> >
> >
> > ...
> > > for me definitely https://issues.apache.org/jira/browse/CASSANDRA-9633
> > >
> > > I am surprised nobody mentioned this in the previous answers, there is
> > > ~50 people waiting for it to happen and multiple people working on it
> > > seriously and wanting that feature to be there for so so long.
> > > ...
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
you are the apple of my eye !

Re: [DISCUSSION] Next release roadmap

[guo Maxwell]

 besides do we need a table level backup and restore solution for cassandra
? https://issues.apache.org/jira/browse/CASSANDRA-15402

Re: [DISCUSSION] Next release roadmap

[guo Maxwell]

“Filter rows from system and system_schema based on users permissions” I
think the issue match this improvement.
https://issues.apache.org/jira/browse/CASSANDRA-15871

David Capwell  于2021年4月9日周五 上午3:56写道：

> Here are two things I descoped from 4.0 and plan to pick up after 4.0
> release
>
> https://issues.apache.org/jira/browse/CASSANDRA-15566 <
> https://issues.apache.org/jira/browse/CASSANDRA-15566> - Repair
> coordinator can hang under some cases.  Aka repair has many cases where it
> can become unresponsive and not reliant; so work to improve it.
> https://issues.apache.org/jira/browse/CASSANDRA-15399 <
> https://issues.apache.org/jira/browse/CASSANDRA-15399> - Add ability to
> track state in repair.  Rather than rely on logs to see what repair is
> doing, add APIs to expose what’s going on with repair, and what repair has
> done.
>
> > On Apr 8, 2021, at 8:22 AM, Benjamin Lerer  wrote:
> >
> > On our side, the list of improvements we plan to deliver for the next
> > release are:
> >
> > Query side improvements:
> >
> >  * Storage Attached Index or SAI. The CEP can be found at
> >
> https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-7%3A+Storage+Attached+Index
> >  * Add support for OR predicates in the CQL where clause
> >  * Allow to aggregate by time intervals (CASSANDRA-11871) and allow UDFs
> > in GROUP BY clause
> >  * Materialized views hardening: Addressing the different Materialized
> > Views issues (see CASSANDRA-15921 and [1] for some of the work involved)
> >
> > Security improvements:
> >
> >  * Add support for Dynamic Data Masking (CEP pending)
> >  * Allow the creation of roles that have the ability to assign arbitrary
> > privileges, or scoped privileges without also granting those roles access
> > to database objects.
> >  * Filter rows from system and system_schema based on users permissions
> >
> > Performance improvements:
> >
> >  * Trie-based index format (CEP pending)
> >  * Trie-based memtables (CEP pending)
> >
> > Safety/Usability improvements:
> >
> >  * Guardrails. The CEP can be found at
> >
> https://cwiki.apache.org/confluence/display/CASSANDRA/%28DRAFT%29+-+CEP-3%3A+Guardrails
> >
> > Pluggability improvements:
> >
> >  * Pluggable schema manager (CEP pending)
> >  * Pluggable filesystem (CEP pending)
> >  * Memtable API (CEP pending). The goal being to allow improvements such
> > as CASSANDRA-13981 to be easily plugged into Cassandra
> >
> > Feedbacks are welcome :-).
> >
> >
> > [1]
> >
> https://lists.apache.org/thread.html/r908b5397dd803132822cabe5ba0755861d97bb5d8603a523591d55c9%40%3Cdev.cassandra.apache.org%3E
> >
> > Le jeu. 8 avr. 2021 à 17:21, Benjamin Lerer  a écrit
> :
> >
> >> Hi Everybody,
> >> Please speak up and tell us what you plan to contribute in the next
> year.
> >>
> >> The goal of this discussion is to allow people to present the
> >> contributions that they have planned for the next Cassandra release.
> >>
> >> That discussion has several benefits:
> >>
> >>   - It will give a greater visibility on who is planning to contribute
> >>   and what their contributions would be. Allowing other contributors to
> join
> >>   the efforts or ask for questions if they wish to.
> >>   -  It will also us to synchronize our efforts when some features
> >>   impact the same part of the code
> >>   - For users, it will provide an idea of what to expect from the next
> >>   release
> >>
> >>
> >> Thanks in advance for all your inputs.
> >>
>
>

-- 
you are the apple of my eye !