Cassandra Contributor Meeting to focus on outstanding 4.0 issues

[Patrick McFadin]

Hi everyone,

First, I want to acknowledge some of the raw conversations today in the
cassandra-dev slack channel. It was probably well overdue and if not for
2020 and what a wonderful year this has been, we might have gotten there
earlier. I really appreciate how everyone who participated kept the
conversation from completely devolving into something that wouldn't get us
anywhere as a project. I also want to say that I hope we don't forget to
pause and give each other a break. People are already tense and on edge
from months of pandemics, politics, social unrest, disasters, child care
issues and a lot of other things on a long list. I find it helpful to
assume that everyone I talk to is stressed out, behind schedule and
probably has a boss pressuring them to do something. We share a really
unique common bond by willfully choosing to work on distributed systems. I
hope we find more reasons to come together than be apart in the future.

If I can even attempt to sum up the important themes from a very long
conversation. There is a tension between the frozen codebase leading to 4.0
and the desire to add new features beyond 4.0.  One way to eliminate that
tension is to get 4.0 wrapped up. What is left to get 4.0 done isn't clear
to many and there is a desire to get clear on the remaining items.

I would like to contribute some of my energy and time to help this along.
Standing on the shoulders of giants here, a LOT of work has already gone
into this topic. Josh, Jordan and Jon (J^3 ?) kept a steady strain on
updating jiras and status. This Kanban is really helpful:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=1661

There is an epic that was discussed in slack:
https://issues.apache.org/jira/browse/CASSANDRA-15536

There are 17 separate jiras as dependencies, of which only 2 are marked as
completed. 3 have no assigned owner. I have put a Zoom call on the
contributor meeting calendar to discuss those three first. I had to weave
it between ApacheCon sessions but hopefully this will help in velocity. Agenda
posted, feel free to alter as you see fit. September 29, 1230PM PST:
https://cwiki.apache.org/confluence/display/CASSANDRA/2020-09-29+Apache+Cassandra+Contributor+Meeting+-+4.0+push+edition

I mentioned this is Slack. We are so close. Beta 2 is out there, Harry just
dropped and Jiras have been closing. I get excited when I think of the
impact this will have. My entire life has now somehow become dependent on
Cassandra working. When I take a pic with my iPhone. Play games on my
PlayStation. Use curb-side pickup at Target. Order food on GrubHub.
Everybody here helped make that happen and there are a lot of great
engineers that will be deploying 4.0 and loving it. Most importantly, I
reliably get food when I'm playing Ghost of Tsushima and take a selfie for
my finsta. #priorities I would love to stick it to 2020 and make "Shipping
4.0" a positive event as we leave this stinking year behind.

Patrick

Re: [DISCUSS] Next steps for Kubernetes operator SIG

[Stefan Miklosovic]

Hi,

Patrick's suggestion seems good to me.

I won't go into specifics here as I need to genuinely prepare for
this. It is quite hard to dig deep into the solutions of others and
bring some constructive criticism because it takes a lot of time to
study it and everybody has some "why's" behind it.

To summarize my goals and concerns:

1) We should be as much "Kubernetes operator idiomatic" as possible.
Industry standards, no custom brain-child of this or that group
because they think it is just cool or they just didn't know any
better. I do NOT say it is like that right now, I just want to be
ruthless here as much as possible when it comes to functionality and
why it is done like that. It is awesome that we have already something
latest (thanks to John) and it adheres to the latest releases. I
personally had a hard time to keep up with all the releases, once I
finished something and I aligned it, after a week or two there was
already another one where things were different, it is a very
fast-moving space and I hope that by time we develop something it will
not be obsolete.

2) It may be easier said than done but it is guaranteed that people
get emotional, it's their precious etc, so please let's go into this
with good intentions, not trying to push one solution over the other
just because they would like to see it there ... I will have an
equally hard time to comply with this point. My plan is to explain
what is _wrong_ with our solution. Where we made mistakes and what
should be done differently but it is "too late" etc. It is quite hard
to describe your work and all effort in this light but without telling
what is wrong we can not decide what is good imho.

3) We should put something together fast enough so we can call it a
release. We can always iterate on it for eternity. But the foundations
need to be there. Here I want to say that I especially like what John
did. I looked through these specs and it was obvious it has been
written with care and attention. It looked _solid_. I am not sure how
hard it is to put all other things on top of that, I truly do not, and
here I think we would have to reinvent that wheel if we want to
proceed because I can not imagine what it would be to retrofit e.g.
CassKop on top of John specs, it is just like putting round pegs into
the square holes, maybe some chunks would be reused easily but
otherwise I worry we will be just on square one.

One specific feeling I have as I read this is that even if there is
the will to create the fourth operator, the respective parties will
not be able to drop their own repository. The whole point behind this
effort, to me, is to have a solid, community driven, stable, modern
and feature complete operator people are truly using. I can see that
once this is real, we will _really_ sunset our operator, redirecting
people to the new operator on main readme doc etc, we truly mean it.
Sure, if somebody comes and bug fix will be needed, we will fix it,
but the whole point of doing this is to stop using what we have
currently, over time, otherwise we are just splitting this space even
more. If CassKop is not sure if they will use it because they do not
know if that operator will be "enough" for them, aren't we just doing
it wrong? If I exaggerate, they should be fine with deleting the whole
repository and using just this Cassandra one we are going to make
otherwise I don't see the point to work on this ...

On Thu, 24 Sep 2020 at 20:45, Joshua McKenzie  wrote:
>
> - choose cass-operator: it is not on offer right now so let’s see if it does
>
>
> We should all talk a lot more, but this is 100% a mistake - I take the
> blame for that. The intention has long been to offer cass-operator for
> donation but it slipped through the cracks and your email yesterday made me
> double-take.
>
> We have since resolved this misalignment. DataStax would be happy to donate
> any and all of cass-operator to the ASF and C* project if it's what we all
> agree best serves our collective Cassandra users. I'm also cognizant that
> an immense amount of effort has gone into CassKop and we seem to have
> something of an embarrassment of riches.
>
> I'm given to understand (haven't dug in personally) that the two operators
> express pretty different opinions when it comes to frameworks, designs,
> supported versions, etc. I think a discrete enumeration of the feature set
> and "identities" of both could really help navigate this conversation going
> forward.
>
> Also - thanks for that context Franck. It's always helpful to know where
> other people are coming from when we're all working together towards a
> common goal.
>
>
> On Thu, Sep 24, 2020 at 12:23 PM,  wrote:
>
> > I can share Orange’s view of the situation, sorry it is a long story!
> >
> > We started CassKop at the end of 2018 after betting on K8S which was not
> > so simple as far as C* was concerned. Lack of support for local storage,
> > IPs that change all the time, different network plugins to try to 

Re: Creating a branch for 5.0 …?

[Jordan West]

On Thu, Sep 24, 2020 at 10:19 AM Joshua McKenzie 
wrote:

> Jordan: thanks for providing that context - it's quite helpful. Was that
> aspect of the conversation captured and shared with the rest of the project
> on the mailing list? It's a shame if not, since that may have contributed
> quite a bit to misalignment and misunderstanding over time.


Unfortuantely, I don't believe it was. The discussion occurred between
committers/contributors/PMC members on the floor in the venue and was
brought to the mailing list for a vote to ensure everyone was included. It
certainly would have been better if we were able to capture more of it (we
took as many notes as we could and translated them into that wiki doc but
there was no ability to record/etc). While its true that even more detail
would have been helpful, many of the folks who are pushing back against
what was discussed there were not in attendance or active in the open
source project at the time.

>
> Fewer and fewer people have the appetite to deal with this bickering and
> exposing anyone new to this seems like a guaranteed way to turn them away
> from the project for good.
>

Speaking for myself, the tone and tenor, in addition to the lack of
progress, in these discussions have absolutely affected my desire to
participate in the open source project (from taking additional reviews that
would speed up 4.0 on my free time, to encouraging new community members to
join, to participating in these discussions entirely). I hope we can find a
better way to communicate.

One suggestion I have for folks participating is to consider what feedback
is best given on the public mailing list and what feedback is better
delivered directly. We don't have to do EVERYTHING on the public mailing
list. Only the parts that pertain to Cassandra. And sometimes we can
resolve our personal differences better when there isn't an additional
audience around.


>
>
> On Thu, Sep 24, 2020 at 12:22 PM, Benedict Elliott Smith <
> bened...@apache.org> wrote:
>
> > The discussion on the present topic has not concluded, and if we are
> > making an exception to 4.0 only then it really needs to.
> >
> > Members of one organisation have been pushing hard for feature
> development
> > to proceed, arguing it harms unnamed third parties. A request that these
> > third parties be asked to participate in the discussion has so far gone
> > unanswered. It is reasonable that this is answered before a vote, since
> > this is the entire basis of the argument in favour of branching.
> >
> > Given this is the basis of argument, I would also propose a less
> > contentious vote, should one be undertaken: to create a cassandra-5.0
> > branch that is open only to contributions from those unaffiliated by
> > employment with any existing committers. This seems to alleviate the
> > concerns precipitating this discussion, while mitigating the concerns of
> > those who are opposed to it.
> >
> > On 24/09/2020, 17:02, "Jake Luciani"  wrote:
> >
> > The vote was to unfreeze new changes at beta, so logically that means
> > non-bugfix work goes into trunk.
> >
> > Jordan, thanks. That is a more recent vote so thanks. That being said,
> > under that line Benedict comments this needs to be discussed. So how
> about
> > we just have a Vote on branching cassandra-4.0 and the issue will be
> > decided?
> >
> > Jake
> >
> > On Thu, Sep 24, 2020 at 11:53 AM Benedict Elliott Smith  > org> wrote:
> >
> > I'm not sure what you are referring to here, that vote said nothing about
> > branching at beta.
> >
> > The most recent vote on the topic anyway was for the Release Lifecycle
> > process, which stipulates branching at GA.
> >
> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> >
> > We can vote to modify this document, or to make an exception, but I am
> > aware of no other vote stipulating anything about the point at which we
> > branch.
> >
> > On 24/09/2020, 16:49, "Jake Luciani"  wrote:
> >
> > Today the community still has in force an explicit vote prohibiting
> >
> > thee
> > merge of this work.
> >
> > You referred to an explicit vote here. I assume that was the one you were
> > referring to? Yes, the community should decide.
> > Call a vote if you think the community thinks we should continue the
> > freeze
> > vs continuing to rely on beliefs about the community.
> >
> > I'm simply pointing out the branching of 4.0 post beta was the plan of
> > last
> > record.
> >
> > Jake
> >
> > On Thu, Sep 24, 2020 at 11:44 AM Benedict Elliott Smith <
> benedict@apache.
> > org>
> > wrote:
> >
> > The community does everything through discussion and consensus.
> >
> > Does that
> >
> > include branching, or not?
> >
> > If there is no consensus, a vote is held. Whether or not you
> >
> > consider the
> >
> > vote from 2018 still valid, you still need to seek the consent of the
> > community for your action today. Or is that not sacrosanct anymore?
> >
> > On 24/09/2020, 16:22, "Jake Luciani"  wrote:
> 

Re: Creating a branch for 5.0 …?

[Dinesh Joshi]

Josh,

> For what it's worth, when I engage with other people working on large forks
> that want to bring their features back to the project, the absolute last
> thing I want to do to them is throw them into email threads like this.

People need to positively engage with the community, have discussions & debate 
their PoV to get their contributions into the official codebase.

The CEP explicitly helps do just that. Creating a 5.0 branch implies that these 
parties just want to drop code rather than actually discuss their features with 
the community. This is the disconnect for me personally.

People who run large forks, should also care about what the community is 
working on. If they open up a CEP or Discuss thread, I don't think it'll be 
remotely as contentious as this thread.

> Fewer and fewer people have the appetite to deal with this bickering and 
> exposing anyone new to this seems like a guaranteed way to turn them away 
> from the project for good.

Debate is healthy and its important for the long-term health of this project.


Dinesh
-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Creating a branch for 5.0 …?

[Brandon Williams]

On Thu, Sep 24, 2020 at 1:14 PM sankalp kohli  wrote:

> I think it is important to think why we are here. We are here as we shipped
> 3.0 with 10s of correctness bug. So the statements should be
> "3.0 shipped with 10s of correctness bugs and that is causing contributions
> to go away and stopping innovation"

I think we're making logical leaps again without any real foundation,
it could be argued that innovation
 is not happening because there is no place for it to happen just as easily.

> "Lack of 4.0 release due to 3.0 shipping with 10s of correctness bugs is
> causing people to think C* is dead."

This is probably true.

> Once we start putting it this way, we can start debating on how not to make
> 4.0 like 3.0 and cause 5.0 to be delayed.

This again seems based on an assumption that allowing feature work
will deter correctness work.  Is there any evidence of that, with the
merge onus removed?

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [DISCUSS] Next steps for Kubernetes operator SIG

[Joshua McKenzie]

- choose cass-operator: it is not on offer right now so let’s see if it does


We should all talk a lot more, but this is 100% a mistake - I take the
blame for that. The intention has long been to offer cass-operator for
donation but it slipped through the cracks and your email yesterday made me
double-take.

We have since resolved this misalignment. DataStax would be happy to donate
any and all of cass-operator to the ASF and C* project if it's what we all
agree best serves our collective Cassandra users. I'm also cognizant that
an immense amount of effort has gone into CassKop and we seem to have
something of an embarrassment of riches.

I'm given to understand (haven't dug in personally) that the two operators
express pretty different opinions when it comes to frameworks, designs,
supported versions, etc. I think a discrete enumeration of the feature set
and "identities" of both could really help navigate this conversation going
forward.

Also - thanks for that context Franck. It's always helpful to know where
other people are coming from when we're all working together towards a
common goal.


On Thu, Sep 24, 2020 at 12:23 PM,  wrote:

> I can share Orange’s view of the situation, sorry it is a long story!
>
> We started CassKop at the end of 2018 after betting on K8S which was not
> so simple as far as C* was concerned. Lack of support for local storage,
> IPs that change all the time, different network plugins to try to implement
> a non standard K8s way of having nodes see each other from different dcs…
> We hesitated with Mesos but could not have both and K8S was already
> tracting so much you could not not choose it.
>
> Anyway, we looked around and did not see anyone with such requirements so
> we said: why not try it ourselves but on github so that we may give it back
> to the community. We have used C* for quite a few years with great success
> on production with massive load and perfect availability. We love C* @
> Orange :) Thanks!
>
> So we started writing support for mono-dc cluster (CassKop) and added the
> multi dc support with MultiCassKop which is another operator included in
> the CassKop repo. For more details we tried to document our designs as much
> as possible here: https://orange-opensource.github.io/casskop/docs/
> 1_concepts/3_design_principes#multi-site-management
>
> In the middle of last year we had some talks with Datastax about working
> together around their new management sidecar. Their position on open source
> was not clear at that time so we said please come back when you have
> decided to go open source with it. Which they did in the beginning of this
> year. But at that time I guess work had started on cass-operator so we kept
> our separate ways.
>
> Since the beginning of the years, we have been working with our OPS team
> to have it in production. It is not simple as the team has to learn K8S and
> trust a newborn operator. This takes time especially as our internal
> cluster has been tweaked for multi-tenancy with obscure options being set
> by our K8s team…
>
> We also developed with Instaclustr the Backup & Restore functionnality (we
> have new CRDs (Custom Resource Definition) for backup and restore and a
> reconcile loop that calls out Instaclustr sidecar for these operations). We
> now support multiple backups in parallel and can write to s3/ google or
> azur (but Stefan could give more details here if needed)
>
> During the SIG calls we mentioned our desire to donate CassKop once it
> satisfies our basics requirements (v1 coming just now but I said it too
> many times already) I am actually not sure Datastax mentioned their desire
> to donate cass-operator but we decided to compare the designs and the
> functionalities based on respective CRDs. The CRD is the interface with the
> user as it is where you describe the cluster that you want to have. These
> talks were very interesting and we found out that the CassKop team had made
> good choices most of the time but was may be too open. Indeed our intention
> was to give all the possibilities for our OPS team to work. This includes :
> - very open topology definition using any configuration of labels to map
> dcs / racks and nodes to labels on clusters (we have labels on dcs / rooms
> / rows and server racks so we can map C* racks to storage or network arrays
> internaly)
> - possibility to have multiple C* nodes on a single K8S host (because
> internal clouds are not really clouds, they have limited resources)
> - custom C* image selection,
> - custom bootstrap script that lets you configure C* as you want using
> ConfigMaps,
> - the ability to mount different volumes wherever they wanted,
> - the possibility to run any number of sidecars alongside C* for custom
> probes in our case
>
> This makes CassKop quite powerful and flexible.
> We made sure that all those options are not enabled by default so one can
> just pop a simple 3 node cluster quickly
>
> On the other hand cass-operator had an interesting 

Re: Creating a branch for 5.0 …?

[sankalp kohli]

Hi,
I hear the following
"Freeze is causing contributions to go away and stopping innovation"
"Lack of 4.0 release is causing people to think C* is dead."

I think it is important to think why we are here. We are here as we shipped
3.0 with 10s of correctness bug. So the statements should be
"3.0 shipped with 10s of correctness bugs and that is causing contributions
to go away and stopping innovation"
"Lack of 4.0 release due to 3.0 shipping with 10s of correctness bugs is
causing people to think C* is dead."

Once we start putting it this way, we can start debating on how not to make
4.0 like 3.0 and cause 5.0 to be delayed. So instead of focusing on
symptoms and major effects 3.0 + correctness bugs has caused, lets talk
about following

1. How can we make 4.0 stable so we dont have to freeze for 2+ years and
dont ship with 10s of correctness bugs.
2. How can we have a stable .0 release instead of people not using it for
10th minor.
3. How can we have a predictable timeline for major releases like 4.0 so
contributors/users knows how to plan around it.

I think it is more important to fix the root cause than fixing the
symptoms.

Thanks,
Sankalp






On Thu, Sep 24, 2020 at 10:50 AM Brandon Williams  wrote:

> On Thu, Sep 24, 2020 at 12:22 PM sankalp kohli 
> wrote:
> >
> > Hi Brandon,
> >   In all respect, we need to discuss and vote before we
> > create a new branch. So it is best if we do that instead of creating
> > branches.
>
> Fair enough.
>
> > Freeze is a symptom not a cause so if we dont like the symptom,
> > we should see how to fix the cause. Are we fine having a database release
> > which will be released with 10s of  correctness bugs and with an implicit
> > assumption that no one will use it in prod till 10th minor.
>
> I think we're making quite a leap to go from adding new features in
> one branch equating to releasing software with more bugs in a
> completely different branch.  I think your concern is that by having
> the project 'bless' a feature-accepting branch, this will detract from
> time they would otherwise spend on stabilizing the release branch (if
> we had one.)  I can't fault you for this, it may be a real
> possibility, nobody knows.  Nothing can be done about changing what
> people decide to work on in a free project, however, and it would be
> naive to think there are not people already out there interested in
> working purely on new features, with no dog in the stability of 4.0
> fight.  I know that's not conducive to your goals (and honestly I'd
> rather they help on the 4.0 side too) but that's just the way things
> are.
>
> > Everyone wants to innovate but security, durability and availability
> comes
> > first.  People who are working on stabilizing 4.0 are doing it as there
> is
> > no other choice. So I request you to kindly cooperate on working with
> them
> > in making 4.0 a stable release.
>
> Sorry, but I respectfully refuse your statement that there is no other
> choice.  This is a free project and there is always a choice for those
> who may wish to donate their time.  I would like to cast a wide enough
> net to allow all their itches to be scratched here for the future
> health of this project.
>
> Kind Regards,
> Brandon
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Creating a branch for 5.0 …?

[Brandon Williams]

On Thu, Sep 24, 2020 at 12:22 PM sankalp kohli  wrote:
>
> Hi Brandon,
>   In all respect, we need to discuss and vote before we
> create a new branch. So it is best if we do that instead of creating
> branches.

Fair enough.

> Freeze is a symptom not a cause so if we dont like the symptom,
> we should see how to fix the cause. Are we fine having a database release
> which will be released with 10s of  correctness bugs and with an implicit
> assumption that no one will use it in prod till 10th minor.

I think we're making quite a leap to go from adding new features in
one branch equating to releasing software with more bugs in a
completely different branch.  I think your concern is that by having
the project 'bless' a feature-accepting branch, this will detract from
time they would otherwise spend on stabilizing the release branch (if
we had one.)  I can't fault you for this, it may be a real
possibility, nobody knows.  Nothing can be done about changing what
people decide to work on in a free project, however, and it would be
naive to think there are not people already out there interested in
working purely on new features, with no dog in the stability of 4.0
fight.  I know that's not conducive to your goals (and honestly I'd
rather they help on the 4.0 side too) but that's just the way things
are.

> Everyone wants to innovate but security, durability and availability comes
> first.  People who are working on stabilizing 4.0 are doing it as there is
> no other choice. So I request you to kindly cooperate on working with them
> in making 4.0 a stable release.

Sorry, but I respectfully refuse your statement that there is no other
choice.  This is a free project and there is always a choice for those
who may wish to donate their time.  I would like to cast a wide enough
net to allow all their itches to be scratched here for the future
health of this project.

Kind Regards,
Brandon

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [DISCUSS] Next steps for Kubernetes operator SIG

[Patrick McFadin]

No problem Franck! I will postpone this week's meeting to next week and we
can continue the discussion on the ML.

Patrick

On Thu, Sep 24, 2020 at 9:23 AM  wrote:

> I can share Orange’s view of the situation, sorry it is a long story!
>
> We started CassKop at the end of 2018 after betting on K8S which was not
> so simple as far as C* was concerned.
> Lack of support for local storage, IPs that change all the time, different
> network plugins to try to implement a non standard K8s way of having nodes
> see each other from different dcs…
> We hesitated with Mesos but could not have both and K8S was already
> tracting so much you could not not choose it.
>
> Anyway, we looked around and did not see anyone with such requirements so
> we said: why not try it ourselves but on github so that we may give it back
> to the community.
> We have used C* for quite a few years with great success on production
> with massive load and perfect availability. We love C* @ Orange :) Thanks!
>
> So we started writing support for mono-dc cluster (CassKop) and added the
> multi dc support with MultiCassKop which is another operator included in
> the CassKop repo.
> For more details we tried to document our designs as much as possible
> here:
> https://orange-opensource.github.io/casskop/docs/1_concepts/3_design_principes#multi-site-management
>
> In the middle of last year we had some talks with Datastax about working
> together around their new management sidecar. Their position on open source
> was not clear at that time so we said please come back when you have
> decided to go open source with it.
> Which they did in the beginning of this year. But at that time I guess
> work had started on cass-operator so we kept our separate ways.
>
> Since the beginning of the years, we have been working with our OPS team
> to have it in production. It is not simple as the team has to learn K8S and
> trust a newborn operator.
> This takes time especially as our internal cluster has been tweaked for
> multi-tenancy with obscure options being set by our K8s team…
>
> We also developed with Instaclustr the Backup & Restore functionnality (we
> have new CRDs (Custom Resource Definition) for backup and restore and a
> reconcile loop that calls out Instaclustr sidecar for these operations).
> We now support multiple backups in parallel and can write to s3/ google or
> azur (but Stefan could give more details here if needed)
>
> During the SIG calls we mentioned our desire to donate CassKop once it
> satisfies our basics requirements (v1 coming just now but I said it too
> many times already)
> I am actually not sure Datastax mentioned their desire to donate
> cass-operator but we decided to compare the designs and the functionalities
> based on respective CRDs.
> The CRD is the interface with the user as it is where you describe the
> cluster that you want to have.
> These talks were very interesting and we found out that the CassKop team
> had made good choices most of the time but was may be too open.
> Indeed our intention was to give all the possibilities for our OPS team to
> work.
> This includes :
> - very open topology definition using any configuration of labels to map
> dcs / racks and nodes to labels on clusters (we have labels on dcs / rooms
> / rows and server racks so we can map C* racks to storage or network arrays
> internaly)
> - possibility to have multiple C* nodes on a single K8S host (because
> internal clouds are not really clouds, they have limited resources)
> - custom C* image selection,
> - custom bootstrap script that lets you configure C* as you want using
> ConfigMaps,
> -  the ability to mount different volumes wherever they wanted,
> -  the possibility to run any number of sidecars alongside C* for custom
> probes in our case
>
> This makes CassKop quite powerful and flexible.
> We made sure that all those options are not enabled by default so one can
> just pop a simple 3 node cluster quickly
>
> On the other hand cass-operator had an interesting way of configuring C*
> just inside the CRD using cass-config. This is simple and elegant so we are
> implementing it as well for the support of C* 4
>
> Now for the future, there are 3 choices in my opinion:
> - start from scratch (or John’s repo) by cherry picking bits from all
> operators. This is possible but will take some time / effort to have
> something usable. And then it will be compared to cass-operator and CassKop.
> I don’t see Orange contributing too much here as we believe CassKop to be
> a much better starting point
> - choose cass-operator: it is not on offer right now so let’s see if it
> does. I think Orange could contribute some bits inherited from CassKop if
> it is agreed by the community. Not sure it would be enough for us to use it.
> - choose CassKop: we would be delighted to donate it and contribute with
> some committers (including the original author who now works for AWS). It
> would then become the community operator but there would be 

Re: Creating a branch for 5.0 …?

[sankalp kohli]

Hi Brandon,
  In all respect, we need to discuss and vote before we
create a new branch. So it is best if we do that instead of creating
branches. Freeze is a symptom not a cause so if we dont like the symptom,
we should see how to fix the cause. Are we fine having a database release
which will be released with 10s of  correctness bugs and with an implicit
assumption that no one will use it in prod till 10th minor.

Everyone wants to innovate but security, durability and availability comes
first.  People who are working on stabilizing 4.0 are doing it as there is
no other choice. So I request you to kindly cooperate on working with them
in making 4.0 a stable release.

https://tinyurl.com/30seriousissues


Thanks,
Sankalp

On Thu, Sep 24, 2020 at 9:30 AM Brandon Williams  wrote:

> On Thu, Sep 24, 2020 at 11:22 AM Benedict Elliott Smith
>  wrote:
> > Given this is the basis of argument, I would also propose a less
> contentious vote, should one be undertaken: to create a cassandra-5.0
> branch that is open only to contributions from those unaffiliated by
> employment with any existing committers.
>
> Are you serious?  What kind of opensource environment is it that
> precludes anyone from contributing based on their employment?
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Creating a branch for 5.0 …?

[Joshua McKenzie]

this is the basis of argument,

This is incorrect. The basis of the argument was Mick's point of view he
expressed as an individual on the project, which is that projects that turn
away contributions or try and force people into contributing in certain
ways are not long for this world. A very basic logical argument with a lot
of individual experience backing it up; anecdata I felt it appropriate to
share knowing there are limitations on how much detail I can disclose. Also
not consistent with the Apache Way, as Jeff pointed out.

You keep trying to bend this to a discussion about my credibility and about
DataStax' incentives. If you think I'm lying to try and shift the project,
please say so and stop hiding behind terms like "mythical" or
"hypothetical".

This is individuals on the project who care deeply about its welfare with
points of view that strongly differ from yours, *as individual humans.*

Much like another *individual human and committer *coming out of the
woodwork to open up a bunch of tickets this week for post 4.0.

Much like me waking up to Brandon this morning with him choosing to do
this *because
he as a human being is worried about the long-term health of the project.*

Feel free to engage with us as the individual humans we are. Please stop
trying to turn this discussion into an argument between talking heads
advocating for what their respective companies are coercing them to do. We
in the Cassandra engineering community are fortunate enough that our skills
are in very high demand; in all likelihood people are working where they
are because they share philosophical perspectives with the people they work
with, not because of some top-down institutional mandate on them.

For what it's worth, when I engage with other people working on large forks
that want to bring their features back to the project, the absolute last
thing I want to do to them is throw them into email threads like this.
Fewer and fewer people have the appetite to deal with this bickering and
exposing anyone new to this seems like a guaranteed way to turn them away
from the project for good.

Jordan: thanks for providing that context - it's quite helpful. Was that
aspect of the conversation captured and shared with the rest of the project
on the mailing list? It's a shame if not, since that may have contributed
quite a bit to misalignment and misunderstanding over time.


On Thu, Sep 24, 2020 at 12:22 PM, Benedict Elliott Smith <
bened...@apache.org> wrote:

> The discussion on the present topic has not concluded, and if we are
> making an exception to 4.0 only then it really needs to.
>
> Members of one organisation have been pushing hard for feature development
> to proceed, arguing it harms unnamed third parties. A request that these
> third parties be asked to participate in the discussion has so far gone
> unanswered. It is reasonable that this is answered before a vote, since
> this is the entire basis of the argument in favour of branching.
>
> Given this is the basis of argument, I would also propose a less
> contentious vote, should one be undertaken: to create a cassandra-5.0
> branch that is open only to contributions from those unaffiliated by
> employment with any existing committers. This seems to alleviate the
> concerns precipitating this discussion, while mitigating the concerns of
> those who are opposed to it.
>
> On 24/09/2020, 17:02, "Jake Luciani"  wrote:
>
> The vote was to unfreeze new changes at beta, so logically that means
> non-bugfix work goes into trunk.
>
> Jordan, thanks. That is a more recent vote so thanks. That being said,
> under that line Benedict comments this needs to be discussed. So how about
> we just have a Vote on branching cassandra-4.0 and the issue will be
> decided?
>
> Jake
>
> On Thu, Sep 24, 2020 at 11:53 AM Benedict Elliott Smith  org> wrote:
>
> I'm not sure what you are referring to here, that vote said nothing about
> branching at beta.
>
> The most recent vote on the topic anyway was for the Release Lifecycle
> process, which stipulates branching at GA.
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>
> We can vote to modify this document, or to make an exception, but I am
> aware of no other vote stipulating anything about the point at which we
> branch.
>
> On 24/09/2020, 16:49, "Jake Luciani"  wrote:
>
> Today the community still has in force an explicit vote prohibiting
>
> thee
> merge of this work.
>
> You referred to an explicit vote here. I assume that was the one you were
> referring to? Yes, the community should decide.
> Call a vote if you think the community thinks we should continue the
> freeze
> vs continuing to rely on beliefs about the community.
>
> I'm simply pointing out the branching of 4.0 post beta was the plan of
> last
> record.
>
> Jake
>
> On Thu, Sep 24, 2020 at 11:44 AM Benedict Elliott Smith < benedict@apache.
> org>
> wrote:
>
> The community does everything through discussion and consensus.
>
> Does that
>

Re: Creating a branch for 5.0 …?

[Brandon Williams]

On Thu, Sep 24, 2020 at 11:22 AM Benedict Elliott Smith
 wrote:
> Given this is the basis of argument, I would also propose a less contentious 
> vote, should one be undertaken: to create a cassandra-5.0 branch that is open 
> only to contributions from those unaffiliated by employment with any existing 
> committers.

Are you serious?  What kind of opensource environment is it that
precludes anyone from contributing based on their employment?

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [DISCUSS] Next steps for Kubernetes operator SIG

[franck.dehay]

I can share Orange’s view of the situation, sorry it is a long story!

We started CassKop at the end of 2018 after betting on K8S which was not so 
simple as far as C* was concerned.
Lack of support for local storage, IPs that change all the time, different 
network plugins to try to implement a non standard K8s way of having nodes see 
each other from different dcs…
We hesitated with Mesos but could not have both and K8S was already tracting so 
much you could not not choose it.

Anyway, we looked around and did not see anyone with such requirements so we 
said: why not try it ourselves but on github so that we may give it back to the 
community.
We have used C* for quite a few years with great success on production with 
massive load and perfect availability. We love C* @ Orange :) Thanks!

So we started writing support for mono-dc cluster (CassKop) and added the multi 
dc support with MultiCassKop which is another operator included in the CassKop 
repo.
For more details we tried to document our designs as much as possible here: 
https://orange-opensource.github.io/casskop/docs/1_concepts/3_design_principes#multi-site-management

In the middle of last year we had some talks with Datastax about working 
together around their new management sidecar. Their position on open source was 
not clear at that time so we said please come back when you have decided to go 
open source with it.
Which they did in the beginning of this year. But at that time I guess work had 
started on cass-operator so we kept our separate ways.

Since the beginning of the years, we have been working with our OPS team to 
have it in production. It is not simple as the team has to learn K8S and trust 
a newborn operator.
This takes time especially as our internal cluster has been tweaked for 
multi-tenancy with obscure options being set by our K8s team…

We also developed with Instaclustr the Backup & Restore functionnality (we have 
new CRDs (Custom Resource Definition) for backup and restore and a reconcile 
loop that calls out Instaclustr sidecar for these operations).
We now support multiple backups in parallel and can write to s3/ google or azur 
(but Stefan could give more details here if needed)

During the SIG calls we mentioned our desire to donate CassKop once it 
satisfies our basics requirements (v1 coming just now but I said it too many 
times already)
I am actually not sure Datastax mentioned their desire to donate cass-operator 
but we decided to compare the designs and the functionalities based on 
respective CRDs.
The CRD is the interface with the user as it is where you describe the cluster 
that you want to have.
These talks were very interesting and we found out that the CassKop team had 
made good choices most of the time but was may be too open.
Indeed our intention was to give all the possibilities for our OPS team to work.
This includes :
- very open topology definition using any configuration of labels to map dcs / 
racks and nodes to labels on clusters (we have labels on dcs / rooms / rows and 
server racks so we can map C* racks to storage or network arrays internaly)
- possibility to have multiple C* nodes on a single K8S host (because internal 
clouds are not really clouds, they have limited resources)
- custom C* image selection,
- custom bootstrap script that lets you configure C* as you want using 
ConfigMaps,
-  the ability to mount different volumes wherever they wanted,
-  the possibility to run any number of sidecars alongside C* for custom probes 
in our case

This makes CassKop quite powerful and flexible.
We made sure that all those options are not enabled by default so one can just 
pop a simple 3 node cluster quickly

On the other hand cass-operator had an interesting way of configuring C* just 
inside the CRD using cass-config. This is simple and elegant so we are 
implementing it as well for the support of C* 4

Now for the future, there are 3 choices in my opinion:
- start from scratch (or John’s repo) by cherry picking bits from all 
operators. This is possible but will take some time / effort to have something 
usable. And then it will be compared to cass-operator and CassKop.
I don’t see Orange contributing too much here as we believe CassKop to be a 
much better starting point
- choose cass-operator: it is not on offer right now so let’s see if it does. I 
think Orange could contribute some bits inherited from CassKop if it is agreed 
by the community. Not sure it would be enough for us to use it.
- choose CassKop: we would be delighted to donate it and contribute with some 
committers (including the original author who now works for AWS). It would then 
become the community operator but there would be cass-operator alongside 
probably.
But Cass-operator is made to make it easier for Datastax to manage customer 
clusters by imposing some configuration. It make sense for their needs, so may 
be 2 operators. We don’t know how backup/restore will be handled here with 
medusa being adapted to 

Re: Creating a branch for 5.0 …?

[Benedict Elliott Smith]

The discussion on the present topic has not concluded, and if we are making an 
exception to 4.0 only then it really needs to.

Members of one organisation have been pushing hard for feature development to 
proceed, arguing it harms unnamed third parties.  A request that these third 
parties be asked to participate in the discussion has so far gone unanswered.  
It is reasonable that this is answered before a vote, since this is the entire 
basis of the argument in favour of branching.

Given this is the basis of argument, I would also propose a less contentious 
vote, should one be undertaken: to create a cassandra-5.0 branch that is open 
only to contributions from those unaffiliated by employment with any existing 
committers.  This seems to alleviate the concerns precipitating this 
discussion, while mitigating the concerns of those who are opposed to it.



On 24/09/2020, 17:02, "Jake Luciani"  wrote:

The vote was to unfreeze new changes at beta, so logically that means
non-bugfix work goes into trunk.

Jordan, thanks.   That is a more recent vote so thanks.  That being said,
under that line Benedict comments this needs to be discussed.
So how about we just have a Vote on branching cassandra-4.0 and the issue
will be decided?

Jake




On Thu, Sep 24, 2020 at 11:53 AM Benedict Elliott Smith 

wrote:

> I'm not sure what you are referring to here, that vote said nothing about
> branching at beta.
>
> The most recent vote on the topic anyway was for the Release Lifecycle
> process, which stipulates branching at GA.
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>
> We can vote to modify this document, or to make an exception, but I am
> aware of no other vote stipulating anything about the point at which we
> branch.
>
>
> On 24/09/2020, 16:49, "Jake Luciani"  wrote:
>
> > Today the community still has in force an explicit vote prohibiting
> thee
> merge of this work.
>
> You referred to an explicit vote here.  I assume that was the one you
> were
> referring to?  Yes, the community should decide.
> Call a vote if you think the community thinks we should continue the
> freeze
> vs continuing to rely on beliefs about the community.
>
> I'm simply pointing out the branching of 4.0 post beta was the plan of
> last
> record.
>
> Jake
>
> On Thu, Sep 24, 2020 at 11:44 AM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > The community does everything through discussion and consensus.
> Does that
> > include branching, or not?
> >
> > If there is no consensus, a vote is held.  Whether or not you
> consider the
> > vote from 2018 still valid, you still need to seek the consent of 
the
> > community for your action today.  Or is that not sacrosanct anymore?
> >
> >
> > On 24/09/2020, 16:22, "Jake Luciani"  wrote:
> >
> > I'm sorry I see no issue with branching 4.0 as it was the thing
> we
> > voted on
> > back in 2018.  If you wish to extend the freeze you should call
> a new
> > vote.
> >
> > On Thu, Sep 24, 2020 at 11:15 AM Benedict Elliott Smith <
> > bened...@apache.org>
> > wrote:
> >
> > > Nobody has any problem with an external repository being
> > maintained.  Just
> > > bear in mind the normal process will need to take place to
> merge to
> > the ASF
> > > repository, and that there may be feedback and review requests
> to
> > address,
> > > so merge order and diffs will probably change.
> > >
> > >
> > > On 24/09/2020, 16:05, "Brandon Williams" 
> wrote:
> > >
> > > On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
> > >  wrote:
> > > >
> > > > You do not have the authority to unilaterally overrule
> the
> > community
> > > process.  This is a serious breach of your responsibilities as
> a
> > member of
> > > the PMC.
> > >
> > > Feel free to complain that I'm creating branches we intend
> to
> > someday,
> > > perhaps even in 2020, release.
> > >
> > > > I have deleted this branch, and will do so again if you
> repeat
> > this.
> > >
> > > This would create some interesting tickets for INFRA, but
> I won't
> > > waste their time with you either. Whether either of us has
> the
> > > authority to do such on ASF infrastructure is irrelevant,
> 

Re: [DISCUSS] Next steps for Kubernetes operator SIG

[Patrick McFadin]

I would like to propose a hybrid a hybrid of what Benedict mentioned.

Let's postpone today's (Sept 24) SIG to the next week, Oct 1. Same time.

I'll keep the same zoom with some modifications. Each group, CassKop and
cass-operator can have time to present the following:

 - State your view of the situation
 - Why they would or would not support a given approach/operator. Be
technically specific.
 - What technical circumstances might lead them to change their mind
 - Your view of a path forward

Each group will get 20 minutes with 10 minutes of q I can moderate.
After the meeting I'll post the video as well as a full transcript (Thank
you Otter.ai!) If there were presentation decks, then post a PDF of those.
I'll kick off the discussion in the dev ML and we can debate here.

This is my proposal for moving things forward if possible. +1, -1 or more
debating?

Patrick

On Wed, Sep 23, 2020 at 12:21 PM Benedict Elliott Smith 
wrote:

> Perhaps it helps to widen the field of discussion to the dev list?
>
> It might help if each of the stakeholder organisations state their view on
> the situation, including why they would or would not support a given
> approach/operator, and what (preferably specific) circumstances might lead
> them to change their mind?
>
> I realise there are meeting logs, but getting a wider discourse with
> non-stakeholder input might help to build a community consensus?  It
> doesn't seem like it can hurt at this point, anyway.
>
>
> On 23/09/2020, 17:13, "John Sanda"  wrote:
>
> I want to point out that pretty much everything being  discussed in
> this
> thread has been discussed at length during the SIG meetings. I think
> it is
> worth noting because we are pretty much still have the same
> conversation.
>
> On Wed, Sep 23, 2020 at 12:03 PM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > I don't think there's anything about a code drop that's not "The
> Apache
> > Way"
> >
> > If there's a consensus (or even strong majority) amongst invested
> parties,
> > I don't see why we could not adopt an operator directly into the
> project.
> >
> > It's possible a green field approach might lead to fewer hard
> feelings, as
> > everyone is in the same boat. Perhaps all operators are also
> suboptimal and
> > could be improved with a rewrite? But I think coordinating a lot of
> > different entities around an empty codebase is particularly
> challenging.  I
> > actually think it could be better for cohesion and collaboration to
> have a
> > suboptimal but substantive starting point.
> >
> >
> > On 23/09/2020, 16:11, "Stefan Miklosovic" <
> > stefan.mikloso...@instaclustr.com> wrote:
> >
> > I think that from Instaclustr it was stated quite clearly
> multiple
> > times that we are "fine to throw it away" if there is something
> better
> > and more wide-spread.Indeed, we have invested a lot of time in
> the
> > operator but it was not useless at all, we gained a lot of quite
> > unique knowledge how to put all pieces together. However, I
> think that
> > this space is going to be quite fragmented and "balkanized",
> which is
> > not always a bad thing, but in a quite narrow area as Kubernetes
> > operator is, I just do not see how 4 operators are going to be
> > beneficial for ordinary people ("official" from community, ours,
> > Datastax one and CassKop (without any significant order)). Sure,
> > innovation and healthy competition is important but to what
> extent ...
> > One can start a Cassandra cluster on Kubernetes just so many
> times
> > differently and nobody really likes a vendor lock-in. People
> wanting
> > to run a cluster on K8S realise that there are three operators,
> each
> > backed by a private business entity, and the community operator
> is not
> > there ... Huh, interesting ... One may even start to question
> what is
> > wrong with these folks that it takes three companies to build
> their
> > own solution.
> >
> > Having said that, to my perception, Cassandra community just
> does not
> > have enough engineers nor contributors to keep 4 operators alive
> at
> > the same time (I wish I was wrong) so the idea of selecting the
> best
> > one or to merge obvious things and approaches together is
> > understandable, even if it meant we eventually sunset ours. In
> > addition, nobody from big players is going to contribute to the
> code
> > base of the other one, for obvious reasons, so channeling and
> > directing this effort into something common for a community
> seems to
> > be the only reasonable way of cooperation.
> >
> > It is quite hard to bootstrap this if the donation of the code
> in big
> > chunks / whole repo is out of question as 

Re: Creating a branch for 5.0 …?

[Jordan West]

The intention was the former. It was discussed during apache con in 2019
and many people expressed the desire to wait until GA. Even some who
initially were opposed to the freeze.


Jordan

On Thu, Sep 24, 2020 at 9:04 AM Joshua McKenzie 
wrote:

>
> https://lists.apache.org/thread.html/5ee66f3986bf8308912c216bd1b5f9aea35443626db9f92cdca4d7b9%40%3Cdev.cassandra.apache.org%3E
>
>
>
> "*From: *sankalp kohli 
>
> *To: *dev 
>
> *Subject: *Re: [VOTE] Branching Change for 4.0 Freeze
>
> *Date: *2018/07/11 21:50:08
>
> *List: *dev@cassandra.apache.org
>
> 
>
>
>
> We will be in this state till beta is reached."
>
>
>
> The release lifecycle doc says: "*A new branch is created for the release
>
> with the new major version, limiting any new feature addition to the new
>
> release branch, with new feature development will continue to happen only
>
> on trunk.*"
>
>
>
> This could be read as "we won't branch until we hit GA" or "we will make
>
> sure we definitely branch at GA so disruptive features don't go into it",
>
> the latter of which is what we've done in all prior releases. I'm curious
>
> if there was any point in that discussion where the intention was made
>
> explicit if anyone has a link.
>
>
>
>
>
>
>
> On Thu, Sep 24, 2020 at 11:53 AM, Benedict Elliott Smith <
>
> bened...@apache.org> wrote:
>
>
>
> > I'm not sure what you are referring to here, that vote said nothing about
>
> > branching at beta.
>
> >
>
> > The most recent vote on the topic anyway was for the Release Lifecycle
>
> > process, which stipulates branching at GA.
>
> >
>
> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>
> >
>
> > We can vote to modify this document, or to make an exception, but I am
>
> > aware of no other vote stipulating anything about the point at which we
>
> > branch.
>
> >
>
> > On 24/09/2020, 16:49, "Jake Luciani"  wrote:
>
> >
>
> > Today the community still has in force an explicit vote prohibiting thee
>
> >
>
> > merge of this work.
>
> >
>
> > You referred to an explicit vote here. I assume that was the one you were
>
> > referring to? Yes, the community should decide.
>
> > Call a vote if you think the community thinks we should continue the
>
> > freeze vs continuing to rely on beliefs about the community.
>
> >
>
> > I'm simply pointing out the branching of 4.0 post beta was the plan of
>
> > last record.
>
> >
>
> > Jake
>
> >
>
> > On Thu, Sep 24, 2020 at 11:44 AM Benedict Elliott Smith 
> > org> wrote:
>
> >
>
> > The community does everything through discussion and consensus. Does that
>
> > include branching, or not?
>
> >
>
> > If there is no consensus, a vote is held. Whether or not you consider the
>
> > vote from 2018 still valid, you still need to seek the consent of the
>
> > community for your action today. Or is that not sacrosanct anymore?
>
> >
>
> > On 24/09/2020, 16:22, "Jake Luciani"  wrote:
>
> >
>
> > I'm sorry I see no issue with branching 4.0 as it was the thing we voted
>
> > on
>
> > back in 2018. If you wish to extend the freeze you should call a new
> vote.
>
> >
>
> > On Thu, Sep 24, 2020 at 11:15 AM Benedict Elliott Smith <
> benedict@apache.
>
> > org>
>
> > wrote:
>
> >
>
> > Nobody has any problem with an external repository being
>
> >
>
> > maintained. Just
>
> >
>
> > bear in mind the normal process will need to take place to merge to
>
> >
>
> > the ASF
>
> >
>
> > repository, and that there may be feedback and review requests to
>
> >
>
> > address,
>
> >
>
> > so merge order and diffs will probably change.
>
> >
>
> > On 24/09/2020, 16:05, "Brandon Williams"  wrote:
>
> >
>
> > On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
>
> >  wrote:
>
> >
>
> > You do not have the authority to unilaterally overrule the
>
> >
>
> > community
>
> >
>
> > process. This is a serious breach of your responsibilities as a
>
> >
>
> > member of
>
> >
>
> > the PMC.
>
> >
>
> > Feel free to complain that I'm creating branches we intend to
>
> >
>
> > someday,
>
> >
>
> > perhaps even in 2020, release.
>
> >
>
> > I have deleted this branch, and will do so again if you repeat
>
> >
>
> > this.
>
> >
>
> > This would create some interesting tickets for INFRA, but I won't waste
>
> > their time with you either. Whether either of us has the authority to do
>
> > such on ASF infrastructure is irrelevant, since
>
> >
>
> > that
>
> >
>
> > is the only thing that can be argued here. The ASL absolutely
>
> >
>
> > allows
>
> >
>
> > people to innovate on their own with the code, so let's just
>
> >
>
> > move the
>
> >
>
> > bits.
>
> >
>
> > Those who wish to innovate,
>
> > https://github.com/driftx/cassandra/tree/cassandra-5.0 is now
>
> >
>
> > open for
>
> >
>
> > business, PRs accepted. This will be maintained to track trunk
>
> >
>
> > on the
>
> >
>
> > ASF servers.
>
> >
>
> > I guess this is the apache way.
>
> >
>
> > 

Re: Creating a branch for 5.0 …?

[Joshua McKenzie]

https://lists.apache.org/thread.html/5ee66f3986bf8308912c216bd1b5f9aea35443626db9f92cdca4d7b9%40%3Cdev.cassandra.apache.org%3E

"*From: *sankalp kohli 
*To: *dev 
*Subject: *Re: [VOTE] Branching Change for 4.0 Freeze
*Date: *2018/07/11 21:50:08
*List: *dev@cassandra.apache.org


We will be in this state till beta is reached."

The release lifecycle doc says: "*A new branch is created for the release
with the new major version, limiting any new feature addition to the new
release branch, with new feature development will continue to happen only
on trunk.*"

This could be read as "we won't branch until we hit GA" or "we will make
sure we definitely branch at GA so disruptive features don't go into it",
the latter of which is what we've done in all prior releases. I'm curious
if there was any point in that discussion where the intention was made
explicit if anyone has a link.



On Thu, Sep 24, 2020 at 11:53 AM, Benedict Elliott Smith <
bened...@apache.org> wrote:

> I'm not sure what you are referring to here, that vote said nothing about
> branching at beta.
>
> The most recent vote on the topic anyway was for the Release Lifecycle
> process, which stipulates branching at GA.
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>
> We can vote to modify this document, or to make an exception, but I am
> aware of no other vote stipulating anything about the point at which we
> branch.
>
> On 24/09/2020, 16:49, "Jake Luciani"  wrote:
>
> Today the community still has in force an explicit vote prohibiting thee
>
> merge of this work.
>
> You referred to an explicit vote here. I assume that was the one you were
> referring to? Yes, the community should decide.
> Call a vote if you think the community thinks we should continue the
> freeze vs continuing to rely on beliefs about the community.
>
> I'm simply pointing out the branching of 4.0 post beta was the plan of
> last record.
>
> Jake
>
> On Thu, Sep 24, 2020 at 11:44 AM Benedict Elliott Smith  org> wrote:
>
> The community does everything through discussion and consensus. Does that
> include branching, or not?
>
> If there is no consensus, a vote is held. Whether or not you consider the
> vote from 2018 still valid, you still need to seek the consent of the
> community for your action today. Or is that not sacrosanct anymore?
>
> On 24/09/2020, 16:22, "Jake Luciani"  wrote:
>
> I'm sorry I see no issue with branching 4.0 as it was the thing we voted
> on
> back in 2018. If you wish to extend the freeze you should call a new vote.
>
> On Thu, Sep 24, 2020 at 11:15 AM Benedict Elliott Smith < benedict@apache.
> org>
> wrote:
>
> Nobody has any problem with an external repository being
>
> maintained. Just
>
> bear in mind the normal process will need to take place to merge to
>
> the ASF
>
> repository, and that there may be feedback and review requests to
>
> address,
>
> so merge order and diffs will probably change.
>
> On 24/09/2020, 16:05, "Brandon Williams"  wrote:
>
> On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
>  wrote:
>
> You do not have the authority to unilaterally overrule the
>
> community
>
> process. This is a serious breach of your responsibilities as a
>
> member of
>
> the PMC.
>
> Feel free to complain that I'm creating branches we intend to
>
> someday,
>
> perhaps even in 2020, release.
>
> I have deleted this branch, and will do so again if you repeat
>
> this.
>
> This would create some interesting tickets for INFRA, but I won't waste
> their time with you either. Whether either of us has the authority to do
> such on ASF infrastructure is irrelevant, since
>
> that
>
> is the only thing that can be argued here. The ASL absolutely
>
> allows
>
> people to innovate on their own with the code, so let's just
>
> move the
>
> bits.
>
> Those who wish to innovate,
> https://github.com/driftx/cassandra/tree/cassandra-5.0 is now
>
> open for
>
> business, PRs accepted. This will be maintained to track trunk
>
> on the
>
> ASF servers.
>
> I guess this is the apache way.
>
> -
>
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For
> additional commands, e-mail: dev-h...@cassandra.apache.org
>
> - To
> unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional
> commands, e-mail: dev-h...@cassandra.apache.org
>
> --
> http://twitter.com/tjake
>
> - To
> unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional
> commands, e-mail: dev-h...@cassandra.apache.org
>
> --
> http://twitter.com/tjake
>
> - To
> unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional
> commands, e-mail: dev-h...@cassandra.apache.org
>

Re: Creating a branch for 5.0 …?

[Jake Luciani]

The vote was to unfreeze new changes at beta, so logically that means
non-bugfix work goes into trunk.

Jordan, thanks.   That is a more recent vote so thanks.  That being said,
under that line Benedict comments this needs to be discussed.
So how about we just have a Vote on branching cassandra-4.0 and the issue
will be decided?

Jake




On Thu, Sep 24, 2020 at 11:53 AM Benedict Elliott Smith 
wrote:

> I'm not sure what you are referring to here, that vote said nothing about
> branching at beta.
>
> The most recent vote on the topic anyway was for the Release Lifecycle
> process, which stipulates branching at GA.
>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>
> We can vote to modify this document, or to make an exception, but I am
> aware of no other vote stipulating anything about the point at which we
> branch.
>
>
> On 24/09/2020, 16:49, "Jake Luciani"  wrote:
>
> > Today the community still has in force an explicit vote prohibiting
> thee
> merge of this work.
>
> You referred to an explicit vote here.  I assume that was the one you
> were
> referring to?  Yes, the community should decide.
> Call a vote if you think the community thinks we should continue the
> freeze
> vs continuing to rely on beliefs about the community.
>
> I'm simply pointing out the branching of 4.0 post beta was the plan of
> last
> record.
>
> Jake
>
> On Thu, Sep 24, 2020 at 11:44 AM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > The community does everything through discussion and consensus.
> Does that
> > include branching, or not?
> >
> > If there is no consensus, a vote is held.  Whether or not you
> consider the
> > vote from 2018 still valid, you still need to seek the consent of the
> > community for your action today.  Or is that not sacrosanct anymore?
> >
> >
> > On 24/09/2020, 16:22, "Jake Luciani"  wrote:
> >
> > I'm sorry I see no issue with branching 4.0 as it was the thing
> we
> > voted on
> > back in 2018.  If you wish to extend the freeze you should call
> a new
> > vote.
> >
> > On Thu, Sep 24, 2020 at 11:15 AM Benedict Elliott Smith <
> > bened...@apache.org>
> > wrote:
> >
> > > Nobody has any problem with an external repository being
> > maintained.  Just
> > > bear in mind the normal process will need to take place to
> merge to
> > the ASF
> > > repository, and that there may be feedback and review requests
> to
> > address,
> > > so merge order and diffs will probably change.
> > >
> > >
> > > On 24/09/2020, 16:05, "Brandon Williams" 
> wrote:
> > >
> > > On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
> > >  wrote:
> > > >
> > > > You do not have the authority to unilaterally overrule
> the
> > community
> > > process.  This is a serious breach of your responsibilities as
> a
> > member of
> > > the PMC.
> > >
> > > Feel free to complain that I'm creating branches we intend
> to
> > someday,
> > > perhaps even in 2020, release.
> > >
> > > > I have deleted this branch, and will do so again if you
> repeat
> > this.
> > >
> > > This would create some interesting tickets for INFRA, but
> I won't
> > > waste their time with you either. Whether either of us has
> the
> > > authority to do such on ASF infrastructure is irrelevant,
> since
> > that
> > > is the only thing that can be argued here.  The ASL
> absolutely
> > allows
> > > people to innovate on their own with the code, so let's
> just
> > move the
> > > bits.
> > >
> > > Those who wish to innovate,
> > > https://github.com/driftx/cassandra/tree/cassandra-5.0 is
> now
> > open for
> > > business, PRs accepted. This will be maintained to track
> trunk
> > on the
> > > ASF servers.
> > >
> > > I guess this is the apache way.
> > >
> > >
> >
> -
> > > To unsubscribe, e-mail:
> dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail:
> dev-h...@cassandra.apache.org
> > >
> > >
> > >
> > >
> > >
> -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> > >
> >
> > --
> > http://twitter.com/tjake
> >
> >
> >
> > -
> > To 

Re: Creating a branch for 5.0 …?

[Benedict Elliott Smith]

I'm not sure what you are referring to here, that vote said nothing about 
branching at beta.

The most recent vote on the topic anyway was for the Release Lifecycle process, 
which stipulates branching at GA.

https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle

We can vote to modify this document, or to make an exception, but I am aware of 
no other vote stipulating anything about the point at which we branch.


On 24/09/2020, 16:49, "Jake Luciani"  wrote:

> Today the community still has in force an explicit vote prohibiting thee
merge of this work.

You referred to an explicit vote here.  I assume that was the one you were
referring to?  Yes, the community should decide.
Call a vote if you think the community thinks we should continue the freeze
vs continuing to rely on beliefs about the community.

I'm simply pointing out the branching of 4.0 post beta was the plan of last
record.

Jake

On Thu, Sep 24, 2020 at 11:44 AM Benedict Elliott Smith 

wrote:

> The community does everything through discussion and consensus.  Does that
> include branching, or not?
>
> If there is no consensus, a vote is held.  Whether or not you consider the
> vote from 2018 still valid, you still need to seek the consent of the
> community for your action today.  Or is that not sacrosanct anymore?
>
>
> On 24/09/2020, 16:22, "Jake Luciani"  wrote:
>
> I'm sorry I see no issue with branching 4.0 as it was the thing we
> voted on
> back in 2018.  If you wish to extend the freeze you should call a new
> vote.
>
> On Thu, Sep 24, 2020 at 11:15 AM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > Nobody has any problem with an external repository being
> maintained.  Just
> > bear in mind the normal process will need to take place to merge to
> the ASF
> > repository, and that there may be feedback and review requests to
> address,
> > so merge order and diffs will probably change.
> >
> >
> > On 24/09/2020, 16:05, "Brandon Williams"  wrote:
> >
> > On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
> >  wrote:
> > >
> > > You do not have the authority to unilaterally overrule the
> community
> > process.  This is a serious breach of your responsibilities as a
> member of
> > the PMC.
> >
> > Feel free to complain that I'm creating branches we intend to
> someday,
> > perhaps even in 2020, release.
> >
> > > I have deleted this branch, and will do so again if you repeat
> this.
> >
> > This would create some interesting tickets for INFRA, but I 
won't
> > waste their time with you either. Whether either of us has the
> > authority to do such on ASF infrastructure is irrelevant, since
> that
> > is the only thing that can be argued here.  The ASL absolutely
> allows
> > people to innovate on their own with the code, so let's just
> move the
> > bits.
> >
> > Those who wish to innovate,
> > https://github.com/driftx/cassandra/tree/cassandra-5.0 is now
> open for
> > business, PRs accepted. This will be maintained to track trunk
> on the
> > ASF servers.
> >
> > I guess this is the apache way.
> >
> >
>  -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
> >
> >
> > 
-
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>
> --
> http://twitter.com/tjake
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
http://twitter.com/tjake



-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Creating a branch for 5.0 …?

[Jake Luciani]

> Today the community still has in force an explicit vote prohibiting thee
merge of this work.

You referred to an explicit vote here.  I assume that was the one you were
referring to?  Yes, the community should decide.
Call a vote if you think the community thinks we should continue the freeze
vs continuing to rely on beliefs about the community.

I'm simply pointing out the branching of 4.0 post beta was the plan of last
record.

Jake

On Thu, Sep 24, 2020 at 11:44 AM Benedict Elliott Smith 
wrote:

> The community does everything through discussion and consensus.  Does that
> include branching, or not?
>
> If there is no consensus, a vote is held.  Whether or not you consider the
> vote from 2018 still valid, you still need to seek the consent of the
> community for your action today.  Or is that not sacrosanct anymore?
>
>
> On 24/09/2020, 16:22, "Jake Luciani"  wrote:
>
> I'm sorry I see no issue with branching 4.0 as it was the thing we
> voted on
> back in 2018.  If you wish to extend the freeze you should call a new
> vote.
>
> On Thu, Sep 24, 2020 at 11:15 AM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > Nobody has any problem with an external repository being
> maintained.  Just
> > bear in mind the normal process will need to take place to merge to
> the ASF
> > repository, and that there may be feedback and review requests to
> address,
> > so merge order and diffs will probably change.
> >
> >
> > On 24/09/2020, 16:05, "Brandon Williams"  wrote:
> >
> > On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
> >  wrote:
> > >
> > > You do not have the authority to unilaterally overrule the
> community
> > process.  This is a serious breach of your responsibilities as a
> member of
> > the PMC.
> >
> > Feel free to complain that I'm creating branches we intend to
> someday,
> > perhaps even in 2020, release.
> >
> > > I have deleted this branch, and will do so again if you repeat
> this.
> >
> > This would create some interesting tickets for INFRA, but I won't
> > waste their time with you either. Whether either of us has the
> > authority to do such on ASF infrastructure is irrelevant, since
> that
> > is the only thing that can be argued here.  The ASL absolutely
> allows
> > people to innovate on their own with the code, so let's just
> move the
> > bits.
> >
> > Those who wish to innovate,
> > https://github.com/driftx/cassandra/tree/cassandra-5.0 is now
> open for
> > business, PRs accepted. This will be maintained to track trunk
> on the
> > ASF servers.
> >
> > I guess this is the apache way.
> >
> >
>  -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>
> --
> http://twitter.com/tjake
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
http://twitter.com/tjake

Re: Creating a branch for 5.0 …?

[Jordan West]

https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle was
voted on after and under GA states: "A new branch is created for the
release with the new major version, limiting any new feature addition to
the new release branch, with new feature development will continue to
happen only on trunk."

Jordan


On Thu, Sep 24, 2020 at 8:12 AM Jake Luciani  wrote:

> >  Today the community still has in force an explicit vote prohibiting thee
> merge of this work.  You must conduct a vote to rescind this decision.
>
> Actually, the vote was defined to hold until beta release:
>
>
> https://lists.apache.org/thread.html/5ee66f3986bf8308912c216bd1b5f9aea35443626db9f92cdca4d7b9%40%3Cdev.cassandra.apache.org%3E
>
> -Jake
>
>
> On Thu, Sep 24, 2020 at 11:05 AM Brandon Williams 
> wrote:
>
> > On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
> >  wrote:
> > >
> > > You do not have the authority to unilaterally overrule the community
> > process.  This is a serious breach of your responsibilities as a member
> of
> > the PMC.
> >
> > Feel free to complain that I'm creating branches we intend to someday,
> > perhaps even in 2020, release.
> >
> > > I have deleted this branch, and will do so again if you repeat this.
> >
> > This would create some interesting tickets for INFRA, but I won't
> > waste their time with you either. Whether either of us has the
> > authority to do such on ASF infrastructure is irrelevant, since that
> > is the only thing that can be argued here.  The ASL absolutely allows
> > people to innovate on their own with the code, so let's just move the
> > bits.
> >
> > Those who wish to innovate,
> > https://github.com/driftx/cassandra/tree/cassandra-5.0 is now open for
> > business, PRs accepted. This will be maintained to track trunk on the
> > ASF servers.
> >
> > I guess this is the apache way.
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>
> --
> http://twitter.com/tjake
>

Re: Creating a branch for 5.0 …?

[Benedict Elliott Smith]

The community does everything through discussion and consensus.  Does that 
include branching, or not?

If there is no consensus, a vote is held.  Whether or not you consider the vote 
from 2018 still valid, you still need to seek the consent of the community for 
your action today.  Or is that not sacrosanct anymore?


On 24/09/2020, 16:22, "Jake Luciani"  wrote:

I'm sorry I see no issue with branching 4.0 as it was the thing we voted on
back in 2018.  If you wish to extend the freeze you should call a new vote.

On Thu, Sep 24, 2020 at 11:15 AM Benedict Elliott Smith 

wrote:

> Nobody has any problem with an external repository being maintained.  Just
> bear in mind the normal process will need to take place to merge to the 
ASF
> repository, and that there may be feedback and review requests to address,
> so merge order and diffs will probably change.
>
>
> On 24/09/2020, 16:05, "Brandon Williams"  wrote:
>
> On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
>  wrote:
> >
> > You do not have the authority to unilaterally overrule the community
> process.  This is a serious breach of your responsibilities as a member of
> the PMC.
>
> Feel free to complain that I'm creating branches we intend to someday,
> perhaps even in 2020, release.
>
> > I have deleted this branch, and will do so again if you repeat this.
>
> This would create some interesting tickets for INFRA, but I won't
> waste their time with you either. Whether either of us has the
> authority to do such on ASF infrastructure is irrelevant, since that
> is the only thing that can be argued here.  The ASL absolutely allows
> people to innovate on their own with the code, so let's just move the
> bits.
>
> Those who wish to innovate,
> https://github.com/driftx/cassandra/tree/cassandra-5.0 is now open for
> business, PRs accepted. This will be maintained to track trunk on the
> ASF servers.
>
> I guess this is the apache way.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
http://twitter.com/tjake



-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Creating a branch for 5.0 …?

[Jake Luciani]

I'm sorry I see no issue with branching 4.0 as it was the thing we voted on
back in 2018.  If you wish to extend the freeze you should call a new vote.

On Thu, Sep 24, 2020 at 11:15 AM Benedict Elliott Smith 
wrote:

> Nobody has any problem with an external repository being maintained.  Just
> bear in mind the normal process will need to take place to merge to the ASF
> repository, and that there may be feedback and review requests to address,
> so merge order and diffs will probably change.
>
>
> On 24/09/2020, 16:05, "Brandon Williams"  wrote:
>
> On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
>  wrote:
> >
> > You do not have the authority to unilaterally overrule the community
> process.  This is a serious breach of your responsibilities as a member of
> the PMC.
>
> Feel free to complain that I'm creating branches we intend to someday,
> perhaps even in 2020, release.
>
> > I have deleted this branch, and will do so again if you repeat this.
>
> This would create some interesting tickets for INFRA, but I won't
> waste their time with you either. Whether either of us has the
> authority to do such on ASF infrastructure is irrelevant, since that
> is the only thing that can be argued here.  The ASL absolutely allows
> people to innovate on their own with the code, so let's just move the
> bits.
>
> Those who wish to innovate,
> https://github.com/driftx/cassandra/tree/cassandra-5.0 is now open for
> business, PRs accepted. This will be maintained to track trunk on the
> ASF servers.
>
> I guess this is the apache way.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
http://twitter.com/tjake

Re: Creating a branch for 5.0 …?

[Brandon Williams]

Indeed, it looks like there is no issue with branching now.

On Thu, Sep 24, 2020 at 10:12 AM Jake Luciani  wrote:
>
> >  Today the community still has in force an explicit vote prohibiting thee
> merge of this work.  You must conduct a vote to rescind this decision.
>
> Actually, the vote was defined to hold until beta release:
>
> https://lists.apache.org/thread.html/5ee66f3986bf8308912c216bd1b5f9aea35443626db9f92cdca4d7b9%40%3Cdev.cassandra.apache.org%3E
>
> -Jake
>
>
> On Thu, Sep 24, 2020 at 11:05 AM Brandon Williams  wrote:
>
> > On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
> >  wrote:
> > >
> > > You do not have the authority to unilaterally overrule the community
> > process.  This is a serious breach of your responsibilities as a member of
> > the PMC.
> >
> > Feel free to complain that I'm creating branches we intend to someday,
> > perhaps even in 2020, release.
> >
> > > I have deleted this branch, and will do so again if you repeat this.
> >
> > This would create some interesting tickets for INFRA, but I won't
> > waste their time with you either. Whether either of us has the
> > authority to do such on ASF infrastructure is irrelevant, since that
> > is the only thing that can be argued here.  The ASL absolutely allows
> > people to innovate on their own with the code, so let's just move the
> > bits.
> >
> > Those who wish to innovate,
> > https://github.com/driftx/cassandra/tree/cassandra-5.0 is now open for
> > business, PRs accepted. This will be maintained to track trunk on the
> > ASF servers.
> >
> > I guess this is the apache way.
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>
> --
> http://twitter.com/tjake

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Creating a branch for 5.0 …?

[Benedict Elliott Smith]

Nobody has any problem with an external repository being maintained.  Just bear 
in mind the normal process will need to take place to merge to the ASF 
repository, and that there may be feedback and review requests to address, so 
merge order and diffs will probably change.


On 24/09/2020, 16:05, "Brandon Williams"  wrote:

On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
 wrote:
>
> You do not have the authority to unilaterally overrule the community 
process.  This is a serious breach of your responsibilities as a member of the 
PMC.

Feel free to complain that I'm creating branches we intend to someday,
perhaps even in 2020, release.

> I have deleted this branch, and will do so again if you repeat this.

This would create some interesting tickets for INFRA, but I won't
waste their time with you either. Whether either of us has the
authority to do such on ASF infrastructure is irrelevant, since that
is the only thing that can be argued here.  The ASL absolutely allows
people to innovate on their own with the code, so let's just move the
bits.

Those who wish to innovate,
https://github.com/driftx/cassandra/tree/cassandra-5.0 is now open for
business, PRs accepted. This will be maintained to track trunk on the
ASF servers.

I guess this is the apache way.

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org




-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Creating a branch for 5.0 …?

[Jake Luciani]

>  Today the community still has in force an explicit vote prohibiting thee
merge of this work.  You must conduct a vote to rescind this decision.

Actually, the vote was defined to hold until beta release:

https://lists.apache.org/thread.html/5ee66f3986bf8308912c216bd1b5f9aea35443626db9f92cdca4d7b9%40%3Cdev.cassandra.apache.org%3E

-Jake


On Thu, Sep 24, 2020 at 11:05 AM Brandon Williams  wrote:

> On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
>  wrote:
> >
> > You do not have the authority to unilaterally overrule the community
> process.  This is a serious breach of your responsibilities as a member of
> the PMC.
>
> Feel free to complain that I'm creating branches we intend to someday,
> perhaps even in 2020, release.
>
> > I have deleted this branch, and will do so again if you repeat this.
>
> This would create some interesting tickets for INFRA, but I won't
> waste their time with you either. Whether either of us has the
> authority to do such on ASF infrastructure is irrelevant, since that
> is the only thing that can be argued here.  The ASL absolutely allows
> people to innovate on their own with the code, so let's just move the
> bits.
>
> Those who wish to innovate,
> https://github.com/driftx/cassandra/tree/cassandra-5.0 is now open for
> business, PRs accepted. This will be maintained to track trunk on the
> ASF servers.
>
> I guess this is the apache way.
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
http://twitter.com/tjake

Re: Creating a branch for 5.0 …?

[Brandon Williams]

On Thu, Sep 24, 2020 at 9:55 AM Benedict Elliott Smith
 wrote:
>
> You do not have the authority to unilaterally overrule the community process. 
>  This is a serious breach of your responsibilities as a member of the PMC.

Feel free to complain that I'm creating branches we intend to someday,
perhaps even in 2020, release.

> I have deleted this branch, and will do so again if you repeat this.

This would create some interesting tickets for INFRA, but I won't
waste their time with you either. Whether either of us has the
authority to do such on ASF infrastructure is irrelevant, since that
is the only thing that can be argued here.  The ASL absolutely allows
people to innovate on their own with the code, so let's just move the
bits.

Those who wish to innovate,
https://github.com/driftx/cassandra/tree/cassandra-5.0 is now open for
business, PRs accepted. This will be maintained to track trunk on the
ASF servers.

I guess this is the apache way.

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Creating a branch for 5.0 …?

[Oleksandr Petrov]

> So, here's what I've done, in an effort to make a space for both of these
groups to operate: the exact same thing we've done for every release in the
past. I created a branch for the 4.0 release.

I agree that everyone is free to work on whatever they want, but it seems
like having a 4.next branch isn't a prerequisite for any new (or ongoing)
development.

I appreciate the cheerful tone in which this message was written, but I
still think we should try and find an agreement on such things before
putting them into action.

On Thu, Sep 24, 2020 at 4:31 PM Brandon Williams  wrote:

> It's been a while now for this thread, but it seems to me that it has
> been established:
>
> 1. This is an opensource project and anyone is free to work on any
> part of it that they choose. Nobody has authority over this other than
> the contributor.
> 2. Some people are concerned that allowing innovation (in code) will
> make 4.0 take longer to release and cause them the pain of merging up
> yet another branch.
>
> So, here's what I've done, in an effort to make a space for both of
> these groups to operate: the exact same thing we've done for every
> release in the past.  I created a branch for the 4.0 release.
>
> WHAT HAVE YOU DONE?!
>
> Alright, calm down.  You're in group 2, don't worry:  you have your
> 4.0 branch! You can completely focus on it, and if that's all you want
> to do, that's fine! YOU DO NOT NEED TO MERGE TO TRUNK.  Those who wish
> to volunteer their time merging from 4.0 to trunk will pick up this
> mantle.  If nobody does and I get exhausted, we'll just abort it and
> delete the branch, no big deal.  One more time to make this crystal
> clear: IF YOU DO NOT WANT TO MERGE FROM 4.0 TO TRUNK, YOU ARE ABSOLVED
> FROM THIS DUTY.  The branch is named, unsurprisingly, 'cassandra-4.0'
>
> As for those who wish to begin working on new features, trunk is now
> open for business.
>
> The merge order is now 2.1->2.2->3.0->3.11->4.0 and _optionally_,
> should you _choose_, trunk.
>
> The show must go on.
>
> On Wed, Sep 16, 2020 at 12:08 PM Dinesh Joshi  wrote:
> >
> > Maybe you should ask these people to bring their contributions or issues
> directly to the dev list. You don’t need to disclose their names or contact
> information.
> >
> > Contributing to the project involves engaging the community. We’re still
> open to discussions even if the patches may not land immediately.
> >
> > If they don’t talk to the dev list and won’t make a case for their
> contribution (assuming it’s a big one) we can’t discuss possible ways
> forward to accept it.
> >
> > It also seems that these folks are interested in contributing new
> features to Cassandra. When the community is working towards stabilizing
> 4.0, contributing to that effort will help build goodwill. We’re averse to
> one off, drive-by contributions. I am not assuming that’s the case but the
> fact is that we’re discussing 5.0 here.
> >
> > Dinesh
> >
> > > On Sep 16, 2020, at 6:00 AM, Joshua McKenzie 
> wrote:
> > >
> > > People aren't lining up waiting to contribute to the project until we
> > > accept non-4.0 quality-based contributions. There is a discrete window
> of
> > > opportunity where we as a project can make a first impression on folks
> > > interested in joining our community, and signals from people, the data
> we
> > > have available about contributors, as well as basic logic are all
> > > consistent that we are turning away new contributors, likely
> permanently.
> > > They're moving on to other projects, since "apparently the Cassandra
> > > project isn't interested in new contributions" (interviewees words 2
> weeks
> > > ago, not mine). Or same sentiment expressed by multiple major companies
> > > looking to find a storage coordination layer to put in front of their
> > > storage offerings, for instance.
> > >
> > > And sorry I can't give you specific names, dates, quotations, and/or
> > > contact information Benedict; it seems this rankles you as you
> continue to
> > > use terms like "hypothetical" and "mythical" to describe the very real
> > > humans I have spoken with over the course of the past year on this
> topic.
> > > If my constraints of confidentiality from the people I've interacted
> with
> > > are unacceptable for you in a discussion like this and you don't trust
> me
> > > enough to know I wouldn't overtly lie to try and shift an Overton
> Window,
> > > we should probably go ahead and agree to disagree on this conversation
> and
> > > let committers go forward and do what they think best for the project.
> > >
> > >
> > >
> > >> On Wed, Sep 16, 2020 at 5:09 AM, Benedict Elliott Smith <
> bened...@apache.org
> > >>> wrote:
> > >>
> > >> I know. I recognise that is a frustrating aspect of this discussion.
> It is
> > >> something hard to move on.
> > >>
> > >> So how about we wait until there's a concrete example we can discuss
> as a
> > >> community? If we don't have one, it doesn't seem pressing.
> > >>
> > >> On 16/09/2020, 

Re: Creating a branch for 5.0 …?

[Benedict Elliott Smith]

You do not have the authority to unilaterally overrule the community process.  
This is a serious breach of your responsibilities as a member of the PMC.

I have deleted this branch, and will do so again if you repeat this.

As discussed, nobody can police what you work on, but the community does decide 
what is merged.  Today the community still has in force an explicit vote 
prohibiting thee merge of this work.  You must conduct a vote to rescind this 
decision.  

Given the proposers of this policy have failed to respond to the most recent 
query on the topic, that would also seem very problematic to me.



On 24/09/2020, 15:31, "Brandon Williams"  wrote:

It's been a while now for this thread, but it seems to me that it has
been established:

1. This is an opensource project and anyone is free to work on any
part of it that they choose. Nobody has authority over this other than
the contributor.
2. Some people are concerned that allowing innovation (in code) will
make 4.0 take longer to release and cause them the pain of merging up
yet another branch.

So, here's what I've done, in an effort to make a space for both of
these groups to operate: the exact same thing we've done for every
release in the past.  I created a branch for the 4.0 release.

WHAT HAVE YOU DONE?!

Alright, calm down.  You're in group 2, don't worry:  you have your
4.0 branch! You can completely focus on it, and if that's all you want
to do, that's fine! YOU DO NOT NEED TO MERGE TO TRUNK.  Those who wish
to volunteer their time merging from 4.0 to trunk will pick up this
mantle.  If nobody does and I get exhausted, we'll just abort it and
delete the branch, no big deal.  One more time to make this crystal
clear: IF YOU DO NOT WANT TO MERGE FROM 4.0 TO TRUNK, YOU ARE ABSOLVED
FROM THIS DUTY.  The branch is named, unsurprisingly, 'cassandra-4.0'

As for those who wish to begin working on new features, trunk is now
open for business.

The merge order is now 2.1->2.2->3.0->3.11->4.0 and _optionally_,
should you _choose_, trunk.

The show must go on.

On Wed, Sep 16, 2020 at 12:08 PM Dinesh Joshi  wrote:
>
> Maybe you should ask these people to bring their contributions or issues 
directly to the dev list. You don’t need to disclose their names or contact 
information.
>
> Contributing to the project involves engaging the community. We’re still 
open to discussions even if the patches may not land immediately.
>
> If they don’t talk to the dev list and won’t make a case for their 
contribution (assuming it’s a big one) we can’t discuss possible ways forward 
to accept it.
>
> It also seems that these folks are interested in contributing new 
features to Cassandra. When the community is working towards stabilizing 4.0, 
contributing to that effort will help build goodwill. We’re averse to one off, 
drive-by contributions. I am not assuming that’s the case but the fact is that 
we’re discussing 5.0 here.
>
> Dinesh
>
> > On Sep 16, 2020, at 6:00 AM, Joshua McKenzie  
wrote:
> >
> > People aren't lining up waiting to contribute to the project until we
> > accept non-4.0 quality-based contributions. There is a discrete window 
of
> > opportunity where we as a project can make a first impression on folks
> > interested in joining our community, and signals from people, the data 
we
> > have available about contributors, as well as basic logic are all
> > consistent that we are turning away new contributors, likely 
permanently.
> > They're moving on to other projects, since "apparently the Cassandra
> > project isn't interested in new contributions" (interviewees words 2 
weeks
> > ago, not mine). Or same sentiment expressed by multiple major companies
> > looking to find a storage coordination layer to put in front of their
> > storage offerings, for instance.
> >
> > And sorry I can't give you specific names, dates, quotations, and/or
> > contact information Benedict; it seems this rankles you as you continue 
to
> > use terms like "hypothetical" and "mythical" to describe the very real
> > humans I have spoken with over the course of the past year on this 
topic.
> > If my constraints of confidentiality from the people I've interacted 
with
> > are unacceptable for you in a discussion like this and you don't trust 
me
> > enough to know I wouldn't overtly lie to try and shift an Overton 
Window,
> > we should probably go ahead and agree to disagree on this conversation 
and
> > let committers go forward and do what they think best for the project.
> >
> >
> >
> >> On Wed, Sep 16, 2020 at 5:09 AM, Benedict Elliott Smith 
 >>> wrote:
> >>
> >> I know. I recognise that is a frustrating aspect of this discussion. 
It is
> >> something hard to move on.
> >>
> >> So how 

Re: Creating a branch for 5.0 …?

[Brandon Williams]

It's been a while now for this thread, but it seems to me that it has
been established:

1. This is an opensource project and anyone is free to work on any
part of it that they choose. Nobody has authority over this other than
the contributor.
2. Some people are concerned that allowing innovation (in code) will
make 4.0 take longer to release and cause them the pain of merging up
yet another branch.

So, here's what I've done, in an effort to make a space for both of
these groups to operate: the exact same thing we've done for every
release in the past.  I created a branch for the 4.0 release.

WHAT HAVE YOU DONE?!

Alright, calm down.  You're in group 2, don't worry:  you have your
4.0 branch! You can completely focus on it, and if that's all you want
to do, that's fine! YOU DO NOT NEED TO MERGE TO TRUNK.  Those who wish
to volunteer their time merging from 4.0 to trunk will pick up this
mantle.  If nobody does and I get exhausted, we'll just abort it and
delete the branch, no big deal.  One more time to make this crystal
clear: IF YOU DO NOT WANT TO MERGE FROM 4.0 TO TRUNK, YOU ARE ABSOLVED
FROM THIS DUTY.  The branch is named, unsurprisingly, 'cassandra-4.0'

As for those who wish to begin working on new features, trunk is now
open for business.

The merge order is now 2.1->2.2->3.0->3.11->4.0 and _optionally_,
should you _choose_, trunk.

The show must go on.

On Wed, Sep 16, 2020 at 12:08 PM Dinesh Joshi  wrote:
>
> Maybe you should ask these people to bring their contributions or issues 
> directly to the dev list. You don’t need to disclose their names or contact 
> information.
>
> Contributing to the project involves engaging the community. We’re still open 
> to discussions even if the patches may not land immediately.
>
> If they don’t talk to the dev list and won’t make a case for their 
> contribution (assuming it’s a big one) we can’t discuss possible ways forward 
> to accept it.
>
> It also seems that these folks are interested in contributing new features to 
> Cassandra. When the community is working towards stabilizing 4.0, 
> contributing to that effort will help build goodwill. We’re averse to one 
> off, drive-by contributions. I am not assuming that’s the case but the fact 
> is that we’re discussing 5.0 here.
>
> Dinesh
>
> > On Sep 16, 2020, at 6:00 AM, Joshua McKenzie  wrote:
> >
> > People aren't lining up waiting to contribute to the project until we
> > accept non-4.0 quality-based contributions. There is a discrete window of
> > opportunity where we as a project can make a first impression on folks
> > interested in joining our community, and signals from people, the data we
> > have available about contributors, as well as basic logic are all
> > consistent that we are turning away new contributors, likely permanently.
> > They're moving on to other projects, since "apparently the Cassandra
> > project isn't interested in new contributions" (interviewees words 2 weeks
> > ago, not mine). Or same sentiment expressed by multiple major companies
> > looking to find a storage coordination layer to put in front of their
> > storage offerings, for instance.
> >
> > And sorry I can't give you specific names, dates, quotations, and/or
> > contact information Benedict; it seems this rankles you as you continue to
> > use terms like "hypothetical" and "mythical" to describe the very real
> > humans I have spoken with over the course of the past year on this topic.
> > If my constraints of confidentiality from the people I've interacted with
> > are unacceptable for you in a discussion like this and you don't trust me
> > enough to know I wouldn't overtly lie to try and shift an Overton Window,
> > we should probably go ahead and agree to disagree on this conversation and
> > let committers go forward and do what they think best for the project.
> >
> >
> >
> >> On Wed, Sep 16, 2020 at 5:09 AM, Benedict Elliott Smith 
> >>  >>> wrote:
> >>
> >> I know. I recognise that is a frustrating aspect of this discussion. It is
> >> something hard to move on.
> >>
> >> So how about we wait until there's a concrete example we can discuss as a
> >> community? If we don't have one, it doesn't seem pressing.
> >>
> >> On 16/09/2020, 08:23, "Mick Semb Wever"  wrote:
> >>
> >> Can you provide some concrete examples of your own?
> >>
> >> On a tangent, I really appreciate the work done in the post-mortem
> >> analysis of the 3.0 storage rewrite and just how long that took to find and
> >> fix bugs it caused. The more of that we do the better our QA process will
> >> become and the more we will feel justified/safe in raising concerns about
> >> large patches coming in at the wrong time/place.
> >>
> >> Ironically, this entire proposal so far rests on hypothetical lost
> >> contributions by hypothetical companies and individuals.
> >>
> >> I know. I recognise that is a frustrating aspect of this discussion. It is
> >> something hard to move on.
> >>
> >> I would also like to take issue with a talking 

Re: [DISCUSS] Next steps for Kubernetes operator SIG

[Benjamin Lerer]

>
> I realise there are meeting logs, but getting a wider discourse with
> non-stakeholder input might help to build a community consensus?  It
> doesn't seem like it can hurt at this point, anyway.
>

+1



On Wed, Sep 23, 2020 at 9:21 PM Benedict Elliott Smith 
wrote:

> Perhaps it helps to widen the field of discussion to the dev list?
>
> It might help if each of the stakeholder organisations state their view on
> the situation, including why they would or would not support a given
> approach/operator, and what (preferably specific) circumstances might lead
> them to change their mind?
>
> I realise there are meeting logs, but getting a wider discourse with
> non-stakeholder input might help to build a community consensus?  It
> doesn't seem like it can hurt at this point, anyway.
>
>
> On 23/09/2020, 17:13, "John Sanda"  wrote:
>
> I want to point out that pretty much everything being  discussed in
> this
> thread has been discussed at length during the SIG meetings. I think
> it is
> worth noting because we are pretty much still have the same
> conversation.
>
> On Wed, Sep 23, 2020 at 12:03 PM Benedict Elliott Smith <
> bened...@apache.org>
> wrote:
>
> > I don't think there's anything about a code drop that's not "The
> Apache
> > Way"
> >
> > If there's a consensus (or even strong majority) amongst invested
> parties,
> > I don't see why we could not adopt an operator directly into the
> project.
> >
> > It's possible a green field approach might lead to fewer hard
> feelings, as
> > everyone is in the same boat. Perhaps all operators are also
> suboptimal and
> > could be improved with a rewrite? But I think coordinating a lot of
> > different entities around an empty codebase is particularly
> challenging.  I
> > actually think it could be better for cohesion and collaboration to
> have a
> > suboptimal but substantive starting point.
> >
> >
> > On 23/09/2020, 16:11, "Stefan Miklosovic" <
> > stefan.mikloso...@instaclustr.com> wrote:
> >
> > I think that from Instaclustr it was stated quite clearly
> multiple
> > times that we are "fine to throw it away" if there is something
> better
> > and more wide-spread.Indeed, we have invested a lot of time in
> the
> > operator but it was not useless at all, we gained a lot of quite
> > unique knowledge how to put all pieces together. However, I
> think that
> > this space is going to be quite fragmented and "balkanized",
> which is
> > not always a bad thing, but in a quite narrow area as Kubernetes
> > operator is, I just do not see how 4 operators are going to be
> > beneficial for ordinary people ("official" from community, ours,
> > Datastax one and CassKop (without any significant order)). Sure,
> > innovation and healthy competition is important but to what
> extent ...
> > One can start a Cassandra cluster on Kubernetes just so many
> times
> > differently and nobody really likes a vendor lock-in. People
> wanting
> > to run a cluster on K8S realise that there are three operators,
> each
> > backed by a private business entity, and the community operator
> is not
> > there ... Huh, interesting ... One may even start to question
> what is
> > wrong with these folks that it takes three companies to build
> their
> > own solution.
> >
> > Having said that, to my perception, Cassandra community just
> does not
> > have enough engineers nor contributors to keep 4 operators alive
> at
> > the same time (I wish I was wrong) so the idea of selecting the
> best
> > one or to merge obvious things and approaches together is
> > understandable, even if it meant we eventually sunset ours. In
> > addition, nobody from big players is going to contribute to the
> code
> > base of the other one, for obvious reasons, so channeling and
> > directing this effort into something common for a community
> seems to
> > be the only reasonable way of cooperation.
> >
> > It is quite hard to bootstrap this if the donation of the code
> in big
> > chunks / whole repo is out of question as it is not the "Apache
> way"
> > (there was some thread running here about this in more depth a
> while
> > ago) and we basically need to start from scratch which is quite
> > demotivating, we are just inventing the wheel and nobody is up
> to it.
> > It is like people are waiting for that to happen so they can
> jump in
> > "once it is the thing" but it will never materialise or at least
> the
> > hurdle to kick it off is unnecessarily high. Nobody is going to
> invest
> > in this heavily if there is already a working operator from
> companies
> > mentioned above. As I understood it, one reason of not 

Re: [DISCUSS] CEP-7 Storage Attached Index

[Jasonstack Zhao Yang]

>> Question is: is this planned as a next step?
>> If yes, how are we going to mark SAI as experimental until it gets
>> row offsets? Also, it is likely that index format is going to change when
>> row offsets are added, so my concern is that we may have to support two
>> versions of a format for a smooth migration.

The goal is to support row-level index when merging SAI, I will update the
CEP about it.

>> I think switching to row
>> offsets also has a huge impact on interaction with SPRC and has some
>> potential for optimisations.

Can you share more details on the optimizations?



On Thu, 24 Sep 2020 at 15:20, Oleksandr Petrov 
wrote:

> > But for improving overall index read performance, I think improving base
> table read perf  (because SAI/SASI executes LOTS of
> SinglePartitionReadCommand after searching on-disk index) is more effective
> than switching from Trie to Prefix BTree.
>
> I haven't suggested switching to Prefix B-Tree or any other structure, the
> question was about rationale and motivation of picking one over the other,
> which I am curious about for personal reasons/interests that lie outside of
> Cassandra. Having this listed in CEP could have been helpful for future
> guidance. It's ok if this question is outside of the CEP scope.
>
> I also agree that there are many areas that require improvement around the
> read/write path and 2i, many of which (even outside of base table format or
> read perf) can yield positive performance results.
>
> > FWIW, I personally look forward to receiving that contribution when the
> time is right.
>
> I am very excited for this contribution, too, and it looks like very solid
> work.
>
> I have one more question, about "Upon resolving partition keys, rows are
> loaded using Cassandra’s internal partition read command across SSTables
> and are post filtered". One of the criticisms of SASI and reasons for
> marking it as experimental was CASSANDRA-11990. I think switching to row
> offsets also has a huge impact on interaction with SPRC and has some
> potential for optimisations. Question is: is this planned as a next step?
> If yes, how are we going to mark SAI as experimental until it gets
> row offsets? Also, it is likely that index format is going to change when
> row offsets are added, so my concern is that we may have to support two
> versions of a format for a smooth migration.
>
>
>
> On Thu, Sep 24, 2020 at 6:53 AM Jasonstack Zhao Yang <
> jasonstack.z...@gmail.com> wrote:
>
> > >> I think CEP should be more upfront with "eventually replace
> > >>  it" bit, since it raises the question about what the people who are
> > using
> > >> other index implementations can expect.
> >
> > Will update the CEP to emphasize: SAI will replace other indexes.
> >
> > >> Unfortunately, I do not have an
> > >> implementation sitting around for a direct comparison, but I can
> imagine
> > >> situations when B-Trees may perform better because of simpler
> > construction.
> > >> Maybe we should even consider prototyping a prefix B-Tree to have a
> more
> > >> fair comparison.
> >
> > As long as prefix BTree supports range/prefix aggregation (which is used
> to
> > speed up
> > range/prefix query when matching entire subtree), we can plug it in and
> > compare. It won't
> > affect the CEP design which focuses on sharing data across indexes and
> > posting aggregation.
> >
> > But for improving overall index read performance, I think improving base
> > table read perf
> >  (because SAI/SASI executes LOTS of SinglePartitionReadCommand after
> > searching on-disk index)
> > is more effective than switching from Trie to Prefix BTree.
> >
> >
> >
> > On Thu, 24 Sep 2020 at 05:33, Benedict Elliott Smith <
> bened...@apache.org>
> > wrote:
> >
> > > FWIW, I personally look forward to receiving that contribution when the
> > > time is right.
> > >
> > > On 23/09/2020, 18:45, "Josh McKenzie"  wrote:
> > >
> > > talking about that would involve some bits of information DataStax
> > > might
> > > not be ready to share?
> > >
> > > At the risk of derailing, I've been poking and prodding this week
> at
> > we
> > > contributors at DS getting our act together w/a draft CEP for
> > donating
> > > the
> > > trie-based indices to the ASF project.
> > >
> > > More to come; the intention is certainly to contribute that code.
> The
> > > lack
> > > of a destination to merge it into (i.e. no 5.0-dev branch) is
> > removing
> > > significant urgency from the process as well (not to open a 3rd
> > > Pandora's
> > > box), but there's certainly an interrelatedness to the
> conversations
> > > going
> > > on.
> > >
> > > ---
> > > Josh McKenzie
> > >
> > >
> > > Sent via Superhuman 
> > >
> > >
> > > On Wed, Sep 23, 2020 at 12:48 PM, Caleb Rackliffe <
> > > calebrackli...@gmail.com>
> > > wrote:
> > >
> > > > As long as we can construct the on-disk indexes
> > efficiently/directly
> 

Re: [DISCUSS] CEP-7 Storage Attached Index

[Oleksandr Petrov]

> But for improving overall index read performance, I think improving base
table read perf  (because SAI/SASI executes LOTS of
SinglePartitionReadCommand after searching on-disk index) is more effective
than switching from Trie to Prefix BTree.

I haven't suggested switching to Prefix B-Tree or any other structure, the
question was about rationale and motivation of picking one over the other,
which I am curious about for personal reasons/interests that lie outside of
Cassandra. Having this listed in CEP could have been helpful for future
guidance. It's ok if this question is outside of the CEP scope.

I also agree that there are many areas that require improvement around the
read/write path and 2i, many of which (even outside of base table format or
read perf) can yield positive performance results.

> FWIW, I personally look forward to receiving that contribution when the
time is right.

I am very excited for this contribution, too, and it looks like very solid
work.

I have one more question, about "Upon resolving partition keys, rows are
loaded using Cassandra’s internal partition read command across SSTables
and are post filtered". One of the criticisms of SASI and reasons for
marking it as experimental was CASSANDRA-11990. I think switching to row
offsets also has a huge impact on interaction with SPRC and has some
potential for optimisations. Question is: is this planned as a next step?
If yes, how are we going to mark SAI as experimental until it gets
row offsets? Also, it is likely that index format is going to change when
row offsets are added, so my concern is that we may have to support two
versions of a format for a smooth migration.



On Thu, Sep 24, 2020 at 6:53 AM Jasonstack Zhao Yang <
jasonstack.z...@gmail.com> wrote:

> >> I think CEP should be more upfront with "eventually replace
> >>  it" bit, since it raises the question about what the people who are
> using
> >> other index implementations can expect.
>
> Will update the CEP to emphasize: SAI will replace other indexes.
>
> >> Unfortunately, I do not have an
> >> implementation sitting around for a direct comparison, but I can imagine
> >> situations when B-Trees may perform better because of simpler
> construction.
> >> Maybe we should even consider prototyping a prefix B-Tree to have a more
> >> fair comparison.
>
> As long as prefix BTree supports range/prefix aggregation (which is used to
> speed up
> range/prefix query when matching entire subtree), we can plug it in and
> compare. It won't
> affect the CEP design which focuses on sharing data across indexes and
> posting aggregation.
>
> But for improving overall index read performance, I think improving base
> table read perf
>  (because SAI/SASI executes LOTS of SinglePartitionReadCommand after
> searching on-disk index)
> is more effective than switching from Trie to Prefix BTree.
>
>
>
> On Thu, 24 Sep 2020 at 05:33, Benedict Elliott Smith 
> wrote:
>
> > FWIW, I personally look forward to receiving that contribution when the
> > time is right.
> >
> > On 23/09/2020, 18:45, "Josh McKenzie"  wrote:
> >
> > talking about that would involve some bits of information DataStax
> > might
> > not be ready to share?
> >
> > At the risk of derailing, I've been poking and prodding this week at
> we
> > contributors at DS getting our act together w/a draft CEP for
> donating
> > the
> > trie-based indices to the ASF project.
> >
> > More to come; the intention is certainly to contribute that code. The
> > lack
> > of a destination to merge it into (i.e. no 5.0-dev branch) is
> removing
> > significant urgency from the process as well (not to open a 3rd
> > Pandora's
> > box), but there's certainly an interrelatedness to the conversations
> > going
> > on.
> >
> > ---
> > Josh McKenzie
> >
> >
> > Sent via Superhuman 
> >
> >
> > On Wed, Sep 23, 2020 at 12:48 PM, Caleb Rackliffe <
> > calebrackli...@gmail.com>
> > wrote:
> >
> > > As long as we can construct the on-disk indexes
> efficiently/directly
> > from
> > > a Memtable-attached index on flush, there's room to try other data
> > > structures. Most of the innovation in SAI is around the layout of
> > postings
> > > (something we can expand on if people are interested) and having a
> > > natively row-oriented design that scales w/ multiple indexed
> columns
> > on
> > > single SSTables. There are some broader implications of using the
> > trie that
> > > reach outside SAI itself, but talking about that would involve some
> > bits of
> > > information DataStax might not be ready to share?
> > >
> > > On Wed, Sep 23, 2020 at 11:00 AM Jeremiah D Jordan <
> jeremiah.jordan@
> > > gmail.com> wrote:
> > >
> > > Short question: looking forward, how are we going to maintain three
> > 2i
> > > implementations: SASI, SAI, and 2i?
> > >
> > > I think one of the goals stated in