Re: [Discuss] num_tokens default in Cassandra 4.0

[Jeremy Hanna]

As discussed, let's go with 16.  Speaking with Anthony privately as well, I
had forgotten that some of the analysis that Branimir had initially done on
the skew and allocation may have been internal to DataStax so I should have
mentioned that previously.  Thanks to Mick, Alex, and Anthony for doing
this analysis and helping back the decision with data.  This will benefit
many that start with Cassandra that don't know that 256 is a bad number and
end up with a hard to change decision later.  I assigned myself to
https://issues.apache.org/jira/browse/CASSANDRA-13701.  Thanks all.

On Wed, Mar 11, 2020 at 6:02 AM Mick Semb Wever  wrote:

>
>
> > I propose we drop it to 16 immediately.  I'll add the production docs
> > in CASSANDRA-15618 with notes on token count, the reasons why you'd want
> 1,
> > 4, or 16.  As a follow up, if we can get a token simulation written we
> can
> > try all sorts of topologies with whatever token algorithms we want.  Once
> > that simulation is written and we've got some reports we can revisit.
>
>
> This works for me, for our first step forward.
> Good docs will always empower users more than any default setting can!
>
> cheers,
> Mick
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Kubernetes operator unification

[Patrick McFadin]

Sure. Let me figure out some timing and propose some times.

On Tue, Mar 31, 2020 at 5:15 PM Nate McCall  wrote:

> Given the large portion of work that's been done in EU by the Orange folks
> vs. that of PST and APAC, I think this might be one for which we do two
> versions: PST morning and evening.
>
> On Wed, Apr 1, 2020 at 12:51 PM Patrick McFadin 
> wrote:
>
> > *Thanks for starting this thread Ben! Definitely agree that having a
> single
> > project-owned Kubernetes operator for Cassandra is preferred over a
> > fragmented ecosystem. I'll echo the same sentiment based on conversations
> > that it appears the community is eager to share experiences and
> > implementations in this space.Speaking for both myself and some other
> > contributors I've been working with, we're super excited to collaborate
> > with the community on this unified/standardized project-based operator.
> It
> > seems that nobody is tied to their own code and are open to one solution
> > that blends the best of all of these operators.Given the current virtual
> > event way of life we are experiencing, would everyone be ok with me
> > organizing a Zoom call for anyone who is interested so we can kick this
> off
> > properly? If we all engage our social networks, I think this could be a
> > great way of growing the community and inviting new contributors to the
> > project. When done, I can post the notes and video in the cwiki. Patrick*
> >
> > On Tue, Mar 31, 2020 at 8:09 AM Jake Luciani  wrote:
> >
> > > Hi Ben!
> > >
> > > Totally agree.  We should collaborate on a unified operator and I think
> > as
> > > deployment on k8s becomes more and more prevalent we need to have
> > > distributed testing in k8s.
> > >
> > > To that end we are working on OSS releasing our distributed testing
> > service
> > > we've developed over the years to make this easier and reproducible.
> > Need a
> > > few more days before that's ready
> > > but it may give us a leg up.  I know Alex Petrov has been working a lot
> > on
> > > the new jvm dtest harness and may have some ideas.
> > >
> > > Jake
> > >
> > > On Tue, Mar 31, 2020 at 12:11 AM Ben Bromhead 
> > wrote:
> > >
> > > > Hi All
> > > >
> > > > With the announcement of a C* Sidecar and K8s operator from Datastax
> > > > (congrats btw), Jake and Stefan discussed moving to a more
> > > > standardised/unified implementation of an Apache Cassandra operator
> for
> > > > Kubernetes. Based on discussions with other folks either using our
> > > > operator, building/running their own or just getting started, there
> > > appears
> > > > to be some broader enthusiasm to a more unified approach outside of
> > just
> > > > that thread.
> > > >
> > > > The current state of play for folks looking to run Apache Cassandra,
> > > > particularly on Kubernetes, is fairly fragmented. There are multiple
> > > > projects all doing similar things from large companies operating C*
> at
> > > > scale on kubernetes, individual contributors and commercialising
> > > entities.
> > > > Each one of these projects also have similar but diverse
> > implementations
> > > > and capabilities. From an end user perspective, it makes it very hard
> > to
> > > > figure out what path to take and from someone who supports these end
> > > users,
> > > > I'd much rather support one implementation than 3 even if it's not
> the
> > > one
> > > > we wrote :)
> > > >
> > > > To that end, I'd like to indicate that we (Instaclustr) are open to
> > > working
> > > > towards a project owned standardized K8s operators/sidecar/etc. How
> > that
> > > > looks and how it gets implemented will surely be the subject of
> debate,
> > > > especially amongst those with existing implementations.
> > > >
> > > > Before engaging in CEP process (
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652201)
> > > > it might be useful to informally discuss an approach to unifying
> > > > implementations.
> > > >
> > > > To that end I'd like to circulate the following ideas to kick off the
> > > > discussion of an approach that might be useful:
> > > >
> > > > We should look to start off with a new implementation in a separate
> > repo
> > > > (as the sidecar project has done), leveraging the experience and
> > > > contributions from existing operator implementations and a framework
> > like
> > > > the operator-framework, with the initial scope of just supporting our
> > > > distributed testing in our CI pipeline.
> > > >
> > > > Targeting our own distributed test cases (e.g. dtests) brings a
> number
> > of
> > > > benefits:
> > > >
> > > >- Defines a common environment and goals that minimizes each
> > > >organisations unique kubernetes challenges.
> > > >- Follows the spirit of the 4.0 release to be more dba/operator
> > > aligned,
> > > >more production ready and easier to get right in a production
> > setting
> > > > OOB
> > > >- Our test environment over time will look more and more like how
> > > 

Re: Idea: Marking required scope for 4.0 GA vs. optional

[Jake Luciani]

I see what you mean, I guess my personal line is: does this work worse than
the previous released version?
Seems like that's a yes in this case :)

On Tue, Mar 31, 2020 at 7:35 PM David Capwell  wrote:

> One ticket I wanted to bring up is CASSANDRA-15566, this ticket is not a
> regression but it is a critical bug.  Personally I feel that only
> regression should go into a freeze so I have a hard time justifying that
> ticket right now (all known failure modes have been failing since at least
> 2.1).  I guess the thing that is hard is where do we draw the line?  One
> person's bug is another person's improvement, so it's easy for different
> perspectives to clash on this.
>
>
> On Tue, Mar 31, 2020, 3:27 PM Joseph Lynch  wrote:
>
> > On Tue, Mar 31, 2020 at 1:27 PM Jake Luciani  wrote:
> > >
> > > Can we agree to move the improvements out to 4.0.x?
> >
> > Generally I've been asked to put performance issues as improvements,
> > e.g. CASSANDRA-15379. To be frank though we can't run ZstdCompressor
> > on real clusters without that patch, and therefore I wouldn't feel
> > great releasing ZstdCompressor in 4.0 without that patch.
> >
> > I'm fine to start calling all performance issues "bugs" since at least
> > in our deployments and I think in many others performance regressions
> > are P0 bugs that cost a lot of $$, or we can just keep calling them
> > improvements and just tag them with the ~right target fix version.
> > Namely 4.0-alpha if the change impacts any public interface in a non
> > backwards compatible way (yaml, properties, cql, jmx etc...), 4.0-beta
> > or later if it does not require changes to public interfaces.
> >
> > -Joey
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>


-- 
http://twitter.com/tjake

Re: Idea: Marking required scope for 4.0 GA vs. optional

[Jake Luciani]

If the performance issue is a regression compared to 3.11 that makes total
sense.
And in the case of ZStd since that's new if its unusable without the
"improvement" then it also makes sense.

I think in both cases though it makes sense to classify these as
performance regression bugs.
I'll take a deeper look at the Improvement tickets, perhaps they all fall
into this category.

Jake





On Tue, Mar 31, 2020 at 6:27 PM Joseph Lynch  wrote:

> On Tue, Mar 31, 2020 at 1:27 PM Jake Luciani  wrote:
> >
> > Can we agree to move the improvements out to 4.0.x?
>
> Generally I've been asked to put performance issues as improvements,
> e.g. CASSANDRA-15379. To be frank though we can't run ZstdCompressor
> on real clusters without that patch, and therefore I wouldn't feel
> great releasing ZstdCompressor in 4.0 without that patch.
>
> I'm fine to start calling all performance issues "bugs" since at least
> in our deployments and I think in many others performance regressions
> are P0 bugs that cost a lot of $$, or we can just keep calling them
> improvements and just tag them with the ~right target fix version.
> Namely 4.0-alpha if the change impacts any public interface in a non
> backwards compatible way (yaml, properties, cql, jmx etc...), 4.0-beta
> or later if it does not require changes to public interfaces.
>
> -Joey
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

-- 
http://twitter.com/tjake

20200331 4.0 status update

[Jon Meredith]

Hello dev@cassandra, here is the state of 4.0 things.

Link to the JIRA board:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA

Josh has added a new 'Needs Attention' filter to show any tasks that
are stalled, need an assignee or need a reviewer.  Makes it easy if
you'd like to find something to work on.

There are currently 19 issues tagged as alpha, with discussion taking
place on the mailing list on how to triage and track existing and
newly created issues so that we can get to a first beta.

[Open vs. Closed last 7 days]
We opened 13 new tickets in the last 7 days, four of them against the
alpha with one already committed and another approved, with the other
two progressing. Three against beta, six against the RC.

We closed 11 for a delta of +2, a minor uptick.

Opened:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=1670
Closed:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=1671

[Needs Assignee]
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=1658=1661

Down from 14 to 11.  Three are related to broader release quality efforts.

[Needs Reviewer]
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=1661=1659

3 alpha / 3 beta / 3 RC

A quick plug for the alpha tickets, patch available for your reviewing pleasure.

TEST https://issues.apache.org/jira/browse/CASSANDRA-15338 Fix flakey
testMessagePurging - org.apache.cassandra.net.ConnectionTest
BUG https://issues.apache.org/jira/browse/CASSANDRA-15674
liveDiskSpaceUsed and totalDiskSpaceUsed get corrupted if
IndexSummaryRedistribution gets interrupted
BUG https://issues.apache.org/jira/browse/CASSANDRA-15660 Unable to
specify -e/--execute flag in cqlsh

[Stalled: >14d no movement]
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA-15601=1661=1694

51 tickets tagged against 4.0 haven't been updated in the last 2
weeks, with four are marked to complete in alpha.

CASSANDRA-15262 server_encryption_options is not backwards compatible with 3.11
CASSANDRA-15624 Avoid lazy initializing shut down instances when
trying to send them messages
CASSANDRA-15551 Fix flaky tests org.apache.cassandra.service.MoveTest
testStateJumpToNormal and
testMoveWithPendingRangesNetworkStrategyRackAwareThirtyNodes
CASSANDRA-15601 Ensure repaired data tracking reads a consistent
amount of data across replicas [patch available!]

If you have any new info on those tickets, please update and share as
we're getting tantalizingly close to completing the alpha tasks.

[Cumulative Flow]
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=reporting=cumulativeFlowDiagram=939=936=931=1505=1506=1514=1509=1512=1507=30

Up and to the right, what more could you ask. Continuing at about the
same rate as the last few weeks.
If you'd like to cheer yourself up about progress towards 4.0, zoom
out the overview over the last 12 months
and you can see how much faster things are moving now.


Once again, thanks for all the hard work everybody. Feedback is
welcome on these emails. Are they a helpful snapshot? Are they
motivating? Is it just noise on the dev list. Please let us know how
they can be improved.

Jon

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Kubernetes operator unification

[Patrick McFadin]

*Thanks for starting this thread Ben! Definitely agree that having a single
project-owned Kubernetes operator for Cassandra is preferred over a
fragmented ecosystem. I'll echo the same sentiment based on conversations
that it appears the community is eager to share experiences and
implementations in this space.Speaking for both myself and some other
contributors I've been working with, we're super excited to collaborate
with the community on this unified/standardized project-based operator. It
seems that nobody is tied to their own code and are open to one solution
that blends the best of all of these operators.Given the current virtual
event way of life we are experiencing, would everyone be ok with me
organizing a Zoom call for anyone who is interested so we can kick this off
properly? If we all engage our social networks, I think this could be a
great way of growing the community and inviting new contributors to the
project. When done, I can post the notes and video in the cwiki. Patrick*

On Tue, Mar 31, 2020 at 8:09 AM Jake Luciani  wrote:

> Hi Ben!
>
> Totally agree.  We should collaborate on a unified operator and I think as
> deployment on k8s becomes more and more prevalent we need to have
> distributed testing in k8s.
>
> To that end we are working on OSS releasing our distributed testing service
> we've developed over the years to make this easier and reproducible. Need a
> few more days before that's ready
> but it may give us a leg up.  I know Alex Petrov has been working a lot on
> the new jvm dtest harness and may have some ideas.
>
> Jake
>
> On Tue, Mar 31, 2020 at 12:11 AM Ben Bromhead  wrote:
>
> > Hi All
> >
> > With the announcement of a C* Sidecar and K8s operator from Datastax
> > (congrats btw), Jake and Stefan discussed moving to a more
> > standardised/unified implementation of an Apache Cassandra operator for
> > Kubernetes. Based on discussions with other folks either using our
> > operator, building/running their own or just getting started, there
> appears
> > to be some broader enthusiasm to a more unified approach outside of just
> > that thread.
> >
> > The current state of play for folks looking to run Apache Cassandra,
> > particularly on Kubernetes, is fairly fragmented. There are multiple
> > projects all doing similar things from large companies operating C* at
> > scale on kubernetes, individual contributors and commercialising
> entities.
> > Each one of these projects also have similar but diverse implementations
> > and capabilities. From an end user perspective, it makes it very hard to
> > figure out what path to take and from someone who supports these end
> users,
> > I'd much rather support one implementation than 3 even if it's not the
> one
> > we wrote :)
> >
> > To that end, I'd like to indicate that we (Instaclustr) are open to
> working
> > towards a project owned standardized K8s operators/sidecar/etc. How that
> > looks and how it gets implemented will surely be the subject of debate,
> > especially amongst those with existing implementations.
> >
> > Before engaging in CEP process (
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652201)
> > it might be useful to informally discuss an approach to unifying
> > implementations.
> >
> > To that end I'd like to circulate the following ideas to kick off the
> > discussion of an approach that might be useful:
> >
> > We should look to start off with a new implementation in a separate repo
> > (as the sidecar project has done), leveraging the experience and
> > contributions from existing operator implementations and a framework like
> > the operator-framework, with the initial scope of just supporting our
> > distributed testing in our CI pipeline.
> >
> > Targeting our own distributed test cases (e.g. dtests) brings a number of
> > benefits:
> >
> >- Defines a common environment and goals that minimizes each
> >organisations unique kubernetes challenges.
> >- Follows the spirit of the 4.0 release to be more dba/operator
> aligned,
> >more production ready and easier to get right in a production setting
> > OOB
> >- Our test environment over time will look more and more like how
> users
> >run Cassandra in production. This will be the biggest win IMHO.
> >- The distributed tests will also serve as functional tests for the
> >operator itself.
> >
> > The main drawback I can see with this approach is it will potentially be
> a
> > longer path to getting a useable project based operator out the door. It
> > will also involve a ton of reworking dtests, which for some is going to a
> > hard blocker. From there we can start to expand and support more and more
> > real life use cases. Hopefully this is not a huge leap as our testing
> > should be covering most of those cases!
> >
> > This is largely my personal gut feel on the approach and I'm looking
> > forward to folks other suggestions!
> >
> > Cheers
> >
> > --
> >
> > Ben Bromhead
> >
> > Instaclustr 

Re: Idea: Marking required scope for 4.0 GA vs. optional

[David Capwell]

One ticket I wanted to bring up is CASSANDRA-15566, this ticket is not a
regression but it is a critical bug.  Personally I feel that only
regression should go into a freeze so I have a hard time justifying that
ticket right now (all known failure modes have been failing since at least
2.1).  I guess the thing that is hard is where do we draw the line?  One
person's bug is another person's improvement, so it's easy for different
perspectives to clash on this.


On Tue, Mar 31, 2020, 3:27 PM Joseph Lynch  wrote:

> On Tue, Mar 31, 2020 at 1:27 PM Jake Luciani  wrote:
> >
> > Can we agree to move the improvements out to 4.0.x?
>
> Generally I've been asked to put performance issues as improvements,
> e.g. CASSANDRA-15379. To be frank though we can't run ZstdCompressor
> on real clusters without that patch, and therefore I wouldn't feel
> great releasing ZstdCompressor in 4.0 without that patch.
>
> I'm fine to start calling all performance issues "bugs" since at least
> in our deployments and I think in many others performance regressions
> are P0 bugs that cost a lot of $$, or we can just keep calling them
> improvements and just tag them with the ~right target fix version.
> Namely 4.0-alpha if the change impacts any public interface in a non
> backwards compatible way (yaml, properties, cql, jmx etc...), 4.0-beta
> or later if it does not require changes to public interfaces.
>
> -Joey
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Idea: Marking required scope for 4.0 GA vs. optional

[Joseph Lynch]

On Tue, Mar 31, 2020 at 1:27 PM Jake Luciani  wrote:
>
> Can we agree to move the improvements out to 4.0.x?

Generally I've been asked to put performance issues as improvements,
e.g. CASSANDRA-15379. To be frank though we can't run ZstdCompressor
on real clusters without that patch, and therefore I wouldn't feel
great releasing ZstdCompressor in 4.0 without that patch.

I'm fine to start calling all performance issues "bugs" since at least
in our deployments and I think in many others performance regressions
are P0 bugs that cost a lot of $$, or we can just keep calling them
improvements and just tag them with the ~right target fix version.
Namely 4.0-alpha if the change impacts any public interface in a non
backwards compatible way (yaml, properties, cql, jmx etc...), 4.0-beta
or later if it does not require changes to public interfaces.

-Joey

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Idea: Marking required scope for 4.0 GA vs. optional

[Joshua McKenzie]

An initial rough attempt to get together a queried view of outstanding
scope for "4.0 GA blockers" is here on this quick filter:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=1719

JQL for a very similar query (pulls in about 10 more: catches sub-tasks for
things that are improvements I think?):
https://issues.apache.org/jira/issues/?filter=12348578

So that should at least help unblock anybody that's looking for "what's a
more refined subset of things that are GA blockers for 4.0". Hopefully
that's helpful.

Be aware - that query does a dumb pull in of things that haven't updated in
two weeks. That might be too aggressive, might be just fine, but be aware
that there's tickets people are probably deeply into that don't need
external prodding captured in that list along with just trivially stuck
things that fell off radars.

~Josh

On Tue, Mar 31, 2020 at 4:27 PM Jake Luciani  wrote:

> I've been starting to look at the work left for 4.0 and was surprised to
> see almost 1/2 the open tickets are Improvements.
> Shouldn't the only criteria for 4.0 at this point should be bugs.
>
> Can we agree to move the improvements out to 4.0.x?
>
> -Jake
>
> On Tue, Mar 31, 2020 at 4:02 PM Ekaterina Dimitrova <
> ekaterina.dimitr...@datastax.com> wrote:
>
> > Hello everyone,
> > I think the project might benefit of one "filter applied" now on the
> > current scope of JIRA tickets.
> > And then, regular triage so focus is not lost.
> >
> > Up to now, I always try to look at the Release Lifecycle [1] criteria
> when
> > I open a new ticket in order to mark what is the target version of the
> > patch. Also, I look from the perspective of the general code freeze.
> > Hope I am doing it right. If those are strictly marked, do we need
> > additional label for blocking tickets? Or we want to differentiate
> between
> > strong blockers and such patches which would be good if they land at a
> > specific release but not mandatory?
> >
> > Why I am in for initial filter now? As I found many tickets opened in
> > 2016...17...etc Things have changed a lot since then and there can be
> > easily different PoV nowadays.
> >
> > I would be more than happy to help in any of these activities, too.
> >
> > Ekaterina
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
> >
> >
> > > Begin forwarded message:
> > >
> > > *From:* David Capwell 
> > > *Date:* 31 March 2020, 15:15:11 GMT-4
> > > *To:* dev@cassandra.apache.org
> > > *Subject:* *Re:  Idea: Marking required scope for 4.0 GA vs. optional*
> > > *Reply-To:* dev@cassandra.apache.org
> > >
> > > What I'm used to is having two buckets for a release: tickets in the
> > > release (if not complete they are blockers), and triage. How this is
> done
> > > isn't important but I do feel it's important to have both.
> > >
> > > Right now I don't see a active triage, but to Josh's point we would
> need
> > to
> > > know who should first. Without answering, let me ask a question;
> should I
> > > (non committer) be adding blockers? If I add a blocker who should
> verify
> > it
> > > should block? What if project members elect non-commiters to do this?
> > >
> > >
> > >
> > > On Mon, Mar 30, 2020, 8:45 AM Joshua McKenzie 
> > > wrote:
> > >
> > > Regardless of how we indicate optional vs. required for rel, are there
> > >
> > > strong opinions on who should set that metadata on tickets? Reporter?
> > >
> > > Assignee? One person? A group of people?
> > >
> > >
> > >
> > > On Sun, Mar 29, 2020 at 10:04 AM Joshua McKenzie  >
> > >
> > > wrote:
> > >
> > >
> > > FWIW, I don't care what we go with as long as we can differentiate
> > >
> > > tickets
> > >
> > > that are optional for the rel vs. tickets that are blockers and filter
> > >
> > > the
> > >
> > > JIRA board on them so people know where they should focus their effort.
> > >
> > >
> > > The rest of it's just paint colors to me.
> > >
> > >
> > > On Sun, Mar 29, 2020 at 9:24 AM Mick Semb Wever 
> wrote:
> > >
> > >
> > >
> > > Alternatively, we could revert to using 4.0.X or 4.X as we once did to
> > >
> > > indicate something is targeting a release vs. blocking on inclusion
> > >
> > > for
> > >
> > > it.
> > >
> > > That seems to be a more "project JIRA hackish idiom", and one that's
> > >
> > > historically been confusing for people. At least with a label it would
> > >
> > > be
> > >
> > > clear with a name like "4.0-blocker", and we could then easily filter
> > >
> > > the
> > >
> > > JIRA board on it.
> > >
> > >
> > >
> > >
> > > Now that I better understand Benedict's previous response (see the
> > >
> > > 'Dev Cassandra 4.0 Dev Work Status' thread¹), I'm leaning with his view
> > >
> > > for
> > >
> > > now.
> > >
> > >
> > > I certainly missed the difference on how tickets ended up resolved as
> > >
> > > `4.0-alpha`. That unresolved `4.x` tickets were non-blockers that could
> > >
> > > still go into `4.0-alpha` if they satisfied the feature freeze and met
> > >
> > 

Re: Idea: Marking required scope for 4.0 GA vs. optional

[Jake Luciani]

I've been starting to look at the work left for 4.0 and was surprised to
see almost 1/2 the open tickets are Improvements.
Shouldn't the only criteria for 4.0 at this point should be bugs.

Can we agree to move the improvements out to 4.0.x?

-Jake

On Tue, Mar 31, 2020 at 4:02 PM Ekaterina Dimitrova <
ekaterina.dimitr...@datastax.com> wrote:

> Hello everyone,
> I think the project might benefit of one "filter applied" now on the
> current scope of JIRA tickets.
> And then, regular triage so focus is not lost.
>
> Up to now, I always try to look at the Release Lifecycle [1] criteria when
> I open a new ticket in order to mark what is the target version of the
> patch. Also, I look from the perspective of the general code freeze.
> Hope I am doing it right. If those are strictly marked, do we need
> additional label for blocking tickets? Or we want to differentiate between
> strong blockers and such patches which would be good if they land at a
> specific release but not mandatory?
>
> Why I am in for initial filter now? As I found many tickets opened in
> 2016...17...etc Things have changed a lot since then and there can be
> easily different PoV nowadays.
>
> I would be more than happy to help in any of these activities, too.
>
> Ekaterina
>
> [1]
> https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle
>
>
> > Begin forwarded message:
> >
> > *From:* David Capwell 
> > *Date:* 31 March 2020, 15:15:11 GMT-4
> > *To:* dev@cassandra.apache.org
> > *Subject:* *Re:  Idea: Marking required scope for 4.0 GA vs. optional*
> > *Reply-To:* dev@cassandra.apache.org
> >
> > What I'm used to is having two buckets for a release: tickets in the
> > release (if not complete they are blockers), and triage. How this is done
> > isn't important but I do feel it's important to have both.
> >
> > Right now I don't see a active triage, but to Josh's point we would need
> to
> > know who should first. Without answering, let me ask a question; should I
> > (non committer) be adding blockers? If I add a blocker who should verify
> it
> > should block? What if project members elect non-commiters to do this?
> >
> >
> >
> > On Mon, Mar 30, 2020, 8:45 AM Joshua McKenzie 
> > wrote:
> >
> > Regardless of how we indicate optional vs. required for rel, are there
> >
> > strong opinions on who should set that metadata on tickets? Reporter?
> >
> > Assignee? One person? A group of people?
> >
> >
> >
> > On Sun, Mar 29, 2020 at 10:04 AM Joshua McKenzie 
> >
> > wrote:
> >
> >
> > FWIW, I don't care what we go with as long as we can differentiate
> >
> > tickets
> >
> > that are optional for the rel vs. tickets that are blockers and filter
> >
> > the
> >
> > JIRA board on them so people know where they should focus their effort.
> >
> >
> > The rest of it's just paint colors to me.
> >
> >
> > On Sun, Mar 29, 2020 at 9:24 AM Mick Semb Wever  wrote:
> >
> >
> >
> > Alternatively, we could revert to using 4.0.X or 4.X as we once did to
> >
> > indicate something is targeting a release vs. blocking on inclusion
> >
> > for
> >
> > it.
> >
> > That seems to be a more "project JIRA hackish idiom", and one that's
> >
> > historically been confusing for people. At least with a label it would
> >
> > be
> >
> > clear with a name like "4.0-blocker", and we could then easily filter
> >
> > the
> >
> > JIRA board on it.
> >
> >
> >
> >
> > Now that I better understand Benedict's previous response (see the
> >
> > 'Dev Cassandra 4.0 Dev Work Status' thread¹), I'm leaning with his view
> >
> > for
> >
> > now.
> >
> >
> > I certainly missed the difference on how tickets ended up resolved as
> >
> > `4.0-alpha`. That unresolved `4.x` tickets were non-blockers that could
> >
> > still go into `4.0-alpha` if they satisfied the feature freeze and met
> >
> > someone's itch, while the unresolved `4.0-alpha` tickets were those the
> >
> > community declared as blockers.
> >
> >
> > Putting aside the discussion on what is a blocker, when it is discovered
> >
> > to
> >
> > be a blocker (and vice versa). My confusion stemmed from the fact there
> >
> > was
> >
> > no specific alpha|beta versions for tickets to get resolved into, eg
> >
> > 4.0-alpha1, 4.0-alpha2, etc. So it wasn't just that the normal  `.x`
> >
> > nomenclature got changed, but having accurate fix versions was also
> >
> > removed. If we added the specific alpha|beta versions then I wouldn't
> >
> > consider the approach to be a "hackish idiom" anymore. The 4.0-alpha,
> >
> > 4.0-beta and 4.0, versions become purely target versions like the `.x`,
> >
> > and
> >
> > no resolved ticket is ever assigned them.
> >
> >
> > The simpler we can make and document this the better. please.
> >
> >
> > regards,
> >
> > Mick
> >
> >
> > ¹)
> >
> >
> >
> >
> >
> https://lists.apache.org/thread.html/rd06dabeaa10849795d15ee77c1a8c400b034dce005ac2d0b9366567a%40%3Cdev.cassandra.apache.org%3E
> >
> >
> >
> >
> >
>


-- 
http://twitter.com/tjake

Re: Idea: Marking required scope for 4.0 GA vs. optional

[Ekaterina Dimitrova]

Hello everyone,
I think the project might benefit of one "filter applied" now on the
current scope of JIRA tickets.
And then, regular triage so focus is not lost.

Up to now, I always try to look at the Release Lifecycle [1] criteria when
I open a new ticket in order to mark what is the target version of the
patch. Also, I look from the perspective of the general code freeze.
Hope I am doing it right. If those are strictly marked, do we need
additional label for blocking tickets? Or we want to differentiate between
strong blockers and such patches which would be good if they land at a
specific release but not mandatory?

Why I am in for initial filter now? As I found many tickets opened in
2016...17...etc Things have changed a lot since then and there can be
easily different PoV nowadays.

I would be more than happy to help in any of these activities, too.

Ekaterina

[1] https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle


> Begin forwarded message:
>
> *From:* David Capwell 
> *Date:* 31 March 2020, 15:15:11 GMT-4
> *To:* dev@cassandra.apache.org
> *Subject:* *Re:  Idea: Marking required scope for 4.0 GA vs. optional*
> *Reply-To:* dev@cassandra.apache.org
>
> What I'm used to is having two buckets for a release: tickets in the
> release (if not complete they are blockers), and triage. How this is done
> isn't important but I do feel it's important to have both.
>
> Right now I don't see a active triage, but to Josh's point we would need to
> know who should first. Without answering, let me ask a question; should I
> (non committer) be adding blockers? If I add a blocker who should verify it
> should block? What if project members elect non-commiters to do this?
>
>
>
> On Mon, Mar 30, 2020, 8:45 AM Joshua McKenzie 
> wrote:
>
> Regardless of how we indicate optional vs. required for rel, are there
>
> strong opinions on who should set that metadata on tickets? Reporter?
>
> Assignee? One person? A group of people?
>
>
>
> On Sun, Mar 29, 2020 at 10:04 AM Joshua McKenzie 
>
> wrote:
>
>
> FWIW, I don't care what we go with as long as we can differentiate
>
> tickets
>
> that are optional for the rel vs. tickets that are blockers and filter
>
> the
>
> JIRA board on them so people know where they should focus their effort.
>
>
> The rest of it's just paint colors to me.
>
>
> On Sun, Mar 29, 2020 at 9:24 AM Mick Semb Wever  wrote:
>
>
>
> Alternatively, we could revert to using 4.0.X or 4.X as we once did to
>
> indicate something is targeting a release vs. blocking on inclusion
>
> for
>
> it.
>
> That seems to be a more "project JIRA hackish idiom", and one that's
>
> historically been confusing for people. At least with a label it would
>
> be
>
> clear with a name like "4.0-blocker", and we could then easily filter
>
> the
>
> JIRA board on it.
>
>
>
>
> Now that I better understand Benedict's previous response (see the
>
> 'Dev Cassandra 4.0 Dev Work Status' thread¹), I'm leaning with his view
>
> for
>
> now.
>
>
> I certainly missed the difference on how tickets ended up resolved as
>
> `4.0-alpha`. That unresolved `4.x` tickets were non-blockers that could
>
> still go into `4.0-alpha` if they satisfied the feature freeze and met
>
> someone's itch, while the unresolved `4.0-alpha` tickets were those the
>
> community declared as blockers.
>
>
> Putting aside the discussion on what is a blocker, when it is discovered
>
> to
>
> be a blocker (and vice versa). My confusion stemmed from the fact there
>
> was
>
> no specific alpha|beta versions for tickets to get resolved into, eg
>
> 4.0-alpha1, 4.0-alpha2, etc. So it wasn't just that the normal  `.x`
>
> nomenclature got changed, but having accurate fix versions was also
>
> removed. If we added the specific alpha|beta versions then I wouldn't
>
> consider the approach to be a "hackish idiom" anymore. The 4.0-alpha,
>
> 4.0-beta and 4.0, versions become purely target versions like the `.x`,
>
> and
>
> no resolved ticket is ever assigned them.
>
>
> The simpler we can make and document this the better. please.
>
>
> regards,
>
> Mick
>
>
> ¹)
>
>
>
>
> https://lists.apache.org/thread.html/rd06dabeaa10849795d15ee77c1a8c400b034dce005ac2d0b9366567a%40%3Cdev.cassandra.apache.org%3E
>
>
>
>
>

Re: Idea: Marking required scope for 4.0 GA vs. optional

[Brandon Williams]

On Tue, Mar 31, 2020 at 2:15 PM David Capwell  wrote:
> Right now I don't see a active triage, but to Josh's point we would need to
> know who should first. Without answering, let me ask a question; should I
> (non committer) be adding blockers?

If you like, yes.  I don't think anyone, committer or otherwise,
should limit themselves in any fashion, ticket-wise.  That said, I
think committers should also feel free to change those statuses, and
generally that should not be fought with action, but only with
commentary.  I think those who tussle with ticket action make up only
a small percentage of the community, however.

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Idea: Marking required scope for 4.0 GA vs. optional

[David Capwell]

What I'm used to is having two buckets for a release: tickets in the
release (if not complete they are blockers), and triage. How this is done
isn't important but I do feel it's important to have both.

Right now I don't see a active triage, but to Josh's point we would need to
know who should first. Without answering, let me ask a question; should I
(non committer) be adding blockers? If I add a blocker who should verify it
should block? What if project members elect non-commiters to do this?



On Mon, Mar 30, 2020, 8:45 AM Joshua McKenzie  wrote:

> Regardless of how we indicate optional vs. required for rel, are there
> strong opinions on who should set that metadata on tickets? Reporter?
> Assignee? One person? A group of people?
>
>
> On Sun, Mar 29, 2020 at 10:04 AM Joshua McKenzie 
> wrote:
>
> > FWIW, I don't care what we go with as long as we can differentiate
> tickets
> > that are optional for the rel vs. tickets that are blockers and filter
> the
> > JIRA board on them so people know where they should focus their effort.
> >
> > The rest of it's just paint colors to me.
> >
> > On Sun, Mar 29, 2020 at 9:24 AM Mick Semb Wever  wrote:
> >
> >> >
> >> > Alternatively, we could revert to using 4.0.X or 4.X as we once did to
> >> > indicate something is targeting a release vs. blocking on inclusion
> for
> >> it.
> >> > That seems to be a more "project JIRA hackish idiom", and one that's
> >> > historically been confusing for people. At least with a label it would
> >> be
> >> > clear with a name like "4.0-blocker", and we could then easily filter
> >> the
> >> > JIRA board on it.
> >> >
> >>
> >>
> >> Now that I better understand Benedict's previous response (see the
> >> 'Dev Cassandra 4.0 Dev Work Status' thread¹), I'm leaning with his view
> >> for
> >> now.
> >>
> >> I certainly missed the difference on how tickets ended up resolved as
> >> `4.0-alpha`. That unresolved `4.x` tickets were non-blockers that could
> >> still go into `4.0-alpha` if they satisfied the feature freeze and met
> >> someone's itch, while the unresolved `4.0-alpha` tickets were those the
> >> community declared as blockers.
> >>
> >> Putting aside the discussion on what is a blocker, when it is discovered
> >> to
> >> be a blocker (and vice versa). My confusion stemmed from the fact there
> >> was
> >> no specific alpha|beta versions for tickets to get resolved into, eg
> >> 4.0-alpha1, 4.0-alpha2, etc. So it wasn't just that the normal  `.x`
> >> nomenclature got changed, but having accurate fix versions was also
> >> removed. If we added the specific alpha|beta versions then I wouldn't
> >> consider the approach to be a "hackish idiom" anymore. The 4.0-alpha,
> >> 4.0-beta and 4.0, versions become purely target versions like the `.x`,
> >> and
> >> no resolved ticket is ever assigned them.
> >>
> >> The simpler we can make and document this the better. please.
> >>
> >> regards,
> >> Mick
> >>
> >> ¹)
> >>
> >>
> https://lists.apache.org/thread.html/rd06dabeaa10849795d15ee77c1a8c400b034dce005ac2d0b9366567a%40%3Cdev.cassandra.apache.org%3E
> >>
> >
>

Re: Kubernetes operator unification

[Jake Luciani]

Hi Ben!

Totally agree.  We should collaborate on a unified operator and I think as
deployment on k8s becomes more and more prevalent we need to have
distributed testing in k8s.

To that end we are working on OSS releasing our distributed testing service
we've developed over the years to make this easier and reproducible. Need a
few more days before that's ready
but it may give us a leg up.  I know Alex Petrov has been working a lot on
the new jvm dtest harness and may have some ideas.

Jake

On Tue, Mar 31, 2020 at 12:11 AM Ben Bromhead  wrote:

> Hi All
>
> With the announcement of a C* Sidecar and K8s operator from Datastax
> (congrats btw), Jake and Stefan discussed moving to a more
> standardised/unified implementation of an Apache Cassandra operator for
> Kubernetes. Based on discussions with other folks either using our
> operator, building/running their own or just getting started, there appears
> to be some broader enthusiasm to a more unified approach outside of just
> that thread.
>
> The current state of play for folks looking to run Apache Cassandra,
> particularly on Kubernetes, is fairly fragmented. There are multiple
> projects all doing similar things from large companies operating C* at
> scale on kubernetes, individual contributors and commercialising entities.
> Each one of these projects also have similar but diverse implementations
> and capabilities. From an end user perspective, it makes it very hard to
> figure out what path to take and from someone who supports these end users,
> I'd much rather support one implementation than 3 even if it's not the one
> we wrote :)
>
> To that end, I'd like to indicate that we (Instaclustr) are open to working
> towards a project owned standardized K8s operators/sidecar/etc. How that
> looks and how it gets implemented will surely be the subject of debate,
> especially amongst those with existing implementations.
>
> Before engaging in CEP process (
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652201)
> it might be useful to informally discuss an approach to unifying
> implementations.
>
> To that end I'd like to circulate the following ideas to kick off the
> discussion of an approach that might be useful:
>
> We should look to start off with a new implementation in a separate repo
> (as the sidecar project has done), leveraging the experience and
> contributions from existing operator implementations and a framework like
> the operator-framework, with the initial scope of just supporting our
> distributed testing in our CI pipeline.
>
> Targeting our own distributed test cases (e.g. dtests) brings a number of
> benefits:
>
>- Defines a common environment and goals that minimizes each
>organisations unique kubernetes challenges.
>- Follows the spirit of the 4.0 release to be more dba/operator aligned,
>more production ready and easier to get right in a production setting
> OOB
>- Our test environment over time will look more and more like how users
>run Cassandra in production. This will be the biggest win IMHO.
>- The distributed tests will also serve as functional tests for the
>operator itself.
>
> The main drawback I can see with this approach is it will potentially be a
> longer path to getting a useable project based operator out the door. It
> will also involve a ton of reworking dtests, which for some is going to a
> hard blocker. From there we can start to expand and support more and more
> real life use cases. Hopefully this is not a huge leap as our testing
> should be covering most of those cases!
>
> This is largely my personal gut feel on the approach and I'm looking
> forward to folks other suggestions!
>
> Cheers
>
> --
>
> Ben Bromhead
>
> Instaclustr | www.instaclustr.com | @instaclustr
>  | (650) 284 9692
>