Re: [VOTE] Release Apache Cassandra 4.0-alpha3

[Anthony Grasso]

+1 (non-binding)

On Fri, 31 Jan 2020 at 08:48, Joshua McKenzie  wrote:

> +1
>
> On Thu, Jan 30, 2020 at 4:31 PM Brandon Williams  wrote:
>
> > +1
> >
> > On Thu, Jan 30, 2020 at 1:47 PM Mick Semb Wever  wrote:
> > >
> > > Proposing the test build of Cassandra 4.0-alpha3 for release.
> > >
> > > sha1: 5f7c88601c65cdf14ee68387ed68203f2603fc29
> > > Git:
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.0-alpha3-tentative
> > > Maven Artifacts:
> >
> https://repository.apache.org/content/repositories/orgapachecassandra-1189/org/apache/cassandra/apache-cassandra/4.0-alpha3/
> > >
> > > The Source and Build Artifacts, and the Debian and RPM packages are
> > available here:
> > https://dist.apache.org/repos/dist/dev/cassandra/4.0-alpha3/
> > >
> > > The vote will be open for 72 hours (longer if needed). Everyone who has
> > tested the build is invited to vote. Votes by PMC members are considered
> > binding. A vote passes if there are at least three binding +1s.
> > >
> > > ** Please note this is my first time as release manager, and the
> release
> > process has been improved to deal with sha256|512 checksums as well as to
> > use the ASF dev dist staging location. So please be extra critical. **
> > >
> > >
> > > [1]: CHANGES.txt:
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/4.0-alpha3-tentative
> > > [2]: NEWS.txt:
> >
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/4.0-alpha3-tentative
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: [Discuss] num_tokens default in Cassandra 4.0

[Jon Haddad]

Yes, I'm against it. We should be using the default value that benefits the
most people, rather than an arbitrary compromise.

Most clusters don't shrink, they stay the same size or grow. I'd say 90% or
more fall in this category.  Let's do the right thing by default and
include good comments that help people make the right decision if they
think they'll be outside the usual case.

On Thu, Jan 30, 2020, 8:07 PM Joseph Lynch  wrote:

> Any objections to the compromise of 16 as proposed in Chris's original
> patch?
>
> -Joey
>
> On Thu, Jan 30, 2020, 3:47 PM Anthony Grasso 
> wrote:
>
> > I think lowering the number of tokens is a great idea! Similar to Jon,
> when
> > I have reduced the number of tokens for clients it has been improvement
> in
> > repair performance.
> >
> > I am concerned that the proposed default value for num_tokens is too low.
> > If you set up a cluster using the proposed defaults, you will get a
> > balanced cluster. However, if you decommission nodes you will start to
> see
> > large imbalances especially for small clusters (< 20 nodes). This is
> > because the allocate_tokens_for_local_replication_factor setting is only
> > applied during the bootstrap process.
> >
> > I have recommended very low values for num_tokens to clients. This was
> > because it was very unlikely that they would reduce their cluster size
> and
> > I warned them of the caveats with using a small value for num_tokens.
> >
> > The proposed num_token default value is fine for devs and operators that
> > know what they are doing. However, the general Cassandra community will
> be
> > unaware of the potential issue with such a low value. We should consider
> > setting num_tokens to 16 - 32 as the default. This will at least help
> > reduce the severity of the imbalance when decommissioning a node whilst
> > still providing the benefits of having a low number of tokens. In
> addition,
> > we can add a comment to num_tokens that clusters over 100 nodes (per
> > datacenter) should consider reducing it down to 4.
> >
> > Cheers,
> > Anthony
> >
> > On Fri, 31 Jan 2020 at 01:58, Jon Haddad  wrote:
> >
> > > Larger clusters is where high token counts do the most damage. That's
> why
> > > it's such a problem. You start out with a small cluster using 256, as
> you
> > > grow into the hundreds it becomes more and more unstable.
> > >
> > >
> > > On Thu, Jan 30, 2020, 8:19 AM onmstester onmstester
> > >  wrote:
> > >
> > > > Shouldn't we consider the cluster size to configure num_tokens?
> > > >
> > > > For example is it OK to use num_tokens=4 for a cluster of more than
> 100
> > > of
> > > > nodes?
> > > >
> > > >
> > > >
> > > > Another question that is not so much relevant to this :
> > > >
> > > > When we use the token assignment algorithm (the new/non-random one)
> > for a
> > > > specific keyspace, why should we use initial token for all the seeds,
> > > isn't
> > > > one seed enough and then just set the keyspace for all other nodes?
> > > >
> > > >
> > > >
> > > > Also i do not understand why should we consider rack topology and
> > number
> > > > of racks for configuration of num_tokens?
> > > >
> > > >
> > > >
> > > > Sent using https://www.zoho.com/mail/
> > > >
> > > >
> > > >
> > > >
> > > >  On Thu, 30 Jan 2020 04:33:57 +0330 Jeremy Hanna <
> > > > jeremy.hanna1...@gmail.com> wrote 
> > > >
> > > >
> > > > The new default wouldn't be retroactively set for 3.x, but the same
> > > > principles apply.  The new algorithm is in 3.x as well as the
> > > > simplification of the configuration.  So no reason not to use the
> same
> > > > configuration on 3.x.
> > > >
> > > > > On Jan 30, 2020, at 4:34 AM, Chen-Becker, Derek  > > > dchen...@amazon.com.INVALID> wrote:
> > > > >
> > > > > Does the same guidance apply to 3.x clusters? I read through the
> JIRA
> > > > ticket linked below, along with tickets that it links to, but it's
> not
> > > > clear that the new allocation algorithm is available in 3.x or if
> there
> > > are
> > > > other reasons that this would be problematic.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Derek
> > > > >
> > > > > On 1/29/20, 9:54 AM, "Jon Haddad" 
> wrote:
> > > > >
> > > > >Ive put a lot of my previous clients on 4 tokens, all of which
> > have
> > > > >resulted in a major improvement.
> > > > >
> > > > >I wouldn't use any more than 4 except under some pretty unusual
> > > > >circumstances.
> > > > >
> > > > >Jon
> > > > >
> > > > >On Wed, Jan 29, 2020, 11:18 AM Ben Bromhead  > > > b...@instaclustr.com> wrote:
> > > > >
> > > > >> +1 to reducing the number of tokens as low as possible for
> > > availability
> > > > >> issues. 4 lgtm
> > > > >>
> > > > >> On Wed, Jan 29, 2020 at 1:14 AM Dinesh Joshi  > > djo...@apache.org>
> > > > wrote:
> > > > >>
> > > > >>> Thanks for restarting this discussion Jeremy. I personally think
> 4
> > is
> > > > a
> > > > >>> good number as a default. I think whatever we pick, we should
> 

Re: [Discuss] num_tokens default in Cassandra 4.0

[Joseph Lynch]

Any objections to the compromise of 16 as proposed in Chris's original
patch?

-Joey

On Thu, Jan 30, 2020, 3:47 PM Anthony Grasso 
wrote:

> I think lowering the number of tokens is a great idea! Similar to Jon, when
> I have reduced the number of tokens for clients it has been improvement in
> repair performance.
>
> I am concerned that the proposed default value for num_tokens is too low.
> If you set up a cluster using the proposed defaults, you will get a
> balanced cluster. However, if you decommission nodes you will start to see
> large imbalances especially for small clusters (< 20 nodes). This is
> because the allocate_tokens_for_local_replication_factor setting is only
> applied during the bootstrap process.
>
> I have recommended very low values for num_tokens to clients. This was
> because it was very unlikely that they would reduce their cluster size and
> I warned them of the caveats with using a small value for num_tokens.
>
> The proposed num_token default value is fine for devs and operators that
> know what they are doing. However, the general Cassandra community will be
> unaware of the potential issue with such a low value. We should consider
> setting num_tokens to 16 - 32 as the default. This will at least help
> reduce the severity of the imbalance when decommissioning a node whilst
> still providing the benefits of having a low number of tokens. In addition,
> we can add a comment to num_tokens that clusters over 100 nodes (per
> datacenter) should consider reducing it down to 4.
>
> Cheers,
> Anthony
>
> On Fri, 31 Jan 2020 at 01:58, Jon Haddad  wrote:
>
> > Larger clusters is where high token counts do the most damage. That's why
> > it's such a problem. You start out with a small cluster using 256, as you
> > grow into the hundreds it becomes more and more unstable.
> >
> >
> > On Thu, Jan 30, 2020, 8:19 AM onmstester onmstester
> >  wrote:
> >
> > > Shouldn't we consider the cluster size to configure num_tokens?
> > >
> > > For example is it OK to use num_tokens=4 for a cluster of more than 100
> > of
> > > nodes?
> > >
> > >
> > >
> > > Another question that is not so much relevant to this :
> > >
> > > When we use the token assignment algorithm (the new/non-random one)
> for a
> > > specific keyspace, why should we use initial token for all the seeds,
> > isn't
> > > one seed enough and then just set the keyspace for all other nodes?
> > >
> > >
> > >
> > > Also i do not understand why should we consider rack topology and
> number
> > > of racks for configuration of num_tokens?
> > >
> > >
> > >
> > > Sent using https://www.zoho.com/mail/
> > >
> > >
> > >
> > >
> > >  On Thu, 30 Jan 2020 04:33:57 +0330 Jeremy Hanna <
> > > jeremy.hanna1...@gmail.com> wrote 
> > >
> > >
> > > The new default wouldn't be retroactively set for 3.x, but the same
> > > principles apply.  The new algorithm is in 3.x as well as the
> > > simplification of the configuration.  So no reason not to use the same
> > > configuration on 3.x.
> > >
> > > > On Jan 30, 2020, at 4:34 AM, Chen-Becker, Derek  > > dchen...@amazon.com.INVALID> wrote:
> > > >
> > > > Does the same guidance apply to 3.x clusters? I read through the JIRA
> > > ticket linked below, along with tickets that it links to, but it's not
> > > clear that the new allocation algorithm is available in 3.x or if there
> > are
> > > other reasons that this would be problematic.
> > > >
> > > > Thanks,
> > > >
> > > > Derek
> > > >
> > > > On 1/29/20, 9:54 AM, "Jon Haddad"  wrote:
> > > >
> > > >Ive put a lot of my previous clients on 4 tokens, all of which
> have
> > > >resulted in a major improvement.
> > > >
> > > >I wouldn't use any more than 4 except under some pretty unusual
> > > >circumstances.
> > > >
> > > >Jon
> > > >
> > > >On Wed, Jan 29, 2020, 11:18 AM Ben Bromhead  > > b...@instaclustr.com> wrote:
> > > >
> > > >> +1 to reducing the number of tokens as low as possible for
> > availability
> > > >> issues. 4 lgtm
> > > >>
> > > >> On Wed, Jan 29, 2020 at 1:14 AM Dinesh Joshi  > djo...@apache.org>
> > > wrote:
> > > >>
> > > >>> Thanks for restarting this discussion Jeremy. I personally think 4
> is
> > > a
> > > >>> good number as a default. I think whatever we pick, we should have
> > > enough
> > > >>> documentation for operators to make sense of the new defaults in
> 4.0.
> > > >>>
> > > >>> Dinesh
> > > >>>
> > >  On Jan 28, 2020, at 9:25 PM, Jeremy Hanna  > > jeremy.hanna1...@gmail.com>
> > > >>> wrote:
> > > 
> > >  I wanted to start a discussion about the default for num_tokens
> that
> > > >>> we'd like for people starting in Cassandra 4.0.  This is for ticket
> > > >>> CASSANDRA-13701 <
> > https://issues.apache.org/jira/browse/CASSANDRA-13701>
> > >
> > > >>> (which has been duplicated a number of times, most recently by me).
> > > 
> > >  TLDR, based on availability concerns, skew concerns, operational
> > > >>> concerns, and based on the fact 

Re: [Discuss] num_tokens default in Cassandra 4.0

[Anthony Grasso]

I think lowering the number of tokens is a great idea! Similar to Jon, when
I have reduced the number of tokens for clients it has been improvement in
repair performance.

I am concerned that the proposed default value for num_tokens is too low.
If you set up a cluster using the proposed defaults, you will get a
balanced cluster. However, if you decommission nodes you will start to see
large imbalances especially for small clusters (< 20 nodes). This is
because the allocate_tokens_for_local_replication_factor setting is only
applied during the bootstrap process.

I have recommended very low values for num_tokens to clients. This was
because it was very unlikely that they would reduce their cluster size and
I warned them of the caveats with using a small value for num_tokens.

The proposed num_token default value is fine for devs and operators that
know what they are doing. However, the general Cassandra community will be
unaware of the potential issue with such a low value. We should consider
setting num_tokens to 16 - 32 as the default. This will at least help
reduce the severity of the imbalance when decommissioning a node whilst
still providing the benefits of having a low number of tokens. In addition,
we can add a comment to num_tokens that clusters over 100 nodes (per
datacenter) should consider reducing it down to 4.

Cheers,
Anthony

On Fri, 31 Jan 2020 at 01:58, Jon Haddad  wrote:

> Larger clusters is where high token counts do the most damage. That's why
> it's such a problem. You start out with a small cluster using 256, as you
> grow into the hundreds it becomes more and more unstable.
>
>
> On Thu, Jan 30, 2020, 8:19 AM onmstester onmstester
>  wrote:
>
> > Shouldn't we consider the cluster size to configure num_tokens?
> >
> > For example is it OK to use num_tokens=4 for a cluster of more than 100
> of
> > nodes?
> >
> >
> >
> > Another question that is not so much relevant to this :
> >
> > When we use the token assignment algorithm (the new/non-random one) for a
> > specific keyspace, why should we use initial token for all the seeds,
> isn't
> > one seed enough and then just set the keyspace for all other nodes?
> >
> >
> >
> > Also i do not understand why should we consider rack topology and number
> > of racks for configuration of num_tokens?
> >
> >
> >
> > Sent using https://www.zoho.com/mail/
> >
> >
> >
> >
> >  On Thu, 30 Jan 2020 04:33:57 +0330 Jeremy Hanna <
> > jeremy.hanna1...@gmail.com> wrote 
> >
> >
> > The new default wouldn't be retroactively set for 3.x, but the same
> > principles apply.  The new algorithm is in 3.x as well as the
> > simplification of the configuration.  So no reason not to use the same
> > configuration on 3.x.
> >
> > > On Jan 30, 2020, at 4:34 AM, Chen-Becker, Derek  > dchen...@amazon.com.INVALID> wrote:
> > >
> > > Does the same guidance apply to 3.x clusters? I read through the JIRA
> > ticket linked below, along with tickets that it links to, but it's not
> > clear that the new allocation algorithm is available in 3.x or if there
> are
> > other reasons that this would be problematic.
> > >
> > > Thanks,
> > >
> > > Derek
> > >
> > > On 1/29/20, 9:54 AM, "Jon Haddad"  wrote:
> > >
> > >Ive put a lot of my previous clients on 4 tokens, all of which have
> > >resulted in a major improvement.
> > >
> > >I wouldn't use any more than 4 except under some pretty unusual
> > >circumstances.
> > >
> > >Jon
> > >
> > >On Wed, Jan 29, 2020, 11:18 AM Ben Bromhead  > b...@instaclustr.com> wrote:
> > >
> > >> +1 to reducing the number of tokens as low as possible for
> availability
> > >> issues. 4 lgtm
> > >>
> > >> On Wed, Jan 29, 2020 at 1:14 AM Dinesh Joshi  djo...@apache.org>
> > wrote:
> > >>
> > >>> Thanks for restarting this discussion Jeremy. I personally think 4 is
> > a
> > >>> good number as a default. I think whatever we pick, we should have
> > enough
> > >>> documentation for operators to make sense of the new defaults in 4.0.
> > >>>
> > >>> Dinesh
> > >>>
> >  On Jan 28, 2020, at 9:25 PM, Jeremy Hanna  > jeremy.hanna1...@gmail.com>
> > >>> wrote:
> > 
> >  I wanted to start a discussion about the default for num_tokens that
> > >>> we'd like for people starting in Cassandra 4.0.  This is for ticket
> > >>> CASSANDRA-13701 <
> https://issues.apache.org/jira/browse/CASSANDRA-13701>
> >
> > >>> (which has been duplicated a number of times, most recently by me).
> > 
> >  TLDR, based on availability concerns, skew concerns, operational
> > >>> concerns, and based on the fact that the new allocation algorithm can
> > be
> > >>> configured fairly simply now, this is a proposal to go with 4 as the
> > new
> > >>> default and the allocate_tokens_for_local_replication_factor set to
> 3.
> > >>> That gives a good experience out of the box for people and is the
> most
> > >>> conservative.  It does assume that racks and DCs have been configured
> > >>> correctly.  We would, 

Re: [VOTE] Release Apache Cassandra 4.0-alpha3

[Joshua McKenzie]

+1

On Thu, Jan 30, 2020 at 4:31 PM Brandon Williams  wrote:

> +1
>
> On Thu, Jan 30, 2020 at 1:47 PM Mick Semb Wever  wrote:
> >
> > Proposing the test build of Cassandra 4.0-alpha3 for release.
> >
> > sha1: 5f7c88601c65cdf14ee68387ed68203f2603fc29
> > Git:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.0-alpha3-tentative
> > Maven Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1189/org/apache/cassandra/apache-cassandra/4.0-alpha3/
> >
> > The Source and Build Artifacts, and the Debian and RPM packages are
> available here:
> https://dist.apache.org/repos/dist/dev/cassandra/4.0-alpha3/
> >
> > The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s.
> >
> > ** Please note this is my first time as release manager, and the release
> process has been improved to deal with sha256|512 checksums as well as to
> use the ASF dev dist staging location. So please be extra critical. **
> >
> >
> > [1]: CHANGES.txt:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/4.0-alpha3-tentative
> > [2]: NEWS.txt:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/4.0-alpha3-tentative
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Testing out JIRA as replacement for cwiki tracking of 4.0 quality testing

[Joshua McKenzie]

>From my 4.0 status progress email earlier today, we still have quite a few
testing initiatives that are lacking Shepherds or tracking tickets in JIRA:
[Areas needing Shepherds] - 6
...

[Areas needing tracking tickets] - 11
...

I went ahead and tried out the format of creating an epic in JIRA as a
central location to collect this information in one place. The link for a
WIP look at this is here: Link:
https://issues.apache.org/jira/browse/CASSANDRA-15536. I don't want to get
too far into prototyping this as if we don't collectively want to go this
route, I don't want to have 11 JIRAs created plus an epic we'd then delete
and spam the list.

My .02: I think it'd improve our ability to collaborate and lower friction
to testing if we could do so on JIRA instead of the cwiki. *I suspect *the
edit access restrictions there plus general UX friction (difficult to have
collab discussion, comment chains, links to things, etc) make the confluent
wiki a worse tool for this job than JIRA. Plus if we do it in JIRA we can
track the outstanding scope in the single board and it's far easier to
visualize everything in one place so we can all know where attention and
resources need to be directed to best move the needle on things.

But that's just my opinion. What does everyone else think? Like the JIRA
route? Hate it? No opinion?

If we do decide we want to go the epic / JIRA route, I'd be happy to
migrate the rest of the information in there for things that haven't been
completed yet on the wiki (ticket creation, assignee/reviewer chains, links
to epic).

So what does everyone think?

Re: [VOTE] Release Apache Cassandra 4.0-alpha3

[Brandon Williams]

+1

On Thu, Jan 30, 2020 at 1:47 PM Mick Semb Wever  wrote:
>
> Proposing the test build of Cassandra 4.0-alpha3 for release.
>
> sha1: 5f7c88601c65cdf14ee68387ed68203f2603fc29
> Git: 
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.0-alpha3-tentative
> Maven Artifacts: 
> https://repository.apache.org/content/repositories/orgapachecassandra-1189/org/apache/cassandra/apache-cassandra/4.0-alpha3/
>
> The Source and Build Artifacts, and the Debian and RPM packages are available 
> here: https://dist.apache.org/repos/dist/dev/cassandra/4.0-alpha3/
>
> The vote will be open for 72 hours (longer if needed). Everyone who has 
> tested the build is invited to vote. Votes by PMC members are considered 
> binding. A vote passes if there are at least three binding +1s.
>
> ** Please note this is my first time as release manager, and the release 
> process has been improved to deal with sha256|512 checksums as well as to use 
> the ASF dev dist staging location. So please be extra critical. **
>
>
> [1]: CHANGES.txt: 
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/4.0-alpha3-tentative
> [2]: NEWS.txt: 
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/4.0-alpha3-tentative
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [VOTE] Release Apache Cassandra 4.0-alpha3

[Nate McCall]

+1
Thanks for getting this started, Mick!

On Fri, Jan 31, 2020 at 8:47 AM Mick Semb Wever  wrote:

> Proposing the test build of Cassandra 4.0-alpha3 for release.
>
> sha1: 5f7c88601c65cdf14ee68387ed68203f2603fc29
> Git:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.0-alpha3-tentative
> Maven Artifacts:
> https://repository.apache.org/content/repositories/orgapachecassandra-1189/org/apache/cassandra/apache-cassandra/4.0-alpha3/
>
> The Source and Build Artifacts, and the Debian and RPM packages are
> available here:
> https://dist.apache.org/repos/dist/dev/cassandra/4.0-alpha3/
>
> The vote will be open for 72 hours (longer if needed). Everyone who has
> tested the build is invited to vote. Votes by PMC members are considered
> binding. A vote passes if there are at least three binding +1s.
>
> ** Please note this is my first time as release manager, and the release
> process has been improved to deal with sha256|512 checksums as well as to
> use the ASF dev dist staging location. So please be extra critical. **
>
>
> [1]: CHANGES.txt:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/4.0-alpha3-tentative
> [2]: NEWS.txt:
> https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/4.0-alpha3-tentative
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

[VOTE] Release Apache Cassandra 4.0-alpha3

[Mick Semb Wever]

Proposing the test build of Cassandra 4.0-alpha3 for release.

sha1: 5f7c88601c65cdf14ee68387ed68203f2603fc29
Git: 
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=shortlog;h=refs/tags/4.0-alpha3-tentative
Maven Artifacts: 
https://repository.apache.org/content/repositories/orgapachecassandra-1189/org/apache/cassandra/apache-cassandra/4.0-alpha3/

The Source and Build Artifacts, and the Debian and RPM packages are available 
here: https://dist.apache.org/repos/dist/dev/cassandra/4.0-alpha3/

The vote will be open for 72 hours (longer if needed). Everyone who has tested 
the build is invited to vote. Votes by PMC members are considered binding. A 
vote passes if there are at least three binding +1s.

** Please note this is my first time as release manager, and the release 
process has been improved to deal with sha256|512 checksums as well as to use 
the ASF dev dist staging location. So please be extra critical. **


[1]: CHANGES.txt: 
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=CHANGES.txt;hb=refs/tags/4.0-alpha3-tentative
[2]: NEWS.txt: 
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=blob_plain;f=NEWS.txt;hb=refs/tags/4.0-alpha3-tentative

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: 4.0 release status progress report, 2020/1/30

[Joshua McKenzie]

>
> I opened an infra ticket this morning

 Well that was quick (it's done). I'll be setting up that test epic today.

On Thu, Jan 30, 2020 at 11:46 AM Joshua McKenzie 
wrote:

> Got a 4.0 progress update for everyone:
>
> First off, the 4.0 board can be found at
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.
>
> The broad view: We have a total of 95 tickets open for 4.0 releases:
> https://issues.apache.org/jira/issues/?filter=12347896
>
> One report I often find interesting as a measure of velocity and momentum
> is the Cumulative Flow Diagram. This shows, for all ticket statuses on a
> board, the relative growth rate of categories and statuses from ToDo all
> the way to Done. Here's the link for this over the past 6 months:
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=reporting=cumulativeFlowDiagram=939=936=931=1505=1506=1514=1509=1512=1507=180
>
> What I take away from this: we had a pretty significant uptick in momentum
> here beginning in January. Very excited to see that!
>
> *Progress*: We closed 11 more tickets this week (38% increase from last!)
> for a rolling total of 37 (up from 26 in the last update) of 132 (up from
> 122 last week) across 4.0-alpha, 4.0-beta, and 4.0. Closed:
> Filter for all Recent or open scope on 4.0 (root of the kanban board):
> https://issues.apache.org/jira/issues/?filter=12347782
>
> *Newly opened tickets*
> We had 6 new tickets open in the last week. 3 of them are closed out, and
> 3 of them are alpha release flagged flakey tests - these are good
> candidates for anyone wanting to get newly involved in dev or branch out
> into a new area of the code-base. All of them are located in the SASIIndex
> code so that could be interesting.
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=1670
>
> *LHF / Failing Tests*: 6 of the 11 failing tests now have an assignee. The
> remaining 5 unassigned tickets (3 of which are the SASI above) can be
> found at
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=1660=1661=1658
>
> *Needs Reviewer*: 7 tickets need a reviewer. This is *up* from 6 last
> week. They can be found at
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=1659
>
> *Available to work*: 5 alpha (the remaining test failures), 4 beta, and 14
> RC issues are unassigned. We had 1 unassigned waiting review that I fixed
> assignee on and 2 tickets stalled w/out assignee in non-backlog status I've
> poked people about on ticket. These unassigned tickets can be found at
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=detail=CASSANDRA-15308=1661=1658
>
> *Ready to Commit*: 6 tickets are marked ready to commit. I've added a JQL
> quick filter to the board for them here:
>
> https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA-14688=1672=1661
>
> *Testing*:  On our 4.0 Quality and Test Plan Wiki:
> https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans
>
> [Areas needing Shepherds] - 6
> - Distributed Read/Write Path: Coordination, Replication, and Read Repair
> - Metrics
> - Cluster Setup and Maintenance
> - Platforms / Runtimes
> - Cluster Upgrade
> - Transient Replication
>
> [Areas needing tracking tickets] - 11
> - Local Read/Write Path: Other Areas
> - Distributed Read/Write Path: Coordination, Replication, and Read Repair
> - Repair
> - Compaction
> - Metrics
> - Tooling: Bundled / First-Party
> - Tooling: External Ecosystem
> - Test Frameworks, Tooling, Infrastructure / Automation
> - Cluster Setup and Maintenance
> - Platforms / Runtimes
> - Cluster Upgrade
>
> *Alpha test: Translating the cwiki on quality / testing into an epic*
> As a reminder, I'm going to be trying out building an epic out of the
> testing efforts enumerated in the confluence wiki (link:
> https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans).
> I'll send out an email to the dev list to see what we think about that
> format and the relative value of unifying workflow and collab tracking.
> We've been blocked on the inability to create epics in the C* project so
> that ball has been in my court; got a bit behind on that this week. I
> opened an infra ticket this morning asking to open up this functionality
> (reference: https://issues.apache.org/jira/browse/INFRA-19797). The
> reason I want to try this out: there's a pretty fat part of scope that's
> got high uncertainty to me (these broad testing efforts) so the more we can
> do to reduce friction to exploring, documenting, and facilitating
> collaboration on that scope the better.
>
> It's great to see that we have enough momentum to justify a weekly email
> about this stuff! Also been really good to see increased discussion both on
> slack and on the dev list. Thanks everyone, and let me know if there's
> anything you'd like to see change about this reporting and tracking!

4.0 release status progress report, 2020/1/30

[Joshua McKenzie]

Got a 4.0 progress update for everyone:

First off, the 4.0 board can be found at
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355.

The broad view: We have a total of 95 tickets open for 4.0 releases:
https://issues.apache.org/jira/issues/?filter=12347896

One report I often find interesting as a measure of velocity and momentum
is the Cumulative Flow Diagram. This shows, for all ticket statuses on a
board, the relative growth rate of categories and statuses from ToDo all
the way to Done. Here's the link for this over the past 6 months:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=reporting=cumulativeFlowDiagram=939=936=931=1505=1506=1514=1509=1512=1507=180

What I take away from this: we had a pretty significant uptick in momentum
here beginning in January. Very excited to see that!

*Progress*: We closed 11 more tickets this week (38% increase from last!)
for a rolling total of 37 (up from 26 in the last update) of 132 (up from
122 last week) across 4.0-alpha, 4.0-beta, and 4.0. Closed:
Filter for all Recent or open scope on 4.0 (root of the kanban board):
https://issues.apache.org/jira/issues/?filter=12347782

*Newly opened tickets*
We had 6 new tickets open in the last week. 3 of them are closed out, and 3
of them are alpha release flagged flakey tests - these are good candidates
for anyone wanting to get newly involved in dev or branch out into a new
area of the code-base. All of them are located in the SASIIndex code so
that could be interesting.
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=1670

*LHF / Failing Tests*: 6 of the 11 failing tests now have an assignee. The
remaining 5 unassigned tickets (3 of which are the SASI above) can be found
at
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA=1660=1661=1658

*Needs Reviewer*: 7 tickets need a reviewer. This is *up* from 6 last
week. They can be found at
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=1659

*Available to work*: 5 alpha (the remaining test failures), 4 beta, and 14
RC issues are unassigned. We had 1 unassigned waiting review that I fixed
assignee on and 2 tickets stalled w/out assignee in non-backlog status I've
poked people about on ticket. These unassigned tickets can be found at
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=detail=CASSANDRA-15308=1661=1658

*Ready to Commit*: 6 tickets are marked ready to commit. I've added a JQL
quick filter to the board for them here:
https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=355=CASSANDRA-14688=1672=1661

*Testing*:  On our 4.0 Quality and Test Plan Wiki:
https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans

[Areas needing Shepherds] - 6
- Distributed Read/Write Path: Coordination, Replication, and Read Repair
- Metrics
- Cluster Setup and Maintenance
- Platforms / Runtimes
- Cluster Upgrade
- Transient Replication

[Areas needing tracking tickets] - 11
- Local Read/Write Path: Other Areas
- Distributed Read/Write Path: Coordination, Replication, and Read Repair
- Repair
- Compaction
- Metrics
- Tooling: Bundled / First-Party
- Tooling: External Ecosystem
- Test Frameworks, Tooling, Infrastructure / Automation
- Cluster Setup and Maintenance
- Platforms / Runtimes
- Cluster Upgrade

*Alpha test: Translating the cwiki on quality / testing into an epic*
As a reminder, I'm going to be trying out building an epic out of the
testing efforts enumerated in the confluence wiki (link:
https://cwiki.apache.org/confluence/display/CASSANDRA/4.0+Quality%3A+Components+and+Test+Plans).
I'll send out an email to the dev list to see what we think about that
format and the relative value of unifying workflow and collab tracking.
We've been blocked on the inability to create epics in the C* project so
that ball has been in my court; got a bit behind on that this week. I
opened an infra ticket this morning asking to open up this functionality
(reference: https://issues.apache.org/jira/browse/INFRA-19797). The reason
I want to try this out: there's a pretty fat part of scope that's got high
uncertainty to me (these broad testing efforts) so the more we can do to
reduce friction to exploring, documenting, and facilitating collaboration
on that scope the better.

It's great to see that we have enough momentum to justify a weekly email
about this stuff! Also been really good to see increased discussion both on
slack and on the dev list. Thanks everyone, and let me know if there's
anything you'd like to see change about this reporting and tracking!

~Josh

Re: [Discuss] num_tokens default in Cassandra 4.0

[Jon Haddad]

Larger clusters is where high token counts do the most damage. That's why
it's such a problem. You start out with a small cluster using 256, as you
grow into the hundreds it becomes more and more unstable.


On Thu, Jan 30, 2020, 8:19 AM onmstester onmstester
 wrote:

> Shouldn't we consider the cluster size to configure num_tokens?
>
> For example is it OK to use num_tokens=4 for a cluster of more than 100 of
> nodes?
>
>
>
> Another question that is not so much relevant to this :
>
> When we use the token assignment algorithm (the new/non-random one) for a
> specific keyspace, why should we use initial token for all the seeds, isn't
> one seed enough and then just set the keyspace for all other nodes?
>
>
>
> Also i do not understand why should we consider rack topology and number
> of racks for configuration of num_tokens?
>
>
>
> Sent using https://www.zoho.com/mail/
>
>
>
>
>  On Thu, 30 Jan 2020 04:33:57 +0330 Jeremy Hanna <
> jeremy.hanna1...@gmail.com> wrote 
>
>
> The new default wouldn't be retroactively set for 3.x, but the same
> principles apply.  The new algorithm is in 3.x as well as the
> simplification of the configuration.  So no reason not to use the same
> configuration on 3.x.
>
> > On Jan 30, 2020, at 4:34 AM, Chen-Becker, Derek  dchen...@amazon.com.INVALID> wrote:
> >
> > Does the same guidance apply to 3.x clusters? I read through the JIRA
> ticket linked below, along with tickets that it links to, but it's not
> clear that the new allocation algorithm is available in 3.x or if there are
> other reasons that this would be problematic.
> >
> > Thanks,
> >
> > Derek
> >
> > On 1/29/20, 9:54 AM, "Jon Haddad"  wrote:
> >
> >Ive put a lot of my previous clients on 4 tokens, all of which have
> >resulted in a major improvement.
> >
> >I wouldn't use any more than 4 except under some pretty unusual
> >circumstances.
> >
> >Jon
> >
> >On Wed, Jan 29, 2020, 11:18 AM Ben Bromhead  b...@instaclustr.com> wrote:
> >
> >> +1 to reducing the number of tokens as low as possible for availability
> >> issues. 4 lgtm
> >>
> >> On Wed, Jan 29, 2020 at 1:14 AM Dinesh Joshi 
> wrote:
> >>
> >>> Thanks for restarting this discussion Jeremy. I personally think 4 is
> a
> >>> good number as a default. I think whatever we pick, we should have
> enough
> >>> documentation for operators to make sense of the new defaults in 4.0.
> >>>
> >>> Dinesh
> >>>
>  On Jan 28, 2020, at 9:25 PM, Jeremy Hanna  jeremy.hanna1...@gmail.com>
> >>> wrote:
> 
>  I wanted to start a discussion about the default for num_tokens that
> >>> we'd like for people starting in Cassandra 4.0.  This is for ticket
> >>> CASSANDRA-13701 
>
> >>> (which has been duplicated a number of times, most recently by me).
> 
>  TLDR, based on availability concerns, skew concerns, operational
> >>> concerns, and based on the fact that the new allocation algorithm can
> be
> >>> configured fairly simply now, this is a proposal to go with 4 as the
> new
> >>> default and the allocate_tokens_for_local_replication_factor set to 3.
> >>> That gives a good experience out of the box for people and is the most
> >>> conservative.  It does assume that racks and DCs have been configured
> >>> correctly.  We would, of course, go into some detail in the NEWS.txt.
> 
>  Joey Lynch and Josh Snyder did an extensive analysis of availability
> >>> concerns with high num_tokens/virtual nodes in their paper <
> >>>
> >>
> http://mail-archives.apache.org/mod_mbox/cassandra-dev/201804.mbox/%3CCALShVHcz5PixXFO_4bZZZNnKcrpph-=5QmCyb0M=w-mhdyl...@mail.gmail.com%3E
> >>> .
> >>> This worsens as clusters grow larger.  I won't quote the paper here
> but
> >> in
> >>> order to have a conservative default and with the accompanying new
> >>> allocation algorithm, I think it makes sense as a default.
> 
>  The difficulties have always been that virtual nodes have been
> >>> beneficial for operations but that 256 is too high for the purposes of
> >>> repair and as Joey and Josh cover, for availability.  Going lower with
> >> the
> >>> original allocation algorithm has produced skew in allocation in its
> >> naive
> >>> distribution.  Enter CASSANDRA-7032 <
> >>> https://issues.apache.org/jira/browse/CASSANDRA-7032> and the new
> token
> >>> allocation algorithm.  CASSANDRA-15260 <
> >>> https://issues.apache.org/jira/browse/CASSANDRA-15260> makes the new
> >>> algorithm operationally simpler.
> 
>  One other item of note - since Joey and Josh's analysis, there have
> >> been
> >>> improvements in streaming and other considerations that can reduce the
> >>> probability of more than one node representing some token range being
> >>> unavailable, but it would still be good to be conservative.
> 
>  Please chime in with any concerns with having num_tokens=4 and
> >>> 

Re: [Discuss] num_tokens default in Cassandra 4.0

[onmstester onmstester]

Shouldn't we consider the cluster size to configure num_tokens? 

For example is it OK to use num_tokens=4 for a cluster of more than 100 of 
nodes?



Another question that is not so much relevant to this :

When we use the token assignment algorithm (the new/non-random one) for a 
specific keyspace, why should we use initial token for all the seeds, isn't one 
seed enough and then just set the keyspace for all other nodes?



Also i do not understand why should we consider rack topology and number of 
racks for configuration of num_tokens?



Sent using https://www.zoho.com/mail/




 On Thu, 30 Jan 2020 04:33:57 +0330 Jeremy Hanna 
 wrote 


The new default wouldn't be retroactively set for 3.x, but the same principles 
apply.  The new algorithm is in 3.x as well as the simplification of the 
configuration.  So no reason not to use the same configuration on 3.x. 
 
> On Jan 30, 2020, at 4:34 AM, Chen-Becker, Derek 
>  wrote: 
> 
> Does the same guidance apply to 3.x clusters? I read through the JIRA ticket 
> linked below, along with tickets that it links to, but it's not clear that 
> the new allocation algorithm is available in 3.x or if there are other 
> reasons that this would be problematic. 
> 
> Thanks, 
> 
> Derek 
> 
> On 1/29/20, 9:54 AM, "Jon Haddad"  wrote: 
> 
>Ive put a lot of my previous clients on 4 tokens, all of which have 
>resulted in a major improvement. 
> 
>I wouldn't use any more than 4 except under some pretty unusual 
>circumstances. 
> 
>Jon 
> 
>On Wed, Jan 29, 2020, 11:18 AM Ben Bromhead  
> wrote: 
> 
>> +1 to reducing the number of tokens as low as possible for availability 
>> issues. 4 lgtm 
>> 
>> On Wed, Jan 29, 2020 at 1:14 AM Dinesh Joshi  
>> wrote: 
>> 
>>> Thanks for restarting this discussion Jeremy. I personally think 4 is a 
>>> good number as a default. I think whatever we pick, we should have enough 
>>> documentation for operators to make sense of the new defaults in 4.0. 
>>> 
>>> Dinesh 
>>> 
 On Jan 28, 2020, at 9:25 PM, Jeremy Hanna 
  
>>> wrote: 
 
 I wanted to start a discussion about the default for num_tokens that 
>>> we'd like for people starting in Cassandra 4.0.  This is for ticket 
>>> CASSANDRA-13701  
>>> (which has been duplicated a number of times, most recently by me). 
 
 TLDR, based on availability concerns, skew concerns, operational 
>>> concerns, and based on the fact that the new allocation algorithm can be 
>>> configured fairly simply now, this is a proposal to go with 4 as the new 
>>> default and the allocate_tokens_for_local_replication_factor set to 3. 
>>> That gives a good experience out of the box for people and is the most 
>>> conservative.  It does assume that racks and DCs have been configured 
>>> correctly.  We would, of course, go into some detail in the NEWS.txt. 
 
 Joey Lynch and Josh Snyder did an extensive analysis of availability 
>>> concerns with high num_tokens/virtual nodes in their paper < 
>>> 
>> http://mail-archives.apache.org/mod_mbox/cassandra-dev/201804.mbox/%3CCALShVHcz5PixXFO_4bZZZNnKcrpph-=5QmCyb0M=w-mhdyl...@mail.gmail.com%3E
>>  
>>> . 
>>> This worsens as clusters grow larger.  I won't quote the paper here but 
>> in 
>>> order to have a conservative default and with the accompanying new 
>>> allocation algorithm, I think it makes sense as a default. 
 
 The difficulties have always been that virtual nodes have been 
>>> beneficial for operations but that 256 is too high for the purposes of 
>>> repair and as Joey and Josh cover, for availability.  Going lower with 
>> the 
>>> original allocation algorithm has produced skew in allocation in its 
>> naive 
>>> distribution.  Enter CASSANDRA-7032 < 
>>> https://issues.apache.org/jira/browse/CASSANDRA-7032> and the new token 
>>> allocation algorithm.  CASSANDRA-15260 < 
>>> https://issues.apache.org/jira/browse/CASSANDRA-15260> makes the new 
>>> algorithm operationally simpler. 
 
 One other item of note - since Joey and Josh's analysis, there have 
>> been 
>>> improvements in streaming and other considerations that can reduce the 
>>> probability of more than one node representing some token range being 
>>> unavailable, but it would still be good to be conservative. 
 
 Please chime in with any concerns with having num_tokens=4 and 
>>> allocate_tokens_for_local_replication_factor=3 and the accompanying 
>>> rationale so we can improve the experience for all users. 
 
 Other resources: 
 
>>> 
>> https://thelastpickle.com/blog/2019/02/21/set-up-a-cluster-with-even-token-distribution.html
>>  
 
>>> 
>> https://docs.datastax.com/en/dse/6.7/dse-admin/datastax_enterprise/config/configVnodes.html
>>  
 
>>> 
>>