Re: [Discuss] Accept GoCQL driver donation

[Alex Lourie]

Same here. I've been working on this project for a bit now, and I'm
planning to continue and contribute.

I also like the idea of this project becoming an officially endorsed golang
Cassandra driver. Makes a lot of sense too.

Alex.


On Sat., 1 Sep. 2018, 01:08 Chris Bannister,  wrote:

> I intend to stay on and continue to contribute.
>
> On Fri, 31 Aug 2018 at 4:37 pm, Jonathan Ellis  wrote:
>
> > On Fri, Aug 31, 2018 at 9:14 AM Nate McCall  wrote:
> >
> > > Hi folks,
> > > So I was recently talking with, Chris Bannister  the gocql [0]
> > > maintainer, and he expressed an interest in donating the driver to the
> > > ASF.
> > >
> >
> > Is he looking to continue to maintain it or is he looking to give it a
> good
> > home when he moves on?
> >
> > We could accept this along the same lines as how we took in the dtest
> > > donation - going through the incubator IP clearance process [1], but
> > > in this case it's much simpler as an individual (Chris) owns the
> > > copyright.
> > >
> >
> > Is that actually the case?  Github says 128 contributors, and I don't see
> > any mention of a CLA in
> > https://github.com/gocql/gocql/blob/master/CONTRIBUTING.md.
> >
> > --
> > Jonathan Ellis
> > co-founder, http://www.datastax.com
> > @spyced
> >
>

Re: [Discuss] Accept GoCQL driver donation

[Dinesh Joshi]

> On Aug 31, 2018, at 4:06 PM, Michael Shuler  wrote:
> 
> Yep! I was going to comment similarly. I would be in full support of a
> committer who focused purely on documentation and website content, for
> instance. It's a matter of trusting a contributor to know what and where
> to commit, as well as where/what/when not to.
> 

+1

Dinesh

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [Discuss] Accept GoCQL driver donation

[Michael Shuler]

On 08/31/2018 05:20 PM, Brandon Williams wrote:
> On Fri, Aug 31, 2018 at 4:24 PM Gary Dusbabek  wrote:
> 
>> At what point is the project comfortable/trusting with granting commit bits
>> to someone who is very familiar with a facet of the project (say, a go
>> client, or dtests), but hasn't made substantial contributions to the core
>> project? I think I'd be fine with it, but I'm curious what others think.
>>
> 
> I believe we already did this for our release maintainer.

Yep! I was going to comment similarly. I would be in full support of a
committer who focused purely on documentation and website content, for
instance. It's a matter of trusting a contributor to know what and where
to commit, as well as where/what/when not to.

-- 
Kind regards,
Michael

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [Discuss] Accept GoCQL driver donation

[Brandon Williams]

On Fri, Aug 31, 2018 at 4:24 PM Gary Dusbabek  wrote:

> It would also make sense to think about how existing maintainers are
> brought in as committers, if appropriate (official vote, etc.). I'm under
> the impression that Apache commit bits are granted in a binary fashion -
> you have access to everything or nothing. <-- Is this incorrect?
>

That is my understanding.


> At what point is the project comfortable/trusting with granting commit bits
> to someone who is very familiar with a facet of the project (say, a go
> client, or dtests), but hasn't made substantial contributions to the core
> project? I think I'd be fine with it, but I'm curious what others think.
>

I believe we already did this for our release maintainer.

Re: [Discuss] Accept GoCQL driver donation

[Gary Dusbabek]

It would also make sense to think about how existing maintainers are
brought in as committers, if appropriate (official vote, etc.). I'm under
the impression that Apache commit bits are granted in a binary fashion -
you have access to everything or nothing. <-- Is this incorrect?

At what point is the project comfortable/trusting with granting commit bits
to someone who is very familiar with a facet of the project (say, a go
client, or dtests), but hasn't made substantial contributions to the core
project? I think I'd be fine with it, but I'm curious what others think.

Gary.


On Fri, Aug 31, 2018 at 11:13 AM Jason Brown  wrote:

> I find this idea interesting and worth a strong discussion.
>
> Something to consider is an argument floating around in the admin tool/side
> car discussion: if we take an existing project wholesale, we inherit all of
> it's design decision and technical debt (every project has these). On the
> other hand, we also get working code that is running in production. I
> haven't looked at the gocql code (and probably won't until we're a bit
> beyond the 4.0 code freeze), so I can't speak to anything beyond a cursory
> reading of the docs.
>
> As we consider bringing in drivers under the project umbrella, we need to
> evaluate each driver implementation's merit on a case-by-case basis. I'm
> not sure how we divvy up that work, or whom to entrust with that work, but
> it's something to keep in mind.
>
> Thanks,
>
> -Jason
>
>
> On Fri, Aug 31, 2018 at 8:38 AM, Chris Bannister 
> wrote:
>
> > I intend to stay on and continue to contribute.
> >
> > On Fri, 31 Aug 2018 at 4:37 pm, Jonathan Ellis 
> wrote:
> >
> > > On Fri, Aug 31, 2018 at 9:14 AM Nate McCall 
> wrote:
> > >
> > > > Hi folks,
> > > > So I was recently talking with, Chris Bannister  the gocql [0]
> > > > maintainer, and he expressed an interest in donating the driver to
> the
> > > > ASF.
> > > >
> > >
> > > Is he looking to continue to maintain it or is he looking to give it a
> > good
> > > home when he moves on?
> > >
> > > We could accept this along the same lines as how we took in the dtest
> > > > donation - going through the incubator IP clearance process [1], but
> > > > in this case it's much simpler as an individual (Chris) owns the
> > > > copyright.
> > > >
> > >
> > > Is that actually the case?  Github says 128 contributors, and I don't
> see
> > > any mention of a CLA in
> > > https://github.com/gocql/gocql/blob/master/CONTRIBUTING.md.
> > >
> > > --
> > > Jonathan Ellis
> > > co-founder, http://www.datastax.com
> > > @spyced
> > >
> >
>

Re: Transient Replication 4.0 status update

[Carl Mueller]

I see, so there are no dedicated transient nodes, just other nodes that
function as witnesses.

This is still very exciting.


On Fri, Aug 31, 2018 at 1:49 PM Ariel Weisberg  wrote:

> Hi,
>
> All nodes being the same (in terms of functionality) is something we
> wanted to stick with at least for now. I think we want a design that
> changes the operational, availability, and consistency story as little as
> possible when it's completed.
>
> Ariel
> On Fri, Aug 31, 2018, at 2:27 PM, Carl Mueller wrote:
> > SOrry to spam this with two messages...
> >
> > This ticket is also interesting because it is very close to what I
> imagined
> > a useful use case of RF4 / RF6: being basically RF3 + hot spare where you
> > marked (in the case of RF4) three nodes as primary and the fourth as hot
> > standby, which may be equivalent if I understand the paper/protocol to
> > RF3+1 transient.
> >
> > On Fri, Aug 31, 2018 at 1:07 PM Carl Mueller <
> carl.muel...@smartthings.com>
> > wrote:
> >
> > > I put these questions on the ticket too... Sorry if some of them are
> > > stupid.
> > >
> > > So are (basically) these transient nodes basically serving as
> centralized
> > > hinted handoff caches rather than having the hinted handoffs
> cluttering up
> > > full replicas, especially nodes that have no concern for the token
> range
> > > involved? I understand that hinted handoffs aren't being replaced by
> this,
> > > but is that kind of the idea?
> > >
> > > Are the transient nodes "sitting around"?
> > >
> > > Will the transient nodes have cheaper/lower hardware requirements?
> > >
> > > During cluster expansion, does the newly streaming node acquiring data
> > > function as a temporary transient node until it becomes a full replica?
> > > Likewise while shrinking, does a previously full replica function as a
> > > transient while it streams off data?
> > >
> > > Can this help vnode expansion with multiple concurrent nodes?
> Admittedly
> > > I'm not familiar with how much work has gone into fixing cluster
> expansion
> > > with vnodes, it is my understanding that you typically expand only one
> node
> > > at a time or in multiples of the datacenter size
> > >
> > > On Mon, Aug 27, 2018 at 12:29 PM Ariel Weisberg 
> wrote:
> > >
> > >> Hi all,
> > >>
> > >> I wanted to give everyone an update on how development of Transient
> > >> Replication is going and where we are going to be as of 9/1. Blake
> > >> Eggleston, Alex Petrov, Benedict Elliott Smith, and myself have been
> > >> working to get TR implemented for 4.0. Up to now we have avoided
> merging
> > >> anything related to TR to trunk because we weren't 100% sure we were
> going
> > >> to make the 9/1 deadline and even minimal TR functionality requires
> > >> significant changes (see 14405).
> > >>
> > >> We focused on getting a minimal set of deployable functionality
> working,
> > >> and want to avoid overselling what's going to work in the first
> version.
> > >> The feature is marked explicitly as experimental and has to be
> enabled via
> > >> a feature flag in cassandra.yaml. The expected audience for TR in 4.0
> is
> > >> more experienced users who are ready to tackle deploying experimental
> > >> functionality. As it is deployed by experienced users and we gain more
> > >> confidence in it and remove caveats the # of users it will be
> appropriate
> > >> for will expand.
> > >>
> > >> For 4.0 it looks like we will be able to merge TR with support for
> normal
> > >> reads and writes without monotonic reads. Monotonic reads require
> blocking
> > >> read repair and blocking read repair with TR requires further changes
> that
> > >> aren't feasible by 9/1.
> > >>
> > >> Future TR support would look something like
> > >>
> > >> 4.0.next:
> > >> * vnodes (https://issues.apache.org/jira/browse/CASSANDRA-14404)
> > >>
> > >> 4.next:
> > >> * Monotonic reads (
> > >> https://issues.apache.org/jira/browse/CASSANDRA-14665)
> > >> * LWT (https://issues.apache.org/jira/browse/CASSANDRA-14547)
> > >> * Batch log (
> https://issues.apache.org/jira/browse/CASSANDRA-14549)
> > >> * Counters (https://issues.apache.org/jira/browse/CASSANDRA-14548
> )
> > >>
> > >> Possibly never:
> > >> * Materialized views
> > >>
> > >> Probably never:
> > >> * Secondary indexes
> > >>
> > >> The most difficult changes to support Transient Replication should be
> > >> behind us. LWT, Batch log, and counters shouldn't be that hard to make
> > >> transient replication aware. Monotonic reads require some changes to
> the
> > >> read path, but are at least conceptually not that hard to support. I
> am
> > >> confident that by 4.next TR will have fewer tradeoffs.
> > >>
> > >> If you want to take a peek the current feature branch is
> > >> https://github.com/aweisberg/cassandra/tree/14409-7 although we will
> be
> > >> moving to 14409-8 to rebase on to trunk.
> > >>
> > >> Regards,
> > >> Ariel
> > >>
> > >> 

Re: [Discuss] Accept GoCQL driver donation

[dinesh.jo...@yahoo.com.INVALID]

I like the idea of having an officially supported Go Driver under ASF. It would 
mean easier contributions.
I don't think we should necessarily limit it to a reference implementation. The 
industry has a strong interest in building server side as well as client 
software in Go.
Dinesh
On Friday, August 31, 2018, 7:14:03 AM PDT, Nate McCall 
 wrote:  
 
 Hi folks,
So I was recently talking with, Chris Bannister  the gocql [0]
maintainer, and he expressed an interest in donating the driver to the
ASF.

We could accept this along the same lines as how we took in the dtest
donation - going through the incubator IP clearance process [1], but
in this case it's much simpler as an individual (Chris) owns the
copyright.

I think the end goal here is to have a reference protocol
implementation controlled by the project at the least, potentially
replace cqlsh with a GoLang statically compiled binary eventually (?).

What are other folks' thoughts about this? (we are discussing, not voting).

[0] https://github.com/gocql/gocql
[1] https://incubator.apache.org/guides/ip_clearance.html

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

  

Re: Transient Replication 4.0 status update

[Ariel Weisberg]

Hi,

All nodes being the same (in terms of functionality) is something we wanted to 
stick with at least for now. I think we want a design that changes the 
operational, availability, and consistency story as little as possible when 
it's completed.

Ariel
On Fri, Aug 31, 2018, at 2:27 PM, Carl Mueller wrote:
> SOrry to spam this with two messages...
> 
> This ticket is also interesting because it is very close to what I imagined
> a useful use case of RF4 / RF6: being basically RF3 + hot spare where you
> marked (in the case of RF4) three nodes as primary and the fourth as hot
> standby, which may be equivalent if I understand the paper/protocol to
> RF3+1 transient.
> 
> On Fri, Aug 31, 2018 at 1:07 PM Carl Mueller 
> wrote:
> 
> > I put these questions on the ticket too... Sorry if some of them are
> > stupid.
> >
> > So are (basically) these transient nodes basically serving as centralized
> > hinted handoff caches rather than having the hinted handoffs cluttering up
> > full replicas, especially nodes that have no concern for the token range
> > involved? I understand that hinted handoffs aren't being replaced by this,
> > but is that kind of the idea?
> >
> > Are the transient nodes "sitting around"?
> >
> > Will the transient nodes have cheaper/lower hardware requirements?
> >
> > During cluster expansion, does the newly streaming node acquiring data
> > function as a temporary transient node until it becomes a full replica?
> > Likewise while shrinking, does a previously full replica function as a
> > transient while it streams off data?
> >
> > Can this help vnode expansion with multiple concurrent nodes? Admittedly
> > I'm not familiar with how much work has gone into fixing cluster expansion
> > with vnodes, it is my understanding that you typically expand only one node
> > at a time or in multiples of the datacenter size
> >
> > On Mon, Aug 27, 2018 at 12:29 PM Ariel Weisberg  wrote:
> >
> >> Hi all,
> >>
> >> I wanted to give everyone an update on how development of Transient
> >> Replication is going and where we are going to be as of 9/1. Blake
> >> Eggleston, Alex Petrov, Benedict Elliott Smith, and myself have been
> >> working to get TR implemented for 4.0. Up to now we have avoided merging
> >> anything related to TR to trunk because we weren't 100% sure we were going
> >> to make the 9/1 deadline and even minimal TR functionality requires
> >> significant changes (see 14405).
> >>
> >> We focused on getting a minimal set of deployable functionality working,
> >> and want to avoid overselling what's going to work in the first version.
> >> The feature is marked explicitly as experimental and has to be enabled via
> >> a feature flag in cassandra.yaml. The expected audience for TR in 4.0 is
> >> more experienced users who are ready to tackle deploying experimental
> >> functionality. As it is deployed by experienced users and we gain more
> >> confidence in it and remove caveats the # of users it will be appropriate
> >> for will expand.
> >>
> >> For 4.0 it looks like we will be able to merge TR with support for normal
> >> reads and writes without monotonic reads. Monotonic reads require blocking
> >> read repair and blocking read repair with TR requires further changes that
> >> aren't feasible by 9/1.
> >>
> >> Future TR support would look something like
> >>
> >> 4.0.next:
> >> * vnodes (https://issues.apache.org/jira/browse/CASSANDRA-14404)
> >>
> >> 4.next:
> >> * Monotonic reads (
> >> https://issues.apache.org/jira/browse/CASSANDRA-14665)
> >> * LWT (https://issues.apache.org/jira/browse/CASSANDRA-14547)
> >> * Batch log (https://issues.apache.org/jira/browse/CASSANDRA-14549)
> >> * Counters (https://issues.apache.org/jira/browse/CASSANDRA-14548)
> >>
> >> Possibly never:
> >> * Materialized views
> >>
> >> Probably never:
> >> * Secondary indexes
> >>
> >> The most difficult changes to support Transient Replication should be
> >> behind us. LWT, Batch log, and counters shouldn't be that hard to make
> >> transient replication aware. Monotonic reads require some changes to the
> >> read path, but are at least conceptually not that hard to support. I am
> >> confident that by 4.next TR will have fewer tradeoffs.
> >>
> >> If you want to take a peek the current feature branch is
> >> https://github.com/aweisberg/cassandra/tree/14409-7 although we will be
> >> moving to 14409-8 to rebase on to trunk.
> >>
> >> Regards,
> >> Ariel
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
> >>

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Transient Replication 4.0 status update

[Ariel Weisberg]

Hi,

There are no transient nodes. All nodes are the same. If you have transient 
replication enabled each node will transiently replicate some ranges instead of 
fully replicating them.

Capacity requirements are reduced evenly across all nodes in the cluster.

Nodes are not temporarily transient replicas during expansion. They need to 
stream data like a full replica for the transient range before they can serve 
reads. There is a pending state similar to how there is a pending state for 
full replicas. Transient replicas also always receive writes when they are 
pending. There may be some room to relax how that is handled, but for now we 
opt to send pending transient ranges a bit more data and avoid reading from 
them when maybe we could.

This doesn't change how expansion works with vnodes. The same restrictions 
still apply. We won't officially support vnodes until we have done more testing 
and really thought through the corner cases. It's quite possible we will relax 
the restriction on creating transient keyspaces with vnodes in 4.0.x.

Ariel

On Fri, Aug 31, 2018, at 2:07 PM, Carl Mueller wrote:
> I put these questions on the ticket too... Sorry if some of them are
> stupid.
> 
> So are (basically) these transient nodes basically serving as centralized
> hinted handoff caches rather than having the hinted handoffs cluttering up
> full replicas, especially nodes that have no concern for the token range
> involved? I understand that hinted handoffs aren't being replaced by this,
> but is that kind of the idea?
> 
> Are the transient nodes "sitting around"?
> 
> Will the transient nodes have cheaper/lower hardware requirements?
> 
> During cluster expansion, does the newly streaming node acquiring data
> function as a temporary transient node until it becomes a full replica?
> Likewise while shrinking, does a previously full replica function as a
> transient while it streams off data?
> 
> Can this help vnode expansion with multiple concurrent nodes? Admittedly
> I'm not familiar with how much work has gone into fixing cluster expansion
> with vnodes, it is my understanding that you typically expand only one node
> at a time or in multiples of the datacenter size
> 
> On Mon, Aug 27, 2018 at 12:29 PM Ariel Weisberg  wrote:
> 
> > Hi all,
> >
> > I wanted to give everyone an update on how development of Transient
> > Replication is going and where we are going to be as of 9/1. Blake
> > Eggleston, Alex Petrov, Benedict Elliott Smith, and myself have been
> > working to get TR implemented for 4.0. Up to now we have avoided merging
> > anything related to TR to trunk because we weren't 100% sure we were going
> > to make the 9/1 deadline and even minimal TR functionality requires
> > significant changes (see 14405).
> >
> > We focused on getting a minimal set of deployable functionality working,
> > and want to avoid overselling what's going to work in the first version.
> > The feature is marked explicitly as experimental and has to be enabled via
> > a feature flag in cassandra.yaml. The expected audience for TR in 4.0 is
> > more experienced users who are ready to tackle deploying experimental
> > functionality. As it is deployed by experienced users and we gain more
> > confidence in it and remove caveats the # of users it will be appropriate
> > for will expand.
> >
> > For 4.0 it looks like we will be able to merge TR with support for normal
> > reads and writes without monotonic reads. Monotonic reads require blocking
> > read repair and blocking read repair with TR requires further changes that
> > aren't feasible by 9/1.
> >
> > Future TR support would look something like
> >
> > 4.0.next:
> > * vnodes (https://issues.apache.org/jira/browse/CASSANDRA-14404)
> >
> > 4.next:
> > * Monotonic reads (
> > https://issues.apache.org/jira/browse/CASSANDRA-14665)
> > * LWT (https://issues.apache.org/jira/browse/CASSANDRA-14547)
> > * Batch log (https://issues.apache.org/jira/browse/CASSANDRA-14549)
> > * Counters (https://issues.apache.org/jira/browse/CASSANDRA-14548)
> >
> > Possibly never:
> > * Materialized views
> >
> > Probably never:
> > * Secondary indexes
> >
> > The most difficult changes to support Transient Replication should be
> > behind us. LWT, Batch log, and counters shouldn't be that hard to make
> > transient replication aware. Monotonic reads require some changes to the
> > read path, but are at least conceptually not that hard to support. I am
> > confident that by 4.next TR will have fewer tradeoffs.
> >
> > If you want to take a peek the current feature branch is
> > https://github.com/aweisberg/cassandra/tree/14409-7 although we will be
> > moving to 14409-8 to rebase on to trunk.
> >
> > Regards,
> > Ariel
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >

Re: Transient Replication 4.0 status update

[Carl Mueller]

SOrry to spam this with two messages...

This ticket is also interesting because it is very close to what I imagined
a useful use case of RF4 / RF6: being basically RF3 + hot spare where you
marked (in the case of RF4) three nodes as primary and the fourth as hot
standby, which may be equivalent if I understand the paper/protocol to
RF3+1 transient.

On Fri, Aug 31, 2018 at 1:07 PM Carl Mueller 
wrote:

> I put these questions on the ticket too... Sorry if some of them are
> stupid.
>
> So are (basically) these transient nodes basically serving as centralized
> hinted handoff caches rather than having the hinted handoffs cluttering up
> full replicas, especially nodes that have no concern for the token range
> involved? I understand that hinted handoffs aren't being replaced by this,
> but is that kind of the idea?
>
> Are the transient nodes "sitting around"?
>
> Will the transient nodes have cheaper/lower hardware requirements?
>
> During cluster expansion, does the newly streaming node acquiring data
> function as a temporary transient node until it becomes a full replica?
> Likewise while shrinking, does a previously full replica function as a
> transient while it streams off data?
>
> Can this help vnode expansion with multiple concurrent nodes? Admittedly
> I'm not familiar with how much work has gone into fixing cluster expansion
> with vnodes, it is my understanding that you typically expand only one node
> at a time or in multiples of the datacenter size
>
> On Mon, Aug 27, 2018 at 12:29 PM Ariel Weisberg  wrote:
>
>> Hi all,
>>
>> I wanted to give everyone an update on how development of Transient
>> Replication is going and where we are going to be as of 9/1. Blake
>> Eggleston, Alex Petrov, Benedict Elliott Smith, and myself have been
>> working to get TR implemented for 4.0. Up to now we have avoided merging
>> anything related to TR to trunk because we weren't 100% sure we were going
>> to make the 9/1 deadline and even minimal TR functionality requires
>> significant changes (see 14405).
>>
>> We focused on getting a minimal set of deployable functionality working,
>> and want to avoid overselling what's going to work in the first version.
>> The feature is marked explicitly as experimental and has to be enabled via
>> a feature flag in cassandra.yaml. The expected audience for TR in 4.0 is
>> more experienced users who are ready to tackle deploying experimental
>> functionality. As it is deployed by experienced users and we gain more
>> confidence in it and remove caveats the # of users it will be appropriate
>> for will expand.
>>
>> For 4.0 it looks like we will be able to merge TR with support for normal
>> reads and writes without monotonic reads. Monotonic reads require blocking
>> read repair and blocking read repair with TR requires further changes that
>> aren't feasible by 9/1.
>>
>> Future TR support would look something like
>>
>> 4.0.next:
>> * vnodes (https://issues.apache.org/jira/browse/CASSANDRA-14404)
>>
>> 4.next:
>> * Monotonic reads (
>> https://issues.apache.org/jira/browse/CASSANDRA-14665)
>> * LWT (https://issues.apache.org/jira/browse/CASSANDRA-14547)
>> * Batch log (https://issues.apache.org/jira/browse/CASSANDRA-14549)
>> * Counters (https://issues.apache.org/jira/browse/CASSANDRA-14548)
>>
>> Possibly never:
>> * Materialized views
>>
>> Probably never:
>> * Secondary indexes
>>
>> The most difficult changes to support Transient Replication should be
>> behind us. LWT, Batch log, and counters shouldn't be that hard to make
>> transient replication aware. Monotonic reads require some changes to the
>> read path, but are at least conceptually not that hard to support. I am
>> confident that by 4.next TR will have fewer tradeoffs.
>>
>> If you want to take a peek the current feature branch is
>> https://github.com/aweisberg/cassandra/tree/14409-7 although we will be
>> moving to 14409-8 to rebase on to trunk.
>>
>> Regards,
>> Ariel
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
>>

Re: Transient Replication 4.0 status update

[Carl Mueller]

I put these questions on the ticket too... Sorry if some of them are
stupid.

So are (basically) these transient nodes basically serving as centralized
hinted handoff caches rather than having the hinted handoffs cluttering up
full replicas, especially nodes that have no concern for the token range
involved? I understand that hinted handoffs aren't being replaced by this,
but is that kind of the idea?

Are the transient nodes "sitting around"?

Will the transient nodes have cheaper/lower hardware requirements?

During cluster expansion, does the newly streaming node acquiring data
function as a temporary transient node until it becomes a full replica?
Likewise while shrinking, does a previously full replica function as a
transient while it streams off data?

Can this help vnode expansion with multiple concurrent nodes? Admittedly
I'm not familiar with how much work has gone into fixing cluster expansion
with vnodes, it is my understanding that you typically expand only one node
at a time or in multiples of the datacenter size

On Mon, Aug 27, 2018 at 12:29 PM Ariel Weisberg  wrote:

> Hi all,
>
> I wanted to give everyone an update on how development of Transient
> Replication is going and where we are going to be as of 9/1. Blake
> Eggleston, Alex Petrov, Benedict Elliott Smith, and myself have been
> working to get TR implemented for 4.0. Up to now we have avoided merging
> anything related to TR to trunk because we weren't 100% sure we were going
> to make the 9/1 deadline and even minimal TR functionality requires
> significant changes (see 14405).
>
> We focused on getting a minimal set of deployable functionality working,
> and want to avoid overselling what's going to work in the first version.
> The feature is marked explicitly as experimental and has to be enabled via
> a feature flag in cassandra.yaml. The expected audience for TR in 4.0 is
> more experienced users who are ready to tackle deploying experimental
> functionality. As it is deployed by experienced users and we gain more
> confidence in it and remove caveats the # of users it will be appropriate
> for will expand.
>
> For 4.0 it looks like we will be able to merge TR with support for normal
> reads and writes without monotonic reads. Monotonic reads require blocking
> read repair and blocking read repair with TR requires further changes that
> aren't feasible by 9/1.
>
> Future TR support would look something like
>
> 4.0.next:
> * vnodes (https://issues.apache.org/jira/browse/CASSANDRA-14404)
>
> 4.next:
> * Monotonic reads (
> https://issues.apache.org/jira/browse/CASSANDRA-14665)
> * LWT (https://issues.apache.org/jira/browse/CASSANDRA-14547)
> * Batch log (https://issues.apache.org/jira/browse/CASSANDRA-14549)
> * Counters (https://issues.apache.org/jira/browse/CASSANDRA-14548)
>
> Possibly never:
> * Materialized views
>
> Probably never:
> * Secondary indexes
>
> The most difficult changes to support Transient Replication should be
> behind us. LWT, Batch log, and counters shouldn't be that hard to make
> transient replication aware. Monotonic reads require some changes to the
> read path, but are at least conceptually not that hard to support. I am
> confident that by 4.next TR will have fewer tradeoffs.
>
> If you want to take a peek the current feature branch is
> https://github.com/aweisberg/cassandra/tree/14409-7 although we will be
> moving to 14409-8 to rebase on to trunk.
>
> Regards,
> Ariel
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Java 11 Z garbage collector

[Jeff Jirsa]

Read heavy workload with wider partitions (like 1-2gb) and disable the key 
cache will be worst case for GC




-- 
Jeff Jirsa


> On Aug 31, 2018, at 10:51 AM, Carl Mueller 
>  wrote:
> 
> I'm assuming that p99 that Rocksandra tries to target is caused by GC
> pauses, does anyone have data patterns or datasets that will generate GC
> pauses in Cassandra to highlight the abilities of Rocksandra (and...
> Scylla?) and perhaps this GC approach?
> 
> On Thu, Aug 30, 2018 at 8:11 PM Carl Mueller 
> wrote:
> 
>> Oh nice, I'll check that out.
>> 
>> On Thu, Aug 30, 2018 at 11:07 AM Jonathan Haddad 
>> wrote:
>> 
>>> Advertised, yes, but so far I haven't found it to be any better than
>>> ParNew + CMS or G1 in the performance tests I did when writing
>>> http://thelastpickle.com/blog/2018/08/16/java11.html.
>>> 
>>> That said, I didn't try it with a huge heap (i think it was 16 or 24GB),
>>> so
>>> maybe it'll do better if I throw 50 GB RAM at it.
>>> 
>>> 
>>> 
>>> On Thu, Aug 30, 2018 at 8:42 AM Carl Mueller
>>>  wrote:
>>> 
 https://www.opsian.com/blog/javas-new-zgc-is-very-exciting/
 
 .. max of 4ms for stop the world, large terabyte heaps, seems promising.
 
 Will this be a major boon to cassandra p99 times? Anyone know the
>>> aspects
 of cassandra that cause the most churn and lead to StopTheWorld GC? I
>>> was
 under the impression that bloom filters, caches, etc are statically
 allocated at startup.
 
>>> 
>>> 
>>> --
>>> Jon Haddad
>>> http://www.rustyrazorblade.com
>>> twitter: rustyrazorblade
>>> 
>> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Java 11 Z garbage collector

[Carl Mueller]

I'm assuming that p99 that Rocksandra tries to target is caused by GC
pauses, does anyone have data patterns or datasets that will generate GC
pauses in Cassandra to highlight the abilities of Rocksandra (and...
Scylla?) and perhaps this GC approach?

On Thu, Aug 30, 2018 at 8:11 PM Carl Mueller 
wrote:

> Oh nice, I'll check that out.
>
> On Thu, Aug 30, 2018 at 11:07 AM Jonathan Haddad 
> wrote:
>
>> Advertised, yes, but so far I haven't found it to be any better than
>> ParNew + CMS or G1 in the performance tests I did when writing
>> http://thelastpickle.com/blog/2018/08/16/java11.html.
>>
>> That said, I didn't try it with a huge heap (i think it was 16 or 24GB),
>> so
>> maybe it'll do better if I throw 50 GB RAM at it.
>>
>>
>>
>> On Thu, Aug 30, 2018 at 8:42 AM Carl Mueller
>>  wrote:
>>
>> > https://www.opsian.com/blog/javas-new-zgc-is-very-exciting/
>> >
>> > .. max of 4ms for stop the world, large terabyte heaps, seems promising.
>> >
>> > Will this be a major boon to cassandra p99 times? Anyone know the
>> aspects
>> > of cassandra that cause the most churn and lead to StopTheWorld GC? I
>> was
>> > under the impression that bloom filters, caches, etc are statically
>> > allocated at startup.
>> >
>>
>>
>> --
>> Jon Haddad
>> http://www.rustyrazorblade.com
>> twitter: rustyrazorblade
>>
>

Re: [Discuss] Accept GoCQL driver donation

[Jason Brown]

I find this idea interesting and worth a strong discussion.

Something to consider is an argument floating around in the admin tool/side
car discussion: if we take an existing project wholesale, we inherit all of
it's design decision and technical debt (every project has these). On the
other hand, we also get working code that is running in production. I
haven't looked at the gocql code (and probably won't until we're a bit
beyond the 4.0 code freeze), so I can't speak to anything beyond a cursory
reading of the docs.

As we consider bringing in drivers under the project umbrella, we need to
evaluate each driver implementation's merit on a case-by-case basis. I'm
not sure how we divvy up that work, or whom to entrust with that work, but
it's something to keep in mind.

Thanks,

-Jason


On Fri, Aug 31, 2018 at 8:38 AM, Chris Bannister 
wrote:

> I intend to stay on and continue to contribute.
>
> On Fri, 31 Aug 2018 at 4:37 pm, Jonathan Ellis  wrote:
>
> > On Fri, Aug 31, 2018 at 9:14 AM Nate McCall  wrote:
> >
> > > Hi folks,
> > > So I was recently talking with, Chris Bannister  the gocql [0]
> > > maintainer, and he expressed an interest in donating the driver to the
> > > ASF.
> > >
> >
> > Is he looking to continue to maintain it or is he looking to give it a
> good
> > home when he moves on?
> >
> > We could accept this along the same lines as how we took in the dtest
> > > donation - going through the incubator IP clearance process [1], but
> > > in this case it's much simpler as an individual (Chris) owns the
> > > copyright.
> > >
> >
> > Is that actually the case?  Github says 128 contributors, and I don't see
> > any mention of a CLA in
> > https://github.com/gocql/gocql/blob/master/CONTRIBUTING.md.
> >
> > --
> > Jonathan Ellis
> > co-founder, http://www.datastax.com
> > @spyced
> >
>

Re: [Discuss] Accept GoCQL driver donation

[Chris Bannister]

I intend to stay on and continue to contribute.

On Fri, 31 Aug 2018 at 4:37 pm, Jonathan Ellis  wrote:

> On Fri, Aug 31, 2018 at 9:14 AM Nate McCall  wrote:
>
> > Hi folks,
> > So I was recently talking with, Chris Bannister  the gocql [0]
> > maintainer, and he expressed an interest in donating the driver to the
> > ASF.
> >
>
> Is he looking to continue to maintain it or is he looking to give it a good
> home when he moves on?
>
> We could accept this along the same lines as how we took in the dtest
> > donation - going through the incubator IP clearance process [1], but
> > in this case it's much simpler as an individual (Chris) owns the
> > copyright.
> >
>
> Is that actually the case?  Github says 128 contributors, and I don't see
> any mention of a CLA in
> https://github.com/gocql/gocql/blob/master/CONTRIBUTING.md.
>
> --
> Jonathan Ellis
> co-founder, http://www.datastax.com
> @spyced
>

Re: [Discuss] Accept GoCQL driver donation

[Jonathan Ellis]

On Fri, Aug 31, 2018 at 9:14 AM Nate McCall  wrote:

> Hi folks,
> So I was recently talking with, Chris Bannister  the gocql [0]
> maintainer, and he expressed an interest in donating the driver to the
> ASF.
>

Is he looking to continue to maintain it or is he looking to give it a good
home when he moves on?

We could accept this along the same lines as how we took in the dtest
> donation - going through the incubator IP clearance process [1], but
> in this case it's much simpler as an individual (Chris) owns the
> copyright.
>

Is that actually the case?  Github says 128 contributors, and I don't see
any mention of a CLA in
https://github.com/gocql/gocql/blob/master/CONTRIBUTING.md.

-- 
Jonathan Ellis
co-founder, http://www.datastax.com
@spyced

Re: [Discuss] Accept GoCQL driver donation

[Jonathan Haddad]

Definitely does not belong in the same repo.

I’m all for folding drivers in / writing our own, just needs active
committers.

On Fri, Aug 31, 2018 at 7:45 AM Michael Shuler 
wrote:

> On 08/31/2018 09:34 AM, Jay Zhuang wrote:
> > That's great. Could that be in the same repo as Cassandra or a
> > separate repo?
>
> For similar reasons as discussed for an admin tool, separate
> repositories are quick and simple to create, as well as allow handling
> contribution, CI, build, release, etc. mechanics more flexibly. I see
> little to no advantage to including allthethings in-tree. If there comes
> a time that particular contrib items do make sense to include in the
> main repository, dropping them in would be no problem.
>
> --
> Michael
>
> > On Fri, Aug 31, 2018 at 7:14 AM Nate McCall  wrote:
> >
> >> Hi folks,
> >> So I was recently talking with, Chris Bannister  the gocql [0]
> >> maintainer, and he expressed an interest in donating the driver to the
> >> ASF.
> >>
> >> We could accept this along the same lines as how we took in the dtest
> >> donation - going through the incubator IP clearance process [1], but
> >> in this case it's much simpler as an individual (Chris) owns the
> >> copyright.
> >>
> >> I think the end goal here is to have a reference protocol
> >> implementation controlled by the project at the least, potentially
> >> replace cqlsh with a GoLang statically compiled binary eventually (?).
> >>
> >> What are other folks' thoughts about this? (we are discussing, not
> voting).
> >>
> >> [0] https://github.com/gocql/gocql
> >> [1] https://incubator.apache.org/guides/ip_clearance.html
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
> >>
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
> --
Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade

Re: [Discuss] Accept GoCQL driver donation

[Sumanth Pasupuleti]

It sounds great to have an official GoCQL driver. I like the mentioned end
goal of having a reference implementation that evolves with the core C*
project.

Thanks,
Sumanth



On Fri, Aug 31, 2018 at 7:45 AM Michael Shuler 
wrote:

> On 08/31/2018 09:34 AM, Jay Zhuang wrote:
> > That's great. Could that be in the same repo as Cassandra or a
> > separate repo?
>
> For similar reasons as discussed for an admin tool, separate
> repositories are quick and simple to create, as well as allow handling
> contribution, CI, build, release, etc. mechanics more flexibly. I see
> little to no advantage to including allthethings in-tree. If there comes
> a time that particular contrib items do make sense to include in the
> main repository, dropping them in would be no problem.
>
> --
> Michael
>
> > On Fri, Aug 31, 2018 at 7:14 AM Nate McCall  wrote:
> >
> >> Hi folks,
> >> So I was recently talking with, Chris Bannister  the gocql [0]
> >> maintainer, and he expressed an interest in donating the driver to the
> >> ASF.
> >>
> >> We could accept this along the same lines as how we took in the dtest
> >> donation - going through the incubator IP clearance process [1], but
> >> in this case it's much simpler as an individual (Chris) owns the
> >> copyright.
> >>
> >> I think the end goal here is to have a reference protocol
> >> implementation controlled by the project at the least, potentially
> >> replace cqlsh with a GoLang statically compiled binary eventually (?).
> >>
> >> What are other folks' thoughts about this? (we are discussing, not
> voting).
> >>
> >> [0] https://github.com/gocql/gocql
> >> [1] https://incubator.apache.org/guides/ip_clearance.html
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>
> >>
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [Discuss] Accept GoCQL driver donation

[Michael Shuler]

On 08/31/2018 09:34 AM, Jay Zhuang wrote:
> That's great. Could that be in the same repo as Cassandra or a
> separate repo?

For similar reasons as discussed for an admin tool, separate
repositories are quick and simple to create, as well as allow handling
contribution, CI, build, release, etc. mechanics more flexibly. I see
little to no advantage to including allthethings in-tree. If there comes
a time that particular contrib items do make sense to include in the
main repository, dropping them in would be no problem.

-- 
Michael

> On Fri, Aug 31, 2018 at 7:14 AM Nate McCall  wrote:
> 
>> Hi folks,
>> So I was recently talking with, Chris Bannister  the gocql [0]
>> maintainer, and he expressed an interest in donating the driver to the
>> ASF.
>>
>> We could accept this along the same lines as how we took in the dtest
>> donation - going through the incubator IP clearance process [1], but
>> in this case it's much simpler as an individual (Chris) owns the
>> copyright.
>>
>> I think the end goal here is to have a reference protocol
>> implementation controlled by the project at the least, potentially
>> replace cqlsh with a GoLang statically compiled binary eventually (?).
>>
>> What are other folks' thoughts about this? (we are discussing, not voting).
>>
>> [0] https://github.com/gocql/gocql
>> [1] https://incubator.apache.org/guides/ip_clearance.html
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>>
>>
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: NGCC 2018?

[Jay Zhuang]

Are we going to have a dev event next month? Or anything this year? We may
also be able to provide space in bay area and help to organize it. (Please
let us know, so we could get final approval for that).

On Fri, Jul 27, 2018 at 10:05 AM Jonathan Haddad  wrote:

> My interpretation of Nate's statement was that since there would be a bunch
> of us at Lynn's event, we might as well do NGCC at the same time.
>
> On Thu, Jul 26, 2018 at 9:03 PM Ben Bromhead  wrote:
>
> > It sounds like there may be an appetite for something, but the NGCC in
> its
> > current format is likely to not be that useful?
> >
> > Is a bay area event focused on C* developers something that is
> interesting
> > for the broader dev community? In whatever format that may be?
> >
> > On Tue, Jul 24, 2018 at 5:02 PM Nate McCall  wrote:
> >
> > > This was discussed amongst the PMC recently. We did not come to a
> > > conclusion and there were not terribly strong feelings either way.
> > >
> > > I don't feel like we need to hustle to get "NGCC" in place,
> > > particularly given our decided focus on 4.0. However, that should not
> > > stop us from doing an additional 'c* developer' event in sept. to
> > > coincide with distributed data summit.
> > >
> > > On Wed, Jul 25, 2018 at 5:03 AM, Patrick McFadin 
> > > wrote:
> > > > Ben,
> > > >
> > > > Lynn Bender had offered a space the day before Distributed Data
> Summit
> > in
> > > > September (http://distributeddatasummit.com/) since we are both
> > platinum
> > > > sponsors. I thought he and Nate had talked about that being a good
> > place
> > > > for NGCC since many of us will be in town already.
> > > >
> > > > Nate, now that I've spoken for you, you can clarify, :D
> > > >
> > > > Patrick
> > > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> > > --
> > Ben Bromhead
> > CTO | Instaclustr 
> > +1 650 284 9692
> > Reliability at Scale
> > Cassandra, Spark, Elasticsearch on AWS, Azure, GCP and Softlayer
> >
>
>
> --
> Jon Haddad
> http://www.rustyrazorblade.com
> twitter: rustyrazorblade
>

Re: [Discuss] Accept GoCQL driver donation

[Jay Zhuang]

That's great. Could that be in the same repo as Cassandra or a
separate repo?

On Fri, Aug 31, 2018 at 7:14 AM Nate McCall  wrote:

> Hi folks,
> So I was recently talking with, Chris Bannister  the gocql [0]
> maintainer, and he expressed an interest in donating the driver to the
> ASF.
>
> We could accept this along the same lines as how we took in the dtest
> donation - going through the incubator IP clearance process [1], but
> in this case it's much simpler as an individual (Chris) owns the
> copyright.
>
> I think the end goal here is to have a reference protocol
> implementation controlled by the project at the least, potentially
> replace cqlsh with a GoLang statically compiled binary eventually (?).
>
> What are other folks' thoughts about this? (we are discussing, not voting).
>
> [0] https://github.com/gocql/gocql
> [1] https://incubator.apache.org/guides/ip_clearance.html
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

[Discuss] Accept GoCQL driver donation

[Nate McCall]

Hi folks,
So I was recently talking with, Chris Bannister  the gocql [0]
maintainer, and he expressed an interest in donating the driver to the
ASF.

We could accept this along the same lines as how we took in the dtest
donation - going through the incubator IP clearance process [1], but
in this case it's much simpler as an individual (Chris) owns the
copyright.

I think the end goal here is to have a reference protocol
implementation controlled by the project at the least, potentially
replace cqlsh with a GoLang statically compiled binary eventually (?).

What are other folks' thoughts about this? (we are discussing, not voting).

[0] https://github.com/gocql/gocql
[1] https://incubator.apache.org/guides/ip_clearance.html

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org