Re: Google Season of Docs 2019 for Apache Cassandra

[Dinesh Joshi]

Thanks, Laxmikant. For those who have confluence accounts have edit permissions 
on the page.

Dinesh

> On Mar 19, 2019, at 3:07 AM, Laxmikant Upadhyay  
> wrote:
> 
> Hi Dinesh,
> 
> I am willing to help as well. Kindly add me to the group as well as i don't
> have permission to edit the list.
> 
> Regards,
> Laxmikant
> 
> 
> 
> On Thu, Mar 14, 2019 at 12:34 PM Dinesh Joshi 
> wrote:
> 
>> Thanks for volunteering Stefan. I have added you.
>> 
>> Dinesh
>> 
>>> On Mar 13, 2019, at 11:55 PM, Stefan Miklosovic <
>> stefan.mikloso...@instaclustr.com> wrote:
>>> 
>>> Hi Dinesh,
>>> 
>>> I was participant of Google summer of Code 2013 under JBoss so I can help
>>> with this and with mentoring as I experienced it from the other side.
>>> 
>>> I wanted to add myself into that list but I can not edit that  "Right
>>> now everyone can view and edit this page if they've got the right space
>>> permissions. "
>>> 
>>> On Thu, 14 Mar 2019 at 17:28, Dinesh Joshi 
>>> wrote:
>>> 
 Thank you everyone for offering to help!
 
 I have put together a confluence page detailing the program and other
 information -
 
>> https://cwiki.apache.org/confluence/display/CASSANDRA/Apache+Cassandra+GSoD+2019+application
 <
 
>> https://cwiki.apache.org/confluence/display/CASSANDRA/Apache+Cassandra+GSoD+2019+application
> 
 
 If you do not have a confluence account, please create one. I think the
 PMCs have the ability to give you write permissions.
 
 Please feel free to add and modify the page. I think the most important
 thing right now is to get together project ideas.
 
 Thanks,
 
 Dinesh
 
> On Mar 13, 2019, at 10:19 AM, Horia Mocioi  wrote:
> 
> I can also help.
> 
> Regards,
> Horia
> 
> On Wed, Mar 13, 2019 at 3:12 PM Aaron Ploetz 
 wrote:
> 
>> I’m willing to help as well.  Feel free to reach out!
>> 
>> Aaron Ploetz
>> 
>>> On Mar 12, 2019, at 8:37 PM, Rahul Singh <
>> rahul.xavier.si...@gmail.com
> 
>> wrote:
>>> 
>>> Cool. I’m willing to help by taking sub sections of the overall
>> effort.
>> The docs need a lot of TLC. Thanks ,
>>> 
>>> Rahul Singh
>>> Principal Architect | 1.202.390.9200 | rahul.si...@datastax.com
 On Mar 12, 2019, 8:58 PM -0400, Ben Slater <
 ben.sla...@instaclustr.com>,
>> wrote:
 Hi Dinesh
 
 Great idea. We should be able to find some Instaclustr people to
>> help
>> with
 technical input (Stefan has already put his hand up).
 
 I’m also happy to help with the application if that’s useful.
 
 Cheers
 Ben
 
 ---
 
 
 *Ben Slater*
 *Chief Product Officer*
 
 
  <
 https://twitter.com/instaclustr
>>> 
 
 
 Read our latest technical blog posts here
 .
 
 This email has been sent on behalf of Instaclustr Pty. Limited
>> (Australia)
 and Instaclustr Inc (USA).
 
 This email and any attachments may contain confidential and legally
 privileged information. If you are not the intended recipient, do
>> not
>> copy
 or disclose its content, but please reply to this email immediately
 and
 highlight the error to the sender and then immediately delete the
>> message.
 
 
 On Wed, 13 Mar 2019 at 08:12, Dinesh Joshi
>>  
 wrote:
 
> Hi all,
> 
> I came across GSoD 2019[1]. This is different from GSoC and focuses
 on
> improving documentation for Open Source projects. I think this
>> would
 be
> beneficial for Cassandra especially with 4.0 coming up. However,
>> working
> with a technical writer will require a substantial time commitment
>> from us
> to bring them up to speed.
> 
> Are there any volunteers to help guide the technical writer if
>> Cassandra
> is picked as a project?
> 
> On a side note, we can put together the application on the
>> Confluence
> wiki. I will create a page and if anybody is interested in helping
 out
>> with
> putting together the application, please feel free to collaborate
>> on
>> it.
> 
> Thanks,
> 
> Dinesh
> 
> [1] https://developers.google.com/season-of-docs/docs/timeline
> 
>> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> 
>> 
>> 

Re: Choosing a supported Python 3 major version for cqlsh

[Joseph Lynch]

Since we'll be maintaining backwards compatibility with python 2.7, we
can't really use python 3 only language features or reserved keywords
anyways so we should probably just target the lowest common denominator (so
3.4 or 3.5 probably) and then after Python 2 is officially EOL in 2020
perhaps we can work on replacing 2.7 support with a newer Python 3 version?

Regarding common distros, I believe that these are the default py2 and py3
versions on CentOS and Ubuntu LTS:

Centos 7: python = python 2, python 2.7.5, python 3.5/3.6 available via SCL
Centos 6: python = python 2, python 2.6.6, python 3.5/3.6 available via SCL
Ubuntu 16.04 (xenial): python = python 2, python 2.7.12, python 3.5.2
Ubuntu 18.04 (bionic): python = python 2, python 2.7.15rc1, python 3.6.7

^^ based on variants of "docker run -it ubuntu:16.04 bin/bash -c 'apt
update && apt install -y python3 && python3 --version' | tail" and "docker
run -it centos:7 python --version" and such.

Cheers,
-Joey

On Tue, Mar 19, 2019 at 11:47 AM Jordan West  wrote:

> On Mon, Mar 18, 2019 at 7:52 PM Michael Shuler 
> wrote:
>
> > On 3/18/19 9:06 PM, Patrick Bannister wrote:
> > > I recommend we pick the longest supported stable release available.
> That
> > > would be Python 3.7, which is planned to get its last release in 2023,
> > four
> > > years from now.
> > > - Python 3.5 was planned to get its last major release yesterday
> > > - Python 3.6 is planned to get its last major release in December 2021,
> > > about three years from now
> > >
> > > Any feedback on picking a tested Python version for cqlshlib? I'm
> > inclined
> > > to focus on Python 3.7 as we push toward finishing the ticket.
> >
> > The correct method of choosing this would be to target runtime
> > functionality on the version in the latest LTS release of the likely
> > most-used OS. Ubuntu 18.04 LTS comes with python-3.6.5. I would think it
> > highly likely that if it runs properly on 3.6, it should run on 3.7
>
>
> In my experience working with a different python project recently this
> isn’t the case. There are reserved keywords that were added between 3.6 and
> 3.7:
> https://docs.python.org/3/whatsnew/3.7.html
>
> Jordan
>
>
>
> > fine. Using some 3.7-only feature/syntax and making it difficult on
> > people to install/use on Ubuntu LTS would be user-unfriendly.
> >
> > https://packages.ubuntu.com/bionic/python3
> >
> > There is not a similar CentOS package search, but I see a couple docs
> > say that python-3.6 is available via the SCL repository for this OS. I
> > do not see a 3.7 installation noted.
> >
> > Shoot for the lowest common denominator in real world usage, not the
> > latest release from upstream. Super strong opinion, here.
> >
> > --
> > Michael
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: Choosing a supported Python 3 major version for cqlsh

[Jordan West]

On Mon, Mar 18, 2019 at 7:52 PM Michael Shuler 
wrote:

> On 3/18/19 9:06 PM, Patrick Bannister wrote:
> > I recommend we pick the longest supported stable release available. That
> > would be Python 3.7, which is planned to get its last release in 2023,
> four
> > years from now.
> > - Python 3.5 was planned to get its last major release yesterday
> > - Python 3.6 is planned to get its last major release in December 2021,
> > about three years from now
> >
> > Any feedback on picking a tested Python version for cqlshlib? I'm
> inclined
> > to focus on Python 3.7 as we push toward finishing the ticket.
>
> The correct method of choosing this would be to target runtime
> functionality on the version in the latest LTS release of the likely
> most-used OS. Ubuntu 18.04 LTS comes with python-3.6.5. I would think it
> highly likely that if it runs properly on 3.6, it should run on 3.7


In my experience working with a different python project recently this
isn’t the case. There are reserved keywords that were added between 3.6 and
3.7:
https://docs.python.org/3/whatsnew/3.7.html

Jordan



> fine. Using some 3.7-only feature/syntax and making it difficult on
> people to install/use on Ubuntu LTS would be user-unfriendly.
>
> https://packages.ubuntu.com/bionic/python3
>
> There is not a similar CentOS package search, but I see a couple docs
> say that python-3.6 is available via the SCL repository for this OS. I
> do not see a 3.7 installation noted.
>
> Shoot for the lowest common denominator in real world usage, not the
> latest release from upstream. Super strong opinion, here.
>
> --
> Michael
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [jira] [Updated] (CASSANDRA-14096) Cassandra 3.11.1 Repair Causes Out of Memory

[Benedict Elliott Smith]

Yes, sorry, I was just confused.

I hadn’t actually made any of the permission changes that we voted on, they 
slipped my mind, so I’m not sure how this even happened.

I’ve removed the Anyone permission from transition, anyway, and given 
jira-users the permissions that Contributors had but they did not.


> On 19 Mar 2019, at 13:45, Joshua McKenzie  wrote:
> 
> I don't recall a vote about anonymous users having transition rights. This
> topic irritates me enough that I'm pretty sure I'd go full-on campaign mode
> against it. :)
> 
> On Tue, Mar 19, 2019 at 9:44 AM Benedict Elliott Smith 
> wrote:
> 
>> Hmm.  Actually, did we?  I’m conflating anonymous with logged-in.
>> 
>>> On 19 Mar 2019, at 13:43, Benedict Elliott Smith 
>> wrote:
>>> 
>>> We can, but we will need to arrange another vote, since we explicitly
>> discussed this and voted in favour of lifting any limitation on anonymous
>> users
>>> 
>>> (FWIW, I remain in favour of limiting anonymous users)
>>> 
>>> 
 On 19 Mar 2019, at 13:35, Joshua McKenzie  wrote:
 
 
>>> 
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>> For additional commands, e-mail: dev-h...@cassandra.apache.org
>> 
>> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [jira] [Updated] (CASSANDRA-14096) Cassandra 3.11.1 Repair Causes Out of Memory

[Joshua McKenzie]

I don't recall a vote about anonymous users having transition rights. This
topic irritates me enough that I'm pretty sure I'd go full-on campaign mode
against it. :)

On Tue, Mar 19, 2019 at 9:44 AM Benedict Elliott Smith 
wrote:

> Hmm.  Actually, did we?  I’m conflating anonymous with logged-in.
>
> > On 19 Mar 2019, at 13:43, Benedict Elliott Smith 
> wrote:
> >
> > We can, but we will need to arrange another vote, since we explicitly
> discussed this and voted in favour of lifting any limitation on anonymous
> users
> >
> > (FWIW, I remain in favour of limiting anonymous users)
> >
> >
> >> On 19 Mar 2019, at 13:35, Joshua McKenzie  wrote:
> >>
> >>
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: [jira] [Updated] (CASSANDRA-14096) Cassandra 3.11.1 Repair Causes Out of Memory

[Benedict Elliott Smith]

Hmm.  Actually, did we?  I’m conflating anonymous with logged-in.

> On 19 Mar 2019, at 13:43, Benedict Elliott Smith  wrote:
> 
> We can, but we will need to arrange another vote, since we explicitly 
> discussed this and voted in favour of lifting any limitation on anonymous 
> users
> 
> (FWIW, I remain in favour of limiting anonymous users)
> 
> 
>> On 19 Mar 2019, at 13:35, Joshua McKenzie  wrote:
>> 
>> 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: [jira] [Updated] (CASSANDRA-14096) Cassandra 3.11.1 Repair Causes Out of Memory

[Benedict Elliott Smith]

We can, but we will need to arrange another vote, since we explicitly discussed 
this and voted in favour of lifting any limitation on anonymous users

(FWIW, I remain in favour of limiting anonymous users)


> On 19 Mar 2019, at 13:35, Joshua McKenzie  wrote:
> 
> 

Re: Google Season of Docs 2019 for Apache Cassandra

[Laxmikant Upadhyay]

Hi Dinesh,

I am willing to help as well. Kindly add me to the group as well as i don't
have permission to edit the list.

Regards,
Laxmikant



On Thu, Mar 14, 2019 at 12:34 PM Dinesh Joshi 
wrote:

> Thanks for volunteering Stefan. I have added you.
>
> Dinesh
>
> > On Mar 13, 2019, at 11:55 PM, Stefan Miklosovic <
> stefan.mikloso...@instaclustr.com> wrote:
> >
> > Hi Dinesh,
> >
> > I was participant of Google summer of Code 2013 under JBoss so I can help
> > with this and with mentoring as I experienced it from the other side.
> >
> > I wanted to add myself into that list but I can not edit that  "Right
> > now everyone can view and edit this page if they've got the right space
> > permissions. "
> >
> > On Thu, 14 Mar 2019 at 17:28, Dinesh Joshi 
> > wrote:
> >
> >> Thank you everyone for offering to help!
> >>
> >> I have put together a confluence page detailing the program and other
> >> information -
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Apache+Cassandra+GSoD+2019+application
> >> <
> >>
> https://cwiki.apache.org/confluence/display/CASSANDRA/Apache+Cassandra+GSoD+2019+application
> >>>
> >>
> >> If you do not have a confluence account, please create one. I think the
> >> PMCs have the ability to give you write permissions.
> >>
> >> Please feel free to add and modify the page. I think the most important
> >> thing right now is to get together project ideas.
> >>
> >> Thanks,
> >>
> >> Dinesh
> >>
> >>> On Mar 13, 2019, at 10:19 AM, Horia Mocioi  wrote:
> >>>
> >>> I can also help.
> >>>
> >>> Regards,
> >>> Horia
> >>>
> >>> On Wed, Mar 13, 2019 at 3:12 PM Aaron Ploetz 
> >> wrote:
> >>>
>  I’m willing to help as well.  Feel free to reach out!
> 
>  Aaron Ploetz
> 
> > On Mar 12, 2019, at 8:37 PM, Rahul Singh <
> rahul.xavier.si...@gmail.com
> >>>
>  wrote:
> >
> > Cool. I’m willing to help by taking sub sections of the overall
> effort.
>  The docs need a lot of TLC. Thanks ,
> >
> > Rahul Singh
> > Principal Architect | 1.202.390.9200 | rahul.si...@datastax.com
> >> On Mar 12, 2019, 8:58 PM -0400, Ben Slater <
> >> ben.sla...@instaclustr.com>,
>  wrote:
> >> Hi Dinesh
> >>
> >> Great idea. We should be able to find some Instaclustr people to
> help
>  with
> >> technical input (Stefan has already put his hand up).
> >>
> >> I’m also happy to help with the application if that’s useful.
> >>
> >> Cheers
> >> Ben
> >>
> >> ---
> >>
> >>
> >> *Ben Slater*
> >> *Chief Product Officer*
> >>
> >>
> >>  <
> >> https://twitter.com/instaclustr
> >
> >> 
> >>
> >> Read our latest technical blog posts here
> >> .
> >>
> >> This email has been sent on behalf of Instaclustr Pty. Limited
>  (Australia)
> >> and Instaclustr Inc (USA).
> >>
> >> This email and any attachments may contain confidential and legally
> >> privileged information. If you are not the intended recipient, do
> not
>  copy
> >> or disclose its content, but please reply to this email immediately
> >> and
> >> highlight the error to the sender and then immediately delete the
>  message.
> >>
> >>
> >> On Wed, 13 Mar 2019 at 08:12, Dinesh Joshi
>  >>>
> >> wrote:
> >>
> >>> Hi all,
> >>>
> >>> I came across GSoD 2019[1]. This is different from GSoC and focuses
> >> on
> >>> improving documentation for Open Source projects. I think this
> would
> >> be
> >>> beneficial for Cassandra especially with 4.0 coming up. However,
>  working
> >>> with a technical writer will require a substantial time commitment
>  from us
> >>> to bring them up to speed.
> >>>
> >>> Are there any volunteers to help guide the technical writer if
>  Cassandra
> >>> is picked as a project?
> >>>
> >>> On a side note, we can put together the application on the
> Confluence
> >>> wiki. I will create a page and if anybody is interested in helping
> >> out
>  with
> >>> putting together the application, please feel free to collaborate
> on
>  it.
> >>>
> >>> Thanks,
> >>>
> >>> Dinesh
> >>>
> >>> [1] https://developers.google.com/season-of-docs/docs/timeline
> >>>
> -
> >>> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> >>> For additional commands, e-mail: dev-h...@cassandra.apache.org
> >>>
> >>>
> 
>  -
>  To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
>  For additional commands, e-mail: dev-h...@cassandra.apache.org
> 
> 
> >>>
> >>> --
> >>> 
> >>
> >>
> >
> > --
> > Stefan Miklosovic
>

Re: Any ideas how to make dtests more stable and reproducible?

[Dinesh Joshi]

Hi Stefan,

The dtests have been typically flaky but are more or less stable in the recent 
past. We are working towards stabilizing them. For the dev workflow locally, I 
typically end up running a subset of the dtests via the pytest runner. I am not 
sure how others run it.

I believe CircleCI results are the most consistent and accurate so far. You can 
refer to this[1] recent sample run. All tests passed. The CircleCI workflow has 
changed recently so it'll look different now but the point being that the tests 
are more or less stable. I would caution you if you're running on a free tier, 
it'll take a lot of time and the test results are unreliable as the free tier 
does not have enough resources. To compare your setup, we typically run the 
dtests in 100 CircleCI containers concurrently. Each container has 8 VCPUs and 
16GB RAM. The run takes 20-30 minutes depending on the resource availability.

Thanks,

Dinesh


[1] https://circleci.com/workflow-run/80804bb2-dafb-445a-acca-53401ca02806 



> On Mar 18, 2019, at 5:46 PM, Stefan Miklosovic 
>  wrote:
> 
> Hi,
> 
> I am running large and "simple" dtests (executed via
> cassandra-builds/build-scripts/cassandra-dtest-pytest.sh) and I find myself
> quite frustrated as I do not know if there are errors because tests are
> flaky or there are legit issues which produced them.
> 
> It is "simple" to check it one by one when tests are stable and there is
> couple of them but when there are hundreds of tests, whole test run takes
> ~7 hours and it is not stable, it is like finding a needle in a haystack.
> Sometimes 15 tests fail, sometimes just 10 ... Sometimes there are
> timeouts, sometimes not.
> 
> For basic dtests I am getting stable three errors out of 900 I think which
> quite good. I supplied one patch here (1) so only two of them are failing
> now consistently (it is not merged yet).
> 
> Can you point me to your builds and what results you are getting there?
> Maybe something is wrong with my setup or these dtests are "expected" to be
> flaky from time to time?
> 
> What stability are you getting with official builds when it comes to
> dtests? How often they are run? As part of every pull request / change? Do
> you commit only on "0 dtests failed"?
> 
> Are there some recommendations as on what setup and machine these tests
> should run? I am running them on c5.9xlarge (36 cores with 64 GB or memory)
> on fairly recent Ubuntu with latest Java 8. I am trying to supply all
> needed parameters and libs in order to start Cassandra smoothly without any
> warnings / errors (there are these checks which check if your environment
> is all fine).
> 
> I am testing current trunk.
> 
> Thanks for any input how to make them more stable if there are some tips
> and tricks.
> 
> (1) https://github.com/apache/cassandra-dtest/pull/47
> 
> Stefan Miklosovic