Re: [DISCUSS] A point of view on Testing Cassandra
> > measure, but I don't think it's the most important one. With > regard > > to > > > measuring against a common standard in the project, this is > roughly > > what I > > > had in mind when proposing "Release Quality Metrics" on the > list in > > 2018. I > > > still think making progress on something like this is essential > > toward > > > defining a quantitative bar for release: > > > > https://www.mail-archive.com/dev@cassandra.apache.org/msg13154.html > > > > > > > Conversely, the ability to repeatedly and thoroughly > validate the > > > correctness of both new and existing functionality in the > system is > > vital > > > to the speed with which we can evolve it's form and function. > > > > > > Strongly agreed. > > > > > > > Utopia (and following section) > > > > > > Some nods to great potential refactors to consider post-4.0 > here. ^ > > > > > > > We should productize a kubernetes-centric, infra agnostic > tool > > that has > > > the following available testing paradigms: > > > > > > This would be an excellent set of capabilities to have. > > > > > > > We need to empower our user community to participate in the > testing > > > process... > > > > > > I really like this point. I took as a thought experiment "what > would > > feel > > > great to be able to say" if one were to write a product > announcement > > for > > > 4.0 and landed on something like "Users of Apache Cassandra can > > preflight > > > their 4.0 upgrade by runing $tool to clone, upgrade, and > compare > > their > > > clusters, ensuring that the upgrade will complete smoothly and > > correctly." > > > > > > > The less friction and less investment we can require from > ecosystem > > > participants, the more we can expect them to engage in desired > > behavior. > > > > > > +1 > > > > > > –– > > > > > > I like the document and there's a lot that has me nodding. > Toward the > > > opening statement on "empirical evidence to quantify relative > > stability," > > > I'd love to revisit discussion on quantifying attributes like > these > > here: > > > > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93324430 > > > > > > – Scott > > > > > > > > > From: David Capwell > > > Sent: Tuesday, July 14, 2020 6:23 PM > > > To: dev@cassandra.apache.org > > > Subject: Re: [DISCUSS] A point of view on Testing Cassandra > > > > > > I am also not fully clear on the motives, but welcome anything > which > > helps > > > bring in better and more robust testing; thanks for starting > this. > > > > > > Since I can not comment in the doc I have to copy/paste and put > > here... =( > > > > > > Reality > > > > ... > > > > investing in improving our smoke and integration testing as > much > > as is > > > > possible with our current constraints seems prudent. > > > > > > > > > This section reads as very anti-adding tests to test/unit; I > am 100% > > in > > > favor of improving/creating our smoke, integration, regression, > > > performance, E2E, etc. testing, but don't think I am as > negative to > > > test/unit, these tests are still valuable and more are welcome. > > > > > > To enumerate a punch list of traits we as engineers need from a > > testing > > > > suite > > > > > > > > > Would be good to speak about portability, accessibility, and > version > > > independents. If a new contributor wants to add te
Re: [DISCUSS] A point of view on Testing Cassandra
> things and/or surfacing adverse conditions within the system upon
> failure.
>
> It's tricky because in the past (in my opinion) we've been pretty
> remiss as
> a project when it comes to a devotion to correctness and rigor. The
> danger
> I'm anecdotally seeing is that if we let that pendulum swing too far
> in the
> other direction without successfully clearly defining what "Done"
looks
> like from a quality perspective, that's an Everest we can all climb
> and die
> on as a project.
>
> On Wed, Jul 15, 2020 at 12:42 AM Scott Andreas
> wrote:
>
> > Thanks for starting discussion!
> >
> > Replying to the thread with what I would have left as comments.
> >
> > ––
> >
> > > As yet, we lack empirical evidence to quantify the relative
> stability or
> > instability of our project compared to a peer cohort
> >
> > I think it's more important that we set a standard for the project
> (e.g.,
> > fundamental conformance to properties of the database) rather than
> > attempting to measure quality relative to other DBs. That might be a
> useful
> > measure, but I don't think it's the most important one. With regard
> to
> > measuring against a common standard in the project, this is roughly
> what I
> > had in mind when proposing "Release Quality Metrics" on the list in
> 2018. I
> > still think making progress on something like this is essential
> toward
> > defining a quantitative bar for release:
> > https://www.mail-archive.com/dev@cassandra.apache.org/msg13154.html
> >
> > > Conversely, the ability to repeatedly and thoroughly validate the
> > correctness of both new and existing functionality in the system is
> vital
> > to the speed with which we can evolve it's form and function.
> >
> > Strongly agreed.
> >
> > > Utopia (and following section)
> >
> > Some nods to great potential refactors to consider post-4.0 here. ^
> >
> > > We should productize a kubernetes-centric, infra agnostic tool
> that has
> > the following available testing paradigms:
> >
> > This would be an excellent set of capabilities to have.
> >
> > > We need to empower our user community to participate in the
testing
> > process...
> >
> > I really like this point. I took as a thought experiment "what would
> feel
> > great to be able to say" if one were to write a product announcement
> for
> > 4.0 and landed on something like "Users of Apache Cassandra can
> preflight
> > their 4.0 upgrade by runing $tool to clone, upgrade, and compare
> their
> > clusters, ensuring that the upgrade will complete smoothly and
> correctly."
> >
> > > The less friction and less investment we can require from
ecosystem
> > participants, the more we can expect them to engage in desired
> behavior.
> >
> > +1
> >
> > ––
> >
> > I like the document and there's a lot that has me nodding. Toward
the
> > opening statement on "empirical evidence to quantify relative
> stability,"
> > I'd love to revisit discussion on quantifying attributes like these
> here:
> >
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93324430
> >
> > – Scott
> >
> >
> > From: David Capwell
> > Sent: Tuesday, July 14, 2020 6:23 PM
> > To: dev@cassandra.apache.org
> > Subject: Re: [DISCUSS] A point of view on Testing Cassandra
> >
> > I am also not fully clear on the motives, but welcome anything which
> helps
> > bring in better and more robust testing; thanks for starting this.
> >
> > Since I can not comment in the doc I have to copy/paste and put
> here... =(
> >
> > Reality
> > > ...
> > > investing in improving our smoke and integra
Re: [DISCUSS] A point of view on Testing Cassandra
feel > > great to be able to say" if one were to write a product announcement > for > > 4.0 and landed on something like "Users of Apache Cassandra can > preflight > > their 4.0 upgrade by runing $tool to clone, upgrade, and compare > their > > clusters, ensuring that the upgrade will complete smoothly and > correctly." > > > > > The less friction and less investment we can require from ecosystem > > participants, the more we can expect them to engage in desired > behavior. > > > > +1 > > > > –– > > > > I like the document and there's a lot that has me nodding. Toward the > > opening statement on "empirical evidence to quantify relative > stability," > > I'd love to revisit discussion on quantifying attributes like these > here: > > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93324430 > > > > – Scott > > > > > > From: David Capwell > > Sent: Tuesday, July 14, 2020 6:23 PM > > To: dev@cassandra.apache.org > > Subject: Re: [DISCUSS] A point of view on Testing Cassandra > > > > I am also not fully clear on the motives, but welcome anything which > helps > > bring in better and more robust testing; thanks for starting this. > > > > Since I can not comment in the doc I have to copy/paste and put > here... =( > > > > Reality > > > ... > > > investing in improving our smoke and integration testing as much > as is > > > possible with our current constraints seems prudent. > > > > > > This section reads as very anti-adding tests to test/unit; I am 100% > in > > favor of improving/creating our smoke, integration, regression, > > performance, E2E, etc. testing, but don't think I am as negative to > > test/unit, these tests are still valuable and more are welcome. > > > > To enumerate a punch list of traits we as engineers need from a > testing > > > suite > > > > > > Would be good to speak about portability, accessibility, and version > > independents. If a new contributor wants to add tests to this suite > they > > need to be able to run it, and it should run within a "reasonable" > time > > frame; one of the big issuers with python dtests is that it takes > 14+ hours > > to run, this makes it no longer accessible to new contributors. > > > > > > On Tue, Jul 14, 2020 at 11:47 AM Joshua McKenzie < > jmcken...@apache.org> > > wrote: > > > > > The purpose is purely to signal a point of view on the state of > testing > > in > > > the codebase, some shortcomings of the architecture, and what a > few of us > > > are doing and further planning to do about it. Kind of a "prompt > > discussion > > > if anyone has a wild allergic reaction to it, or encourage > collaboration > > if > > > they have a wild positive reaction" sort of thing. Maybe a > spiritual > > > "CEP-lite". :) > > > > > > I would advocate that we be very selective about the topics on > which we > > > strive for a consistent shared point of view as a project. There > are a > > lot > > > of us and we all have different experiences and different points > of view > > > that lead to different perspectives and value systems. Agreeing on > > discrete > > > definitions of done, 100% - that's table stakes. But agreeing on > how we > > get > > > there, my personal take is we'd all be well served to spend our > energy > > > Doing the Work and expressing these complementary positions rather > than > > > trying to bend everyone to one consistent point of view. > > > > > > Let a thousand flowers bloom, as someone wise recently told me. :) > > > > > > That said, this work will be happening in an open source repo with > a > > > permissive license (almost certainly ASLv2), likely using github > issues, > > so > > > anyone that wants to collaborate on it would be most welcome. I > can make > > > sure Gianluca, Charles, Berenguer, and others bring that to this ML > > thread > > > onc
Re: [DISCUSS] A point of view on Testing Cassandra
, 2020 at 12:42 AM Scott Andreas wrote: > Thanks for starting discussion! > > Replying to the thread with what I would have left as comments. > > –– > > > As yet, we lack empirical evidence to quantify the relative stability or > instability of our project compared to a peer cohort > > I think it's more important that we set a standard for the project (e.g., > fundamental conformance to properties of the database) rather than > attempting to measure quality relative to other DBs. That might be a useful > measure, but I don't think it's the most important one. With regard to > measuring against a common standard in the project, this is roughly what I > had in mind when proposing "Release Quality Metrics" on the list in 2018. I > still think making progress on something like this is essential toward > defining a quantitative bar for release: > https://www.mail-archive.com/dev@cassandra.apache.org/msg13154.html > > > Conversely, the ability to repeatedly and thoroughly validate the > correctness of both new and existing functionality in the system is vital > to the speed with which we can evolve it's form and function. > > Strongly agreed. > > > Utopia (and following section) > > Some nods to great potential refactors to consider post-4.0 here. ^ > > > We should productize a kubernetes-centric, infra agnostic tool that has > the following available testing paradigms: > > This would be an excellent set of capabilities to have. > > > We need to empower our user community to participate in the testing > process... > > I really like this point. I took as a thought experiment "what would feel > great to be able to say" if one were to write a product announcement for > 4.0 and landed on something like "Users of Apache Cassandra can preflight > their 4.0 upgrade by runing $tool to clone, upgrade, and compare their > clusters, ensuring that the upgrade will complete smoothly and correctly." > > > The less friction and less investment we can require from ecosystem > participants, the more we can expect them to engage in desired behavior. > > +1 > > –– > > I like the document and there's a lot that has me nodding. Toward the > opening statement on "empirical evidence to quantify relative stability," > I'd love to revisit discussion on quantifying attributes like these here: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93324430 > > – Scott > > > From: David Capwell > Sent: Tuesday, July 14, 2020 6:23 PM > To: dev@cassandra.apache.org > Subject: Re: [DISCUSS] A point of view on Testing Cassandra > > I am also not fully clear on the motives, but welcome anything which helps > bring in better and more robust testing; thanks for starting this. > > Since I can not comment in the doc I have to copy/paste and put here... =( > > Reality > > ... > > investing in improving our smoke and integration testing as much as is > > possible with our current constraints seems prudent. > > > This section reads as very anti-adding tests to test/unit; I am 100% in > favor of improving/creating our smoke, integration, regression, > performance, E2E, etc. testing, but don't think I am as negative to > test/unit, these tests are still valuable and more are welcome. > > To enumerate a punch list of traits we as engineers need from a testing > > suite > > > Would be good to speak about portability, accessibility, and version > independents. If a new contributor wants to add tests to this suite they > need to be able to run it, and it should run within a "reasonable" time > frame; one of the big issuers with python dtests is that it takes 14+ hours > to run, this makes it no longer accessible to new contributors. > > > On Tue, Jul 14, 2020 at 11:47 AM Joshua McKenzie > wrote: > > > The purpose is purely to signal a point of view on the state of testing > in > > the codebase, some shortcomings of the architecture, and what a few of us > > are doing and further planning to do about it. Kind of a "prompt > discussion > > if anyone has a wild allergic reaction to it, or encourage collaboration > if > > they have a wild positive reaction" sort o
Re: [DISCUSS] A point of view on Testing Cassandra
xcellent set of capabilities to have. > > > We need to empower our user community to participate in the testing > process... > > I really like this point. I took as a thought experiment "what would feel > great to be able to say" if one were to write a product announcement for > 4.0 and landed on something like "Users of Apache Cassandra can preflight > their 4.0 upgrade by runing $tool to clone, upgrade, and compare their > clusters, ensuring that the upgrade will complete smoothly and correctly." > > > The less friction and less investment we can require from ecosystem > participants, the more we can expect them to engage in desired behavior. > > +1 > > –– > > I like the document and there's a lot that has me nodding. Toward the > opening statement on "empirical evidence to quantify relative stability," > I'd love to revisit discussion on quantifying attributes like these here: > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93324430 > > – Scott > > > From: David Capwell > Sent: Tuesday, July 14, 2020 6:23 PM > To: dev@cassandra.apache.org > Subject: Re: [DISCUSS] A point of view on Testing Cassandra > > I am also not fully clear on the motives, but welcome anything which helps > bring in better and more robust testing; thanks for starting this. > > Since I can not comment in the doc I have to copy/paste and put here... =( > > Reality > > ... > > investing in improving our smoke and integration testing as much as is > > possible with our current constraints seems prudent. > > > This section reads as very anti-adding tests to test/unit; I am 100% in > favor of improving/creating our smoke, integration, regression, > performance, E2E, etc. testing, but don't think I am as negative to > test/unit, these tests are still valuable and more are welcome. > > To enumerate a punch list of traits we as engineers need from a testing > > suite > > > Would be good to speak about portability, accessibility, and version > independents. If a new contributor wants to add tests to this suite they > need to be able to run it, and it should run within a "reasonable" time > frame; one of the big issuers with python dtests is that it takes 14+ hours > to run, this makes it no longer accessible to new contributors. > > > On Tue, Jul 14, 2020 at 11:47 AM Joshua McKenzie > wrote: > > > The purpose is purely to signal a point of view on the state of testing > in > > the codebase, some shortcomings of the architecture, and what a few of us > > are doing and further planning to do about it. Kind of a "prompt > discussion > > if anyone has a wild allergic reaction to it, or encourage collaboration > if > > they have a wild positive reaction" sort of thing. Maybe a spiritual > > "CEP-lite". :) > > > > I would advocate that we be very selective about the topics on which we > > strive for a consistent shared point of view as a project. There are a > lot > > of us and we all have different experiences and different points of view > > that lead to different perspectives and value systems. Agreeing on > discrete > > definitions of done, 100% - that's table stakes. But agreeing on how we > get > > there, my personal take is we'd all be well served to spend our energy > > Doing the Work and expressing these complementary positions rather than > > trying to bend everyone to one consistent point of view. > > > > Let a thousand flowers bloom, as someone wise recently told me. :) > > > > That said, this work will be happening in an open source repo with a > > permissive license (almost certainly ASLv2), likely using github issues, > so > > anyone that wants to collaborate on it would be most welcome. I can make > > sure Gianluca, Charles, Berenguer, and others bring that to this ML > thread > > once we've started open-sourcing things. > > > > On Tue, Jul 14, 2020 at 4:25 AM Benedict Elliott Smith < > > bened...@apache.org> > > wrote: > > > > > It does raise the bar to critiquing the document though, but perhaps > > > that's also a feature. > > > > > > Perhaps we can first discuss the purpose of the document? It seems to > be > > a > > > mix of mission statement for the project, as well as your own near term > > > roadmap? Should we interpret it only as an advertisement of your own > > view > > > of the problems the project faces, as a start to dialogue, or is the > > > purpose to solicit feedback? > > &g
Re: [DISCUSS] A point of view on Testing Cassandra
Thanks for starting discussion! Replying to the thread with what I would have left as comments. –– > As yet, we lack empirical evidence to quantify the relative stability or > instability of our project compared to a peer cohort I think it's more important that we set a standard for the project (e.g., fundamental conformance to properties of the database) rather than attempting to measure quality relative to other DBs. That might be a useful measure, but I don't think it's the most important one. With regard to measuring against a common standard in the project, this is roughly what I had in mind when proposing "Release Quality Metrics" on the list in 2018. I still think making progress on something like this is essential toward defining a quantitative bar for release: https://www.mail-archive.com/dev@cassandra.apache.org/msg13154.html > Conversely, the ability to repeatedly and thoroughly validate the correctness > of both new and existing functionality in the system is vital to the speed > with which we can evolve it's form and function. Strongly agreed. > Utopia (and following section) Some nods to great potential refactors to consider post-4.0 here. ^ > We should productize a kubernetes-centric, infra agnostic tool that has the > following available testing paradigms: This would be an excellent set of capabilities to have. > We need to empower our user community to participate in the testing process... I really like this point. I took as a thought experiment "what would feel great to be able to say" if one were to write a product announcement for 4.0 and landed on something like "Users of Apache Cassandra can preflight their 4.0 upgrade by runing $tool to clone, upgrade, and compare their clusters, ensuring that the upgrade will complete smoothly and correctly." > The less friction and less investment we can require from ecosystem > participants, the more we can expect them to engage in desired behavior. +1 –– I like the document and there's a lot that has me nodding. Toward the opening statement on "empirical evidence to quantify relative stability," I'd love to revisit discussion on quantifying attributes like these here: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=93324430 – Scott ____ From: David Capwell Sent: Tuesday, July 14, 2020 6:23 PM To: dev@cassandra.apache.org Subject: Re: [DISCUSS] A point of view on Testing Cassandra I am also not fully clear on the motives, but welcome anything which helps bring in better and more robust testing; thanks for starting this. Since I can not comment in the doc I have to copy/paste and put here... =( Reality > ... > investing in improving our smoke and integration testing as much as is > possible with our current constraints seems prudent. This section reads as very anti-adding tests to test/unit; I am 100% in favor of improving/creating our smoke, integration, regression, performance, E2E, etc. testing, but don't think I am as negative to test/unit, these tests are still valuable and more are welcome. To enumerate a punch list of traits we as engineers need from a testing > suite Would be good to speak about portability, accessibility, and version independents. If a new contributor wants to add tests to this suite they need to be able to run it, and it should run within a "reasonable" time frame; one of the big issuers with python dtests is that it takes 14+ hours to run, this makes it no longer accessible to new contributors. On Tue, Jul 14, 2020 at 11:47 AM Joshua McKenzie wrote: > The purpose is purely to signal a point of view on the state of testing in > the codebase, some shortcomings of the architecture, and what a few of us > are doing and further planning to do about it. Kind of a "prompt discussion > if anyone has a wild allergic reaction to it, or encourage collaboration if > they have a wild positive reaction" sort of thing. Maybe a spiritual > "CEP-lite". :) > > I would advocate that we be very selective about the topics on which we > strive for a consistent shared point of view as a project. There are a lot > of us and we all have different experiences and different points of view > that lead to different perspectives and value systems. Agreeing on discrete > definitions of done, 100% - that's table stakes. But agreeing on how we get > there, my personal take is we'd all be well served to spend our energy > Doing the Work and expressing these complementary positions rather than > trying to bend everyone to one consistent point of view. > > Let a thousand flowers bloom, as someone wise recently told me. :) > > That said, this work will be happening in an open source repo with a > permissive license (almost certai
Re: [DISCUSS] A point of view on Testing Cassandra
I am also not fully clear on the motives, but welcome anything which helps bring in better and more robust testing; thanks for starting this. Since I can not comment in the doc I have to copy/paste and put here... =( Reality > ... > investing in improving our smoke and integration testing as much as is > possible with our current constraints seems prudent. This section reads as very anti-adding tests to test/unit; I am 100% in favor of improving/creating our smoke, integration, regression, performance, E2E, etc. testing, but don't think I am as negative to test/unit, these tests are still valuable and more are welcome. To enumerate a punch list of traits we as engineers need from a testing > suite Would be good to speak about portability, accessibility, and version independents. If a new contributor wants to add tests to this suite they need to be able to run it, and it should run within a "reasonable" time frame; one of the big issuers with python dtests is that it takes 14+ hours to run, this makes it no longer accessible to new contributors. On Tue, Jul 14, 2020 at 11:47 AM Joshua McKenzie wrote: > The purpose is purely to signal a point of view on the state of testing in > the codebase, some shortcomings of the architecture, and what a few of us > are doing and further planning to do about it. Kind of a "prompt discussion > if anyone has a wild allergic reaction to it, or encourage collaboration if > they have a wild positive reaction" sort of thing. Maybe a spiritual > "CEP-lite". :) > > I would advocate that we be very selective about the topics on which we > strive for a consistent shared point of view as a project. There are a lot > of us and we all have different experiences and different points of view > that lead to different perspectives and value systems. Agreeing on discrete > definitions of done, 100% - that's table stakes. But agreeing on how we get > there, my personal take is we'd all be well served to spend our energy > Doing the Work and expressing these complementary positions rather than > trying to bend everyone to one consistent point of view. > > Let a thousand flowers bloom, as someone wise recently told me. :) > > That said, this work will be happening in an open source repo with a > permissive license (almost certainly ASLv2), likely using github issues, so > anyone that wants to collaborate on it would be most welcome. I can make > sure Gianluca, Charles, Berenguer, and others bring that to this ML thread > once we've started open-sourcing things. > > On Tue, Jul 14, 2020 at 4:25 AM Benedict Elliott Smith < > bened...@apache.org> > wrote: > > > It does raise the bar to critiquing the document though, but perhaps > > that's also a feature. > > > > Perhaps we can first discuss the purpose of the document? It seems to be > a > > mix of mission statement for the project, as well as your own near term > > roadmap? Should we interpret it only as an advertisement of your own > view > > of the problems the project faces, as a start to dialogue, or is the > > purpose to solicit feedback? > > > > Would it be helpful to work towards a similar document the whole > community > > endorses, with a shared mission statement, and a (perhaps loosely > defined) > > shared roadmap? > > > > I'd like to call out some specific things in the document that I am > > personally excited by: the project has long lacked a coherent, repeatable > > approach to performance testing and regressions; combined with easy > > visualisation tools this would be a huge win. The FQL sampling with data > > distribution inference is also something that has been discussed > privately > > elsewhere, and would be hugely advantageous to the former, so that we can > > discover representative workloads. > > > > Thanks for taking the time to put this together, and start this dialogue. > > > > > > On 13/07/2020, 23:41, "Joshua McKenzie" wrote: > > > > > > > > Can you please allow comments on the doc so we can leave feedback. > > > > > > > > > > Doc is view only; figured we could keep this to the ML. > > > > > That's a feature, not a bug. > > > > Happy to chat here or on slack w/anyone. This is a complex topic so > > long-form or high bandwidth communication is a better fit than gdoc > > comments. They rapidly become unwieldy. > > > > On Mon, Jul 13, 2020 at 6:17 PM sankalp kohli < > kohlisank...@gmail.com> > > wrote: > > > > > Can you please allow comments on the doc so we can leave feedback. > > > > > > On Mon, Jul 13, 2020 at 2:16 PM Joshua McKenzie < > > jmcken...@apache.org> > > > wrote: > > > > > > > Link: > > > > > > > > > > > > > > https://docs.google.com/document/d/1ktuBWpD2NLurB9PUvmbwGgrXsgnyU58koOseZAfaFBQ/edit# > > > > > > > > > > > > Myself and a few other contributors are working with this point > of > > view > > > as > > > > our frame of where we're going to work on improving testing on > the > > >
Re: [DISCUSS] A point of view on Testing Cassandra
The purpose is purely to signal a point of view on the state of testing in the codebase, some shortcomings of the architecture, and what a few of us are doing and further planning to do about it. Kind of a "prompt discussion if anyone has a wild allergic reaction to it, or encourage collaboration if they have a wild positive reaction" sort of thing. Maybe a spiritual "CEP-lite". :) I would advocate that we be very selective about the topics on which we strive for a consistent shared point of view as a project. There are a lot of us and we all have different experiences and different points of view that lead to different perspectives and value systems. Agreeing on discrete definitions of done, 100% - that's table stakes. But agreeing on how we get there, my personal take is we'd all be well served to spend our energy Doing the Work and expressing these complementary positions rather than trying to bend everyone to one consistent point of view. Let a thousand flowers bloom, as someone wise recently told me. :) That said, this work will be happening in an open source repo with a permissive license (almost certainly ASLv2), likely using github issues, so anyone that wants to collaborate on it would be most welcome. I can make sure Gianluca, Charles, Berenguer, and others bring that to this ML thread once we've started open-sourcing things. On Tue, Jul 14, 2020 at 4:25 AM Benedict Elliott Smith wrote: > It does raise the bar to critiquing the document though, but perhaps > that's also a feature. > > Perhaps we can first discuss the purpose of the document? It seems to be a > mix of mission statement for the project, as well as your own near term > roadmap? Should we interpret it only as an advertisement of your own view > of the problems the project faces, as a start to dialogue, or is the > purpose to solicit feedback? > > Would it be helpful to work towards a similar document the whole community > endorses, with a shared mission statement, and a (perhaps loosely defined) > shared roadmap? > > I'd like to call out some specific things in the document that I am > personally excited by: the project has long lacked a coherent, repeatable > approach to performance testing and regressions; combined with easy > visualisation tools this would be a huge win. The FQL sampling with data > distribution inference is also something that has been discussed privately > elsewhere, and would be hugely advantageous to the former, so that we can > discover representative workloads. > > Thanks for taking the time to put this together, and start this dialogue. > > > On 13/07/2020, 23:41, "Joshua McKenzie" wrote: > > > > > Can you please allow comments on the doc so we can leave feedback. > > > > > > Doc is view only; figured we could keep this to the ML. > > > That's a feature, not a bug. > > Happy to chat here or on slack w/anyone. This is a complex topic so > long-form or high bandwidth communication is a better fit than gdoc > comments. They rapidly become unwieldy. > > On Mon, Jul 13, 2020 at 6:17 PM sankalp kohli > wrote: > > > Can you please allow comments on the doc so we can leave feedback. > > > > On Mon, Jul 13, 2020 at 2:16 PM Joshua McKenzie < > jmcken...@apache.org> > > wrote: > > > > > Link: > > > > > > > > > https://docs.google.com/document/d/1ktuBWpD2NLurB9PUvmbwGgrXsgnyU58koOseZAfaFBQ/edit# > > > > > > > > > Myself and a few other contributors are working with this point of > view > > as > > > our frame of where we're going to work on improving testing on the > > project. > > > I figured it might be useful to foster collaboration more broadly > in the > > > community as well as provide people with the opportunity to > discuss work > > > they're doing they may not yet have had a chance to bring up or > open > > > source. While fallout is already open-sourced, expect the schema > > anonymizer > > > and some of the cassandra-diff + nosqlbench framework effort to be > > > open-sourced / openly worked on soon. Anyone that's interested in > > > collaborating, that would be highly welcome. > > > > > > Doc is view only; figured we could keep this to the ML. > > > > > > Thanks. > > > > > > ~Josh > > > > > > > > > - > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
Re: [DISCUSS] A point of view on Testing Cassandra
It does raise the bar to critiquing the document though, but perhaps that's also a feature. Perhaps we can first discuss the purpose of the document? It seems to be a mix of mission statement for the project, as well as your own near term roadmap? Should we interpret it only as an advertisement of your own view of the problems the project faces, as a start to dialogue, or is the purpose to solicit feedback? Would it be helpful to work towards a similar document the whole community endorses, with a shared mission statement, and a (perhaps loosely defined) shared roadmap? I'd like to call out some specific things in the document that I am personally excited by: the project has long lacked a coherent, repeatable approach to performance testing and regressions; combined with easy visualisation tools this would be a huge win. The FQL sampling with data distribution inference is also something that has been discussed privately elsewhere, and would be hugely advantageous to the former, so that we can discover representative workloads. Thanks for taking the time to put this together, and start this dialogue. On 13/07/2020, 23:41, "Joshua McKenzie" wrote: > > Can you please allow comments on the doc so we can leave feedback. > > Doc is view only; figured we could keep this to the ML. > That's a feature, not a bug. Happy to chat here or on slack w/anyone. This is a complex topic so long-form or high bandwidth communication is a better fit than gdoc comments. They rapidly become unwieldy. On Mon, Jul 13, 2020 at 6:17 PM sankalp kohli wrote: > Can you please allow comments on the doc so we can leave feedback. > > On Mon, Jul 13, 2020 at 2:16 PM Joshua McKenzie > wrote: > > > Link: > > > > > https://docs.google.com/document/d/1ktuBWpD2NLurB9PUvmbwGgrXsgnyU58koOseZAfaFBQ/edit# > > > > > > Myself and a few other contributors are working with this point of view > as > > our frame of where we're going to work on improving testing on the > project. > > I figured it might be useful to foster collaboration more broadly in the > > community as well as provide people with the opportunity to discuss work > > they're doing they may not yet have had a chance to bring up or open > > source. While fallout is already open-sourced, expect the schema > anonymizer > > and some of the cassandra-diff + nosqlbench framework effort to be > > open-sourced / openly worked on soon. Anyone that's interested in > > collaborating, that would be highly welcome. > > > > Doc is view only; figured we could keep this to the ML. > > > > Thanks. > > > > ~Josh > > > - To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org For additional commands, e-mail: dev-h...@cassandra.apache.org
Re: [DISCUSS] A point of view on Testing Cassandra
> > Can you please allow comments on the doc so we can leave feedback. > > Doc is view only; figured we could keep this to the ML. > That's a feature, not a bug. Happy to chat here or on slack w/anyone. This is a complex topic so long-form or high bandwidth communication is a better fit than gdoc comments. They rapidly become unwieldy. On Mon, Jul 13, 2020 at 6:17 PM sankalp kohli wrote: > Can you please allow comments on the doc so we can leave feedback. > > On Mon, Jul 13, 2020 at 2:16 PM Joshua McKenzie > wrote: > > > Link: > > > > > https://docs.google.com/document/d/1ktuBWpD2NLurB9PUvmbwGgrXsgnyU58koOseZAfaFBQ/edit# > > > > > > Myself and a few other contributors are working with this point of view > as > > our frame of where we're going to work on improving testing on the > project. > > I figured it might be useful to foster collaboration more broadly in the > > community as well as provide people with the opportunity to discuss work > > they're doing they may not yet have had a chance to bring up or open > > source. While fallout is already open-sourced, expect the schema > anonymizer > > and some of the cassandra-diff + nosqlbench framework effort to be > > open-sourced / openly worked on soon. Anyone that's interested in > > collaborating, that would be highly welcome. > > > > Doc is view only; figured we could keep this to the ML. > > > > Thanks. > > > > ~Josh > > >
Re: [DISCUSS] A point of view on Testing Cassandra
Can you please allow comments on the doc so we can leave feedback. On Mon, Jul 13, 2020 at 2:16 PM Joshua McKenzie wrote: > Link: > > https://docs.google.com/document/d/1ktuBWpD2NLurB9PUvmbwGgrXsgnyU58koOseZAfaFBQ/edit# > > > Myself and a few other contributors are working with this point of view as > our frame of where we're going to work on improving testing on the project. > I figured it might be useful to foster collaboration more broadly in the > community as well as provide people with the opportunity to discuss work > they're doing they may not yet have had a chance to bring up or open > source. While fallout is already open-sourced, expect the schema anonymizer > and some of the cassandra-diff + nosqlbench framework effort to be > open-sourced / openly worked on soon. Anyone that's interested in > collaborating, that would be highly welcome. > > Doc is view only; figured we could keep this to the ML. > > Thanks. > > ~Josh >
[DISCUSS] A point of view on Testing Cassandra
Link: https://docs.google.com/document/d/1ktuBWpD2NLurB9PUvmbwGgrXsgnyU58koOseZAfaFBQ/edit# Myself and a few other contributors are working with this point of view as our frame of where we're going to work on improving testing on the project. I figured it might be useful to foster collaboration more broadly in the community as well as provide people with the opportunity to discuss work they're doing they may not yet have had a chance to bring up or open source. While fallout is already open-sourced, expect the schema anonymizer and some of the cassandra-diff + nosqlbench framework effort to be open-sourced / openly worked on soon. Anyone that's interested in collaborating, that would be highly welcome. Doc is view only; figured we could keep this to the ML. Thanks. ~Josh
