I am also not fully clear on the motives, but welcome anything which helps bring in better and more robust testing; thanks for starting this.
Since I can not comment in the doc I have to copy/paste and put here... =( Reality > ... > investing in improving our smoke and integration testing as much as is > possible with our current constraints seems prudent. This section reads as very anti-adding tests to test/unit; I am 100% in favor of improving/creating our smoke, integration, regression, performance, E2E, etc. testing, but don't think I am as negative to test/unit, these tests are still valuable and more are welcome. To enumerate a punch list of traits we as engineers need from a testing > suite Would be good to speak about portability, accessibility, and version independents. If a new contributor wants to add tests to this suite they need to be able to run it, and it should run within a "reasonable" time frame; one of the big issuers with python dtests is that it takes 14+ hours to run, this makes it no longer accessible to new contributors. On Tue, Jul 14, 2020 at 11:47 AM Joshua McKenzie <jmcken...@apache.org> wrote: > The purpose is purely to signal a point of view on the state of testing in > the codebase, some shortcomings of the architecture, and what a few of us > are doing and further planning to do about it. Kind of a "prompt discussion > if anyone has a wild allergic reaction to it, or encourage collaboration if > they have a wild positive reaction" sort of thing. Maybe a spiritual > "CEP-lite". :) > > I would advocate that we be very selective about the topics on which we > strive for a consistent shared point of view as a project. There are a lot > of us and we all have different experiences and different points of view > that lead to different perspectives and value systems. Agreeing on discrete > definitions of done, 100% - that's table stakes. But agreeing on how we get > there, my personal take is we'd all be well served to spend our energy > Doing the Work and expressing these complementary positions rather than > trying to bend everyone to one consistent point of view. > > Let a thousand flowers bloom, as someone wise recently told me. :) > > That said, this work will be happening in an open source repo with a > permissive license (almost certainly ASLv2), likely using github issues, so > anyone that wants to collaborate on it would be most welcome. I can make > sure Gianluca, Charles, Berenguer, and others bring that to this ML thread > once we've started open-sourcing things. > > On Tue, Jul 14, 2020 at 4:25 AM Benedict Elliott Smith < > bened...@apache.org> > wrote: > > > It does raise the bar to critiquing the document though, but perhaps > > that's also a feature. > > > > Perhaps we can first discuss the purpose of the document? It seems to be > a > > mix of mission statement for the project, as well as your own near term > > roadmap? Should we interpret it only as an advertisement of your own > view > > of the problems the project faces, as a start to dialogue, or is the > > purpose to solicit feedback? > > > > Would it be helpful to work towards a similar document the whole > community > > endorses, with a shared mission statement, and a (perhaps loosely > defined) > > shared roadmap? > > > > I'd like to call out some specific things in the document that I am > > personally excited by: the project has long lacked a coherent, repeatable > > approach to performance testing and regressions; combined with easy > > visualisation tools this would be a huge win. The FQL sampling with data > > distribution inference is also something that has been discussed > privately > > elsewhere, and would be hugely advantageous to the former, so that we can > > discover representative workloads. > > > > Thanks for taking the time to put this together, and start this dialogue. > > > > > > On 13/07/2020, 23:41, "Joshua McKenzie" <jmcken...@apache.org> wrote: > > > > > > > > Can you please allow comments on the doc so we can leave feedback. > > > > > > > > > > Doc is view only; figured we could keep this to the ML. > > > > > That's a feature, not a bug. > > > > Happy to chat here or on slack w/anyone. This is a complex topic so > > long-form or high bandwidth communication is a better fit than gdoc > > comments. They rapidly become unwieldy. > > > > On Mon, Jul 13, 2020 at 6:17 PM sankalp kohli < > kohlisank...@gmail.com> > > wrote: > > > > > Can you please allow comments on the doc so we can leave feedback. > > > > > > On Mon, Jul 13, 2020 at 2:16 PM Joshua McKenzie < > > jmcken...@apache.org> > > > wrote: > > > > > > > Link: > > > > > > > > > > > > > > https://docs.google.com/document/d/1ktuBWpD2NLurB9PUvmbwGgrXsgnyU58koOseZAfaFBQ/edit# > > > > > > > > > > > > Myself and a few other contributors are working with this point > of > > view > > > as > > > > our frame of where we're going to work on improving testing on > the > > > project. > > > > I figured it might be useful to foster collaboration more broadly > > in the > > > > community as well as provide people with the opportunity to > > discuss work > > > > they're doing they may not yet have had a chance to bring up or > > open > > > > source. While fallout is already open-sourced, expect the schema > > > anonymizer > > > > and some of the cassandra-diff + nosqlbench framework effort to > be > > > > open-sourced / openly worked on soon. Anyone that's interested in > > > > collaborating, that would be highly welcome. > > > > > > > > Doc is view only; figured we could keep this to the ML. > > > > > > > > Thanks. > > > > > > > > ~Josh > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > > For additional commands, e-mail: dev-h...@cassandra.apache.org > > > > >
