Re: [VOTE] KIP-25 System test improvements
Hi Gwen, Yes, it looks like it passed with a lazy majority :) I should be submitting a patch with a few initial tests today or tomorrow. Cheers, Geoff On Mon, Jun 15, 2015 at 2:10 PM, Gwen Shapira wrote: > I counted 3 binding votes (Jun, Neha, Joe) and two non-binding (Ewen and > I). > So I assume the KIP passed :) > > What's the next step? Are we expecting a patch with few initial tests soon? > > On Thu, Jun 11, 2015 at 5:41 PM, Jun Rao wrote: > > +1 > > > > Thanks, > > > > Jun > > > > On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson > > wrote: > > > >> Hi Kafka, > >> > >> After a few rounds of discussion on KIP-25, there doesn't seem to be > >> opposition, so I'd like to propose a vote. > >> > >> Thanks, > >> Geoff > >> > >> On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson > >> wrote: > >> > >> > Hi KIP-25 thread, > >> > > >> > I consolidated some of the questions from this thread and elsewhere. > >> > > >> > Q: Can we see a map of what system-test currently tests, which ones we > >> > want to replace and JIRAs for replacing? > >> > A: Initial draft here: > >> > > >> > https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests > >> > > >> > Q: Will ducktape be maintained separately as a github repo? > >> > A: Yes https://github.com/confluentinc/ducktape > >> > > >> > Q: How easy is viewing the test results and logs, how will test > output be > >> > structured? > >> > A: Hierarchical structure as outlined here: > >> > https://github.com/confluentinc/ducktape/wiki/Design-overview#output > >> > > >> > Q: Does it support code coverage? If not, how easy/ difficult would > it be > >> > to support? > >> > A: It does not, and we have no immediate plans to support this. > >> Difficulty > >> > unclear. > >> > > >> > Q: It would be nice if each Kafka version that we release will also > >> > have a separate "tests" artifact that users can download, untar and > >> easily > >> > run against a Kafka cluster of the same version. > >> > A: This seems reasonable and not too much extra work. Definitely open > to > >> > discussion on this. > >> > > >> > Q: Why not share running services across multiple tests? > >> > A: Prefer to optimize for simplicity and correctness over what might > be a > >> > questionable improvement in run-time. > >> > > >> > Q: Are regressions - in the road map? > >> > A: yes > >> > > >> > Q: Are Jepsen style tests involving network failures in the road map? > >> > A: yes > >> > > >> > Thanks much, > >> > Geoff > >> > > >> > > >> > > >> >
Re: [VOTE] KIP-25 System test improvements
I counted 3 binding votes (Jun, Neha, Joe) and two non-binding (Ewen and I). So I assume the KIP passed :) What's the next step? Are we expecting a patch with few initial tests soon? On Thu, Jun 11, 2015 at 5:41 PM, Jun Rao wrote: > +1 > > Thanks, > > Jun > > On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson > wrote: > >> Hi Kafka, >> >> After a few rounds of discussion on KIP-25, there doesn't seem to be >> opposition, so I'd like to propose a vote. >> >> Thanks, >> Geoff >> >> On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson >> wrote: >> >> > Hi KIP-25 thread, >> > >> > I consolidated some of the questions from this thread and elsewhere. >> > >> > Q: Can we see a map of what system-test currently tests, which ones we >> > want to replace and JIRAs for replacing? >> > A: Initial draft here: >> > >> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests >> > >> > Q: Will ducktape be maintained separately as a github repo? >> > A: Yes https://github.com/confluentinc/ducktape >> > >> > Q: How easy is viewing the test results and logs, how will test output be >> > structured? >> > A: Hierarchical structure as outlined here: >> > https://github.com/confluentinc/ducktape/wiki/Design-overview#output >> > >> > Q: Does it support code coverage? If not, how easy/ difficult would it be >> > to support? >> > A: It does not, and we have no immediate plans to support this. >> Difficulty >> > unclear. >> > >> > Q: It would be nice if each Kafka version that we release will also >> > have a separate "tests" artifact that users can download, untar and >> easily >> > run against a Kafka cluster of the same version. >> > A: This seems reasonable and not too much extra work. Definitely open to >> > discussion on this. >> > >> > Q: Why not share running services across multiple tests? >> > A: Prefer to optimize for simplicity and correctness over what might be a >> > questionable improvement in run-time. >> > >> > Q: Are regressions - in the road map? >> > A: yes >> > >> > Q: Are Jepsen style tests involving network failures in the road map? >> > A: yes >> > >> > Thanks much, >> > Geoff >> > >> > >> > >>
Re: [VOTE] KIP-25 System test improvements
+1 Thanks, Jun On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote: > Hi Kafka, > > After a few rounds of discussion on KIP-25, there doesn't seem to be > opposition, so I'd like to propose a vote. > > Thanks, > Geoff > > On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson > wrote: > > > Hi KIP-25 thread, > > > > I consolidated some of the questions from this thread and elsewhere. > > > > Q: Can we see a map of what system-test currently tests, which ones we > > want to replace and JIRAs for replacing? > > A: Initial draft here: > > > https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests > > > > Q: Will ducktape be maintained separately as a github repo? > > A: Yes https://github.com/confluentinc/ducktape > > > > Q: How easy is viewing the test results and logs, how will test output be > > structured? > > A: Hierarchical structure as outlined here: > > https://github.com/confluentinc/ducktape/wiki/Design-overview#output > > > > Q: Does it support code coverage? If not, how easy/ difficult would it be > > to support? > > A: It does not, and we have no immediate plans to support this. > Difficulty > > unclear. > > > > Q: It would be nice if each Kafka version that we release will also > > have a separate "tests" artifact that users can download, untar and > easily > > run against a Kafka cluster of the same version. > > A: This seems reasonable and not too much extra work. Definitely open to > > discussion on this. > > > > Q: Why not share running services across multiple tests? > > A: Prefer to optimize for simplicity and correctness over what might be a > > questionable improvement in run-time. > > > > Q: Are regressions - in the road map? > > A: yes > > > > Q: Are Jepsen style tests involving network failures in the road map? > > A: yes > > > > Thanks much, > > Geoff > > > > > > >
Re: [VOTE] KIP-25 System test improvements
+1 Thanks, Jun On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote: > Hi Kafka, > > After a few rounds of discussion on KIP-25, there doesn't seem to be > opposition, so I'd like to propose a vote. > > Thanks, > Geoff > > On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson > wrote: > > > Hi KIP-25 thread, > > > > I consolidated some of the questions from this thread and elsewhere. > > > > Q: Can we see a map of what system-test currently tests, which ones we > > want to replace and JIRAs for replacing? > > A: Initial draft here: > > > https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests > > > > Q: Will ducktape be maintained separately as a github repo? > > A: Yes https://github.com/confluentinc/ducktape > > > > Q: How easy is viewing the test results and logs, how will test output be > > structured? > > A: Hierarchical structure as outlined here: > > https://github.com/confluentinc/ducktape/wiki/Design-overview#output > > > > Q: Does it support code coverage? If not, how easy/ difficult would it be > > to support? > > A: It does not, and we have no immediate plans to support this. > Difficulty > > unclear. > > > > Q: It would be nice if each Kafka version that we release will also > > have a separate "tests" artifact that users can download, untar and > easily > > run against a Kafka cluster of the same version. > > A: This seems reasonable and not too much extra work. Definitely open to > > discussion on this. > > > > Q: Why not share running services across multiple tests? > > A: Prefer to optimize for simplicity and correctness over what might be a > > questionable improvement in run-time. > > > > Q: Are regressions - in the road map? > > A: yes > > > > Q: Are Jepsen style tests involving network failures in the road map? > > A: yes > > > > Thanks much, > > Geoff > > > > > > >
Re: [VOTE] KIP-25 System test improvements
+1 -Ewen On Thu, Jun 11, 2015 at 1:47 AM, Joe Stein wrote: > +1 > > ~ Joestein > On Jun 10, 2015 10:21 PM, "Neha Narkhede" wrote: > > > +1. Thanks Geoff! > > > > > > > > > > > > On Wed, Jun 10, 2015 at 6:20 PM -0700, "Gwen Shapira" < > > gshap...@cloudera.com> wrote: > > > > > > > > > > > > > > > > > > > > > > +1 (non-binding. Actually, since this is non-binding anyway, lets make > > it +100. I'm so so excited about having a usable testing framework) > > > > On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote: > > > Hi Kafka, > > > > > > After a few rounds of discussion on KIP-25, there doesn't seem to be > > > opposition, so I'd like to propose a vote. > > > > > > Thanks, > > > Geoff > > > > > > On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson > > > wrote: > > > > > >> Hi KIP-25 thread, > > >> > > >> I consolidated some of the questions from this thread and elsewhere. > > >> > > >> Q: Can we see a map of what system-test currently tests, which ones we > > >> want to replace and JIRAs for replacing? > > >> A: Initial draft here: > > >> > > > https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests > > >> > > >> Q: Will ducktape be maintained separately as a github repo? > > >> A: Yes https://github.com/confluentinc/ducktape > > >> > > >> Q: How easy is viewing the test results and logs, how will test output > > be > > >> structured? > > >> A: Hierarchical structure as outlined here: > > >> https://github.com/confluentinc/ducktape/wiki/Design-overview#output > > >> > > >> Q: Does it support code coverage? If not, how easy/ difficult would it > > be > > >> to support? > > >> A: It does not, and we have no immediate plans to support this. > > Difficulty > > >> unclear. > > >> > > >> Q: It would be nice if each Kafka version that we release will also > > >> have a separate "tests" artifact that users can download, untar and > > easily > > >> run against a Kafka cluster of the same version. > > >> A: This seems reasonable and not too much extra work. Definitely open > to > > >> discussion on this. > > >> > > >> Q: Why not share running services across multiple tests? > > >> A: Prefer to optimize for simplicity and correctness over what might > be > > a > > >> questionable improvement in run-time. > > >> > > >> Q: Are regressions - in the road map? > > >> A: yes > > >> > > >> Q: Are Jepsen style tests involving network failures in the road map? > > >> A: yes > > >> > > >> Thanks much, > > >> Geoff > > >> > > >> > > >> > -- Thanks, Ewen
Re: [VOTE] KIP-25 System test improvements
+1 ~ Joestein On Jun 10, 2015 10:21 PM, "Neha Narkhede" wrote: > +1. Thanks Geoff! > > > > > > On Wed, Jun 10, 2015 at 6:20 PM -0700, "Gwen Shapira" < > gshap...@cloudera.com> wrote: > > > > > > > > > > > +1 (non-binding. Actually, since this is non-binding anyway, lets make > it +100. I'm so so excited about having a usable testing framework) > > On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote: > > Hi Kafka, > > > > After a few rounds of discussion on KIP-25, there doesn't seem to be > > opposition, so I'd like to propose a vote. > > > > Thanks, > > Geoff > > > > On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson > > wrote: > > > >> Hi KIP-25 thread, > >> > >> I consolidated some of the questions from this thread and elsewhere. > >> > >> Q: Can we see a map of what system-test currently tests, which ones we > >> want to replace and JIRAs for replacing? > >> A: Initial draft here: > >> > https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests > >> > >> Q: Will ducktape be maintained separately as a github repo? > >> A: Yes https://github.com/confluentinc/ducktape > >> > >> Q: How easy is viewing the test results and logs, how will test output > be > >> structured? > >> A: Hierarchical structure as outlined here: > >> https://github.com/confluentinc/ducktape/wiki/Design-overview#output > >> > >> Q: Does it support code coverage? If not, how easy/ difficult would it > be > >> to support? > >> A: It does not, and we have no immediate plans to support this. > Difficulty > >> unclear. > >> > >> Q: It would be nice if each Kafka version that we release will also > >> have a separate "tests" artifact that users can download, untar and > easily > >> run against a Kafka cluster of the same version. > >> A: This seems reasonable and not too much extra work. Definitely open to > >> discussion on this. > >> > >> Q: Why not share running services across multiple tests? > >> A: Prefer to optimize for simplicity and correctness over what might be > a > >> questionable improvement in run-time. > >> > >> Q: Are regressions - in the road map? > >> A: yes > >> > >> Q: Are Jepsen style tests involving network failures in the road map? > >> A: yes > >> > >> Thanks much, > >> Geoff > >> > >> > >>
Re: [VOTE] KIP-25 System test improvements
+1. Thanks Geoff! On Wed, Jun 10, 2015 at 6:20 PM -0700, "Gwen Shapira" wrote: +1 (non-binding. Actually, since this is non-binding anyway, lets make it +100. I'm so so excited about having a usable testing framework) On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote: > Hi Kafka, > > After a few rounds of discussion on KIP-25, there doesn't seem to be > opposition, so I'd like to propose a vote. > > Thanks, > Geoff > > On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson > wrote: > >> Hi KIP-25 thread, >> >> I consolidated some of the questions from this thread and elsewhere. >> >> Q: Can we see a map of what system-test currently tests, which ones we >> want to replace and JIRAs for replacing? >> A: Initial draft here: >> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests >> >> Q: Will ducktape be maintained separately as a github repo? >> A: Yes https://github.com/confluentinc/ducktape >> >> Q: How easy is viewing the test results and logs, how will test output be >> structured? >> A: Hierarchical structure as outlined here: >> https://github.com/confluentinc/ducktape/wiki/Design-overview#output >> >> Q: Does it support code coverage? If not, how easy/ difficult would it be >> to support? >> A: It does not, and we have no immediate plans to support this. Difficulty >> unclear. >> >> Q: It would be nice if each Kafka version that we release will also >> have a separate "tests" artifact that users can download, untar and easily >> run against a Kafka cluster of the same version. >> A: This seems reasonable and not too much extra work. Definitely open to >> discussion on this. >> >> Q: Why not share running services across multiple tests? >> A: Prefer to optimize for simplicity and correctness over what might be a >> questionable improvement in run-time. >> >> Q: Are regressions - in the road map? >> A: yes >> >> Q: Are Jepsen style tests involving network failures in the road map? >> A: yes >> >> Thanks much, >> Geoff >> >> >>
Re: [VOTE] KIP-25 System test improvements
+1 (non-binding. Actually, since this is non-binding anyway, lets make it +100. I'm so so excited about having a usable testing framework) On Wed, Jun 10, 2015 at 6:10 PM, Geoffrey Anderson wrote: > Hi Kafka, > > After a few rounds of discussion on KIP-25, there doesn't seem to be > opposition, so I'd like to propose a vote. > > Thanks, > Geoff > > On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson > wrote: > >> Hi KIP-25 thread, >> >> I consolidated some of the questions from this thread and elsewhere. >> >> Q: Can we see a map of what system-test currently tests, which ones we >> want to replace and JIRAs for replacing? >> A: Initial draft here: >> https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests >> >> Q: Will ducktape be maintained separately as a github repo? >> A: Yes https://github.com/confluentinc/ducktape >> >> Q: How easy is viewing the test results and logs, how will test output be >> structured? >> A: Hierarchical structure as outlined here: >> https://github.com/confluentinc/ducktape/wiki/Design-overview#output >> >> Q: Does it support code coverage? If not, how easy/ difficult would it be >> to support? >> A: It does not, and we have no immediate plans to support this. Difficulty >> unclear. >> >> Q: It would be nice if each Kafka version that we release will also >> have a separate "tests" artifact that users can download, untar and easily >> run against a Kafka cluster of the same version. >> A: This seems reasonable and not too much extra work. Definitely open to >> discussion on this. >> >> Q: Why not share running services across multiple tests? >> A: Prefer to optimize for simplicity and correctness over what might be a >> questionable improvement in run-time. >> >> Q: Are regressions - in the road map? >> A: yes >> >> Q: Are Jepsen style tests involving network failures in the road map? >> A: yes >> >> Thanks much, >> Geoff >> >> >>
[VOTE] KIP-25 System test improvements
Hi Kafka, After a few rounds of discussion on KIP-25, there doesn't seem to be opposition, so I'd like to propose a vote. Thanks, Geoff On Mon, Jun 8, 2015 at 10:56 PM, Geoffrey Anderson wrote: > Hi KIP-25 thread, > > I consolidated some of the questions from this thread and elsewhere. > > Q: Can we see a map of what system-test currently tests, which ones we > want to replace and JIRAs for replacing? > A: Initial draft here: > https://cwiki.apache.org/confluence/display/KAFKA/Roadmap+-+port+existing+system+tests > > Q: Will ducktape be maintained separately as a github repo? > A: Yes https://github.com/confluentinc/ducktape > > Q: How easy is viewing the test results and logs, how will test output be > structured? > A: Hierarchical structure as outlined here: > https://github.com/confluentinc/ducktape/wiki/Design-overview#output > > Q: Does it support code coverage? If not, how easy/ difficult would it be > to support? > A: It does not, and we have no immediate plans to support this. Difficulty > unclear. > > Q: It would be nice if each Kafka version that we release will also > have a separate "tests" artifact that users can download, untar and easily > run against a Kafka cluster of the same version. > A: This seems reasonable and not too much extra work. Definitely open to > discussion on this. > > Q: Why not share running services across multiple tests? > A: Prefer to optimize for simplicity and correctness over what might be a > questionable improvement in run-time. > > Q: Are regressions - in the road map? > A: yes > > Q: Are Jepsen style tests involving network failures in the road map? > A: yes > > Thanks much, > Geoff > > >