Gwen, Just for clarification, you were suggesting we should or should not include MM improvement in 0.8.3? I personally would prefer it (KAFKA-1650 and KAFKA-1997) to go into 0.8.3.
I see Joe has made a pass over the tickets and mark them 0.8.3. We can probably do another pass and consider adding: 1) Purgatory improvement (KAFKA-1989). 2) Compression improvement (KAFKA-527). 3) Some unit test failures (KAFKA-1501, I think we are pretty close in getting it fixed). 4) any other tickets? Guozhang On Wed, Mar 11, 2015 at 9:40 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > With regard to mm, I was kind of assuming just based on the amount of work > that that would go in for sure, but yeah I agree it is important. > > -Jay > > On Wed, Mar 11, 2015 at 9:39 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > > > What I was trying to say was let's do a real release whenever either > > consumer or authn is done whichever happens first (or both if they can > > happen close together)--not sure which is more likely to slip. > > > > WRT the beta thing I think the question for people is whether the beta > > period was helpful or not in getting a more stable release? We could > either > > do a beta release again or we could just do a normal release and call the > > consumer feature "experimental" or whatever...basically something to get > it > > in peoples hands before it is supposed to work perfectly and never change > > again. > > > > -Jay > > > > > > On Wed, Mar 11, 2015 at 9:27 PM, Gwen Shapira <gshap...@cloudera.com> > > wrote: > > > >> So basically you are suggesting - lets do a beta release whenever we > >> feel the new consumer is done? > >> > >> This can definitely work. > >> > >> I'd prefer holding for MM improvements too. IMO, its not just more > >> improvements like flush() and compression optimization. > >> Current MirrorMaker can lose data, which makes it pretty useless for > >> its job. We hear lots of requests for robust MM from our customers, so > >> I can imagine its pretty important to the Kafka community (unless I > >> have a completely skewed sample). > >> > >> Gwen > >> > >> > >> > >> On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > >> > Yeah the real question is always what will we block on? > >> > > >> > I don't think we should try to hold back smaller changes. In this > >> bucket I > >> > would include most things you described: mm improvements, replica > >> > assignment tool improvements, flush, purgatory improvements, > compression > >> > optimization, etc. Likely these will all get done in time as well as > >> many > >> > things that kind of pop up from users but probably aren't worth doing > a > >> > release for on their own. If one of them slips that fine. I also don't > >> > think we should try to hold back work that is done if it isn't on a > >> list. > >> > > >> > I would consider either SSL+SASL or the consumer worthy of a release > on > >> its > >> > own. If they finish close to the same time that is great. We can maybe > >> just > >> > assess as these evolve where the other one is at and make a call > >> whether it > >> > will be one or both? > >> > > >> > -Jay > >> > > >> > On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira <gshap...@cloudera.com> > >> wrote: > >> > > >> >> If we are going in terms of features, I can see the following > features > >> >> getting in in the next month or two: > >> >> > >> >> * New consumer > >> >> * Improved Mirror Maker (I've seen tons of interest) > >> >> * Centralized admin requests (aka KIP-4) > >> >> * Nicer replica-reassignment tool > >> >> * SSL (and perhaps also SASL)? > >> >> > >> >> I think this collection will make a nice release. Perhaps we can cap > >> >> it there and focus (as a community) on getting these in, we can have > a > >> >> release without too much scope creep in the not-very-distant-future? > >> >> Even just 3 out of these 5 will still make a nice incremental > >> >> improvement. > >> >> > >> >> Gwen > >> >> > >> >> > >> >> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <jay.kr...@gmail.com> > >> wrote: > >> >> > Yeah I'd be in favor of a quicker, smaller release but I think as > >> long as > >> >> > we have these big things in flight we should probably keep the > >> release > >> >> > criteria feature-based rather than time-based, though (e.g. "when X > >> >> works" > >> >> > not "every other month). > >> >> > > >> >> > Ideally the next release would have at least a "beta" version of > the > >> new > >> >> > consumer. I think having a new hunk of code like that available but > >> >> marked > >> >> > as "beta" is maybe a good way to go, as it gets it into peoples > >> hands for > >> >> > testing. This way we can declare the API not fully locked down > until > >> the > >> >> > final release too, since mostly users only look at stuff after we > >> release > >> >> > it. Maybe we can try to construct a schedule around this? > >> >> > > >> >> > -Jay > >> >> > > >> >> > > >> >> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <joe.st...@stealth.ly> > >> wrote: > >> >> > > >> >> >> There hasn't been any public discussion about the 0.8.3 release > >> plan. > >> >> >> > >> >> >> There seems to be a lot of work in flight, work with patches and > >> review > >> >> >> that could/should get committed but now just pending KIPS, work > >> without > >> >> KIP > >> >> >> but that is in trunk already (e.g. the new Consumer) that would be > >> the > >> >> the > >> >> >> release but missing the KIP for the release... > >> >> >> > >> >> >> What does this mean for the 0.8.3 release? What are we trying to > >> get out > >> >> >> and when? > >> >> >> > >> >> >> Also looking at > >> >> >> > >> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan > >> >> >> there > >> >> >> seems to be things we are getting earlier (which is great of > >> course) so > >> >> are > >> >> >> we going to try to up the version and go with 0.9.0? > >> >> >> > >> >> >> 0.8.2.0 ended up getting very bloated and that delayed it much > >> longer > >> >> than > >> >> >> we had originally communicated to the community and want to make > >> sure we > >> >> >> take that feedback from the community and try to improve upon it. > >> >> >> > >> >> >> Thanks! > >> >> >> > >> >> >> ~ Joe Stein > >> >> >> - - - - - - - - - - - - - - - - - > >> >> >> > >> >> >> http://www.stealth.ly > >> >> >> - - - - - - - - - - - - - - - - - > >> >> >> > >> >> > >> > > > > > -- -- Guozhang
