So basically you are suggesting - lets do a beta release whenever we feel the new consumer is done?
This can definitely work. I'd prefer holding for MM improvements too. IMO, its not just more improvements like flush() and compression optimization. Current MirrorMaker can lose data, which makes it pretty useless for its job. We hear lots of requests for robust MM from our customers, so I can imagine its pretty important to the Kafka community (unless I have a completely skewed sample). Gwen On Wed, Mar 11, 2015 at 9:18 PM, Jay Kreps <jay.kr...@gmail.com> wrote: > Yeah the real question is always what will we block on? > > I don't think we should try to hold back smaller changes. In this bucket I > would include most things you described: mm improvements, replica > assignment tool improvements, flush, purgatory improvements, compression > optimization, etc. Likely these will all get done in time as well as many > things that kind of pop up from users but probably aren't worth doing a > release for on their own. If one of them slips that fine. I also don't > think we should try to hold back work that is done if it isn't on a list. > > I would consider either SSL+SASL or the consumer worthy of a release on its > own. If they finish close to the same time that is great. We can maybe just > assess as these evolve where the other one is at and make a call whether it > will be one or both? > > -Jay > > On Wed, Mar 11, 2015 at 8:51 PM, Gwen Shapira <gshap...@cloudera.com> wrote: > >> If we are going in terms of features, I can see the following features >> getting in in the next month or two: >> >> * New consumer >> * Improved Mirror Maker (I've seen tons of interest) >> * Centralized admin requests (aka KIP-4) >> * Nicer replica-reassignment tool >> * SSL (and perhaps also SASL)? >> >> I think this collection will make a nice release. Perhaps we can cap >> it there and focus (as a community) on getting these in, we can have a >> release without too much scope creep in the not-very-distant-future? >> Even just 3 out of these 5 will still make a nice incremental >> improvement. >> >> Gwen >> >> >> On Wed, Mar 11, 2015 at 8:29 PM, Jay Kreps <jay.kr...@gmail.com> wrote: >> > Yeah I'd be in favor of a quicker, smaller release but I think as long as >> > we have these big things in flight we should probably keep the release >> > criteria feature-based rather than time-based, though (e.g. "when X >> works" >> > not "every other month). >> > >> > Ideally the next release would have at least a "beta" version of the new >> > consumer. I think having a new hunk of code like that available but >> marked >> > as "beta" is maybe a good way to go, as it gets it into peoples hands for >> > testing. This way we can declare the API not fully locked down until the >> > final release too, since mostly users only look at stuff after we release >> > it. Maybe we can try to construct a schedule around this? >> > >> > -Jay >> > >> > >> > On Wed, Mar 11, 2015 at 7:55 PM, Joe Stein <joe.st...@stealth.ly> wrote: >> > >> >> There hasn't been any public discussion about the 0.8.3 release plan. >> >> >> >> There seems to be a lot of work in flight, work with patches and review >> >> that could/should get committed but now just pending KIPS, work without >> KIP >> >> but that is in trunk already (e.g. the new Consumer) that would be the >> the >> >> release but missing the KIP for the release... >> >> >> >> What does this mean for the 0.8.3 release? What are we trying to get out >> >> and when? >> >> >> >> Also looking at >> >> https://cwiki.apache.org/confluence/display/KAFKA/Future+release+plan >> >> there >> >> seems to be things we are getting earlier (which is great of course) so >> are >> >> we going to try to up the version and go with 0.9.0? >> >> >> >> 0.8.2.0 ended up getting very bloated and that delayed it much longer >> than >> >> we had originally communicated to the community and want to make sure we >> >> take that feedback from the community and try to improve upon it. >> >> >> >> Thanks! >> >> >> >> ~ Joe Stein >> >> - - - - - - - - - - - - - - - - - >> >> >> >> http://www.stealth.ly >> >> - - - - - - - - - - - - - - - - - >> >> >>
