Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Jun Rao]

Ok, thanks everyone for the feedback. Based on the feedback, I'd recommend
that we do an 0.8.2.2 release including just the two snappy bug fixes.

Jun

On Tue, Aug 18, 2015 at 12:59 PM, Gwen Shapira  wrote:

> Any objections if I leave KAFKA-2114 (setting min.insync.replicas default)
> out?
>
> The test code is using changes that were done after 0.8.2.x cut-off, which
> makes it difficult to cherry-pick.
>
> Gwen
>
>
>
> On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira  wrote:
>
> > Jun,
> >
> > KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
> > cherrypick just this fix.
> > I suggest leaving this out since there is a 0.8.2.x workaround in the
> JIRA.
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao  wrote:
> >
> >> Gwen,
> >>
> >> Thanks for putting the list together.
> >>
> >> I'd recommend that we exclude the following:
> >> KAFKA-1702: This is for the old producer and is only a problem if there
> >> are
> >> some unexpected exceptions (e.g. UnknownClass).
> >> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
> >> KAFKA-1724: The patch there is never committed since the fix is included
> >> in
> >> another jira (a much larger patch).
> >> KAFKA-2241: This doesn't seem be a common problem. It only happens when
> >> the
> >> fetch request blocks on the broker for an extended period of time, which
> >> should be rare.
> >>
> >> I'd also recommend that we include the following:
> >> KAFKA-2147: This impacts the memory size of the purgatory and a number
> of
> >> people have experienced that. The fix is small and has been tested in
> >> production usage. It hasn't been committed though since the issue is
> >> already fixed in trunk and we weren't planning for an 0.8.2.2 release
> >> then.
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira 
> wrote:
> >>
> >> > Thanks for creating a list, Grant!
> >> >
> >> > I placed it on the wiki with a quick evaluation of the content and
> >> whether
> >> > it should be in 0.8.2.2:
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >> >
> >> > I'm attempting to only cherrypick fixes that are both important for
> >> large
> >> > number of users (or very critical to some users) and very safe (mostly
> >> > judged by the size of the change, but not only)
> >> >
> >> > If your favorite bugfix is missing from the list, or is there but
> marked
> >> > "No", please let us know (in this thread) what we are missing and why
> >> it is
> >> > both important and safe.
> >> > Also, if I accidentally included something you consider unsafe, speak
> >> up!
> >> >
> >> > Gwen
> >> >
> >> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
> >> wrote:
> >> >
> >> > > +dev
> >> > >
> >> > > Adding dev list back in. Somehow it got dropped.
> >> > >
> >> > >
> >> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke 
> >> > wrote:
> >> > >
> >> > > > Below is a list of candidate bug fix jiras marked fixed for
> 0.8.3. I
> >> > > don't
> >> > > > suspect all of these will (or should) make it into the release but
> >> this
> >> > > > should be a relatively complete list to work from:
> >> > > >
> >> > > >- KAFKA-2114  >:
> >> > > Unable
> >> > > >to change min.insync.replicas default
> >> > > >- KAFKA-1702  >:
> >> > > >Messages silently Lost by producer
> >> > > >- KAFKA-2012  >:
> >> > > >Broker should automatically handle corrupt index files
> >> > > >- KAFKA-2406  >:
> >> > ISR
> >> > > >propagation should be throttled to avoid overwhelming
> controller.
> >> > > >- KAFKA-2336  >:
> >> > > >Changing offsets.topic.num.partitions after the offset topic is
> >> > > created
> >> > > >breaks consumer group partition assignment
> >> > > >- KAFKA-2337  >:
> >> > > Verify
> >> > > >that metric names will not collide when creating new topics
> >> > > >- KAFKA-2393  >:
> >> > > >Correctly Handle InvalidTopicException in
> >> > KafkaApis.getTopicMetadata()
> >> > > >- KAFKA-2189  >:
> >> > > Snappy
> >> > > >compression of message batches less efficient in 0.8.2.1
> >> > > >- KAFKA-2308  >:
> >> > New
> >> > > >producer + Snappy face un-compression errors after broker
> restart
> >> > > >- KAFKA-2042  >:
> >> > New
> >> > > >producer metadata update always get all topics.
> >> > > >- KAFKA-1367  >:
> >> > > Broker
> >> > > >topic

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Jun Rao]

Yes, I think it's fine to leave out both KAFKA-2147 and KAFKA-2114.

Thanks,

Jun

On Tue, Aug 18, 2015 at 12:59 PM, Gwen Shapira  wrote:

> Any objections if I leave KAFKA-2114 (setting min.insync.replicas default)
> out?
>
> The test code is using changes that were done after 0.8.2.x cut-off, which
> makes it difficult to cherry-pick.
>
> Gwen
>
>
>
> On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira  wrote:
>
> > Jun,
> >
> > KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
> > cherrypick just this fix.
> > I suggest leaving this out since there is a 0.8.2.x workaround in the
> JIRA.
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao  wrote:
> >
> >> Gwen,
> >>
> >> Thanks for putting the list together.
> >>
> >> I'd recommend that we exclude the following:
> >> KAFKA-1702: This is for the old producer and is only a problem if there
> >> are
> >> some unexpected exceptions (e.g. UnknownClass).
> >> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
> >> KAFKA-1724: The patch there is never committed since the fix is included
> >> in
> >> another jira (a much larger patch).
> >> KAFKA-2241: This doesn't seem be a common problem. It only happens when
> >> the
> >> fetch request blocks on the broker for an extended period of time, which
> >> should be rare.
> >>
> >> I'd also recommend that we include the following:
> >> KAFKA-2147: This impacts the memory size of the purgatory and a number
> of
> >> people have experienced that. The fix is small and has been tested in
> >> production usage. It hasn't been committed though since the issue is
> >> already fixed in trunk and we weren't planning for an 0.8.2.2 release
> >> then.
> >>
> >> Thanks,
> >>
> >> Jun
> >>
> >> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira 
> wrote:
> >>
> >> > Thanks for creating a list, Grant!
> >> >
> >> > I placed it on the wiki with a quick evaluation of the content and
> >> whether
> >> > it should be in 0.8.2.2:
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >> >
> >> > I'm attempting to only cherrypick fixes that are both important for
> >> large
> >> > number of users (or very critical to some users) and very safe (mostly
> >> > judged by the size of the change, but not only)
> >> >
> >> > If your favorite bugfix is missing from the list, or is there but
> marked
> >> > "No", please let us know (in this thread) what we are missing and why
> >> it is
> >> > both important and safe.
> >> > Also, if I accidentally included something you consider unsafe, speak
> >> up!
> >> >
> >> > Gwen
> >> >
> >> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
> >> wrote:
> >> >
> >> > > +dev
> >> > >
> >> > > Adding dev list back in. Somehow it got dropped.
> >> > >
> >> > >
> >> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke 
> >> > wrote:
> >> > >
> >> > > > Below is a list of candidate bug fix jiras marked fixed for
> 0.8.3. I
> >> > > don't
> >> > > > suspect all of these will (or should) make it into the release but
> >> this
> >> > > > should be a relatively complete list to work from:
> >> > > >
> >> > > >- KAFKA-2114  >:
> >> > > Unable
> >> > > >to change min.insync.replicas default
> >> > > >- KAFKA-1702  >:
> >> > > >Messages silently Lost by producer
> >> > > >- KAFKA-2012  >:
> >> > > >Broker should automatically handle corrupt index files
> >> > > >- KAFKA-2406  >:
> >> > ISR
> >> > > >propagation should be throttled to avoid overwhelming
> controller.
> >> > > >- KAFKA-2336  >:
> >> > > >Changing offsets.topic.num.partitions after the offset topic is
> >> > > created
> >> > > >breaks consumer group partition assignment
> >> > > >- KAFKA-2337  >:
> >> > > Verify
> >> > > >that metric names will not collide when creating new topics
> >> > > >- KAFKA-2393  >:
> >> > > >Correctly Handle InvalidTopicException in
> >> > KafkaApis.getTopicMetadata()
> >> > > >- KAFKA-2189  >:
> >> > > Snappy
> >> > > >compression of message batches less efficient in 0.8.2.1
> >> > > >- KAFKA-2308  >:
> >> > New
> >> > > >producer + Snappy face un-compression errors after broker
> restart
> >> > > >- KAFKA-2042  >:
> >> > New
> >> > > >producer metadata update always get all topics.
> >> > > >- KAFKA-1367  >:
> >> > > Broker
> >> > > >topic metadata not kept in sync with ZooKeeper
> >> > > >- KAFKA-972 <

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Gwen Shapira]

Any objections if I leave KAFKA-2114 (setting min.insync.replicas default)
out?

The test code is using changes that were done after 0.8.2.x cut-off, which
makes it difficult to cherry-pick.

Gwen



On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira  wrote:

> Jun,
>
> KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
> cherrypick just this fix.
> I suggest leaving this out since there is a 0.8.2.x workaround in the JIRA.
>
> Gwen
>
> On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao  wrote:
>
>> Gwen,
>>
>> Thanks for putting the list together.
>>
>> I'd recommend that we exclude the following:
>> KAFKA-1702: This is for the old producer and is only a problem if there
>> are
>> some unexpected exceptions (e.g. UnknownClass).
>> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
>> KAFKA-1724: The patch there is never committed since the fix is included
>> in
>> another jira (a much larger patch).
>> KAFKA-2241: This doesn't seem be a common problem. It only happens when
>> the
>> fetch request blocks on the broker for an extended period of time, which
>> should be rare.
>>
>> I'd also recommend that we include the following:
>> KAFKA-2147: This impacts the memory size of the purgatory and a number of
>> people have experienced that. The fix is small and has been tested in
>> production usage. It hasn't been committed though since the issue is
>> already fixed in trunk and we weren't planning for an 0.8.2.2 release
>> then.
>>
>> Thanks,
>>
>> Jun
>>
>> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira  wrote:
>>
>> > Thanks for creating a list, Grant!
>> >
>> > I placed it on the wiki with a quick evaluation of the content and
>> whether
>> > it should be in 0.8.2.2:
>> >
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>> >
>> > I'm attempting to only cherrypick fixes that are both important for
>> large
>> > number of users (or very critical to some users) and very safe (mostly
>> > judged by the size of the change, but not only)
>> >
>> > If your favorite bugfix is missing from the list, or is there but marked
>> > "No", please let us know (in this thread) what we are missing and why
>> it is
>> > both important and safe.
>> > Also, if I accidentally included something you consider unsafe, speak
>> up!
>> >
>> > Gwen
>> >
>> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
>> wrote:
>> >
>> > > +dev
>> > >
>> > > Adding dev list back in. Somehow it got dropped.
>> > >
>> > >
>> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke 
>> > wrote:
>> > >
>> > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
>> > > don't
>> > > > suspect all of these will (or should) make it into the release but
>> this
>> > > > should be a relatively complete list to work from:
>> > > >
>> > > >- KAFKA-2114 :
>> > > Unable
>> > > >to change min.insync.replicas default
>> > > >- KAFKA-1702 :
>> > > >Messages silently Lost by producer
>> > > >- KAFKA-2012 :
>> > > >Broker should automatically handle corrupt index files
>> > > >- KAFKA-2406 :
>> > ISR
>> > > >propagation should be throttled to avoid overwhelming controller.
>> > > >- KAFKA-2336 :
>> > > >Changing offsets.topic.num.partitions after the offset topic is
>> > > created
>> > > >breaks consumer group partition assignment
>> > > >- KAFKA-2337 :
>> > > Verify
>> > > >that metric names will not collide when creating new topics
>> > > >- KAFKA-2393 :
>> > > >Correctly Handle InvalidTopicException in
>> > KafkaApis.getTopicMetadata()
>> > > >- KAFKA-2189 :
>> > > Snappy
>> > > >compression of message batches less efficient in 0.8.2.1
>> > > >- KAFKA-2308 :
>> > New
>> > > >producer + Snappy face un-compression errors after broker restart
>> > > >- KAFKA-2042 :
>> > New
>> > > >producer metadata update always get all topics.
>> > > >- KAFKA-1367 :
>> > > Broker
>> > > >topic metadata not kept in sync with ZooKeeper
>> > > >- KAFKA-972 :
>> > > MetadataRequest
>> > > >returns stale list of brokers
>> > > >- KAFKA-1867 :
>> > > liveBroker
>> > > >list not updated on a cluster with no topics
>> > > >- KAFKA-1650 :
>> > > Mirror
>> > > >Maker could lose data on unclean shutdown.
>> > > >- KAFKA

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Gwen Shapira]

Jun,

KAFKA-2147 doesn't seem to have a commit associated with it, so I can't
cherrypick just this fix.
I suggest leaving this out since there is a 0.8.2.x workaround in the JIRA.

Gwen

On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao  wrote:

> Gwen,
>
> Thanks for putting the list together.
>
> I'd recommend that we exclude the following:
> KAFKA-1702: This is for the old producer and is only a problem if there are
> some unexpected exceptions (e.g. UnknownClass).
> KAFKA-2336: Most people don't change offsets.topic.num.partitions.
> KAFKA-1724: The patch there is never committed since the fix is included in
> another jira (a much larger patch).
> KAFKA-2241: This doesn't seem be a common problem. It only happens when the
> fetch request blocks on the broker for an extended period of time, which
> should be rare.
>
> I'd also recommend that we include the following:
> KAFKA-2147: This impacts the memory size of the purgatory and a number of
> people have experienced that. The fix is small and has been tested in
> production usage. It hasn't been committed though since the issue is
> already fixed in trunk and we weren't planning for an 0.8.2.2 release then.
>
> Thanks,
>
> Jun
>
> On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira  wrote:
>
> > Thanks for creating a list, Grant!
> >
> > I placed it on the wiki with a quick evaluation of the content and
> whether
> > it should be in 0.8.2.2:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >
> > I'm attempting to only cherrypick fixes that are both important for large
> > number of users (or very critical to some users) and very safe (mostly
> > judged by the size of the change, but not only)
> >
> > If your favorite bugfix is missing from the list, or is there but marked
> > "No", please let us know (in this thread) what we are missing and why it
> is
> > both important and safe.
> > Also, if I accidentally included something you consider unsafe, speak up!
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
> wrote:
> >
> > > +dev
> > >
> > > Adding dev list back in. Somehow it got dropped.
> > >
> > >
> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke 
> > wrote:
> > >
> > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > > don't
> > > > suspect all of these will (or should) make it into the release but
> this
> > > > should be a relatively complete list to work from:
> > > >
> > > >- KAFKA-2114 :
> > > Unable
> > > >to change min.insync.replicas default
> > > >- KAFKA-1702 :
> > > >Messages silently Lost by producer
> > > >- KAFKA-2012 :
> > > >Broker should automatically handle corrupt index files
> > > >- KAFKA-2406 :
> > ISR
> > > >propagation should be throttled to avoid overwhelming controller.
> > > >- KAFKA-2336 :
> > > >Changing offsets.topic.num.partitions after the offset topic is
> > > created
> > > >breaks consumer group partition assignment
> > > >- KAFKA-2337 :
> > > Verify
> > > >that metric names will not collide when creating new topics
> > > >- KAFKA-2393 :
> > > >Correctly Handle InvalidTopicException in
> > KafkaApis.getTopicMetadata()
> > > >- KAFKA-2189 :
> > > Snappy
> > > >compression of message batches less efficient in 0.8.2.1
> > > >- KAFKA-2308 :
> > New
> > > >producer + Snappy face un-compression errors after broker restart
> > > >- KAFKA-2042 :
> > New
> > > >producer metadata update always get all topics.
> > > >- KAFKA-1367 :
> > > Broker
> > > >topic metadata not kept in sync with ZooKeeper
> > > >- KAFKA-972 :
> > > MetadataRequest
> > > >returns stale list of brokers
> > > >- KAFKA-1867 :
> > > liveBroker
> > > >list not updated on a cluster with no topics
> > > >- KAFKA-1650 :
> > > Mirror
> > > >Maker could lose data on unclean shutdown.
> > > >- KAFKA-2009 :
> > Fix
> > > >UncheckedOffset.removeOffset synchronization and trace logging
> issue
> > > in
> > > >mirror maker
> > > >- KAFKA-2407 :
> > Only
> > > >create a log directory when it will be used
> > > >- KAFKA-2327 :

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Andrew Otto]

I agree: keep it simple :)  The latest stable version of Kafka right now has a 
critical bug in it.  Fixing that would be good enough.  0.8.2.2 should probably 
just a maintenance/bugfix release.


> On Aug 18, 2015, at 14:29, Edward Ribeiro  wrote:
> 
> I sort of follow Daniel Nelson on this issue: cut a 0.8.2.2 but not include
> much thing besides Snappy fixes. I mean, include a couple of additional
> critical bug fixes, if really urgent, and that's it.
> 
> On Tue, Aug 18, 2015 at 3:25 PM, Daniel Nelson 
> wrote:
> 
>> I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think
>> that it needs to include anything other than the fix for Snappy that kicked
>> off this discussion in the first place. If there are additional critical
>> issues that can be included without delaying the process, I see no downside.
>> 
>>> On Aug 18, 2015, at 10:54 AM, Neha Narkhede  wrote:
>>> 
>>> How about looking at the scope for the 0.8.3 release first before we cut
>>> yet another point release off of 0.8.2.2? Each release includes some
>>> overhead and if there is a larger release in the works, it might be worth
>>> working on getting that. My take is that the 2 things the community has
>>> been waiting for is SSL support and the new consumer and we have been
>>> promising to get 0.8.3 with both those features for several months now.
>>> 
>>> Looking at the progress on both, it seems we are very close to getting
>> both
>>> those checked in and it looks like we should get there in another 5-6
>>> weeks. Furthermore, both of these features are large and I anticipate us
>>> receiving feedback and bugs that will require a couple of point releases
>> on
>>> top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
>>> get the community to use the newly released features, gather feedback and
>>> do point releases incorporating that feedback and iterate on it.
>>> 
>>> We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
>>> to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
>>> makes sense.
>>> 
>>> Thanks,
>>> Neha
>>> 
>>> On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira 
>> wrote:
>>> 
 Thanks Jun.
 
 I updated the list with your suggestions.
 If anyone feels we are missing a critical patch for 0.8.2.2, please
>> speak
 up.
 
 Gwen
 
 On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao  wrote:
 
> Hi, Grant,
> 
> I took a look at that list. None of those is really critical as you
>> said.
> So, I'd suggest that we not include those to minimize the scope of the
> release.
> 
> Thanks,
> 
> Jun
> 
> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke 
 wrote:
> 
>> Thanks Gwen.
>> 
>> I updated a few small things on the wiki page.
>> 
>> Below is a list of jiras I think could also be marked as included. All
 of
>> these, though not super critical, seem like fairly small and low risk
>> changes that help avoid potentially confusing issues or errors for
 users.
>> 
>> KAFKA-2012
>> KAFKA-972
>> KAFKA-2337 & KAFKA-2393
>> KAFKA-1867
>> KAFKA-2407
>> KAFKA-2234
>> KAFKA-1866
>> KAFKA-2345 & KAFKA-2355
>> 
>> thoughts?
>> 
>> Thank you,
>> Grant
>> 
>> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira 
 wrote:
>> 
>>> Thanks for creating a list, Grant!
>>> 
>>> I placed it on the wiki with a quick evaluation of the content and
>> whether
>>> it should be in 0.8.2.2:
>>> 
>>> 
>> 
> 
 
>> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>>> 
>>> I'm attempting to only cherrypick fixes that are both important for
> large
>>> number of users (or very critical to some users) and very safe
 (mostly
>>> judged by the size of the change, but not only)
>>> 
>>> If your favorite bugfix is missing from the list, or is there but
> marked
>>> "No", please let us know (in this thread) what we are missing and why
> it
>> is
>>> both important and safe.
>>> Also, if I accidentally included something you consider unsafe, speak
> up!
>>> 
>>> Gwen
>>> 
>>> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
>> wrote:
>>> 
 +dev
 
 Adding dev list back in. Somehow it got dropped.
 
 
 On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke  
>>> wrote:
 
> Below is a list of candidate bug fix jiras marked fixed for
 0.8.3.
> I
 don't
> suspect all of these will (or should) make it into the release
 but
>> this
> should be a relatively complete list to work from:
> 
>  - KAFKA-2114 <
 https://issues.apache.org/jira/browse/KAFKA-2114
>> :
 Unable
>  to change min.insync.replicas default
>  - KAFKA-1702 <
 https:

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Edward Ribeiro]

I sort of follow Daniel Nelson on this issue: cut a 0.8.2.2 but not include
much thing besides Snappy fixes. I mean, include a couple of additional
critical bug fixes, if really urgent, and that's it.

On Tue, Aug 18, 2015 at 3:25 PM, Daniel Nelson 
wrote:

> I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think
> that it needs to include anything other than the fix for Snappy that kicked
> off this discussion in the first place. If there are additional critical
> issues that can be included without delaying the process, I see no downside.
>
> > On Aug 18, 2015, at 10:54 AM, Neha Narkhede  wrote:
> >
> > How about looking at the scope for the 0.8.3 release first before we cut
> > yet another point release off of 0.8.2.2? Each release includes some
> > overhead and if there is a larger release in the works, it might be worth
> > working on getting that. My take is that the 2 things the community has
> > been waiting for is SSL support and the new consumer and we have been
> > promising to get 0.8.3 with both those features for several months now.
> >
> > Looking at the progress on both, it seems we are very close to getting
> both
> > those checked in and it looks like we should get there in another 5-6
> > weeks. Furthermore, both of these features are large and I anticipate us
> > receiving feedback and bugs that will require a couple of point releases
> on
> > top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
> > get the community to use the newly released features, gather feedback and
> > do point releases incorporating that feedback and iterate on it.
> >
> > We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
> > to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
> > makes sense.
> >
> > Thanks,
> > Neha
> >
> > On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira 
> wrote:
> >
> >> Thanks Jun.
> >>
> >> I updated the list with your suggestions.
> >> If anyone feels we are missing a critical patch for 0.8.2.2, please
> speak
> >> up.
> >>
> >> Gwen
> >>
> >> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao  wrote:
> >>
> >>> Hi, Grant,
> >>>
> >>> I took a look at that list. None of those is really critical as you
> said.
> >>> So, I'd suggest that we not include those to minimize the scope of the
> >>> release.
> >>>
> >>> Thanks,
> >>>
> >>> Jun
> >>>
> >>> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke 
> >> wrote:
> >>>
>  Thanks Gwen.
> 
>  I updated a few small things on the wiki page.
> 
>  Below is a list of jiras I think could also be marked as included. All
> >> of
>  these, though not super critical, seem like fairly small and low risk
>  changes that help avoid potentially confusing issues or errors for
> >> users.
> 
>  KAFKA-2012
>  KAFKA-972
>  KAFKA-2337 & KAFKA-2393
>  KAFKA-1867
>  KAFKA-2407
>  KAFKA-2234
>  KAFKA-1866
>  KAFKA-2345 & KAFKA-2355
> 
>  thoughts?
> 
>  Thank you,
>  Grant
> 
>  On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira 
> >> wrote:
> 
> > Thanks for creating a list, Grant!
> >
> > I placed it on the wiki with a quick evaluation of the content and
>  whether
> > it should be in 0.8.2.2:
> >
> >
> 
> >>>
> >>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >
> > I'm attempting to only cherrypick fixes that are both important for
> >>> large
> > number of users (or very critical to some users) and very safe
> >> (mostly
> > judged by the size of the change, but not only)
> >
> > If your favorite bugfix is missing from the list, or is there but
> >>> marked
> > "No", please let us know (in this thread) what we are missing and why
> >>> it
>  is
> > both important and safe.
> > Also, if I accidentally included something you consider unsafe, speak
> >>> up!
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
>  wrote:
> >
> >> +dev
> >>
> >> Adding dev list back in. Somehow it got dropped.
> >>
> >>
> >> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke  >>>
> > wrote:
> >>
> >>> Below is a list of candidate bug fix jiras marked fixed for
> >> 0.8.3.
> >>> I
> >> don't
> >>> suspect all of these will (or should) make it into the release
> >> but
>  this
> >>> should be a relatively complete list to work from:
> >>>
> >>>   - KAFKA-2114 <
> >> https://issues.apache.org/jira/browse/KAFKA-2114
>  :
> >> Unable
> >>>   to change min.insync.replicas default
> >>>   - KAFKA-1702 <
> >> https://issues.apache.org/jira/browse/KAFKA-1702
>  :
> >>>   Messages silently Lost by producer
> >>>   - KAFKA-2012 <
> >> https://issues.apache.org/jira/browse/KAFKA-2012
>  :
> >>>   Broker should automatically handle corrupt index files
> >>>   - KAFKA-2406 <
> >> https://issues.apache.org/j

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Daniel Nelson]

I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think that it 
needs to include anything other than the fix for Snappy that kicked off this 
discussion in the first place. If there are additional critical issues that can 
be included without delaying the process, I see no downside.

> On Aug 18, 2015, at 10:54 AM, Neha Narkhede  wrote:
> 
> How about looking at the scope for the 0.8.3 release first before we cut
> yet another point release off of 0.8.2.2? Each release includes some
> overhead and if there is a larger release in the works, it might be worth
> working on getting that. My take is that the 2 things the community has
> been waiting for is SSL support and the new consumer and we have been
> promising to get 0.8.3 with both those features for several months now.
> 
> Looking at the progress on both, it seems we are very close to getting both
> those checked in and it looks like we should get there in another 5-6
> weeks. Furthermore, both of these features are large and I anticipate us
> receiving feedback and bugs that will require a couple of point releases on
> top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
> get the community to use the newly released features, gather feedback and
> do point releases incorporating that feedback and iterate on it.
> 
> We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
> to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
> makes sense.
> 
> Thanks,
> Neha
> 
> On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira  wrote:
> 
>> Thanks Jun.
>> 
>> I updated the list with your suggestions.
>> If anyone feels we are missing a critical patch for 0.8.2.2, please speak
>> up.
>> 
>> Gwen
>> 
>> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao  wrote:
>> 
>>> Hi, Grant,
>>> 
>>> I took a look at that list. None of those is really critical as you said.
>>> So, I'd suggest that we not include those to minimize the scope of the
>>> release.
>>> 
>>> Thanks,
>>> 
>>> Jun
>>> 
>>> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke 
>> wrote:
>>> 
 Thanks Gwen.
 
 I updated a few small things on the wiki page.
 
 Below is a list of jiras I think could also be marked as included. All
>> of
 these, though not super critical, seem like fairly small and low risk
 changes that help avoid potentially confusing issues or errors for
>> users.
 
 KAFKA-2012
 KAFKA-972
 KAFKA-2337 & KAFKA-2393
 KAFKA-1867
 KAFKA-2407
 KAFKA-2234
 KAFKA-1866
 KAFKA-2345 & KAFKA-2355
 
 thoughts?
 
 Thank you,
 Grant
 
 On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira 
>> wrote:
 
> Thanks for creating a list, Grant!
> 
> I placed it on the wiki with a quick evaluation of the content and
 whether
> it should be in 0.8.2.2:
> 
> 
 
>>> 
>> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> 
> I'm attempting to only cherrypick fixes that are both important for
>>> large
> number of users (or very critical to some users) and very safe
>> (mostly
> judged by the size of the change, but not only)
> 
> If your favorite bugfix is missing from the list, or is there but
>>> marked
> "No", please let us know (in this thread) what we are missing and why
>>> it
 is
> both important and safe.
> Also, if I accidentally included something you consider unsafe, speak
>>> up!
> 
> Gwen
> 
> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
 wrote:
> 
>> +dev
>> 
>> Adding dev list back in. Somehow it got dropped.
>> 
>> 
>> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke >> 
> wrote:
>> 
>>> Below is a list of candidate bug fix jiras marked fixed for
>> 0.8.3.
>>> I
>> don't
>>> suspect all of these will (or should) make it into the release
>> but
 this
>>> should be a relatively complete list to work from:
>>> 
>>>   - KAFKA-2114 <
>> https://issues.apache.org/jira/browse/KAFKA-2114
 :
>> Unable
>>>   to change min.insync.replicas default
>>>   - KAFKA-1702 <
>> https://issues.apache.org/jira/browse/KAFKA-1702
 :
>>>   Messages silently Lost by producer
>>>   - KAFKA-2012 <
>> https://issues.apache.org/jira/browse/KAFKA-2012
 :
>>>   Broker should automatically handle corrupt index files
>>>   - KAFKA-2406 <
>> https://issues.apache.org/jira/browse/KAFKA-2406
 :
> ISR
>>>   propagation should be throttled to avoid overwhelming
>>> controller.
>>>   - KAFKA-2336 <
>> https://issues.apache.org/jira/browse/KAFKA-2336
 :
>>>   Changing offsets.topic.num.partitions after the offset topic
>> is
>> created
>>>   breaks consumer group partition assignment
>>>   - KAFKA-2337 <
>> https://issues.apache.org/jira/browse/KAFKA-2337
 :
>> Verify
>>>   that metric names will not collide when creating new topics
>>

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Gwen Shapira]

I completely agree that community focus on 0.8.3 is important and don't
want to take away from that effort.

That said, not everyone will upgrade to 0.8.3 immediately (unfortunately),
and there is still value in releasing bugfixes on previous releases.

I'm keeping the scope very small, to make the new version safe and make the
release easier and less distracting for us. If this interferes with 0.8.3,
we will adjust course to make sure it doesn't.

Gwen

On Tue, Aug 18, 2015 at 10:54 AM, Neha Narkhede  wrote:

> How about looking at the scope for the 0.8.3 release first before we cut
> yet another point release off of 0.8.2.2? Each release includes some
> overhead and if there is a larger release in the works, it might be worth
> working on getting that. My take is that the 2 things the community has
> been waiting for is SSL support and the new consumer and we have been
> promising to get 0.8.3 with both those features for several months now.
>
> Looking at the progress on both, it seems we are very close to getting both
> those checked in and it looks like we should get there in another 5-6
> weeks. Furthermore, both of these features are large and I anticipate us
> receiving feedback and bugs that will require a couple of point releases on
> top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
> get the community to use the newly released features, gather feedback and
> do point releases incorporating that feedback and iterate on it.
>
> We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
> to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
> makes sense.
>
> Thanks,
> Neha
>
> On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira  wrote:
>
> > Thanks Jun.
> >
> > I updated the list with your suggestions.
> > If anyone feels we are missing a critical patch for 0.8.2.2, please speak
> > up.
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao  wrote:
> >
> > > Hi, Grant,
> > >
> > > I took a look at that list. None of those is really critical as you
> said.
> > > So, I'd suggest that we not include those to minimize the scope of the
> > > release.
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> > > On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke 
> > wrote:
> > >
> > > > Thanks Gwen.
> > > >
> > > > I updated a few small things on the wiki page.
> > > >
> > > > Below is a list of jiras I think could also be marked as included.
> All
> > of
> > > > these, though not super critical, seem like fairly small and low risk
> > > > changes that help avoid potentially confusing issues or errors for
> > users.
> > > >
> > > > KAFKA-2012
> > > > KAFKA-972
> > > > KAFKA-2337 & KAFKA-2393
> > > > KAFKA-1867
> > > > KAFKA-2407
> > > > KAFKA-2234
> > > > KAFKA-1866
> > > > KAFKA-2345 & KAFKA-2355
> > > >
> > > > thoughts?
> > > >
> > > > Thank you,
> > > > Grant
> > > >
> > > > On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira 
> > wrote:
> > > >
> > > > > Thanks for creating a list, Grant!
> > > > >
> > > > > I placed it on the wiki with a quick evaluation of the content and
> > > > whether
> > > > > it should be in 0.8.2.2:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> > > > >
> > > > > I'm attempting to only cherrypick fixes that are both important for
> > > large
> > > > > number of users (or very critical to some users) and very safe
> > (mostly
> > > > > judged by the size of the change, but not only)
> > > > >
> > > > > If your favorite bugfix is missing from the list, or is there but
> > > marked
> > > > > "No", please let us know (in this thread) what we are missing and
> why
> > > it
> > > > is
> > > > > both important and safe.
> > > > > Also, if I accidentally included something you consider unsafe,
> speak
> > > up!
> > > > >
> > > > > Gwen
> > > > >
> > > > > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
> > > > wrote:
> > > > >
> > > > > > +dev
> > > > > >
> > > > > > Adding dev list back in. Somehow it got dropped.
> > > > > >
> > > > > >
> > > > > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <
> ghe...@cloudera.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Below is a list of candidate bug fix jiras marked fixed for
> > 0.8.3.
> > > I
> > > > > > don't
> > > > > > > suspect all of these will (or should) make it into the release
> > but
> > > > this
> > > > > > > should be a relatively complete list to work from:
> > > > > > >
> > > > > > >- KAFKA-2114 <
> > https://issues.apache.org/jira/browse/KAFKA-2114
> > > >:
> > > > > > Unable
> > > > > > >to change min.insync.replicas default
> > > > > > >- KAFKA-1702 <
> > https://issues.apache.org/jira/browse/KAFKA-1702
> > > >:
> > > > > > >Messages silently Lost by producer
> > > > > > >- KAFKA-2012 <
> > https://issues.apache.org/jira/browse/KAFKA-2012
> > > >:
> > > > > > >Broker should automatically handle corrupt index files
> > > > > > >- KAFKA-2406 <
> > https://issues.apache.org/jira/bro

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Neha Narkhede]

How about looking at the scope for the 0.8.3 release first before we cut
yet another point release off of 0.8.2.2? Each release includes some
overhead and if there is a larger release in the works, it might be worth
working on getting that. My take is that the 2 things the community has
been waiting for is SSL support and the new consumer and we have been
promising to get 0.8.3 with both those features for several months now.

Looking at the progress on both, it seems we are very close to getting both
those checked in and it looks like we should get there in another 5-6
weeks. Furthermore, both of these features are large and I anticipate us
receiving feedback and bugs that will require a couple of point releases on
top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and
get the community to use the newly released features, gather feedback and
do point releases incorporating that feedback and iterate on it.

We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us
to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still
makes sense.

Thanks,
Neha

On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira  wrote:

> Thanks Jun.
>
> I updated the list with your suggestions.
> If anyone feels we are missing a critical patch for 0.8.2.2, please speak
> up.
>
> Gwen
>
> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao  wrote:
>
> > Hi, Grant,
> >
> > I took a look at that list. None of those is really critical as you said.
> > So, I'd suggest that we not include those to minimize the scope of the
> > release.
> >
> > Thanks,
> >
> > Jun
> >
> > On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke 
> wrote:
> >
> > > Thanks Gwen.
> > >
> > > I updated a few small things on the wiki page.
> > >
> > > Below is a list of jiras I think could also be marked as included. All
> of
> > > these, though not super critical, seem like fairly small and low risk
> > > changes that help avoid potentially confusing issues or errors for
> users.
> > >
> > > KAFKA-2012
> > > KAFKA-972
> > > KAFKA-2337 & KAFKA-2393
> > > KAFKA-1867
> > > KAFKA-2407
> > > KAFKA-2234
> > > KAFKA-1866
> > > KAFKA-2345 & KAFKA-2355
> > >
> > > thoughts?
> > >
> > > Thank you,
> > > Grant
> > >
> > > On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira 
> wrote:
> > >
> > > > Thanks for creating a list, Grant!
> > > >
> > > > I placed it on the wiki with a quick evaluation of the content and
> > > whether
> > > > it should be in 0.8.2.2:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> > > >
> > > > I'm attempting to only cherrypick fixes that are both important for
> > large
> > > > number of users (or very critical to some users) and very safe
> (mostly
> > > > judged by the size of the change, but not only)
> > > >
> > > > If your favorite bugfix is missing from the list, or is there but
> > marked
> > > > "No", please let us know (in this thread) what we are missing and why
> > it
> > > is
> > > > both important and safe.
> > > > Also, if I accidentally included something you consider unsafe, speak
> > up!
> > > >
> > > > Gwen
> > > >
> > > > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
> > > wrote:
> > > >
> > > > > +dev
> > > > >
> > > > > Adding dev list back in. Somehow it got dropped.
> > > > >
> > > > >
> > > > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke  >
> > > > wrote:
> > > > >
> > > > > > Below is a list of candidate bug fix jiras marked fixed for
> 0.8.3.
> > I
> > > > > don't
> > > > > > suspect all of these will (or should) make it into the release
> but
> > > this
> > > > > > should be a relatively complete list to work from:
> > > > > >
> > > > > >- KAFKA-2114 <
> https://issues.apache.org/jira/browse/KAFKA-2114
> > >:
> > > > > Unable
> > > > > >to change min.insync.replicas default
> > > > > >- KAFKA-1702 <
> https://issues.apache.org/jira/browse/KAFKA-1702
> > >:
> > > > > >Messages silently Lost by producer
> > > > > >- KAFKA-2012 <
> https://issues.apache.org/jira/browse/KAFKA-2012
> > >:
> > > > > >Broker should automatically handle corrupt index files
> > > > > >- KAFKA-2406 <
> https://issues.apache.org/jira/browse/KAFKA-2406
> > >:
> > > > ISR
> > > > > >propagation should be throttled to avoid overwhelming
> > controller.
> > > > > >- KAFKA-2336 <
> https://issues.apache.org/jira/browse/KAFKA-2336
> > >:
> > > > > >Changing offsets.topic.num.partitions after the offset topic
> is
> > > > > created
> > > > > >breaks consumer group partition assignment
> > > > > >- KAFKA-2337 <
> https://issues.apache.org/jira/browse/KAFKA-2337
> > >:
> > > > > Verify
> > > > > >that metric names will not collide when creating new topics
> > > > > >- KAFKA-2393 <
> https://issues.apache.org/jira/browse/KAFKA-2393
> > >:
> > > > > >Correctly Handle InvalidTopicException in
> > > > KafkaApis.getTopicMetadata()
> > > > > >- KAFKA-2189 <
> https://issues.apache.org/jira/browse/KAFKA-2189
> > >:
> > > > 

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Gwen Shapira]

Thanks Jun.

I updated the list with your suggestions.
If anyone feels we are missing a critical patch for 0.8.2.2, please speak
up.

Gwen

On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao  wrote:

> Hi, Grant,
>
> I took a look at that list. None of those is really critical as you said.
> So, I'd suggest that we not include those to minimize the scope of the
> release.
>
> Thanks,
>
> Jun
>
> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke  wrote:
>
> > Thanks Gwen.
> >
> > I updated a few small things on the wiki page.
> >
> > Below is a list of jiras I think could also be marked as included. All of
> > these, though not super critical, seem like fairly small and low risk
> > changes that help avoid potentially confusing issues or errors for users.
> >
> > KAFKA-2012
> > KAFKA-972
> > KAFKA-2337 & KAFKA-2393
> > KAFKA-1867
> > KAFKA-2407
> > KAFKA-2234
> > KAFKA-1866
> > KAFKA-2345 & KAFKA-2355
> >
> > thoughts?
> >
> > Thank you,
> > Grant
> >
> > On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira  wrote:
> >
> > > Thanks for creating a list, Grant!
> > >
> > > I placed it on the wiki with a quick evaluation of the content and
> > whether
> > > it should be in 0.8.2.2:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> > >
> > > I'm attempting to only cherrypick fixes that are both important for
> large
> > > number of users (or very critical to some users) and very safe (mostly
> > > judged by the size of the change, but not only)
> > >
> > > If your favorite bugfix is missing from the list, or is there but
> marked
> > > "No", please let us know (in this thread) what we are missing and why
> it
> > is
> > > both important and safe.
> > > Also, if I accidentally included something you consider unsafe, speak
> up!
> > >
> > > Gwen
> > >
> > > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
> > wrote:
> > >
> > > > +dev
> > > >
> > > > Adding dev list back in. Somehow it got dropped.
> > > >
> > > >
> > > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke 
> > > wrote:
> > > >
> > > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3.
> I
> > > > don't
> > > > > suspect all of these will (or should) make it into the release but
> > this
> > > > > should be a relatively complete list to work from:
> > > > >
> > > > >- KAFKA-2114  >:
> > > > Unable
> > > > >to change min.insync.replicas default
> > > > >- KAFKA-1702  >:
> > > > >Messages silently Lost by producer
> > > > >- KAFKA-2012  >:
> > > > >Broker should automatically handle corrupt index files
> > > > >- KAFKA-2406  >:
> > > ISR
> > > > >propagation should be throttled to avoid overwhelming
> controller.
> > > > >- KAFKA-2336  >:
> > > > >Changing offsets.topic.num.partitions after the offset topic is
> > > > created
> > > > >breaks consumer group partition assignment
> > > > >- KAFKA-2337  >:
> > > > Verify
> > > > >that metric names will not collide when creating new topics
> > > > >- KAFKA-2393  >:
> > > > >Correctly Handle InvalidTopicException in
> > > KafkaApis.getTopicMetadata()
> > > > >- KAFKA-2189  >:
> > > > Snappy
> > > > >compression of message batches less efficient in 0.8.2.1
> > > > >- KAFKA-2308  >:
> > > New
> > > > >producer + Snappy face un-compression errors after broker
> restart
> > > > >- KAFKA-2042  >:
> > > New
> > > > >producer metadata update always get all topics.
> > > > >- KAFKA-1367  >:
> > > > Broker
> > > > >topic metadata not kept in sync with ZooKeeper
> > > > >- KAFKA-972 :
> > > > MetadataRequest
> > > > >returns stale list of brokers
> > > > >- KAFKA-1867  >:
> > > > liveBroker
> > > > >list not updated on a cluster with no topics
> > > > >- KAFKA-1650  >:
> > > > Mirror
> > > > >Maker could lose data on unclean shutdown.
> > > > >- KAFKA-2009  >:
> > > Fix
> > > > >UncheckedOffset.removeOffset synchronization and trace logging
> > issue
> > > > in
> > > > >mirror maker
> > > > >- KAFKA-2407  >:
> > > Only
> > > > >create a log directory when it will be used
> > > > >- KAFKA-2327 

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Jun Rao]

Hi, Grant,

I took a look at that list. None of those is really critical as you said.
So, I'd suggest that we not include those to minimize the scope of the
release.

Thanks,

Jun

On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke  wrote:

> Thanks Gwen.
>
> I updated a few small things on the wiki page.
>
> Below is a list of jiras I think could also be marked as included. All of
> these, though not super critical, seem like fairly small and low risk
> changes that help avoid potentially confusing issues or errors for users.
>
> KAFKA-2012
> KAFKA-972
> KAFKA-2337 & KAFKA-2393
> KAFKA-1867
> KAFKA-2407
> KAFKA-2234
> KAFKA-1866
> KAFKA-2345 & KAFKA-2355
>
> thoughts?
>
> Thank you,
> Grant
>
> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira  wrote:
>
> > Thanks for creating a list, Grant!
> >
> > I placed it on the wiki with a quick evaluation of the content and
> whether
> > it should be in 0.8.2.2:
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
> >
> > I'm attempting to only cherrypick fixes that are both important for large
> > number of users (or very critical to some users) and very safe (mostly
> > judged by the size of the change, but not only)
> >
> > If your favorite bugfix is missing from the list, or is there but marked
> > "No", please let us know (in this thread) what we are missing and why it
> is
> > both important and safe.
> > Also, if I accidentally included something you consider unsafe, speak up!
> >
> > Gwen
> >
> > On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke 
> wrote:
> >
> > > +dev
> > >
> > > Adding dev list back in. Somehow it got dropped.
> > >
> > >
> > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke 
> > wrote:
> > >
> > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > > don't
> > > > suspect all of these will (or should) make it into the release but
> this
> > > > should be a relatively complete list to work from:
> > > >
> > > >- KAFKA-2114 :
> > > Unable
> > > >to change min.insync.replicas default
> > > >- KAFKA-1702 :
> > > >Messages silently Lost by producer
> > > >- KAFKA-2012 :
> > > >Broker should automatically handle corrupt index files
> > > >- KAFKA-2406 :
> > ISR
> > > >propagation should be throttled to avoid overwhelming controller.
> > > >- KAFKA-2336 :
> > > >Changing offsets.topic.num.partitions after the offset topic is
> > > created
> > > >breaks consumer group partition assignment
> > > >- KAFKA-2337 :
> > > Verify
> > > >that metric names will not collide when creating new topics
> > > >- KAFKA-2393 :
> > > >Correctly Handle InvalidTopicException in
> > KafkaApis.getTopicMetadata()
> > > >- KAFKA-2189 :
> > > Snappy
> > > >compression of message batches less efficient in 0.8.2.1
> > > >- KAFKA-2308 :
> > New
> > > >producer + Snappy face un-compression errors after broker restart
> > > >- KAFKA-2042 :
> > New
> > > >producer metadata update always get all topics.
> > > >- KAFKA-1367 :
> > > Broker
> > > >topic metadata not kept in sync with ZooKeeper
> > > >- KAFKA-972 :
> > > MetadataRequest
> > > >returns stale list of brokers
> > > >- KAFKA-1867 :
> > > liveBroker
> > > >list not updated on a cluster with no topics
> > > >- KAFKA-1650 :
> > > Mirror
> > > >Maker could lose data on unclean shutdown.
> > > >- KAFKA-2009 :
> > Fix
> > > >UncheckedOffset.removeOffset synchronization and trace logging
> issue
> > > in
> > > >mirror maker
> > > >- KAFKA-2407 :
> > Only
> > > >create a log directory when it will be used
> > > >- KAFKA-2327 :
> > > >broker doesn't start if config defines advertised.host but not
> > > >advertised.port
> > > >- KAFKA-1788: producer record can stay in RecordAccumulator
> forever
> > if
> > > >leader is no available
> > > >- KAFKA-2234 :
> > > >Partition reassignment of a nonexistent topic prevents future
> > > reassignments
> > > >- KAFKA-2096 :
> > > >Enable keepalive

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Jun Rao]

Gwen,

Thanks for putting the list together.

I'd recommend that we exclude the following:
KAFKA-1702: This is for the old producer and is only a problem if there are
some unexpected exceptions (e.g. UnknownClass).
KAFKA-2336: Most people don't change offsets.topic.num.partitions.
KAFKA-1724: The patch there is never committed since the fix is included in
another jira (a much larger patch).
KAFKA-2241: This doesn't seem be a common problem. It only happens when the
fetch request blocks on the broker for an extended period of time, which
should be rare.

I'd also recommend that we include the following:
KAFKA-2147: This impacts the memory size of the purgatory and a number of
people have experienced that. The fix is small and has been tested in
production usage. It hasn't been committed though since the issue is
already fixed in trunk and we weren't planning for an 0.8.2.2 release then.

Thanks,

Jun

On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira  wrote:

> Thanks for creating a list, Grant!
>
> I placed it on the wiki with a quick evaluation of the content and whether
> it should be in 0.8.2.2:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>
> I'm attempting to only cherrypick fixes that are both important for large
> number of users (or very critical to some users) and very safe (mostly
> judged by the size of the change, but not only)
>
> If your favorite bugfix is missing from the list, or is there but marked
> "No", please let us know (in this thread) what we are missing and why it is
> both important and safe.
> Also, if I accidentally included something you consider unsafe, speak up!
>
> Gwen
>
> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke  wrote:
>
> > +dev
> >
> > Adding dev list back in. Somehow it got dropped.
> >
> >
> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke 
> wrote:
> >
> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > don't
> > > suspect all of these will (or should) make it into the release but this
> > > should be a relatively complete list to work from:
> > >
> > >- KAFKA-2114 :
> > Unable
> > >to change min.insync.replicas default
> > >- KAFKA-1702 :
> > >Messages silently Lost by producer
> > >- KAFKA-2012 :
> > >Broker should automatically handle corrupt index files
> > >- KAFKA-2406 :
> ISR
> > >propagation should be throttled to avoid overwhelming controller.
> > >- KAFKA-2336 :
> > >Changing offsets.topic.num.partitions after the offset topic is
> > created
> > >breaks consumer group partition assignment
> > >- KAFKA-2337 :
> > Verify
> > >that metric names will not collide when creating new topics
> > >- KAFKA-2393 :
> > >Correctly Handle InvalidTopicException in
> KafkaApis.getTopicMetadata()
> > >- KAFKA-2189 :
> > Snappy
> > >compression of message batches less efficient in 0.8.2.1
> > >- KAFKA-2308 :
> New
> > >producer + Snappy face un-compression errors after broker restart
> > >- KAFKA-2042 :
> New
> > >producer metadata update always get all topics.
> > >- KAFKA-1367 :
> > Broker
> > >topic metadata not kept in sync with ZooKeeper
> > >- KAFKA-972 :
> > MetadataRequest
> > >returns stale list of brokers
> > >- KAFKA-1867 :
> > liveBroker
> > >list not updated on a cluster with no topics
> > >- KAFKA-1650 :
> > Mirror
> > >Maker could lose data on unclean shutdown.
> > >- KAFKA-2009 :
> Fix
> > >UncheckedOffset.removeOffset synchronization and trace logging issue
> > in
> > >mirror maker
> > >- KAFKA-2407 :
> Only
> > >create a log directory when it will be used
> > >- KAFKA-2327 :
> > >broker doesn't start if config defines advertised.host but not
> > >advertised.port
> > >- KAFKA-1788: producer record can stay in RecordAccumulator forever
> if
> > >leader is no available
> > >- KAFKA-2234 :
> > >Partition reassignment of a nonexistent topic prevents future
> > reassignments
> > >- KAFKA-2096 :
> > >En

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Grant Henke]

Thanks Gwen.

I updated a few small things on the wiki page.

Below is a list of jiras I think could also be marked as included. All of
these, though not super critical, seem like fairly small and low risk
changes that help avoid potentially confusing issues or errors for users.

KAFKA-2012
KAFKA-972
KAFKA-2337 & KAFKA-2393
KAFKA-1867
KAFKA-2407
KAFKA-2234
KAFKA-1866
KAFKA-2345 & KAFKA-2355

thoughts?

Thank you,
Grant

On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira  wrote:

> Thanks for creating a list, Grant!
>
> I placed it on the wiki with a quick evaluation of the content and whether
> it should be in 0.8.2.2:
>
> https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2
>
> I'm attempting to only cherrypick fixes that are both important for large
> number of users (or very critical to some users) and very safe (mostly
> judged by the size of the change, but not only)
>
> If your favorite bugfix is missing from the list, or is there but marked
> "No", please let us know (in this thread) what we are missing and why it is
> both important and safe.
> Also, if I accidentally included something you consider unsafe, speak up!
>
> Gwen
>
> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke  wrote:
>
> > +dev
> >
> > Adding dev list back in. Somehow it got dropped.
> >
> >
> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke 
> wrote:
> >
> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > don't
> > > suspect all of these will (or should) make it into the release but this
> > > should be a relatively complete list to work from:
> > >
> > >- KAFKA-2114 :
> > Unable
> > >to change min.insync.replicas default
> > >- KAFKA-1702 :
> > >Messages silently Lost by producer
> > >- KAFKA-2012 :
> > >Broker should automatically handle corrupt index files
> > >- KAFKA-2406 :
> ISR
> > >propagation should be throttled to avoid overwhelming controller.
> > >- KAFKA-2336 :
> > >Changing offsets.topic.num.partitions after the offset topic is
> > created
> > >breaks consumer group partition assignment
> > >- KAFKA-2337 :
> > Verify
> > >that metric names will not collide when creating new topics
> > >- KAFKA-2393 :
> > >Correctly Handle InvalidTopicException in
> KafkaApis.getTopicMetadata()
> > >- KAFKA-2189 :
> > Snappy
> > >compression of message batches less efficient in 0.8.2.1
> > >- KAFKA-2308 :
> New
> > >producer + Snappy face un-compression errors after broker restart
> > >- KAFKA-2042 :
> New
> > >producer metadata update always get all topics.
> > >- KAFKA-1367 :
> > Broker
> > >topic metadata not kept in sync with ZooKeeper
> > >- KAFKA-972 :
> > MetadataRequest
> > >returns stale list of brokers
> > >- KAFKA-1867 :
> > liveBroker
> > >list not updated on a cluster with no topics
> > >- KAFKA-1650 :
> > Mirror
> > >Maker could lose data on unclean shutdown.
> > >- KAFKA-2009 :
> Fix
> > >UncheckedOffset.removeOffset synchronization and trace logging issue
> > in
> > >mirror maker
> > >- KAFKA-2407 :
> Only
> > >create a log directory when it will be used
> > >- KAFKA-2327 :
> > >broker doesn't start if config defines advertised.host but not
> > >advertised.port
> > >- KAFKA-1788: producer record can stay in RecordAccumulator forever
> if
> > >leader is no available
> > >- KAFKA-2234 :
> > >Partition reassignment of a nonexistent topic prevents future
> > reassignments
> > >- KAFKA-2096 :
> > >Enable keepalive socket option for broker to prevent socket leak
> > >- KAFKA-1057 :
> Trim
> > >whitespaces from user specified configs
> > >- KAFKA-1641 :
> Log
> > >cleaner exits if last cleaned offset is lower than earliest offset
> > >- KAFKA-1648 :
> > Round
> > >robin consumer balance throws an NPE when there 

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Gwen Shapira]

Thanks for creating a list, Grant!

I placed it on the wiki with a quick evaluation of the content and whether
it should be in 0.8.2.2:
https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2

I'm attempting to only cherrypick fixes that are both important for large
number of users (or very critical to some users) and very safe (mostly
judged by the size of the change, but not only)

If your favorite bugfix is missing from the list, or is there but marked
"No", please let us know (in this thread) what we are missing and why it is
both important and safe.
Also, if I accidentally included something you consider unsafe, speak up!

Gwen

On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke  wrote:

> +dev
>
> Adding dev list back in. Somehow it got dropped.
>
>
> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke  wrote:
>
> > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> don't
> > suspect all of these will (or should) make it into the release but this
> > should be a relatively complete list to work from:
> >
> >- KAFKA-2114 :
> Unable
> >to change min.insync.replicas default
> >- KAFKA-1702 :
> >Messages silently Lost by producer
> >- KAFKA-2012 :
> >Broker should automatically handle corrupt index files
> >- KAFKA-2406 : ISR
> >propagation should be throttled to avoid overwhelming controller.
> >- KAFKA-2336 :
> >Changing offsets.topic.num.partitions after the offset topic is
> created
> >breaks consumer group partition assignment
> >- KAFKA-2337 :
> Verify
> >that metric names will not collide when creating new topics
> >- KAFKA-2393 :
> >Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
> >- KAFKA-2189 :
> Snappy
> >compression of message batches less efficient in 0.8.2.1
> >- KAFKA-2308 : New
> >producer + Snappy face un-compression errors after broker restart
> >- KAFKA-2042 : New
> >producer metadata update always get all topics.
> >- KAFKA-1367 :
> Broker
> >topic metadata not kept in sync with ZooKeeper
> >- KAFKA-972 :
> MetadataRequest
> >returns stale list of brokers
> >- KAFKA-1867 :
> liveBroker
> >list not updated on a cluster with no topics
> >- KAFKA-1650 :
> Mirror
> >Maker could lose data on unclean shutdown.
> >- KAFKA-2009 : Fix
> >UncheckedOffset.removeOffset synchronization and trace logging issue
> in
> >mirror maker
> >- KAFKA-2407 : Only
> >create a log directory when it will be used
> >- KAFKA-2327 :
> >broker doesn't start if config defines advertised.host but not
> >advertised.port
> >- KAFKA-1788: producer record can stay in RecordAccumulator forever if
> >leader is no available
> >- KAFKA-2234 :
> >Partition reassignment of a nonexistent topic prevents future
> reassignments
> >- KAFKA-2096 :
> >Enable keepalive socket option for broker to prevent socket leak
> >- KAFKA-1057 : Trim
> >whitespaces from user specified configs
> >- KAFKA-1641 : Log
> >cleaner exits if last cleaned offset is lower than earliest offset
> >- KAFKA-1648 :
> Round
> >robin consumer balance throws an NPE when there are no topics
> >- KAFKA-1724 :
> >Errors after reboot in single node setup
> >- KAFKA-1758 :
> >corrupt recovery file prevents startup
> >- KAFKA-1866 :
> >LogStartOffset gauge throws exceptions after log.delete()
> >- KAFKA-1883 :
> NullPointerException
> >in RequestSendThread
> >- KAFKA-1896 :
> >Record size funcition of record in mirror maker hit NPE when the
> message
> >valu

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Ashish Singh]

+1 for 0.8.2.2 release with fixes for KAFKA-2189, 2114 and 2308.

On Mon, Aug 17, 2015 at 9:16 AM, Flavio Junqueira  wrote:

> It is pretty typical that Apache projects have a release manager for each
> release:
>
> http://www.apache.org/dev/release-publishing.html <
> http://www.apache.org/dev/release-publishing.html>
>
> It doesn't have to be the same person every time, though, not even for the
> same branch.
>
> -Flavio
>
>
> > On 17 Aug 2015, at 05:01, Gwen Shapira  wrote:
> >
> > BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> > manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> > branch and once enough bug fixes happened (or if sufficiently critical
> > fixes happened) to roll out a new maintenance release (with every 3 month
> > as a reasonable bugfix release target).
>
>


-- 

Regards,
Ashish

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Flavio Junqueira]

It is pretty typical that Apache projects have a release manager for each 
release:

http://www.apache.org/dev/release-publishing.html 


It doesn't have to be the same person every time, though, not even for the same 
branch.

-Flavio


> On 17 Aug 2015, at 05:01, Gwen Shapira  wrote:
> 
> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> branch and once enough bug fixes happened (or if sufficiently critical
> fixes happened) to roll out a new maintenance release (with every 3 month
> as a reasonable bugfix release target).

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Gwen Shapira]

but +1 for 0.8.2 patch that marks the new consumer API as unstable (or
unimplemented ;)


On Mon, Aug 17, 2015 at 9:12 AM, Gwen Shapira  wrote:

> The network refactoring portion was not tested well enough yet for me to
> feel comfortable pushing it into a bugfix release. The new purgatory and
> MirrorMaker changes are also pretty big.
>
> The whole goal of bugfix releases is to make sure it is more stable than
> the previous releases.
>
> On Mon, Aug 17, 2015 at 8:54 AM, Stevo Slavić  wrote:
>
>> Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk,
>> with new consumer API appropriately annotated/documented as unstable?
>>
>> On Mon, Aug 17, 2015, 17:17 Grant Henke  wrote:
>>
>> > +dev
>> >
>> > Adding dev list back in. Somehow it got dropped.
>> >
>> >
>> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke 
>> wrote:
>> >
>> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
>> > don't
>> > > suspect all of these will (or should) make it into the release but
>> this
>> > > should be a relatively complete list to work from:
>> > >
>> > >- KAFKA-2114 :
>> > Unable
>> > >to change min.insync.replicas default
>> > >- KAFKA-1702 :
>> > >Messages silently Lost by producer
>> > >- KAFKA-2012 :
>> > >Broker should automatically handle corrupt index files
>> > >- KAFKA-2406 :
>> ISR
>> > >propagation should be throttled to avoid overwhelming controller.
>> > >- KAFKA-2336 :
>> > >Changing offsets.topic.num.partitions after the offset topic is
>> > created
>> > >breaks consumer group partition assignment
>> > >- KAFKA-2337 :
>> > Verify
>> > >that metric names will not collide when creating new topics
>> > >- KAFKA-2393 :
>> > >Correctly Handle InvalidTopicException in
>> KafkaApis.getTopicMetadata()
>> > >- KAFKA-2189 :
>> > Snappy
>> > >compression of message batches less efficient in 0.8.2.1
>> > >- KAFKA-2308 :
>> New
>> > >producer + Snappy face un-compression errors after broker restart
>> > >- KAFKA-2042 :
>> New
>> > >producer metadata update always get all topics.
>> > >- KAFKA-1367 :
>> > Broker
>> > >topic metadata not kept in sync with ZooKeeper
>> > >- KAFKA-972 :
>> > MetadataRequest
>> > >returns stale list of brokers
>> > >- KAFKA-1867 :
>> > liveBroker
>> > >list not updated on a cluster with no topics
>> > >- KAFKA-1650 :
>> > Mirror
>> > >Maker could lose data on unclean shutdown.
>> > >- KAFKA-2009 :
>> Fix
>> > >UncheckedOffset.removeOffset synchronization and trace logging
>> issue
>> > in
>> > >mirror maker
>> > >- KAFKA-2407 :
>> Only
>> > >create a log directory when it will be used
>> > >- KAFKA-2327 :
>> > >broker doesn't start if config defines advertised.host but not
>> > >advertised.port
>> > >- KAFKA-1788: producer record can stay in RecordAccumulator
>> forever if
>> > >leader is no available
>> > >- KAFKA-2234 :
>> > >Partition reassignment of a nonexistent topic prevents future
>> > reassignments
>> > >- KAFKA-2096 :
>> > >Enable keepalive socket option for broker to prevent socket leak
>> > >- KAFKA-1057 :
>> Trim
>> > >whitespaces from user specified configs
>> > >- KAFKA-1641 :
>> Log
>> > >cleaner exits if last cleaned offset is lower than earliest offset
>> > >- KAFKA-1648 :
>> > Round
>> > >robin consumer balance throws an NPE when there are no topics
>> > >- KAFKA-1724 :
>> > >Errors after reboot in single node setup
>> > >- KAFKA-1758 :
>> > >corrupt recovery file prevents startup
>> > >- KAFKA-1866 :
>> > >LogStartOffset gauge throws exceptions after log.delete()
>> > >- KAFKA-1883 

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Gwen Shapira]

The network refactoring portion was not tested well enough yet for me to
feel comfortable pushing it into a bugfix release. The new purgatory and
MirrorMaker changes are also pretty big.

The whole goal of bugfix releases is to make sure it is more stable than
the previous releases.

On Mon, Aug 17, 2015 at 8:54 AM, Stevo Slavić  wrote:

> Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk,
> with new consumer API appropriately annotated/documented as unstable?
>
> On Mon, Aug 17, 2015, 17:17 Grant Henke  wrote:
>
> > +dev
> >
> > Adding dev list back in. Somehow it got dropped.
> >
> >
> > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke 
> wrote:
> >
> > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> > don't
> > > suspect all of these will (or should) make it into the release but this
> > > should be a relatively complete list to work from:
> > >
> > >- KAFKA-2114 :
> > Unable
> > >to change min.insync.replicas default
> > >- KAFKA-1702 :
> > >Messages silently Lost by producer
> > >- KAFKA-2012 :
> > >Broker should automatically handle corrupt index files
> > >- KAFKA-2406 :
> ISR
> > >propagation should be throttled to avoid overwhelming controller.
> > >- KAFKA-2336 :
> > >Changing offsets.topic.num.partitions after the offset topic is
> > created
> > >breaks consumer group partition assignment
> > >- KAFKA-2337 :
> > Verify
> > >that metric names will not collide when creating new topics
> > >- KAFKA-2393 :
> > >Correctly Handle InvalidTopicException in
> KafkaApis.getTopicMetadata()
> > >- KAFKA-2189 :
> > Snappy
> > >compression of message batches less efficient in 0.8.2.1
> > >- KAFKA-2308 :
> New
> > >producer + Snappy face un-compression errors after broker restart
> > >- KAFKA-2042 :
> New
> > >producer metadata update always get all topics.
> > >- KAFKA-1367 :
> > Broker
> > >topic metadata not kept in sync with ZooKeeper
> > >- KAFKA-972 :
> > MetadataRequest
> > >returns stale list of brokers
> > >- KAFKA-1867 :
> > liveBroker
> > >list not updated on a cluster with no topics
> > >- KAFKA-1650 :
> > Mirror
> > >Maker could lose data on unclean shutdown.
> > >- KAFKA-2009 :
> Fix
> > >UncheckedOffset.removeOffset synchronization and trace logging issue
> > in
> > >mirror maker
> > >- KAFKA-2407 :
> Only
> > >create a log directory when it will be used
> > >- KAFKA-2327 :
> > >broker doesn't start if config defines advertised.host but not
> > >advertised.port
> > >- KAFKA-1788: producer record can stay in RecordAccumulator forever
> if
> > >leader is no available
> > >- KAFKA-2234 :
> > >Partition reassignment of a nonexistent topic prevents future
> > reassignments
> > >- KAFKA-2096 :
> > >Enable keepalive socket option for broker to prevent socket leak
> > >- KAFKA-1057 :
> Trim
> > >whitespaces from user specified configs
> > >- KAFKA-1641 :
> Log
> > >cleaner exits if last cleaned offset is lower than earliest offset
> > >- KAFKA-1648 :
> > Round
> > >robin consumer balance throws an NPE when there are no topics
> > >- KAFKA-1724 :
> > >Errors after reboot in single node setup
> > >- KAFKA-1758 :
> > >corrupt recovery file prevents startup
> > >- KAFKA-1866 :
> > >LogStartOffset gauge throws exceptions after log.delete()
> > >- KAFKA-1883 :
> > NullPointerException
> > >in RequestSendThread
> > >- KAFKA-1896 :
> > >Record size funcition of record in mirror maker hit NPE when the
> > message

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Stevo Slavić]

Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk,
with new consumer API appropriately annotated/documented as unstable?

On Mon, Aug 17, 2015, 17:17 Grant Henke  wrote:

> +dev
>
> Adding dev list back in. Somehow it got dropped.
>
>
> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke  wrote:
>
> > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I
> don't
> > suspect all of these will (or should) make it into the release but this
> > should be a relatively complete list to work from:
> >
> >- KAFKA-2114 :
> Unable
> >to change min.insync.replicas default
> >- KAFKA-1702 :
> >Messages silently Lost by producer
> >- KAFKA-2012 :
> >Broker should automatically handle corrupt index files
> >- KAFKA-2406 : ISR
> >propagation should be throttled to avoid overwhelming controller.
> >- KAFKA-2336 :
> >Changing offsets.topic.num.partitions after the offset topic is
> created
> >breaks consumer group partition assignment
> >- KAFKA-2337 :
> Verify
> >that metric names will not collide when creating new topics
> >- KAFKA-2393 :
> >Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
> >- KAFKA-2189 :
> Snappy
> >compression of message batches less efficient in 0.8.2.1
> >- KAFKA-2308 : New
> >producer + Snappy face un-compression errors after broker restart
> >- KAFKA-2042 : New
> >producer metadata update always get all topics.
> >- KAFKA-1367 :
> Broker
> >topic metadata not kept in sync with ZooKeeper
> >- KAFKA-972 :
> MetadataRequest
> >returns stale list of brokers
> >- KAFKA-1867 :
> liveBroker
> >list not updated on a cluster with no topics
> >- KAFKA-1650 :
> Mirror
> >Maker could lose data on unclean shutdown.
> >- KAFKA-2009 : Fix
> >UncheckedOffset.removeOffset synchronization and trace logging issue
> in
> >mirror maker
> >- KAFKA-2407 : Only
> >create a log directory when it will be used
> >- KAFKA-2327 :
> >broker doesn't start if config defines advertised.host but not
> >advertised.port
> >- KAFKA-1788: producer record can stay in RecordAccumulator forever if
> >leader is no available
> >- KAFKA-2234 :
> >Partition reassignment of a nonexistent topic prevents future
> reassignments
> >- KAFKA-2096 :
> >Enable keepalive socket option for broker to prevent socket leak
> >- KAFKA-1057 : Trim
> >whitespaces from user specified configs
> >- KAFKA-1641 : Log
> >cleaner exits if last cleaned offset is lower than earliest offset
> >- KAFKA-1648 :
> Round
> >robin consumer balance throws an NPE when there are no topics
> >- KAFKA-1724 :
> >Errors after reboot in single node setup
> >- KAFKA-1758 :
> >corrupt recovery file prevents startup
> >- KAFKA-1866 :
> >LogStartOffset gauge throws exceptions after log.delete()
> >- KAFKA-1883 :
> NullPointerException
> >in RequestSendThread
> >- KAFKA-1896 :
> >Record size funcition of record in mirror maker hit NPE when the
> message
> >value is null.
> >- KAFKA-2101 :
> >Metric metadata-age is reset on a failed update
> >- KAFKA-2112 : make
> >overflowWheel volatile
> >- KAFKA-2117 :
> >OffsetManager uses incorrect field for metadata
> >- KAFKA-2164 :
> >ReplicaFetcherThread: suspicious log message on reset offset
> >- KAFK

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Grant Henke]

Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't
suspect all of these will (or should) make it into the release but this
should be a relatively complete list to work from:

   - KAFKA-2114 : Unable
   to change min.insync.replicas default
   - KAFKA-1702 :
   Messages silently Lost by producer
   - KAFKA-2012 : Broker
   should automatically handle corrupt index files
   - KAFKA-2406 : ISR
   propagation should be throttled to avoid overwhelming controller.
   - KAFKA-2336 :
   Changing offsets.topic.num.partitions after the offset topic is created
   breaks consumer group partition assignment
   - KAFKA-2337 : Verify
   that metric names will not collide when creating new topics
   - KAFKA-2393 :
   Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
   - KAFKA-2189 : Snappy
   compression of message batches less efficient in 0.8.2.1
   - KAFKA-2308 : New
   producer + Snappy face un-compression errors after broker restart
   - KAFKA-2042 : New
   producer metadata update always get all topics.
   - KAFKA-1367 : Broker
   topic metadata not kept in sync with ZooKeeper
   - KAFKA-972 :
MetadataRequest
   returns stale list of brokers
   - KAFKA-1867 : liveBroker
   list not updated on a cluster with no topics
   - KAFKA-1650 : Mirror
   Maker could lose data on unclean shutdown.
   - KAFKA-2009 : Fix
   UncheckedOffset.removeOffset synchronization and trace logging issue in
   mirror maker
   - KAFKA-2407 : Only
   create a log directory when it will be used
   - KAFKA-2327 : broker
   doesn't start if config defines advertised.host but not advertised.port
   - KAFKA-1788: producer record can stay in RecordAccumulator forever if
   leader is no available
   - KAFKA-2234 :
   Partition reassignment of a nonexistent topic prevents future reassignments
   - KAFKA-2096 : Enable
   keepalive socket option for broker to prevent socket leak
   - KAFKA-1057 : Trim
   whitespaces from user specified configs
   - KAFKA-1641 : Log
   cleaner exits if last cleaned offset is lower than earliest offset
   - KAFKA-1648 : Round
   robin consumer balance throws an NPE when there are no topics
   - KAFKA-1724 : Errors
   after reboot in single node setup
   - KAFKA-1758 : corrupt
   recovery file prevents startup
   - KAFKA-1866 :
   LogStartOffset gauge throws exceptions after log.delete()
   - KAFKA-1883 :
NullPointerException
   in RequestSendThread
   - KAFKA-1896 : Record
   size funcition of record in mirror maker hit NPE when the message value is
   null.
   - KAFKA-2101 : Metric
   metadata-age is reset on a failed update
   - KAFKA-2112 : make
   overflowWheel volatile
   - KAFKA-2117 :
   OffsetManager uses incorrect field for metadata
   - KAFKA-2164 :
   ReplicaFetcherThread: suspicious log message on reset offset
   - KAFKA-1668 :
   TopicCommand doesn't warn if --topic argument doesn't match any topics
   - KAFKA-2198 :
   kafka-topics.sh exits with 0 status on failures
   - KAFKA-2235 :
   LogCleaner offset map overflow
   - KAFKA-2241 :
   AbstractFetcherThread.shutdown() should not block on
   ReadableByteChannel.read(buffer)
   - KAFKA-2272 :
   listeners endpoint parsing fails if the hostname has 

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Grant Henke]

+dev

Adding dev list back in. Somehow it got dropped.


On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke  wrote:

> Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't
> suspect all of these will (or should) make it into the release but this
> should be a relatively complete list to work from:
>
>- KAFKA-2114 : Unable
>to change min.insync.replicas default
>- KAFKA-1702 :
>Messages silently Lost by producer
>- KAFKA-2012 :
>Broker should automatically handle corrupt index files
>- KAFKA-2406 : ISR
>propagation should be throttled to avoid overwhelming controller.
>- KAFKA-2336 :
>Changing offsets.topic.num.partitions after the offset topic is created
>breaks consumer group partition assignment
>- KAFKA-2337 : Verify
>that metric names will not collide when creating new topics
>- KAFKA-2393 :
>Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata()
>- KAFKA-2189 : Snappy
>compression of message batches less efficient in 0.8.2.1
>- KAFKA-2308 : New
>producer + Snappy face un-compression errors after broker restart
>- KAFKA-2042 : New
>producer metadata update always get all topics.
>- KAFKA-1367 : Broker
>topic metadata not kept in sync with ZooKeeper
>- KAFKA-972 : 
> MetadataRequest
>returns stale list of brokers
>- KAFKA-1867 : liveBroker
>list not updated on a cluster with no topics
>- KAFKA-1650 : Mirror
>Maker could lose data on unclean shutdown.
>- KAFKA-2009 : Fix
>UncheckedOffset.removeOffset synchronization and trace logging issue in
>mirror maker
>- KAFKA-2407 : Only
>create a log directory when it will be used
>- KAFKA-2327 :
>broker doesn't start if config defines advertised.host but not
>advertised.port
>- KAFKA-1788: producer record can stay in RecordAccumulator forever if
>leader is no available
>- KAFKA-2234 :
>Partition reassignment of a nonexistent topic prevents future reassignments
>- KAFKA-2096 :
>Enable keepalive socket option for broker to prevent socket leak
>- KAFKA-1057 : Trim
>whitespaces from user specified configs
>- KAFKA-1641 : Log
>cleaner exits if last cleaned offset is lower than earliest offset
>- KAFKA-1648 : Round
>robin consumer balance throws an NPE when there are no topics
>- KAFKA-1724 :
>Errors after reboot in single node setup
>- KAFKA-1758 :
>corrupt recovery file prevents startup
>- KAFKA-1866 :
>LogStartOffset gauge throws exceptions after log.delete()
>- KAFKA-1883 : 
> NullPointerException
>in RequestSendThread
>- KAFKA-1896 :
>Record size funcition of record in mirror maker hit NPE when the message
>value is null.
>- KAFKA-2101 :
>Metric metadata-age is reset on a failed update
>- KAFKA-2112 : make
>overflowWheel volatile
>- KAFKA-2117 :
>OffsetManager uses incorrect field for metadata
>- KAFKA-2164 :
>ReplicaFetcherThread: suspicious log message on reset offset
>- KAFKA-1668 :
>TopicCommand doesn't warn if --topic argument doesn't match any topics
>- KAFKA-2198 :
>kafka-topics.sh exits with 0 status on failures
>- KAFKA-2235 :
>LogCleaner offset map overflow
>- K

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Grant Henke]

+1 to that suggestion. Though I suspect that requires a committer to do.
Making it part of the standard commit process could work too.
On Aug 16, 2015 11:01 PM, "Gwen Shapira"  wrote:

> BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
> manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
> branch and once enough bug fixes happened (or if sufficiently critical
> fixes happened) to roll out a new maintenance release (with every 3 month
> as a reasonable bugfix release target).
>
> This will add some predictability regarding how fast we release fixes for
> bugs.
>
> Gwen
>
> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman 
> wrote:
>
> > +1 for the release and also including
> >
> > https://issues.apache.org/jira/browse/KAFKA-2114
> >
> > Thanks
> >
> > Jeff
> >
> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić  wrote:
> >
> > > +1 (non-binding) for 0.8.2.2 release
> > >
> > > Would be nice to include in that release new producer resiliency bug
> > fixes
> > > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > > https://issues.apache.org/jira/browse/KAFKA-2120
> > >
> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira 
> wrote:
> > >
> > > > Will be nice to include Kafka-2308 and fix two critical snappy issues
> > in
> > > > the maintenance release.
> > > >
> > > > Gwen
> > > > On Aug 14, 2015 6:16 AM, "Grant Henke"  wrote:
> > > >
> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> > > > >
> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> > ku...@nmsworks.co.in
> > > >
> > > > > wrote:
> > > > >
> > > > > > +1  for 0.8.2.2 release
> > > > > >
> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma 
> > > > wrote:
> > > > > >
> > > > > > > I think this is a good idea as the change is minimal on our
> side
> > > and
> > > > it
> > > > > > has
> > > > > > > been tested in production for some time by the reporter.
> > > > > > >
> > > > > > > Best,
> > > > > > > Ismael
> > > > > > >
> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao 
> > wrote:
> > > > > > >
> > > > > > > > Hi, Everyone,
> > > > > > > >
> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people have
> > > > reported
> > > > > an
> > > > > > > > issue with snappy compression (
> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189).
> Basically,
> > if
> > > > > they
> > > > > > > use
> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space
> increase.
> > > The
> > > > > > issue
> > > > > > > > has since been fixed in trunk (just a snappy jar upgrade).
> > Since
> > > > > 0.8.3
> > > > > > is
> > > > > > > > still a few months away, it may make sense to do an 0.8.2.2
> > > release
> > > > > > just
> > > > > > > to
> > > > > > > > fix this issue. Any objections?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Jun
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Grant Henke
> > > > > Software Engineer | Cloudera
> > > > > gr...@cloudera.com | twitter.com/gchenke |
> > linkedin.com/in/granthenke
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Jeff Holoman
> > Systems Engineer
> >
>

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Gwen Shapira]

BTW. I think it will be great for Apache Kafka to have a 0.8.2 "release
manager" who's role is to cherrypick low-risk bug-fixes into the 0.8.2
branch and once enough bug fixes happened (or if sufficiently critical
fixes happened) to roll out a new maintenance release (with every 3 month
as a reasonable bugfix release target).

This will add some predictability regarding how fast we release fixes for
bugs.

Gwen

On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman  wrote:

> +1 for the release and also including
>
> https://issues.apache.org/jira/browse/KAFKA-2114
>
> Thanks
>
> Jeff
>
> On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić  wrote:
>
> > +1 (non-binding) for 0.8.2.2 release
> >
> > Would be nice to include in that release new producer resiliency bug
> fixes
> > https://issues.apache.org/jira/browse/KAFKA-1788 and
> > https://issues.apache.org/jira/browse/KAFKA-2120
> >
> > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira  wrote:
> >
> > > Will be nice to include Kafka-2308 and fix two critical snappy issues
> in
> > > the maintenance release.
> > >
> > > Gwen
> > > On Aug 14, 2015 6:16 AM, "Grant Henke"  wrote:
> > >
> > > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> > > >
> > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy <
> ku...@nmsworks.co.in
> > >
> > > > wrote:
> > > >
> > > > > +1  for 0.8.2.2 release
> > > > >
> > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma 
> > > wrote:
> > > > >
> > > > > > I think this is a good idea as the change is minimal on our side
> > and
> > > it
> > > > > has
> > > > > > been tested in production for some time by the reporter.
> > > > > >
> > > > > > Best,
> > > > > > Ismael
> > > > > >
> > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao 
> wrote:
> > > > > >
> > > > > > > Hi, Everyone,
> > > > > > >
> > > > > > > Since the release of Kafka 0.8.2.1, a number of people have
> > > reported
> > > > an
> > > > > > > issue with snappy compression (
> > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically,
> if
> > > > they
> > > > > > use
> > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase.
> > The
> > > > > issue
> > > > > > > has since been fixed in trunk (just a snappy jar upgrade).
> Since
> > > > 0.8.3
> > > > > is
> > > > > > > still a few months away, it may make sense to do an 0.8.2.2
> > release
> > > > > just
> > > > > > to
> > > > > > > fix this issue. Any objections?
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Jun
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Grant Henke
> > > > Software Engineer | Cloudera
> > > > gr...@cloudera.com | twitter.com/gchenke |
> linkedin.com/in/granthenke
> > > >
> > >
> >
>
>
>
> --
> Jeff Holoman
> Systems Engineer
>

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Jeff Holoman]

+1 for the release and also including

https://issues.apache.org/jira/browse/KAFKA-2114

Thanks

Jeff

On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić  wrote:

> +1 (non-binding) for 0.8.2.2 release
>
> Would be nice to include in that release new producer resiliency bug fixes
> https://issues.apache.org/jira/browse/KAFKA-1788 and
> https://issues.apache.org/jira/browse/KAFKA-2120
>
> On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira  wrote:
>
> > Will be nice to include Kafka-2308 and fix two critical snappy issues in
> > the maintenance release.
> >
> > Gwen
> > On Aug 14, 2015 6:16 AM, "Grant Henke"  wrote:
> >
> > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> > >
> > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy  >
> > > wrote:
> > >
> > > > +1  for 0.8.2.2 release
> > > >
> > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma 
> > wrote:
> > > >
> > > > > I think this is a good idea as the change is minimal on our side
> and
> > it
> > > > has
> > > > > been tested in production for some time by the reporter.
> > > > >
> > > > > Best,
> > > > > Ismael
> > > > >
> > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao  wrote:
> > > > >
> > > > > > Hi, Everyone,
> > > > > >
> > > > > > Since the release of Kafka 0.8.2.1, a number of people have
> > reported
> > > an
> > > > > > issue with snappy compression (
> > > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> > > they
> > > > > use
> > > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase.
> The
> > > > issue
> > > > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> > > 0.8.3
> > > > is
> > > > > > still a few months away, it may make sense to do an 0.8.2.2
> release
> > > > just
> > > > > to
> > > > > > fix this issue. Any objections?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>



-- 
Jeff Holoman
Systems Engineer

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Stevo Slavić]

+1 (non-binding) for 0.8.2.2 release

Would be nice to include in that release new producer resiliency bug fixes
https://issues.apache.org/jira/browse/KAFKA-1788 and
https://issues.apache.org/jira/browse/KAFKA-2120

On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira  wrote:

> Will be nice to include Kafka-2308 and fix two critical snappy issues in
> the maintenance release.
>
> Gwen
> On Aug 14, 2015 6:16 AM, "Grant Henke"  wrote:
>
> > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> >
> > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy 
> > wrote:
> >
> > > +1  for 0.8.2.2 release
> > >
> > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma 
> wrote:
> > >
> > > > I think this is a good idea as the change is minimal on our side and
> it
> > > has
> > > > been tested in production for some time by the reporter.
> > > >
> > > > Best,
> > > > Ismael
> > > >
> > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao  wrote:
> > > >
> > > > > Hi, Everyone,
> > > > >
> > > > > Since the release of Kafka 0.8.2.1, a number of people have
> reported
> > an
> > > > > issue with snappy compression (
> > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> > they
> > > > use
> > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> > > issue
> > > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> > 0.8.3
> > > is
> > > > > still a few months away, it may make sense to do an 0.8.2.2 release
> > > just
> > > > to
> > > > > fix this issue. Any objections?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Joe Stein]

Is this something also that should be in 0.8.2.2
https://issues.apache.org/jira/browse/KAFKA-2421 wasn't sure what the lz4
usage has been seems to break on some jdks.

+1 on the snappy fixes

~ Joe Stein

On Fri, Aug 14, 2015 at 5:39 PM, Guozhang Wang  wrote:

> +1 for both KAFKA-2189 and 2308.
>
> On Fri, Aug 14, 2015 at 7:03 AM, Gwen Shapira  wrote:
>
> > Will be nice to include Kafka-2308 and fix two critical snappy issues in
> > the maintenance release.
> >
> > Gwen
> > On Aug 14, 2015 6:16 AM, "Grant Henke"  wrote:
> >
> > > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> > >
> > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy  >
> > > wrote:
> > >
> > > > +1  for 0.8.2.2 release
> > > >
> > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma 
> > wrote:
> > > >
> > > > > I think this is a good idea as the change is minimal on our side
> and
> > it
> > > > has
> > > > > been tested in production for some time by the reporter.
> > > > >
> > > > > Best,
> > > > > Ismael
> > > > >
> > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao  wrote:
> > > > >
> > > > > > Hi, Everyone,
> > > > > >
> > > > > > Since the release of Kafka 0.8.2.1, a number of people have
> > reported
> > > an
> > > > > > issue with snappy compression (
> > > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> > > they
> > > > > use
> > > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase.
> The
> > > > issue
> > > > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> > > 0.8.3
> > > > is
> > > > > > still a few months away, it may make sense to do an 0.8.2.2
> release
> > > > just
> > > > > to
> > > > > > fix this issue. Any objections?
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Jun
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Grant Henke
> > > Software Engineer | Cloudera
> > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> > >
> >
>
>
>
> --
> -- Guozhang
>

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Abhijith]

Thanks.  Any idea when we can expect 0.8.3?

Sent from my phone

> On Aug 14, 2015, at 5:39 PM, Guozhang Wang  wrote:
> 
> +1 for both KAFKA-2189 and 2308.
> 
>> On Fri, Aug 14, 2015 at 7:03 AM, Gwen Shapira  wrote:
>> 
>> Will be nice to include Kafka-2308 and fix two critical snappy issues in
>> the maintenance release.
>> 
>> Gwen
>>> On Aug 14, 2015 6:16 AM, "Grant Henke"  wrote:
>>> 
>>> Just to clarify. Will KAFKA-2189 be the only patch in the release?
>>> 
>>> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy 
>>> wrote:
>>> 
 +1  for 0.8.2.2 release
 
 On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma 
>> wrote:
 
> I think this is a good idea as the change is minimal on our side and
>> it
 has
> been tested in production for some time by the reporter.
> 
> Best,
> Ismael
> 
>> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao  wrote:
>> 
>> Hi, Everyone,
>> 
>> Since the release of Kafka 0.8.2.1, a number of people have
>> reported
>>> an
>> issue with snappy compression (
>> https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
>>> they
> use
>> snappy in 0.8.2.1, they will experience a 2-3X space increase. The
 issue
>> has since been fixed in trunk (just a snappy jar upgrade). Since
>>> 0.8.3
 is
>> still a few months away, it may make sense to do an 0.8.2.2 release
 just
> to
>> fix this issue. Any objections?
>> 
>> Thanks,
>> 
>> Jun
>>> 
>>> 
>>> 
>>> --
>>> Grant Henke
>>> Software Engineer | Cloudera
>>> gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> 
> 
> 
> -- 
> -- Guozhang

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Guozhang Wang]

+1 for both KAFKA-2189 and 2308.

On Fri, Aug 14, 2015 at 7:03 AM, Gwen Shapira  wrote:

> Will be nice to include Kafka-2308 and fix two critical snappy issues in
> the maintenance release.
>
> Gwen
> On Aug 14, 2015 6:16 AM, "Grant Henke"  wrote:
>
> > Just to clarify. Will KAFKA-2189 be the only patch in the release?
> >
> > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy 
> > wrote:
> >
> > > +1  for 0.8.2.2 release
> > >
> > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma 
> wrote:
> > >
> > > > I think this is a good idea as the change is minimal on our side and
> it
> > > has
> > > > been tested in production for some time by the reporter.
> > > >
> > > > Best,
> > > > Ismael
> > > >
> > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao  wrote:
> > > >
> > > > > Hi, Everyone,
> > > > >
> > > > > Since the release of Kafka 0.8.2.1, a number of people have
> reported
> > an
> > > > > issue with snappy compression (
> > > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> > they
> > > > use
> > > > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> > > issue
> > > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> > 0.8.3
> > > is
> > > > > still a few months away, it may make sense to do an 0.8.2.2 release
> > > just
> > > > to
> > > > > fix this issue. Any objections?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Jun
> > > > >
> > > >
> > >
> >
> >
> >
> > --
> > Grant Henke
> > Software Engineer | Cloudera
> > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
> >
>



-- 
-- Guozhang

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Gwen Shapira]

Will be nice to include Kafka-2308 and fix two critical snappy issues in
the maintenance release.

Gwen
On Aug 14, 2015 6:16 AM, "Grant Henke"  wrote:

> Just to clarify. Will KAFKA-2189 be the only patch in the release?
>
> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy 
> wrote:
>
> > +1  for 0.8.2.2 release
> >
> > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma  wrote:
> >
> > > I think this is a good idea as the change is minimal on our side and it
> > has
> > > been tested in production for some time by the reporter.
> > >
> > > Best,
> > > Ismael
> > >
> > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao  wrote:
> > >
> > > > Hi, Everyone,
> > > >
> > > > Since the release of Kafka 0.8.2.1, a number of people have reported
> an
> > > > issue with snappy compression (
> > > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if
> they
> > > use
> > > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> > issue
> > > > has since been fixed in trunk (just a snappy jar upgrade). Since
> 0.8.3
> > is
> > > > still a few months away, it may make sense to do an 0.8.2.2 release
> > just
> > > to
> > > > fix this issue. Any objections?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > >
> >
>
>
>
> --
> Grant Henke
> Software Engineer | Cloudera
> gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
>

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Grant Henke]

Just to clarify. Will KAFKA-2189 be the only patch in the release?

On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy 
wrote:

> +1  for 0.8.2.2 release
>
> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma  wrote:
>
> > I think this is a good idea as the change is minimal on our side and it
> has
> > been tested in production for some time by the reporter.
> >
> > Best,
> > Ismael
> >
> > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao  wrote:
> >
> > > Hi, Everyone,
> > >
> > > Since the release of Kafka 0.8.2.1, a number of people have reported an
> > > issue with snappy compression (
> > > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they
> > use
> > > snappy in 0.8.2.1, they will experience a 2-3X space increase. The
> issue
> > > has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3
> is
> > > still a few months away, it may make sense to do an 0.8.2.2 release
> just
> > to
> > > fix this issue. Any objections?
> > >
> > > Thanks,
> > >
> > > Jun
> > >
> >
>



-- 
Grant Henke
Software Engineer | Cloudera
gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Manikumar Reddy]

+1  for 0.8.2.2 release

On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma  wrote:

> I think this is a good idea as the change is minimal on our side and it has
> been tested in production for some time by the reporter.
>
> Best,
> Ismael
>
> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao  wrote:
>
> > Hi, Everyone,
> >
> > Since the release of Kafka 0.8.2.1, a number of people have reported an
> > issue with snappy compression (
> > https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they
> use
> > snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue
> > has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is
> > still a few months away, it may make sense to do an 0.8.2.2 release just
> to
> > fix this issue. Any objections?
> >
> > Thanks,
> >
> > Jun
> >
>

Re: [DISCUSSION] Kafka 0.8.2.2 release?

[Ismael Juma]

I think this is a good idea as the change is minimal on our side and it has
been tested in production for some time by the reporter.

Best,
Ismael

On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao  wrote:

> Hi, Everyone,
>
> Since the release of Kafka 0.8.2.1, a number of people have reported an
> issue with snappy compression (
> https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use
> snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue
> has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is
> still a few months away, it may make sense to do an 0.8.2.2 release just to
> fix this issue. Any objections?
>
> Thanks,
>
> Jun
>