Re: [DISCUSSION] Kafka 0.8.2.2 release?
Ok, thanks everyone for the feedback. Based on the feedback, I'd recommend that we do an 0.8.2.2 release including just the two snappy bug fixes. Jun On Tue, Aug 18, 2015 at 12:59 PM, Gwen Shapira g...@confluent.io wrote: Any objections if I leave KAFKA-2114 (setting min.insync.replicas default) out? The test code is using changes that were done after 0.8.2.x cut-off, which makes it difficult to cherry-pick. Gwen On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira g...@confluent.io wrote: Jun, KAFKA-2147 doesn't seem to have a commit associated with it, so I can't cherrypick just this fix. I suggest leaving this out since there is a 0.8.2.x workaround in the JIRA. Gwen On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao j...@confluent.io wrote: Gwen, Thanks for putting the list together. I'd recommend that we exclude the following: KAFKA-1702: This is for the old producer and is only a problem if there are some unexpected exceptions (e.g. UnknownClass). KAFKA-2336: Most people don't change offsets.topic.num.partitions. KAFKA-1724: The patch there is never committed since the fix is included in another jira (a much larger patch). KAFKA-2241: This doesn't seem be a common problem. It only happens when the fetch request blocks on the broker for an extended period of time, which should be rare. I'd also recommend that we include the following: KAFKA-2147: This impacts the memory size of the purgatory and a number of people have experienced that. The fix is small and has been tested in production usage. It hasn't been committed though since the issue is already fixed in trunk and we weren't planning for an 0.8.2.2 release then. Thanks, Jun On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira g...@confluent.io wrote: Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114 : Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702 : Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012 : Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406 : ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336 : Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337 : Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393 : Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189 : Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308 : New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042 : New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367 : Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867 : liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650 : Mirror
Re: [DISCUSSION] Kafka 0.8.2.2 release?
I agree: keep it simple :) The latest stable version of Kafka right now has a critical bug in it. Fixing that would be good enough. 0.8.2.2 should probably just a maintenance/bugfix release. On Aug 18, 2015, at 14:29, Edward Ribeiro edward.ribe...@gmail.com wrote: I sort of follow Daniel Nelson on this issue: cut a 0.8.2.2 but not include much thing besides Snappy fixes. I mean, include a couple of additional critical bug fixes, if really urgent, and that's it. On Tue, Aug 18, 2015 at 3:25 PM, Daniel Nelson daniel.nel...@vungle.com wrote: I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think that it needs to include anything other than the fix for Snappy that kicked off this discussion in the first place. If there are additional critical issues that can be included without delaying the process, I see no downside. On Aug 18, 2015, at 10:54 AM, Neha Narkhede n...@confluent.io wrote: How about looking at the scope for the 0.8.3 release first before we cut yet another point release off of 0.8.2.2? Each release includes some overhead and if there is a larger release in the works, it might be worth working on getting that. My take is that the 2 things the community has been waiting for is SSL support and the new consumer and we have been promising to get 0.8.3 with both those features for several months now. Looking at the progress on both, it seems we are very close to getting both those checked in and it looks like we should get there in another 5-6 weeks. Furthermore, both of these features are large and I anticipate us receiving feedback and bugs that will require a couple of point releases on top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and get the community to use the newly released features, gather feedback and do point releases incorporating that feedback and iterate on it. We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still makes sense. Thanks, Neha On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira g...@confluent.io wrote: Thanks Jun. I updated the list with your suggestions. If anyone feels we are missing a critical patch for 0.8.2.2, please speak up. Gwen On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao j...@confluent.io wrote: Hi, Grant, I took a look at that list. None of those is really critical as you said. So, I'd suggest that we not include those to minimize the scope of the release. Thanks, Jun On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke ghe...@cloudera.com wrote: Thanks Gwen. I updated a few small things on the wiki page. Below is a list of jiras I think could also be marked as included. All of these, though not super critical, seem like fairly small and low risk changes that help avoid potentially confusing issues or errors for users. KAFKA-2012 KAFKA-972 KAFKA-2337 KAFKA-2393 KAFKA-1867 KAFKA-2407 KAFKA-2234 KAFKA-1866 KAFKA-2345 KAFKA-2355 thoughts? Thank you, Grant On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira g...@confluent.io wrote: Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114 : Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702 : Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012 : Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406 : ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336 : Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337
Re: [DISCUSSION] Kafka 0.8.2.2 release?
How about looking at the scope for the 0.8.3 release first before we cut yet another point release off of 0.8.2.2? Each release includes some overhead and if there is a larger release in the works, it might be worth working on getting that. My take is that the 2 things the community has been waiting for is SSL support and the new consumer and we have been promising to get 0.8.3 with both those features for several months now. Looking at the progress on both, it seems we are very close to getting both those checked in and it looks like we should get there in another 5-6 weeks. Furthermore, both of these features are large and I anticipate us receiving feedback and bugs that will require a couple of point releases on top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and get the community to use the newly released features, gather feedback and do point releases incorporating that feedback and iterate on it. We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still makes sense. Thanks, Neha On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira g...@confluent.io wrote: Thanks Jun. I updated the list with your suggestions. If anyone feels we are missing a critical patch for 0.8.2.2, please speak up. Gwen On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao j...@confluent.io wrote: Hi, Grant, I took a look at that list. None of those is really critical as you said. So, I'd suggest that we not include those to minimize the scope of the release. Thanks, Jun On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke ghe...@cloudera.com wrote: Thanks Gwen. I updated a few small things on the wiki page. Below is a list of jiras I think could also be marked as included. All of these, though not super critical, seem like fairly small and low risk changes that help avoid potentially confusing issues or errors for users. KAFKA-2012 KAFKA-972 KAFKA-2337 KAFKA-2393 KAFKA-1867 KAFKA-2407 KAFKA-2234 KAFKA-1866 KAFKA-2345 KAFKA-2355 thoughts? Thank you, Grant On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira g...@confluent.io wrote: Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114 : Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702 : Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012 : Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406 : ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336 : Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337 : Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393 : Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189 : Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308 : New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042 : New producer metadata
Re: [DISCUSSION] Kafka 0.8.2.2 release?
I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think that it needs to include anything other than the fix for Snappy that kicked off this discussion in the first place. If there are additional critical issues that can be included without delaying the process, I see no downside. On Aug 18, 2015, at 10:54 AM, Neha Narkhede n...@confluent.io wrote: How about looking at the scope for the 0.8.3 release first before we cut yet another point release off of 0.8.2.2? Each release includes some overhead and if there is a larger release in the works, it might be worth working on getting that. My take is that the 2 things the community has been waiting for is SSL support and the new consumer and we have been promising to get 0.8.3 with both those features for several months now. Looking at the progress on both, it seems we are very close to getting both those checked in and it looks like we should get there in another 5-6 weeks. Furthermore, both of these features are large and I anticipate us receiving feedback and bugs that will require a couple of point releases on top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and get the community to use the newly released features, gather feedback and do point releases incorporating that feedback and iterate on it. We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still makes sense. Thanks, Neha On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira g...@confluent.io wrote: Thanks Jun. I updated the list with your suggestions. If anyone feels we are missing a critical patch for 0.8.2.2, please speak up. Gwen On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao j...@confluent.io wrote: Hi, Grant, I took a look at that list. None of those is really critical as you said. So, I'd suggest that we not include those to minimize the scope of the release. Thanks, Jun On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke ghe...@cloudera.com wrote: Thanks Gwen. I updated a few small things on the wiki page. Below is a list of jiras I think could also be marked as included. All of these, though not super critical, seem like fairly small and low risk changes that help avoid potentially confusing issues or errors for users. KAFKA-2012 KAFKA-972 KAFKA-2337 KAFKA-2393 KAFKA-1867 KAFKA-2407 KAFKA-2234 KAFKA-1866 KAFKA-2345 KAFKA-2355 thoughts? Thank you, Grant On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira g...@confluent.io wrote: Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114 : Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702 : Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012 : Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406 : ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336 : Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337 : Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393 : Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189 : Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308 : New producer + Snappy face un-compression errors after broker restart - KAFKA-2042
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Thanks Jun. I updated the list with your suggestions. If anyone feels we are missing a critical patch for 0.8.2.2, please speak up. Gwen On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao j...@confluent.io wrote: Hi, Grant, I took a look at that list. None of those is really critical as you said. So, I'd suggest that we not include those to minimize the scope of the release. Thanks, Jun On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke ghe...@cloudera.com wrote: Thanks Gwen. I updated a few small things on the wiki page. Below is a list of jiras I think could also be marked as included. All of these, though not super critical, seem like fairly small and low risk changes that help avoid potentially confusing issues or errors for users. KAFKA-2012 KAFKA-972 KAFKA-2337 KAFKA-2393 KAFKA-1867 KAFKA-2407 KAFKA-2234 KAFKA-1866 KAFKA-2345 KAFKA-2355 thoughts? Thank you, Grant On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira g...@confluent.io wrote: Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114 : Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702 : Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012 : Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406 : ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336 : Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337 : Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393 : Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189 : Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308 : New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042 : New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367 : Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867 : liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650 : Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009 : Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407 : Only create a log directory when it will be used - KAFKA-2327 https://issues.apache.org/jira/browse/KAFKA-2327 : broker doesn't start if config defines advertised.host but not advertised.port - KAFKA-1788: producer record can stay in RecordAccumulator forever if leader is no available - KAFKA-2234 https://issues.apache.org/jira/browse/KAFKA-2234 : Partition reassignment of a nonexistent topic prevents future reassignments -
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Jun, KAFKA-2147 doesn't seem to have a commit associated with it, so I can't cherrypick just this fix. I suggest leaving this out since there is a 0.8.2.x workaround in the JIRA. Gwen On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao j...@confluent.io wrote: Gwen, Thanks for putting the list together. I'd recommend that we exclude the following: KAFKA-1702: This is for the old producer and is only a problem if there are some unexpected exceptions (e.g. UnknownClass). KAFKA-2336: Most people don't change offsets.topic.num.partitions. KAFKA-1724: The patch there is never committed since the fix is included in another jira (a much larger patch). KAFKA-2241: This doesn't seem be a common problem. It only happens when the fetch request blocks on the broker for an extended period of time, which should be rare. I'd also recommend that we include the following: KAFKA-2147: This impacts the memory size of the purgatory and a number of people have experienced that. The fix is small and has been tested in production usage. It hasn't been committed though since the issue is already fixed in trunk and we weren't planning for an 0.8.2.2 release then. Thanks, Jun On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira g...@confluent.io wrote: Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114: Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702: Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012: Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406: ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337: Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189: Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308: New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042: New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367: Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867: liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650: Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009: Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407: Only create a log directory when it will be used - KAFKA-2327 https://issues.apache.org/jira/browse/KAFKA-2327: broker doesn't start if config defines advertised.host but not advertised.port - KAFKA-1788: producer record can stay in RecordAccumulator forever if leader is no available - KAFKA-2234
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Any objections if I leave KAFKA-2114 (setting min.insync.replicas default) out? The test code is using changes that were done after 0.8.2.x cut-off, which makes it difficult to cherry-pick. Gwen On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira g...@confluent.io wrote: Jun, KAFKA-2147 doesn't seem to have a commit associated with it, so I can't cherrypick just this fix. I suggest leaving this out since there is a 0.8.2.x workaround in the JIRA. Gwen On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao j...@confluent.io wrote: Gwen, Thanks for putting the list together. I'd recommend that we exclude the following: KAFKA-1702: This is for the old producer and is only a problem if there are some unexpected exceptions (e.g. UnknownClass). KAFKA-2336: Most people don't change offsets.topic.num.partitions. KAFKA-1724: The patch there is never committed since the fix is included in another jira (a much larger patch). KAFKA-2241: This doesn't seem be a common problem. It only happens when the fetch request blocks on the broker for an extended period of time, which should be rare. I'd also recommend that we include the following: KAFKA-2147: This impacts the memory size of the purgatory and a number of people have experienced that. The fix is small and has been tested in production usage. It hasn't been committed though since the issue is already fixed in trunk and we weren't planning for an 0.8.2.2 release then. Thanks, Jun On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira g...@confluent.io wrote: Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114: Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702: Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012: Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406: ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337: Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189: Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308: New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042: New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367: Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867: liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650: Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009: Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407: Only create a log directory when it will be used - KAFKA-2327
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Yes, I think it's fine to leave out both KAFKA-2147 and KAFKA-2114. Thanks, Jun On Tue, Aug 18, 2015 at 12:59 PM, Gwen Shapira g...@confluent.io wrote: Any objections if I leave KAFKA-2114 (setting min.insync.replicas default) out? The test code is using changes that were done after 0.8.2.x cut-off, which makes it difficult to cherry-pick. Gwen On Tue, Aug 18, 2015 at 12:16 PM, Gwen Shapira g...@confluent.io wrote: Jun, KAFKA-2147 doesn't seem to have a commit associated with it, so I can't cherrypick just this fix. I suggest leaving this out since there is a 0.8.2.x workaround in the JIRA. Gwen On Mon, Aug 17, 2015 at 5:24 PM, Jun Rao j...@confluent.io wrote: Gwen, Thanks for putting the list together. I'd recommend that we exclude the following: KAFKA-1702: This is for the old producer and is only a problem if there are some unexpected exceptions (e.g. UnknownClass). KAFKA-2336: Most people don't change offsets.topic.num.partitions. KAFKA-1724: The patch there is never committed since the fix is included in another jira (a much larger patch). KAFKA-2241: This doesn't seem be a common problem. It only happens when the fetch request blocks on the broker for an extended period of time, which should be rare. I'd also recommend that we include the following: KAFKA-2147: This impacts the memory size of the purgatory and a number of people have experienced that. The fix is small and has been tested in production usage. It hasn't been committed though since the issue is already fixed in trunk and we weren't planning for an 0.8.2.2 release then. Thanks, Jun On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira g...@confluent.io wrote: Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114 : Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702 : Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012 : Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406 : ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336 : Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337 : Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393 : Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189 : Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308 : New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042 : New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367 : Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867 : liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650 : Mirror Maker could lose data on unclean shutdown. - KAFKA-2009
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114: Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702: Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012: Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406: ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337: Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189: Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308: New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042: New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367: Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867: liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650: Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009: Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407: Only create a log directory when it will be used - KAFKA-2327 https://issues.apache.org/jira/browse/KAFKA-2327: broker doesn't start if config defines advertised.host but not advertised.port - KAFKA-1788: producer record can stay in RecordAccumulator forever if leader is no available - KAFKA-2234 https://issues.apache.org/jira/browse/KAFKA-2234: Partition reassignment of a nonexistent topic prevents future reassignments - KAFKA-2096 https://issues.apache.org/jira/browse/KAFKA-2096: Enable keepalive socket option for broker to prevent socket leak - KAFKA-1057 https://issues.apache.org/jira/browse/KAFKA-1057: Trim whitespaces from user specified configs - KAFKA-1641 https://issues.apache.org/jira/browse/KAFKA-1641: Log cleaner exits if last cleaned offset is lower than earliest offset - KAFKA-1648 https://issues.apache.org/jira/browse/KAFKA-1648: Round robin consumer balance throws an NPE when there are no topics - KAFKA-1724 https://issues.apache.org/jira/browse/KAFKA-1724: Errors after reboot in single node setup - KAFKA-1758 https://issues.apache.org/jira/browse/KAFKA-1758: corrupt recovery file prevents startup - KAFKA-1866 https://issues.apache.org/jira/browse/KAFKA-1866: LogStartOffset gauge throws exceptions after log.delete() - KAFKA-1883 https://issues.apache.org/jira/browse/KAFKA-1883: NullPointerException in RequestSendThread - KAFKA-1896 https://issues.apache.org/jira/browse/KAFKA-1896: Record size funcition of record in mirror maker hit NPE when the message value is null. - KAFKA-2101 https://issues.apache.org/jira/browse/KAFKA-2101: Metric metadata-age is reset on a failed update - KAFKA-2112
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Hi, Grant, I took a look at that list. None of those is really critical as you said. So, I'd suggest that we not include those to minimize the scope of the release. Thanks, Jun On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke ghe...@cloudera.com wrote: Thanks Gwen. I updated a few small things on the wiki page. Below is a list of jiras I think could also be marked as included. All of these, though not super critical, seem like fairly small and low risk changes that help avoid potentially confusing issues or errors for users. KAFKA-2012 KAFKA-972 KAFKA-2337 KAFKA-2393 KAFKA-1867 KAFKA-2407 KAFKA-2234 KAFKA-1866 KAFKA-2345 KAFKA-2355 thoughts? Thank you, Grant On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira g...@confluent.io wrote: Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114: Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702: Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012: Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406: ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337: Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189: Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308: New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042: New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367: Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867: liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650: Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009: Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407: Only create a log directory when it will be used - KAFKA-2327 https://issues.apache.org/jira/browse/KAFKA-2327: broker doesn't start if config defines advertised.host but not advertised.port - KAFKA-1788: producer record can stay in RecordAccumulator forever if leader is no available - KAFKA-2234 https://issues.apache.org/jira/browse/KAFKA-2234: Partition reassignment of a nonexistent topic prevents future reassignments - KAFKA-2096 https://issues.apache.org/jira/browse/KAFKA-2096: Enable keepalive socket option for broker to prevent socket leak - KAFKA-1057 https://issues.apache.org/jira/browse/KAFKA-1057: Trim whitespaces from user specified configs - KAFKA-1641 https://issues.apache.org/jira/browse/KAFKA-1641: Log cleaner exits if last cleaned offset is
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Gwen, Thanks for putting the list together. I'd recommend that we exclude the following: KAFKA-1702: This is for the old producer and is only a problem if there are some unexpected exceptions (e.g. UnknownClass). KAFKA-2336: Most people don't change offsets.topic.num.partitions. KAFKA-1724: The patch there is never committed since the fix is included in another jira (a much larger patch). KAFKA-2241: This doesn't seem be a common problem. It only happens when the fetch request blocks on the broker for an extended period of time, which should be rare. I'd also recommend that we include the following: KAFKA-2147: This impacts the memory size of the purgatory and a number of people have experienced that. The fix is small and has been tested in production usage. It hasn't been committed though since the issue is already fixed in trunk and we weren't planning for an 0.8.2.2 release then. Thanks, Jun On Mon, Aug 17, 2015 at 2:56 PM, Gwen Shapira g...@confluent.io wrote: Thanks for creating a list, Grant! I placed it on the wiki with a quick evaluation of the content and whether it should be in 0.8.2.2: https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 I'm attempting to only cherrypick fixes that are both important for large number of users (or very critical to some users) and very safe (mostly judged by the size of the change, but not only) If your favorite bugfix is missing from the list, or is there but marked No, please let us know (in this thread) what we are missing and why it is both important and safe. Also, if I accidentally included something you consider unsafe, speak up! Gwen On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114: Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702: Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012: Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406: ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337: Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189: Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308: New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042: New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367: Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867: liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650: Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009: Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407: Only create a log directory when it will be used - KAFKA-2327 https://issues.apache.org/jira/browse/KAFKA-2327: broker doesn't start if config defines advertised.host but not advertised.port - KAFKA-1788: producer record can stay in RecordAccumulator forever if leader is no available - KAFKA-2234 https://issues.apache.org/jira/browse/KAFKA-2234: Partition reassignment of a nonexistent topic prevents future reassignments - KAFKA-2096 https://issues.apache.org/jira/browse/KAFKA-2096: Enable keepalive socket option for broker to prevent socket leak - KAFKA-1057 https://issues.apache.org/jira/browse/KAFKA-1057: Trim whitespaces from user specified configs
Re: [DISCUSSION] Kafka 0.8.2.2 release?
+1 for 0.8.2.2 release with fixes for KAFKA-2189, 2114 and 2308. On Mon, Aug 17, 2015 at 9:16 AM, Flavio Junqueira f...@apache.org wrote: It is pretty typical that Apache projects have a release manager for each release: http://www.apache.org/dev/release-publishing.html http://www.apache.org/dev/release-publishing.html It doesn't have to be the same person every time, though, not even for the same branch. -Flavio On 17 Aug 2015, at 05:01, Gwen Shapira g...@confluent.io wrote: BTW. I think it will be great for Apache Kafka to have a 0.8.2 release manager who's role is to cherrypick low-risk bug-fixes into the 0.8.2 branch and once enough bug fixes happened (or if sufficiently critical fixes happened) to roll out a new maintenance release (with every 3 month as a reasonable bugfix release target). -- Regards, Ashish
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk, with new consumer API appropriately annotated/documented as unstable? On Mon, Aug 17, 2015, 17:17 Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114: Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702: Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012: Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406: ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337: Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189: Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308: New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042: New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367: Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867: liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650: Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009: Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407: Only create a log directory when it will be used - KAFKA-2327 https://issues.apache.org/jira/browse/KAFKA-2327: broker doesn't start if config defines advertised.host but not advertised.port - KAFKA-1788: producer record can stay in RecordAccumulator forever if leader is no available - KAFKA-2234 https://issues.apache.org/jira/browse/KAFKA-2234: Partition reassignment of a nonexistent topic prevents future reassignments - KAFKA-2096 https://issues.apache.org/jira/browse/KAFKA-2096: Enable keepalive socket option for broker to prevent socket leak - KAFKA-1057 https://issues.apache.org/jira/browse/KAFKA-1057: Trim whitespaces from user specified configs - KAFKA-1641 https://issues.apache.org/jira/browse/KAFKA-1641: Log cleaner exits if last cleaned offset is lower than earliest offset - KAFKA-1648 https://issues.apache.org/jira/browse/KAFKA-1648: Round robin consumer balance throws an NPE when there are no topics - KAFKA-1724 https://issues.apache.org/jira/browse/KAFKA-1724: Errors after reboot in single node setup - KAFKA-1758 https://issues.apache.org/jira/browse/KAFKA-1758: corrupt recovery file prevents startup - KAFKA-1866 https://issues.apache.org/jira/browse/KAFKA-1866: LogStartOffset gauge throws exceptions after log.delete() - KAFKA-1883 https://issues.apache.org/jira/browse/KAFKA-1883: NullPointerException in RequestSendThread - KAFKA-1896 https://issues.apache.org/jira/browse/KAFKA-1896: Record size funcition of record in mirror maker hit NPE when the message value is null. - KAFKA-2101 https://issues.apache.org/jira/browse/KAFKA-2101: Metric metadata-age is reset on a failed update - KAFKA-2112 https://issues.apache.org/jira/browse/KAFKA-2112: make overflowWheel volatile - KAFKA-2117 https://issues.apache.org/jira/browse/KAFKA-2117: OffsetManager uses incorrect field for metadata - KAFKA-2164 https://issues.apache.org/jira/browse/KAFKA-2164: ReplicaFetcherThread: suspicious log message on reset offset - KAFKA-1668 https://issues.apache.org/jira/browse/KAFKA-1668: TopicCommand doesn't warn if --topic argument doesn't match any topics - KAFKA-2198
Re: [DISCUSSION] Kafka 0.8.2.2 release?
+dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114: Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702: Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012: Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406: ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337: Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189: Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308: New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042: New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367: Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867: liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650: Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009: Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407: Only create a log directory when it will be used - KAFKA-2327 https://issues.apache.org/jira/browse/KAFKA-2327: broker doesn't start if config defines advertised.host but not advertised.port - KAFKA-1788: producer record can stay in RecordAccumulator forever if leader is no available - KAFKA-2234 https://issues.apache.org/jira/browse/KAFKA-2234: Partition reassignment of a nonexistent topic prevents future reassignments - KAFKA-2096 https://issues.apache.org/jira/browse/KAFKA-2096: Enable keepalive socket option for broker to prevent socket leak - KAFKA-1057 https://issues.apache.org/jira/browse/KAFKA-1057: Trim whitespaces from user specified configs - KAFKA-1641 https://issues.apache.org/jira/browse/KAFKA-1641: Log cleaner exits if last cleaned offset is lower than earliest offset - KAFKA-1648 https://issues.apache.org/jira/browse/KAFKA-1648: Round robin consumer balance throws an NPE when there are no topics - KAFKA-1724 https://issues.apache.org/jira/browse/KAFKA-1724: Errors after reboot in single node setup - KAFKA-1758 https://issues.apache.org/jira/browse/KAFKA-1758: corrupt recovery file prevents startup - KAFKA-1866 https://issues.apache.org/jira/browse/KAFKA-1866: LogStartOffset gauge throws exceptions after log.delete() - KAFKA-1883 https://issues.apache.org/jira/browse/KAFKA-1883: NullPointerException in RequestSendThread - KAFKA-1896 https://issues.apache.org/jira/browse/KAFKA-1896: Record size funcition of record in mirror maker hit NPE when the message value is null. - KAFKA-2101 https://issues.apache.org/jira/browse/KAFKA-2101: Metric metadata-age is reset on a failed update - KAFKA-2112 https://issues.apache.org/jira/browse/KAFKA-2112: make overflowWheel volatile - KAFKA-2117 https://issues.apache.org/jira/browse/KAFKA-2117: OffsetManager uses incorrect field for metadata - KAFKA-2164 https://issues.apache.org/jira/browse/KAFKA-2164: ReplicaFetcherThread: suspicious log message on reset offset - KAFKA-1668 https://issues.apache.org/jira/browse/KAFKA-1668: TopicCommand doesn't warn if --topic argument doesn't match any topics - KAFKA-2198 https://issues.apache.org/jira/browse/KAFKA-2198: kafka-topics.sh exits with 0 status on failures - KAFKA-2235 https://issues.apache.org/jira/browse/KAFKA-2235: LogCleaner offset map overflow - KAFKA-2241 https://issues.apache.org/jira/browse/KAFKA-2241: AbstractFetcherThread.shutdown() should not block on
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114: Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702: Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012: Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406: ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337: Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189: Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308: New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042: New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367: Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867: liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650: Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009: Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407: Only create a log directory when it will be used - KAFKA-2327 https://issues.apache.org/jira/browse/KAFKA-2327: broker doesn't start if config defines advertised.host but not advertised.port - KAFKA-1788: producer record can stay in RecordAccumulator forever if leader is no available - KAFKA-2234 https://issues.apache.org/jira/browse/KAFKA-2234: Partition reassignment of a nonexistent topic prevents future reassignments - KAFKA-2096 https://issues.apache.org/jira/browse/KAFKA-2096: Enable keepalive socket option for broker to prevent socket leak - KAFKA-1057 https://issues.apache.org/jira/browse/KAFKA-1057: Trim whitespaces from user specified configs - KAFKA-1641 https://issues.apache.org/jira/browse/KAFKA-1641: Log cleaner exits if last cleaned offset is lower than earliest offset - KAFKA-1648 https://issues.apache.org/jira/browse/KAFKA-1648: Round robin consumer balance throws an NPE when there are no topics - KAFKA-1724 https://issues.apache.org/jira/browse/KAFKA-1724: Errors after reboot in single node setup - KAFKA-1758 https://issues.apache.org/jira/browse/KAFKA-1758: corrupt recovery file prevents startup - KAFKA-1866 https://issues.apache.org/jira/browse/KAFKA-1866: LogStartOffset gauge throws exceptions after log.delete() - KAFKA-1883 https://issues.apache.org/jira/browse/KAFKA-1883: NullPointerException in RequestSendThread - KAFKA-1896 https://issues.apache.org/jira/browse/KAFKA-1896: Record size funcition of record in mirror maker hit NPE when the message value is null. - KAFKA-2101 https://issues.apache.org/jira/browse/KAFKA-2101: Metric metadata-age is reset on a failed update - KAFKA-2112 https://issues.apache.org/jira/browse/KAFKA-2112: make overflowWheel volatile - KAFKA-2117 https://issues.apache.org/jira/browse/KAFKA-2117: OffsetManager uses incorrect field for metadata - KAFKA-2164 https://issues.apache.org/jira/browse/KAFKA-2164: ReplicaFetcherThread: suspicious log message on reset offset - KAFKA-1668 https://issues.apache.org/jira/browse/KAFKA-1668: TopicCommand doesn't warn if --topic argument doesn't match any topics - KAFKA-2198 https://issues.apache.org/jira/browse/KAFKA-2198: kafka-topics.sh exits with 0 status on failures - KAFKA-2235 https://issues.apache.org/jira/browse/KAFKA-2235: LogCleaner offset map overflow - KAFKA-2241 https://issues.apache.org/jira/browse/KAFKA-2241: AbstractFetcherThread.shutdown() should not block on ReadableByteChannel.read(buffer) - KAFKA-2272 https://issues.apache.org/jira/browse/KAFKA-2272: listeners endpoint parsing fails if the hostname has capital letter - KAFKA-2345
Re: [DISCUSSION] Kafka 0.8.2.2 release?
The network refactoring portion was not tested well enough yet for me to feel comfortable pushing it into a bugfix release. The new purgatory and MirrorMaker changes are also pretty big. The whole goal of bugfix releases is to make sure it is more stable than the previous releases. On Mon, Aug 17, 2015 at 8:54 AM, Stevo Slavić ssla...@gmail.com wrote: Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk, with new consumer API appropriately annotated/documented as unstable? On Mon, Aug 17, 2015, 17:17 Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114: Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702: Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012: Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406: ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337: Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189: Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308: New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042: New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367: Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867: liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650: Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009: Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407: Only create a log directory when it will be used - KAFKA-2327 https://issues.apache.org/jira/browse/KAFKA-2327: broker doesn't start if config defines advertised.host but not advertised.port - KAFKA-1788: producer record can stay in RecordAccumulator forever if leader is no available - KAFKA-2234 https://issues.apache.org/jira/browse/KAFKA-2234: Partition reassignment of a nonexistent topic prevents future reassignments - KAFKA-2096 https://issues.apache.org/jira/browse/KAFKA-2096: Enable keepalive socket option for broker to prevent socket leak - KAFKA-1057 https://issues.apache.org/jira/browse/KAFKA-1057: Trim whitespaces from user specified configs - KAFKA-1641 https://issues.apache.org/jira/browse/KAFKA-1641: Log cleaner exits if last cleaned offset is lower than earliest offset - KAFKA-1648 https://issues.apache.org/jira/browse/KAFKA-1648: Round robin consumer balance throws an NPE when there are no topics - KAFKA-1724 https://issues.apache.org/jira/browse/KAFKA-1724: Errors after reboot in single node setup - KAFKA-1758 https://issues.apache.org/jira/browse/KAFKA-1758: corrupt recovery file prevents startup - KAFKA-1866 https://issues.apache.org/jira/browse/KAFKA-1866: LogStartOffset gauge throws exceptions after log.delete() - KAFKA-1883 https://issues.apache.org/jira/browse/KAFKA-1883: NullPointerException in RequestSendThread - KAFKA-1896 https://issues.apache.org/jira/browse/KAFKA-1896: Record size funcition of record in mirror maker hit NPE when the message value is null. - KAFKA-2101 https://issues.apache.org/jira/browse/KAFKA-2101: Metric metadata-age is reset on a failed update - KAFKA-2112 https://issues.apache.org/jira/browse/KAFKA-2112: make overflowWheel
Re: [DISCUSSION] Kafka 0.8.2.2 release?
It is pretty typical that Apache projects have a release manager for each release: http://www.apache.org/dev/release-publishing.html http://www.apache.org/dev/release-publishing.html It doesn't have to be the same person every time, though, not even for the same branch. -Flavio On 17 Aug 2015, at 05:01, Gwen Shapira g...@confluent.io wrote: BTW. I think it will be great for Apache Kafka to have a 0.8.2 release manager who's role is to cherrypick low-risk bug-fixes into the 0.8.2 branch and once enough bug fixes happened (or if sufficiently critical fixes happened) to roll out a new maintenance release (with every 3 month as a reasonable bugfix release target).
Re: [DISCUSSION] Kafka 0.8.2.2 release?
but +1 for 0.8.2 patch that marks the new consumer API as unstable (or unimplemented ;) On Mon, Aug 17, 2015 at 9:12 AM, Gwen Shapira g...@confluent.io wrote: The network refactoring portion was not tested well enough yet for me to feel comfortable pushing it into a bugfix release. The new purgatory and MirrorMaker changes are also pretty big. The whole goal of bugfix releases is to make sure it is more stable than the previous releases. On Mon, Aug 17, 2015 at 8:54 AM, Stevo Slavić ssla...@gmail.com wrote: Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk, with new consumer API appropriately annotated/documented as unstable? On Mon, Aug 17, 2015, 17:17 Grant Henke ghe...@cloudera.com wrote: +dev Adding dev list back in. Somehow it got dropped. On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke ghe...@cloudera.com wrote: Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I don't suspect all of these will (or should) make it into the release but this should be a relatively complete list to work from: - KAFKA-2114 https://issues.apache.org/jira/browse/KAFKA-2114: Unable to change min.insync.replicas default - KAFKA-1702 https://issues.apache.org/jira/browse/KAFKA-1702: Messages silently Lost by producer - KAFKA-2012 https://issues.apache.org/jira/browse/KAFKA-2012: Broker should automatically handle corrupt index files - KAFKA-2406 https://issues.apache.org/jira/browse/KAFKA-2406: ISR propagation should be throttled to avoid overwhelming controller. - KAFKA-2336 https://issues.apache.org/jira/browse/KAFKA-2336: Changing offsets.topic.num.partitions after the offset topic is created breaks consumer group partition assignment - KAFKA-2337 https://issues.apache.org/jira/browse/KAFKA-2337: Verify that metric names will not collide when creating new topics - KAFKA-2393 https://issues.apache.org/jira/browse/KAFKA-2393: Correctly Handle InvalidTopicException in KafkaApis.getTopicMetadata() - KAFKA-2189 https://issues.apache.org/jira/browse/KAFKA-2189: Snappy compression of message batches less efficient in 0.8.2.1 - KAFKA-2308 https://issues.apache.org/jira/browse/KAFKA-2308: New producer + Snappy face un-compression errors after broker restart - KAFKA-2042 https://issues.apache.org/jira/browse/KAFKA-2042: New producer metadata update always get all topics. - KAFKA-1367 https://issues.apache.org/jira/browse/KAFKA-1367: Broker topic metadata not kept in sync with ZooKeeper - KAFKA-972 https://issues.apache.org/jira/browse/KAFKA-972: MetadataRequest returns stale list of brokers - KAFKA-1867 https://issues.apache.org/jira/browse/KAFKA-1867: liveBroker list not updated on a cluster with no topics - KAFKA-1650 https://issues.apache.org/jira/browse/KAFKA-1650: Mirror Maker could lose data on unclean shutdown. - KAFKA-2009 https://issues.apache.org/jira/browse/KAFKA-2009: Fix UncheckedOffset.removeOffset synchronization and trace logging issue in mirror maker - KAFKA-2407 https://issues.apache.org/jira/browse/KAFKA-2407: Only create a log directory when it will be used - KAFKA-2327 https://issues.apache.org/jira/browse/KAFKA-2327: broker doesn't start if config defines advertised.host but not advertised.port - KAFKA-1788: producer record can stay in RecordAccumulator forever if leader is no available - KAFKA-2234 https://issues.apache.org/jira/browse/KAFKA-2234: Partition reassignment of a nonexistent topic prevents future reassignments - KAFKA-2096 https://issues.apache.org/jira/browse/KAFKA-2096: Enable keepalive socket option for broker to prevent socket leak - KAFKA-1057 https://issues.apache.org/jira/browse/KAFKA-1057: Trim whitespaces from user specified configs - KAFKA-1641 https://issues.apache.org/jira/browse/KAFKA-1641: Log cleaner exits if last cleaned offset is lower than earliest offset - KAFKA-1648 https://issues.apache.org/jira/browse/KAFKA-1648: Round robin consumer balance throws an NPE when there are no topics - KAFKA-1724 https://issues.apache.org/jira/browse/KAFKA-1724: Errors after reboot in single node setup - KAFKA-1758 https://issues.apache.org/jira/browse/KAFKA-1758: corrupt recovery file prevents startup - KAFKA-1866 https://issues.apache.org/jira/browse/KAFKA-1866: LogStartOffset gauge throws exceptions after log.delete() - KAFKA-1883 https://issues.apache.org/jira/browse/KAFKA-1883: NullPointerException in RequestSendThread - KAFKA-1896 https://issues.apache.org/jira/browse/KAFKA-1896: Record size funcition of record in mirror maker hit NPE when the message value is null. - KAFKA-2101
Re: [DISCUSSION] Kafka 0.8.2.2 release?
BTW. I think it will be great for Apache Kafka to have a 0.8.2 release manager who's role is to cherrypick low-risk bug-fixes into the 0.8.2 branch and once enough bug fixes happened (or if sufficiently critical fixes happened) to roll out a new maintenance release (with every 3 month as a reasonable bugfix release target). This will add some predictability regarding how fast we release fixes for bugs. Gwen On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman jholo...@cloudera.com wrote: +1 for the release and also including https://issues.apache.org/jira/browse/KAFKA-2114 Thanks Jeff On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić ssla...@gmail.com wrote: +1 (non-binding) for 0.8.2.2 release Would be nice to include in that release new producer resiliency bug fixes https://issues.apache.org/jira/browse/KAFKA-1788 and https://issues.apache.org/jira/browse/KAFKA-2120 On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira g...@confluent.io wrote: Will be nice to include Kafka-2308 and fix two critical snappy issues in the maintenance release. Gwen On Aug 14, 2015 6:16 AM, Grant Henke ghe...@cloudera.com wrote: Just to clarify. Will KAFKA-2189 be the only patch in the release? On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy ku...@nmsworks.co.in wrote: +1 for 0.8.2.2 release On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma ism...@juma.me.uk wrote: I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao j...@confluent.io wrote: Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is still a few months away, it may make sense to do an 0.8.2.2 release just to fix this issue. Any objections? Thanks, Jun -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke -- Jeff Holoman Systems Engineer
Re: [DISCUSSION] Kafka 0.8.2.2 release?
+1 for the release and also including https://issues.apache.org/jira/browse/KAFKA-2114 Thanks Jeff On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić ssla...@gmail.com wrote: +1 (non-binding) for 0.8.2.2 release Would be nice to include in that release new producer resiliency bug fixes https://issues.apache.org/jira/browse/KAFKA-1788 and https://issues.apache.org/jira/browse/KAFKA-2120 On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira g...@confluent.io wrote: Will be nice to include Kafka-2308 and fix two critical snappy issues in the maintenance release. Gwen On Aug 14, 2015 6:16 AM, Grant Henke ghe...@cloudera.com wrote: Just to clarify. Will KAFKA-2189 be the only patch in the release? On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy ku...@nmsworks.co.in wrote: +1 for 0.8.2.2 release On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma ism...@juma.me.uk wrote: I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao j...@confluent.io wrote: Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is still a few months away, it may make sense to do an 0.8.2.2 release just to fix this issue. Any objections? Thanks, Jun -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke -- Jeff Holoman Systems Engineer
Re: [DISCUSSION] Kafka 0.8.2.2 release?
+1 to that suggestion. Though I suspect that requires a committer to do. Making it part of the standard commit process could work too. On Aug 16, 2015 11:01 PM, Gwen Shapira g...@confluent.io wrote: BTW. I think it will be great for Apache Kafka to have a 0.8.2 release manager who's role is to cherrypick low-risk bug-fixes into the 0.8.2 branch and once enough bug fixes happened (or if sufficiently critical fixes happened) to roll out a new maintenance release (with every 3 month as a reasonable bugfix release target). This will add some predictability regarding how fast we release fixes for bugs. Gwen On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman jholo...@cloudera.com wrote: +1 for the release and also including https://issues.apache.org/jira/browse/KAFKA-2114 Thanks Jeff On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić ssla...@gmail.com wrote: +1 (non-binding) for 0.8.2.2 release Would be nice to include in that release new producer resiliency bug fixes https://issues.apache.org/jira/browse/KAFKA-1788 and https://issues.apache.org/jira/browse/KAFKA-2120 On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira g...@confluent.io wrote: Will be nice to include Kafka-2308 and fix two critical snappy issues in the maintenance release. Gwen On Aug 14, 2015 6:16 AM, Grant Henke ghe...@cloudera.com wrote: Just to clarify. Will KAFKA-2189 be the only patch in the release? On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy ku...@nmsworks.co.in wrote: +1 for 0.8.2.2 release On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma ism...@juma.me.uk wrote: I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao j...@confluent.io wrote: Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is still a few months away, it may make sense to do an 0.8.2.2 release just to fix this issue. Any objections? Thanks, Jun -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke -- Jeff Holoman Systems Engineer
Re: [DISCUSSION] Kafka 0.8.2.2 release?
+1 (non-binding) for 0.8.2.2 release Would be nice to include in that release new producer resiliency bug fixes https://issues.apache.org/jira/browse/KAFKA-1788 and https://issues.apache.org/jira/browse/KAFKA-2120 On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira g...@confluent.io wrote: Will be nice to include Kafka-2308 and fix two critical snappy issues in the maintenance release. Gwen On Aug 14, 2015 6:16 AM, Grant Henke ghe...@cloudera.com wrote: Just to clarify. Will KAFKA-2189 be the only patch in the release? On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy ku...@nmsworks.co.in wrote: +1 for 0.8.2.2 release On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma ism...@juma.me.uk wrote: I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao j...@confluent.io wrote: Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is still a few months away, it may make sense to do an 0.8.2.2 release just to fix this issue. Any objections? Thanks, Jun -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Is this something also that should be in 0.8.2.2 https://issues.apache.org/jira/browse/KAFKA-2421 wasn't sure what the lz4 usage has been seems to break on some jdks. +1 on the snappy fixes ~ Joe Stein On Fri, Aug 14, 2015 at 5:39 PM, Guozhang Wang wangg...@gmail.com wrote: +1 for both KAFKA-2189 and 2308. On Fri, Aug 14, 2015 at 7:03 AM, Gwen Shapira g...@confluent.io wrote: Will be nice to include Kafka-2308 and fix two critical snappy issues in the maintenance release. Gwen On Aug 14, 2015 6:16 AM, Grant Henke ghe...@cloudera.com wrote: Just to clarify. Will KAFKA-2189 be the only patch in the release? On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy ku...@nmsworks.co.in wrote: +1 for 0.8.2.2 release On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma ism...@juma.me.uk wrote: I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao j...@confluent.io wrote: Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is still a few months away, it may make sense to do an 0.8.2.2 release just to fix this issue. Any objections? Thanks, Jun -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke -- -- Guozhang
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Thanks. Any idea when we can expect 0.8.3? Sent from my phone On Aug 14, 2015, at 5:39 PM, Guozhang Wang wangg...@gmail.com wrote: +1 for both KAFKA-2189 and 2308. On Fri, Aug 14, 2015 at 7:03 AM, Gwen Shapira g...@confluent.io wrote: Will be nice to include Kafka-2308 and fix two critical snappy issues in the maintenance release. Gwen On Aug 14, 2015 6:16 AM, Grant Henke ghe...@cloudera.com wrote: Just to clarify. Will KAFKA-2189 be the only patch in the release? On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy ku...@nmsworks.co.in wrote: +1 for 0.8.2.2 release On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma ism...@juma.me.uk wrote: I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao j...@confluent.io wrote: Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is still a few months away, it may make sense to do an 0.8.2.2 release just to fix this issue. Any objections? Thanks, Jun -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke -- -- Guozhang
Re: [DISCUSSION] Kafka 0.8.2.2 release?
+1 for both KAFKA-2189 and 2308. On Fri, Aug 14, 2015 at 7:03 AM, Gwen Shapira g...@confluent.io wrote: Will be nice to include Kafka-2308 and fix two critical snappy issues in the maintenance release. Gwen On Aug 14, 2015 6:16 AM, Grant Henke ghe...@cloudera.com wrote: Just to clarify. Will KAFKA-2189 be the only patch in the release? On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy ku...@nmsworks.co.in wrote: +1 for 0.8.2.2 release On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma ism...@juma.me.uk wrote: I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao j...@confluent.io wrote: Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is still a few months away, it may make sense to do an 0.8.2.2 release just to fix this issue. Any objections? Thanks, Jun -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke -- -- Guozhang
Re: [DISCUSSION] Kafka 0.8.2.2 release?
I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao j...@confluent.io wrote: Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is still a few months away, it may make sense to do an 0.8.2.2 release just to fix this issue. Any objections? Thanks, Jun
Re: [DISCUSSION] Kafka 0.8.2.2 release?
+1 for 0.8.2.2 release On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma ism...@juma.me.uk wrote: I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao j...@confluent.io wrote: Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is still a few months away, it may make sense to do an 0.8.2.2 release just to fix this issue. Any objections? Thanks, Jun
Re: [DISCUSSION] Kafka 0.8.2.2 release?
Just to clarify. Will KAFKA-2189 be the only patch in the release? On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy ku...@nmsworks.co.in wrote: +1 for 0.8.2.2 release On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma ism...@juma.me.uk wrote: I think this is a good idea as the change is minimal on our side and it has been tested in production for some time by the reporter. Best, Ismael On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao j...@confluent.io wrote: Hi, Everyone, Since the release of Kafka 0.8.2.1, a number of people have reported an issue with snappy compression ( https://issues.apache.org/jira/browse/KAFKA-2189). Basically, if they use snappy in 0.8.2.1, they will experience a 2-3X space increase. The issue has since been fixed in trunk (just a snappy jar upgrade). Since 0.8.3 is still a few months away, it may make sense to do an 0.8.2.2 release just to fix this issue. Any objections? Thanks, Jun -- Grant Henke Software Engineer | Cloudera gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke
