I sort of follow Daniel Nelson on this issue: cut a 0.8.2.2 but not include much thing besides Snappy fixes. I mean, include a couple of additional critical bug fixes, if really urgent, and that's it.
On Tue, Aug 18, 2015 at 3:25 PM, Daniel Nelson <daniel.nel...@vungle.com> wrote: > I am strongly in favor of cutting a 0.8.2.2 release, but I don’t think > that it needs to include anything other than the fix for Snappy that kicked > off this discussion in the first place. If there are additional critical > issues that can be included without delaying the process, I see no downside. > > > On Aug 18, 2015, at 10:54 AM, Neha Narkhede <n...@confluent.io> wrote: > > > > How about looking at the scope for the 0.8.3 release first before we cut > > yet another point release off of 0.8.2.2? Each release includes some > > overhead and if there is a larger release in the works, it might be worth > > working on getting that. My take is that the 2 things the community has > > been waiting for is SSL support and the new consumer and we have been > > promising to get 0.8.3 with both those features for several months now. > > > > Looking at the progress on both, it seems we are very close to getting > both > > those checked in and it looks like we should get there in another 5-6 > > weeks. Furthermore, both of these features are large and I anticipate us > > receiving feedback and bugs that will require a couple of point releases > on > > top of 0.8.3 anyway. One possibility is to work on 0.8.3 together now and > > get the community to use the newly released features, gather feedback and > > do point releases incorporating that feedback and iterate on it. > > > > We could absolutely do both 0.8.2.2 and 0.8.3. What I'd ask for is for us > > to look at the 0.8.3 timeline too and make a call whether 0.8.2.2 still > > makes sense. > > > > Thanks, > > Neha > > > > On Tue, Aug 18, 2015 at 10:24 AM, Gwen Shapira <g...@confluent.io> > wrote: > > > >> Thanks Jun. > >> > >> I updated the list with your suggestions. > >> If anyone feels we are missing a critical patch for 0.8.2.2, please > speak > >> up. > >> > >> Gwen > >> > >> On Mon, Aug 17, 2015 at 5:40 PM, Jun Rao <j...@confluent.io> wrote: > >> > >>> Hi, Grant, > >>> > >>> I took a look at that list. None of those is really critical as you > said. > >>> So, I'd suggest that we not include those to minimize the scope of the > >>> release. > >>> > >>> Thanks, > >>> > >>> Jun > >>> > >>> On Mon, Aug 17, 2015 at 5:16 PM, Grant Henke <ghe...@cloudera.com> > >> wrote: > >>> > >>>> Thanks Gwen. > >>>> > >>>> I updated a few small things on the wiki page. > >>>> > >>>> Below is a list of jiras I think could also be marked as included. All > >> of > >>>> these, though not super critical, seem like fairly small and low risk > >>>> changes that help avoid potentially confusing issues or errors for > >> users. > >>>> > >>>> KAFKA-2012 > >>>> KAFKA-972 > >>>> KAFKA-2337 & KAFKA-2393 > >>>> KAFKA-1867 > >>>> KAFKA-2407 > >>>> KAFKA-2234 > >>>> KAFKA-1866 > >>>> KAFKA-2345 & KAFKA-2355 > >>>> > >>>> thoughts? > >>>> > >>>> Thank you, > >>>> Grant > >>>> > >>>> On Mon, Aug 17, 2015 at 4:56 PM, Gwen Shapira <g...@confluent.io> > >> wrote: > >>>> > >>>>> Thanks for creating a list, Grant! > >>>>> > >>>>> I placed it on the wiki with a quick evaluation of the content and > >>>> whether > >>>>> it should be in 0.8.2.2: > >>>>> > >>>>> > >>>> > >>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/Proposed+patches+for+0.8.2.2 > >>>>> > >>>>> I'm attempting to only cherrypick fixes that are both important for > >>> large > >>>>> number of users (or very critical to some users) and very safe > >> (mostly > >>>>> judged by the size of the change, but not only) > >>>>> > >>>>> If your favorite bugfix is missing from the list, or is there but > >>> marked > >>>>> "No", please let us know (in this thread) what we are missing and why > >>> it > >>>> is > >>>>> both important and safe. > >>>>> Also, if I accidentally included something you consider unsafe, speak > >>> up! > >>>>> > >>>>> Gwen > >>>>> > >>>>> On Mon, Aug 17, 2015 at 8:17 AM, Grant Henke <ghe...@cloudera.com> > >>>> wrote: > >>>>> > >>>>>> +dev > >>>>>> > >>>>>> Adding dev list back in. Somehow it got dropped. > >>>>>> > >>>>>> > >>>>>> On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <ghe...@cloudera.com > >>> > >>>>> wrote: > >>>>>> > >>>>>>> Below is a list of candidate bug fix jiras marked fixed for > >> 0.8.3. > >>> I > >>>>>> don't > >>>>>>> suspect all of these will (or should) make it into the release > >> but > >>>> this > >>>>>>> should be a relatively complete list to work from: > >>>>>>> > >>>>>>> - KAFKA-2114 < > >> https://issues.apache.org/jira/browse/KAFKA-2114 > >>>> : > >>>>>> Unable > >>>>>>> to change min.insync.replicas default > >>>>>>> - KAFKA-1702 < > >> https://issues.apache.org/jira/browse/KAFKA-1702 > >>>> : > >>>>>>> Messages silently Lost by producer > >>>>>>> - KAFKA-2012 < > >> https://issues.apache.org/jira/browse/KAFKA-2012 > >>>> : > >>>>>>> Broker should automatically handle corrupt index files > >>>>>>> - KAFKA-2406 < > >> https://issues.apache.org/jira/browse/KAFKA-2406 > >>>> : > >>>>> ISR > >>>>>>> propagation should be throttled to avoid overwhelming > >>> controller. > >>>>>>> - KAFKA-2336 < > >> https://issues.apache.org/jira/browse/KAFKA-2336 > >>>> : > >>>>>>> Changing offsets.topic.num.partitions after the offset topic > >> is > >>>>>> created > >>>>>>> breaks consumer group partition assignment > >>>>>>> - KAFKA-2337 < > >> https://issues.apache.org/jira/browse/KAFKA-2337 > >>>> : > >>>>>> Verify > >>>>>>> that metric names will not collide when creating new topics > >>>>>>> - KAFKA-2393 < > >> https://issues.apache.org/jira/browse/KAFKA-2393 > >>>> : > >>>>>>> Correctly Handle InvalidTopicException in > >>>>> KafkaApis.getTopicMetadata() > >>>>>>> - KAFKA-2189 < > >> https://issues.apache.org/jira/browse/KAFKA-2189 > >>>> : > >>>>>> Snappy > >>>>>>> compression of message batches less efficient in 0.8.2.1 > >>>>>>> - KAFKA-2308 < > >> https://issues.apache.org/jira/browse/KAFKA-2308 > >>>> : > >>>>> New > >>>>>>> producer + Snappy face un-compression errors after broker > >>> restart > >>>>>>> - KAFKA-2042 < > >> https://issues.apache.org/jira/browse/KAFKA-2042 > >>>> : > >>>>> New > >>>>>>> producer metadata update always get all topics. > >>>>>>> - KAFKA-1367 < > >> https://issues.apache.org/jira/browse/KAFKA-1367 > >>>> : > >>>>>> Broker > >>>>>>> topic metadata not kept in sync with ZooKeeper > >>>>>>> - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972 > >>> : > >>>>>> MetadataRequest > >>>>>>> returns stale list of brokers > >>>>>>> - KAFKA-1867 < > >> https://issues.apache.org/jira/browse/KAFKA-1867 > >>>> : > >>>>>> liveBroker > >>>>>>> list not updated on a cluster with no topics > >>>>>>> - KAFKA-1650 < > >> https://issues.apache.org/jira/browse/KAFKA-1650 > >>>> : > >>>>>> Mirror > >>>>>>> Maker could lose data on unclean shutdown. > >>>>>>> - KAFKA-2009 < > >> https://issues.apache.org/jira/browse/KAFKA-2009 > >>>> : > >>>>> Fix > >>>>>>> UncheckedOffset.removeOffset synchronization and trace logging > >>>> issue > >>>>>> in > >>>>>>> mirror maker > >>>>>>> - KAFKA-2407 < > >> https://issues.apache.org/jira/browse/KAFKA-2407 > >>>> : > >>>>> Only > >>>>>>> create a log directory when it will be used > >>>>>>> - KAFKA-2327 < > >> https://issues.apache.org/jira/browse/KAFKA-2327 > >>>> : > >>>>>>> broker doesn't start if config defines advertised.host but not > >>>>>>> advertised.port > >>>>>>> - KAFKA-1788: producer record can stay in RecordAccumulator > >>>> forever > >>>>> if > >>>>>>> leader is no available > >>>>>>> - KAFKA-2234 < > >> https://issues.apache.org/jira/browse/KAFKA-2234 > >>>> : > >>>>>>> Partition reassignment of a nonexistent topic prevents future > >>>>>> reassignments > >>>>>>> - KAFKA-2096 < > >> https://issues.apache.org/jira/browse/KAFKA-2096 > >>>> : > >>>>>>> Enable keepalive socket option for broker to prevent socket > >> leak > >>>>>>> - KAFKA-1057 < > >> https://issues.apache.org/jira/browse/KAFKA-1057 > >>>> : > >>>>> Trim > >>>>>>> whitespaces from user specified configs > >>>>>>> - KAFKA-1641 < > >> https://issues.apache.org/jira/browse/KAFKA-1641 > >>>> : > >>>>> Log > >>>>>>> cleaner exits if last cleaned offset is lower than earliest > >>> offset > >>>>>>> - KAFKA-1648 < > >> https://issues.apache.org/jira/browse/KAFKA-1648 > >>>> : > >>>>>> Round > >>>>>>> robin consumer balance throws an NPE when there are no topics > >>>>>>> - KAFKA-1724 < > >> https://issues.apache.org/jira/browse/KAFKA-1724 > >>>> : > >>>>>>> Errors after reboot in single node setup > >>>>>>> - KAFKA-1758 < > >> https://issues.apache.org/jira/browse/KAFKA-1758 > >>>> : > >>>>>>> corrupt recovery file prevents startup > >>>>>>> - KAFKA-1866 < > >> https://issues.apache.org/jira/browse/KAFKA-1866 > >>>> : > >>>>>>> LogStartOffset gauge throws exceptions after log.delete() > >>>>>>> - KAFKA-1883 < > >> https://issues.apache.org/jira/browse/KAFKA-1883 > >>>> : > >>>>>> NullPointerException > >>>>>>> in RequestSendThread > >>>>>>> - KAFKA-1896 < > >> https://issues.apache.org/jira/browse/KAFKA-1896 > >>>> : > >>>>>>> Record size funcition of record in mirror maker hit NPE when > >> the > >>>>>> message > >>>>>>> value is null. > >>>>>>> - KAFKA-2101 < > >> https://issues.apache.org/jira/browse/KAFKA-2101 > >>>> : > >>>>>>> Metric metadata-age is reset on a failed update > >>>>>>> - KAFKA-2112 < > >> https://issues.apache.org/jira/browse/KAFKA-2112 > >>>> : > >>>>> make > >>>>>>> overflowWheel volatile > >>>>>>> - KAFKA-2117 < > >> https://issues.apache.org/jira/browse/KAFKA-2117 > >>>> : > >>>>>>> OffsetManager uses incorrect field for metadata > >>>>>>> - KAFKA-2164 < > >> https://issues.apache.org/jira/browse/KAFKA-2164 > >>>> : > >>>>>>> ReplicaFetcherThread: suspicious log message on reset offset > >>>>>>> - KAFKA-1668 < > >> https://issues.apache.org/jira/browse/KAFKA-1668 > >>>> : > >>>>>>> TopicCommand doesn't warn if --topic argument doesn't match > >> any > >>>>> topics > >>>>>>> - KAFKA-2198 < > >> https://issues.apache.org/jira/browse/KAFKA-2198 > >>>> : > >>>>>>> kafka-topics.sh exits with 0 status on failures > >>>>>>> - KAFKA-2235 < > >> https://issues.apache.org/jira/browse/KAFKA-2235 > >>>> : > >>>>>>> LogCleaner offset map overflow > >>>>>>> - KAFKA-2241 < > >> https://issues.apache.org/jira/browse/KAFKA-2241 > >>>> : > >>>>>>> AbstractFetcherThread.shutdown() should not block on > >>>>>>> ReadableByteChannel.read(buffer) > >>>>>>> - KAFKA-2272 < > >> https://issues.apache.org/jira/browse/KAFKA-2272 > >>>> : > >>>>>>> listeners endpoint parsing fails if the hostname has capital > >>>> letter > >>>>>>> - KAFKA-2345 < > >> https://issues.apache.org/jira/browse/KAFKA-2345 > >>>> : > >>>>>>> Attempt to delete a topic already marked for deletion throws > >>>>>>> ZkNodeExistsException > >>>>>>> - KAFKA-2353 < > >> https://issues.apache.org/jira/browse/KAFKA-2353 > >>>> : > >>>>>>> SocketServer.Processor should catch exception and close the > >>> socket > >>>>>> properly > >>>>>>> in configureNewConnections. > >>>>>>> - KAFKA-1836 < > >> https://issues.apache.org/jira/browse/KAFKA-1836 > >>>> : > >>>>>>> metadata.fetch.timeout.ms set to zero blocks forever > >>>>>>> - KAFKA-2317 < > >> https://issues.apache.org/jira/browse/KAFKA-2317 > >>>> : > >>>>>> De-register > >>>>>>> isrChangeNotificationListener on controller resignation > >>>>>>> > >>>>>>> Note: KAFKA-2120 < > >> https://issues.apache.org/jira/browse/KAFKA-2120 > >>>> > >>>> & > >>>>>>> KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> > >> were > >>>>>>> mentioned in previous emails, but are not in the list because > >> they > >>>> are > >>>>>> not > >>>>>>> committed yet. > >>>>>>> > >>>>>>> Hope that helps the effort. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> Grant > >>>>>>> > >>>>>>> On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke < > >> ghe...@cloudera.com > >>>> > >>>>>> wrote: > >>>>>>> > >>>>>>>> +1 to that suggestion. Though I suspect that requires a > >> committer > >>> to > >>>>> do. > >>>>>>>> Making it part of the standard commit process could work too. > >>>>>>>> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <g...@confluent.io> > >>> wrote: > >>>>>>>> > >>>>>>>>> BTW. I think it will be great for Apache Kafka to have a 0.8.2 > >>>>> "release > >>>>>>>>> manager" who's role is to cherrypick low-risk bug-fixes into > >> the > >>>>> 0.8.2 > >>>>>>>>> branch and once enough bug fixes happened (or if sufficiently > >>>>> critical > >>>>>>>>> fixes happened) to roll out a new maintenance release (with > >>> every 3 > >>>>>> month > >>>>>>>>> as a reasonable bugfix release target). > >>>>>>>>> > >>>>>>>>> This will add some predictability regarding how fast we release > >>>> fixes > >>>>>> for > >>>>>>>>> bugs. > >>>>>>>>> > >>>>>>>>> Gwen > >>>>>>>>> > >>>>>>>>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman < > >>>> jholo...@cloudera.com > >>>>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> +1 for the release and also including > >>>>>>>>>> > >>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2114 > >>>>>>>>>> > >>>>>>>>>> Thanks > >>>>>>>>>> > >>>>>>>>>> Jeff > >>>>>>>>>> > >>>>>>>>>> On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić < > >>> ssla...@gmail.com > >>>>> > >>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> +1 (non-binding) for 0.8.2.2 release > >>>>>>>>>>> > >>>>>>>>>>> Would be nice to include in that release new producer > >>>> resiliency > >>>>>> bug > >>>>>>>>>> fixes > >>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-1788 and > >>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2120 > >>>>>>>>>>> > >>>>>>>>>>> On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira < > >>>> g...@confluent.io > >>>>>> > >>>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>>> Will be nice to include Kafka-2308 and fix two critical > >>>> snappy > >>>>>>>>> issues > >>>>>>>>>> in > >>>>>>>>>>>> the maintenance release. > >>>>>>>>>>>> > >>>>>>>>>>>> Gwen > >>>>>>>>>>>> On Aug 14, 2015 6:16 AM, "Grant Henke" < > >>> ghe...@cloudera.com> > >>>>>>>>> wrote: > >>>>>>>>>>>> > >>>>>>>>>>>>> Just to clarify. Will KAFKA-2189 be the only patch in > >> the > >>>>>>>>> release? > >>>>>>>>>>>>> > >>>>>>>>>>>>> On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy < > >>>>>>>>>> ku...@nmsworks.co.in > >>>>>>>>>>>> > >>>>>>>>>>>>> wrote: > >>>>>>>>>>>>> > >>>>>>>>>>>>>> +1 for 0.8.2.2 release > >>>>>>>>>>>>>> > >>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma < > >>>>>>>>> ism...@juma.me.uk> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>>> > >>>>>>>>>>>>>>> I think this is a good idea as the change is > >> minimal > >>> on > >>>>> our > >>>>>>>>> side > >>>>>>>>>>> and > >>>>>>>>>>>> it > >>>>>>>>>>>>>> has > >>>>>>>>>>>>>>> been tested in production for some time by the > >>>> reporter. > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> Best, > >>>>>>>>>>>>>>> Ismael > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao < > >>>>> j...@confluent.io > >>>>>>> > >>>>>>>>>> wrote: > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Hi, Everyone, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Since the release of Kafka 0.8.2.1, a number of > >>>> people > >>>>>> have > >>>>>>>>>>>> reported > >>>>>>>>>>>>> an > >>>>>>>>>>>>>>>> issue with snappy compression ( > >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/KAFKA-2189 > >> ). > >>>>>>>>> Basically, > >>>>>>>>>> if > >>>>>>>>>>>>> they > >>>>>>>>>>>>>>> use > >>>>>>>>>>>>>>>> snappy in 0.8.2.1, they will experience a 2-3X > >>> space > >>>>>>>>> increase. > >>>>>>>>>>> The > >>>>>>>>>>>>>> issue > >>>>>>>>>>>>>>>> has since been fixed in trunk (just a snappy jar > >>>>>> upgrade). > >>>>>>>>>> Since > >>>>>>>>>>>>> 0.8.3 > >>>>>>>>>>>>>> is > >>>>>>>>>>>>>>>> still a few months away, it may make sense to do > >> an > >>>>>> 0.8.2.2 > >>>>>>>>>>> release > >>>>>>>>>>>>>> just > >>>>>>>>>>>>>>> to > >>>>>>>>>>>>>>>> fix this issue. Any objections? > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Thanks, > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>>> Jun > >>>>>>>>>>>>>>>> > >>>>>>>>>>>>>>> > >>>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> > >>>>>>>>>>>>> -- > >>>>>>>>>>>>> Grant Henke > >>>>>>>>>>>>> Software Engineer | Cloudera > >>>>>>>>>>>>> gr...@cloudera.com | twitter.com/gchenke | > >>>>>>>>>> linkedin.com/in/granthenke > >>>>>>>>>>>>> > >>>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> -- > >>>>>>>>>> Jeff Holoman > >>>>>>>>>> Systems Engineer > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> Grant Henke > >>>>>>> Software Engineer | Cloudera > >>>>>>> gr...@cloudera.com | twitter.com/gchenke | > >>>> linkedin.com/in/granthenke > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Grant Henke > >>>>>> Software Engineer | Cloudera > >>>>>> gr...@cloudera.com | twitter.com/gchenke | > >>> linkedin.com/in/granthenke > >>>>>> > >>>>> > >>>> > >>>> > >>>> > >>>> -- > >>>> Grant Henke > >>>> Software Engineer | Cloudera > >>>> gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke > >>>> > >>> > >> > > > > > > > > -- > > Thanks, > > Neha > >
