The network refactoring portion was not tested well enough yet for me to feel comfortable pushing it into a bugfix release. The new purgatory and MirrorMaker changes are also pretty big.
The whole goal of bugfix releases is to make sure it is more stable than the previous releases. On Mon, Aug 17, 2015 at 8:54 AM, Stevo Slavić <ssla...@gmail.com> wrote: > Instead of cherrypicking, why not just make 0.8.2.2 of off current trunk, > with new consumer API appropriately annotated/documented as unstable? > > On Mon, Aug 17, 2015, 17:17 Grant Henke <ghe...@cloudera.com> wrote: > > > +dev > > > > Adding dev list back in. Somehow it got dropped. > > > > > > On Mon, Aug 17, 2015 at 10:16 AM, Grant Henke <ghe...@cloudera.com> > wrote: > > > > > Below is a list of candidate bug fix jiras marked fixed for 0.8.3. I > > don't > > > suspect all of these will (or should) make it into the release but this > > > should be a relatively complete list to work from: > > > > > > - KAFKA-2114 <https://issues.apache.org/jira/browse/KAFKA-2114>: > > Unable > > > to change min.insync.replicas default > > > - KAFKA-1702 <https://issues.apache.org/jira/browse/KAFKA-1702>: > > > Messages silently Lost by producer > > > - KAFKA-2012 <https://issues.apache.org/jira/browse/KAFKA-2012>: > > > Broker should automatically handle corrupt index files > > > - KAFKA-2406 <https://issues.apache.org/jira/browse/KAFKA-2406>: > ISR > > > propagation should be throttled to avoid overwhelming controller. > > > - KAFKA-2336 <https://issues.apache.org/jira/browse/KAFKA-2336>: > > > Changing offsets.topic.num.partitions after the offset topic is > > created > > > breaks consumer group partition assignment > > > - KAFKA-2337 <https://issues.apache.org/jira/browse/KAFKA-2337>: > > Verify > > > that metric names will not collide when creating new topics > > > - KAFKA-2393 <https://issues.apache.org/jira/browse/KAFKA-2393>: > > > Correctly Handle InvalidTopicException in > KafkaApis.getTopicMetadata() > > > - KAFKA-2189 <https://issues.apache.org/jira/browse/KAFKA-2189>: > > Snappy > > > compression of message batches less efficient in 0.8.2.1 > > > - KAFKA-2308 <https://issues.apache.org/jira/browse/KAFKA-2308>: > New > > > producer + Snappy face un-compression errors after broker restart > > > - KAFKA-2042 <https://issues.apache.org/jira/browse/KAFKA-2042>: > New > > > producer metadata update always get all topics. > > > - KAFKA-1367 <https://issues.apache.org/jira/browse/KAFKA-1367>: > > Broker > > > topic metadata not kept in sync with ZooKeeper > > > - KAFKA-972 <https://issues.apache.org/jira/browse/KAFKA-972>: > > MetadataRequest > > > returns stale list of brokers > > > - KAFKA-1867 <https://issues.apache.org/jira/browse/KAFKA-1867>: > > liveBroker > > > list not updated on a cluster with no topics > > > - KAFKA-1650 <https://issues.apache.org/jira/browse/KAFKA-1650>: > > Mirror > > > Maker could lose data on unclean shutdown. > > > - KAFKA-2009 <https://issues.apache.org/jira/browse/KAFKA-2009>: > Fix > > > UncheckedOffset.removeOffset synchronization and trace logging issue > > in > > > mirror maker > > > - KAFKA-2407 <https://issues.apache.org/jira/browse/KAFKA-2407>: > Only > > > create a log directory when it will be used > > > - KAFKA-2327 <https://issues.apache.org/jira/browse/KAFKA-2327>: > > > broker doesn't start if config defines advertised.host but not > > > advertised.port > > > - KAFKA-1788: producer record can stay in RecordAccumulator forever > if > > > leader is no available > > > - KAFKA-2234 <https://issues.apache.org/jira/browse/KAFKA-2234>: > > > Partition reassignment of a nonexistent topic prevents future > > reassignments > > > - KAFKA-2096 <https://issues.apache.org/jira/browse/KAFKA-2096>: > > > Enable keepalive socket option for broker to prevent socket leak > > > - KAFKA-1057 <https://issues.apache.org/jira/browse/KAFKA-1057>: > Trim > > > whitespaces from user specified configs > > > - KAFKA-1641 <https://issues.apache.org/jira/browse/KAFKA-1641>: > Log > > > cleaner exits if last cleaned offset is lower than earliest offset > > > - KAFKA-1648 <https://issues.apache.org/jira/browse/KAFKA-1648>: > > Round > > > robin consumer balance throws an NPE when there are no topics > > > - KAFKA-1724 <https://issues.apache.org/jira/browse/KAFKA-1724>: > > > Errors after reboot in single node setup > > > - KAFKA-1758 <https://issues.apache.org/jira/browse/KAFKA-1758>: > > > corrupt recovery file prevents startup > > > - KAFKA-1866 <https://issues.apache.org/jira/browse/KAFKA-1866>: > > > LogStartOffset gauge throws exceptions after log.delete() > > > - KAFKA-1883 <https://issues.apache.org/jira/browse/KAFKA-1883>: > > NullPointerException > > > in RequestSendThread > > > - KAFKA-1896 <https://issues.apache.org/jira/browse/KAFKA-1896>: > > > Record size funcition of record in mirror maker hit NPE when the > > message > > > value is null. > > > - KAFKA-2101 <https://issues.apache.org/jira/browse/KAFKA-2101>: > > > Metric metadata-age is reset on a failed update > > > - KAFKA-2112 <https://issues.apache.org/jira/browse/KAFKA-2112>: > make > > > overflowWheel volatile > > > - KAFKA-2117 <https://issues.apache.org/jira/browse/KAFKA-2117>: > > > OffsetManager uses incorrect field for metadata > > > - KAFKA-2164 <https://issues.apache.org/jira/browse/KAFKA-2164>: > > > ReplicaFetcherThread: suspicious log message on reset offset > > > - KAFKA-1668 <https://issues.apache.org/jira/browse/KAFKA-1668>: > > > TopicCommand doesn't warn if --topic argument doesn't match any > topics > > > - KAFKA-2198 <https://issues.apache.org/jira/browse/KAFKA-2198>: > > > kafka-topics.sh exits with 0 status on failures > > > - KAFKA-2235 <https://issues.apache.org/jira/browse/KAFKA-2235>: > > > LogCleaner offset map overflow > > > - KAFKA-2241 <https://issues.apache.org/jira/browse/KAFKA-2241>: > > > AbstractFetcherThread.shutdown() should not block on > > > ReadableByteChannel.read(buffer) > > > - KAFKA-2272 <https://issues.apache.org/jira/browse/KAFKA-2272>: > > > listeners endpoint parsing fails if the hostname has capital letter > > > - KAFKA-2345 <https://issues.apache.org/jira/browse/KAFKA-2345>: > > > Attempt to delete a topic already marked for deletion throws > > > ZkNodeExistsException > > > - KAFKA-2353 <https://issues.apache.org/jira/browse/KAFKA-2353>: > > > SocketServer.Processor should catch exception and close the socket > > properly > > > in configureNewConnections. > > > - KAFKA-1836 <https://issues.apache.org/jira/browse/KAFKA-1836>: > > > metadata.fetch.timeout.ms set to zero blocks forever > > > - KAFKA-2317 <https://issues.apache.org/jira/browse/KAFKA-2317>: > > De-register > > > isrChangeNotificationListener on controller resignation > > > > > > Note: KAFKA-2120 <https://issues.apache.org/jira/browse/KAFKA-2120> & > > > KAFKA-2421 <https://issues.apache.org/jira/browse/KAFKA-2421> were > > > mentioned in previous emails, but are not in the list because they are > > not > > > committed yet. > > > > > > Hope that helps the effort. > > > > > > Thanks, > > > Grant > > > > > > On Mon, Aug 17, 2015 at 12:09 AM, Grant Henke <ghe...@cloudera.com> > > wrote: > > > > > >> +1 to that suggestion. Though I suspect that requires a committer to > do. > > >> Making it part of the standard commit process could work too. > > >> On Aug 16, 2015 11:01 PM, "Gwen Shapira" <g...@confluent.io> wrote: > > >> > > >>> BTW. I think it will be great for Apache Kafka to have a 0.8.2 > "release > > >>> manager" who's role is to cherrypick low-risk bug-fixes into the > 0.8.2 > > >>> branch and once enough bug fixes happened (or if sufficiently > critical > > >>> fixes happened) to roll out a new maintenance release (with every 3 > > month > > >>> as a reasonable bugfix release target). > > >>> > > >>> This will add some predictability regarding how fast we release fixes > > for > > >>> bugs. > > >>> > > >>> Gwen > > >>> > > >>> On Sun, Aug 16, 2015 at 8:09 PM, Jeff Holoman <jholo...@cloudera.com > > > > >>> wrote: > > >>> > > >>> > +1 for the release and also including > > >>> > > > >>> > https://issues.apache.org/jira/browse/KAFKA-2114 > > >>> > > > >>> > Thanks > > >>> > > > >>> > Jeff > > >>> > > > >>> > On Sun, Aug 16, 2015 at 2:51 PM, Stevo Slavić <ssla...@gmail.com> > > >>> wrote: > > >>> > > > >>> > > +1 (non-binding) for 0.8.2.2 release > > >>> > > > > >>> > > Would be nice to include in that release new producer resiliency > > bug > > >>> > fixes > > >>> > > https://issues.apache.org/jira/browse/KAFKA-1788 and > > >>> > > https://issues.apache.org/jira/browse/KAFKA-2120 > > >>> > > > > >>> > > On Fri, Aug 14, 2015 at 4:03 PM, Gwen Shapira <g...@confluent.io > > > > >>> wrote: > > >>> > > > > >>> > > > Will be nice to include Kafka-2308 and fix two critical snappy > > >>> issues > > >>> > in > > >>> > > > the maintenance release. > > >>> > > > > > >>> > > > Gwen > > >>> > > > On Aug 14, 2015 6:16 AM, "Grant Henke" <ghe...@cloudera.com> > > >>> wrote: > > >>> > > > > > >>> > > > > Just to clarify. Will KAFKA-2189 be the only patch in the > > >>> release? > > >>> > > > > > > >>> > > > > On Fri, Aug 14, 2015 at 7:35 AM, Manikumar Reddy < > > >>> > ku...@nmsworks.co.in > > >>> > > > > > >>> > > > > wrote: > > >>> > > > > > > >>> > > > > > +1 for 0.8.2.2 release > > >>> > > > > > > > >>> > > > > > On Fri, Aug 14, 2015 at 5:49 PM, Ismael Juma < > > >>> ism...@juma.me.uk> > > >>> > > > wrote: > > >>> > > > > > > > >>> > > > > > > I think this is a good idea as the change is minimal on > our > > >>> side > > >>> > > and > > >>> > > > it > > >>> > > > > > has > > >>> > > > > > > been tested in production for some time by the reporter. > > >>> > > > > > > > > >>> > > > > > > Best, > > >>> > > > > > > Ismael > > >>> > > > > > > > > >>> > > > > > > On Fri, Aug 14, 2015 at 1:15 PM, Jun Rao < > j...@confluent.io > > > > > >>> > wrote: > > >>> > > > > > > > > >>> > > > > > > > Hi, Everyone, > > >>> > > > > > > > > > >>> > > > > > > > Since the release of Kafka 0.8.2.1, a number of people > > have > > >>> > > > reported > > >>> > > > > an > > >>> > > > > > > > issue with snappy compression ( > > >>> > > > > > > > https://issues.apache.org/jira/browse/KAFKA-2189). > > >>> Basically, > > >>> > if > > >>> > > > > they > > >>> > > > > > > use > > >>> > > > > > > > snappy in 0.8.2.1, they will experience a 2-3X space > > >>> increase. > > >>> > > The > > >>> > > > > > issue > > >>> > > > > > > > has since been fixed in trunk (just a snappy jar > > upgrade). > > >>> > Since > > >>> > > > > 0.8.3 > > >>> > > > > > is > > >>> > > > > > > > still a few months away, it may make sense to do an > > 0.8.2.2 > > >>> > > release > > >>> > > > > > just > > >>> > > > > > > to > > >>> > > > > > > > fix this issue. Any objections? > > >>> > > > > > > > > > >>> > > > > > > > Thanks, > > >>> > > > > > > > > > >>> > > > > > > > Jun > > >>> > > > > > > > > > >>> > > > > > > > > >>> > > > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > > >>> > > > > -- > > >>> > > > > Grant Henke > > >>> > > > > Software Engineer | Cloudera > > >>> > > > > gr...@cloudera.com | twitter.com/gchenke | > > >>> > linkedin.com/in/granthenke > > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > >>> > > > >>> > > > >>> > -- > > >>> > Jeff Holoman > > >>> > Systems Engineer > > >>> > > > >>> > > >> > > > > > > > > > -- > > > Grant Henke > > > Software Engineer | Cloudera > > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke > > > > > > > > > > > -- > > Grant Henke > > Software Engineer | Cloudera > > gr...@cloudera.com | twitter.com/gchenke | linkedin.com/in/granthenke > > >
