Erik, Open and honest opinions are always welcome. Even if they may come across as harsh (and yours did not). We value your opinion, and that of the mesos community.
I’d like to get these issues resolved and would ask others to help where they can. You’ve been very forthcoming with information to help resolve this. Please don’t stop. The more information the better. Thanks again, -Taylor > On Jan 24, 2018, at 8:12 PM, Erik Weathers <eweath...@groupon.com.INVALID> > wrote: > > Hi Taylor, > > I apologize that this objection is a bit long, incomplete, and late for > this release vote. Also, I by no means intend this as an attack on the > good folks that develop and maintain storm-kafka-client. That being said, > I'm uncomfortable with the situation of storm-kafka-client in the > 1.0.x-branch (and possibly 1.1.x-branch too) and I wonder if everyone is > aware of the situation. There seem to be a number of important > storm-kafka-client changes that haven't been backported to the > 1.0.x-branch. e.g., > > (1) We discovered in storm-1.0.3 that a spout can get stuck forever if its > stored offsets fall behind the earliest available and > FirstPollOffsetStrategy is set to UNCOMMITTED_EARLIEST or > UNCOMMITTED_LATEST. We believe this behavior to be fixed in newer > branches, but not in 1.0.x-branch. The issue is here (note that the > fetchOffset doesn't get updated in the case that it was out of bounds for > the seek): > * https://github.com/apache/storm/blob/v1.0.6/external/ > storm-kafka-client/src/main/java/org/apache/storm/kafka/ > spout/KafkaSpout.java#L188-L192 > > (2) storm-1.0.x is using kafka-clients-0.9.0.1, which isn't acceptable when > using Kafka 0.10 due to the performance impact on the Kafka cluster. I > believe, perhaps naively, that we could use kafka-clients-0.10.x just fine > in storm-kafka-client-1.0.x, even when speaking to a Kafka 0.9 cluster. > (Obviously we'd have to fix any issues with the Kafka API usage from > storm-kafka-clients.) > > > Notably, this situation is especially problematic for storm-on-mesos, since > we *cannot* run anything newer than storm-1.0.x for the daemons (Nimbus, > Supervisor, Worker), because of a fundamental change that was made to the > storm-core scheduling logic. > > I haven't yet spent the time to fully analyze the set of changes made to > storm-kafka-client in the various active branches, so I apologize for a > lack of links to existing PRs and Jira tickets. I also need to file > tickets for some of this stuff, as I believe issue (1) above was fixed in > newer branches as part of a large patch for another issue. > > I also suppose that this isn't super actionable, since it might not be fair > to hold back the entire storm release just for these issues. However, if > we can get these issues fixed in 1.0.x-branch I hope we can cut the 1.0.7 > release soon thereafter. > > Thanks! > > - Erik > >> On Wed, Jan 24, 2018 at 10:41 AM, P. Taylor Goetz <ptgo...@gmail.com> wrote: >> >> This is a call to vote on releasing Apache Storm 1.0.6 (rc1) >> >> Full list of changes in this release: >> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1. >> 0.6-rc1/RELEASE_NOTES.html >> >> The tag/commit to be voted upon is v1.0.6: >> >> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=tree;h >> =24a421e34a71353dc6c750b1f026d06df8ead3f2;hb=bce45993f8622e4 >> d3e9ccba96cc78e4ef76e48ae >> >> The source archive being voted upon can be found here: >> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1. >> 0.6-rc1/apache-storm-1.0.6-src.tar.gz >> >> Other release files, signatures and digests can be found here: >> >> https://dist.apache.org/repos/dist/dev/storm/apache-storm-1.0.6-rc1/ >> >> The release artifacts are signed with the following key: >> >> https://git-wip-us.apache.org/repos/asf?p=storm.git;a=blob_p >> lain;f=KEYS;hb=22b832708295fa2c15c4f3c70ac0d2bc6fded4bd >> >> The Nexus staging repository for this release is: >> >> https://repository.apache.org/content/repositories/orgapachestorm-1054 >> >> Please vote on releasing this package as Apache Storm 1.0.6. >> >> When voting, please list the actions taken to verify the release. >> >> This vote will be open for at least 72 hours. >> >> [ ] +1 Release this package as Apache Storm 1.0.6 >> [ ] 0 No opinion >> [ ] -1 Do not release this package because... >> >> Thanks to everyone who contributed to this release. >> >> -Taylor >>
