I am in favor of not incubating storm-kafka-client but rather keep it as part of the storm project. We can consider supporting a separate branch, but before we agree to go that route I would like to hear lessons learned from community members that have been part of similar transitions in other projects.
As for not back-porting all the storm-kafka-client changes onto 1.0.x and 1.1.x branches, I expressed my opinion in the JIRA when the this discussion first came up. Basically, I don’t think it is feasible to back-port everything. Furthermore, 1.2.0 will be a minor version for which it is reasonable to expect users of earlier versions to upgrade to without major hassle. In any software it is expected that there will be divergence across minor versions. New versions become available because they have improvements and and sometimes new, small, features in it. If the user wants to benefit from those improvements, she/he will have to upgrade to the most recent version. Compatibility should be accounted for across versions, but as Stig mentioned, for the most part we believe it has been. Although it is possible to copy the 1.x storm-kafka-client changes to 1.1.x and 1.0.x, that same argument could be made for every other connector or storm component, and I am not sure it’s a good principle. I just think that it’s natural and beneficial to the community and the product that users keep upgrading (especially across backwards compatible versions), rather that keeping on patching things up. A few more opinions on storm-kafka-client - Can we identify the pros and cons of keeping Kafka 0.9 dependencies, and unless there are very strong reasons not to do so, simply upgrade to a more recent version. - Although storm-kafka-client was not labeled as beta, technically it was beta, and in practice it was used by users as beta. As far as I know, users that used it from the beginning were still exploring it, and I don’t think that until a lot of the improvements came in it made it to production. So Beta is a nice label to have behind a component that can cover for some necessary non backwards compatible changes. However, I don’t think it would have made a world of difference in practice for storm-kafka-client. - The Kaka community has at times made non backward compatible changes. If Kafka itself makes such changes, is it that concerning if storm-kafka-client has to make or or two such non backward compatible changes? Thanks, Hugo On Jan 25, 2018, at 9:47 AM, P. Taylor Goetz <ptgo...@gmail.com<mailto:ptgo...@gmail.com>> wrote: On Jan 25, 2018, at 12:32 PM, Bobby Evans <ev...@oath.com.INVALID<mailto:ev...@oath.com.INVALID>> wrote: To make this happen though someone is going to need to step up and take lead on this. We will need a plan on what to do (is it becoming a separate repo with a separate release cycle but still a part of the storm project?) Do we plan to spin it off to be an incubator project on its own? I don’t think it’s necessary to spin it off as a separate incubator project, that seems like overkill and would be weird from a community perspective. The path of least resistance might be a separate git repo, or just keeping it in the current Storm repo and decoupling it from the main build/release process so it can be independently released. -Taylor
