Agree that release should include Netty upgrade (STORM-2918)
> On Feb 1, 2018, at 3:39 AM, Satish Duggana <satish.dugg...@gmail.com> wrote:
>
> -1 to include STORM-2918 <https://issues.apache.org/jira/browse/STORM-2918>
>
> Thanks,
> Satish.
>
>
> On Thu, Feb 1, 2018 at 2:28 PM, Artem Ervits <artemerv...@gmail.com> wrote:
>
>> -1 (non-binding)
>> Please include https://issues.apache.org/jira/browse/STORM-2918
>>
>> On Jan 29, 2018 12:03 AM, "Jungtaek Lim" <kabh...@gmail.com> wrote:
>>
>>> We have two different topics in cancelled vote thread. Let's initiate
>>> another threads to continue discussion.
>>>
>>> - Jungtaek Lim (HeartSaVioR)
>>>
>>> 2018년 1월 27일 (토) 오후 11:56, Stig Rohde Døssing <stigdoess...@gmail.com>님이
>>> 작성:
>>>
>>>> I don't think storm-kafka-client needs its own project, but I like the
>>> idea
>>>> of decoupling storm-kafka-client from the release cadence of Storm.
>>>> Primarily because it would allow us to release more frequently, but
>> also
>>>> because I think most of the time there's no tight coupling between the
>>>> Storm minor/patch version and storm-kafka-client. As far as I know
>>>> storm-kafka-client 1.2.0 runs fine on other Storm 1.x versions. If
>>>> storm-kafka-client released independently, we could probably get away
>>> with
>>>> maintaining only 1.x and 2.x compatible versions. I also see some value
>>> in
>>>> being able to do breaking changes independently of Storm's major
>> version,
>>>> without breaking semantic versioning. People clearly expect us to not
>>>> introduce breaking changes in minor/patch releases (e.g. Alexandre when
>>>> storm-kafka-client 1.2.0 wouldn't run with his 1.1.1 topology a while
>>>> back), but we've had to do that a few times to avoid postponing fixes
>>> until
>>>> Storm 2.0.0. Releasing independently would also address Erik's concern
>>> that
>>>> we're releasing a "new" storm-kafka-client 1.1.2 that has known issues.
>>>>
>>>> If we want to decouple storm-kafka-client's release cycle from Storm, I
>>>> don't think we can keep it in the Storm repo. It would get confusing
>> with
>>>> branches and release tags. We might try to decouple it in the same way
>>>> Maven have decoupled their plugins from the main Maven repo, where the
>>> code
>>>> for a given plugin is in a separate repository, but the plugin is still
>>>> part of the Maven project and uses the Maven mailing list for
>> discussion
>>>> and release announcements.
>>>>
>>>> My main concern for moving storm-kafka-client to another repository
>> would
>>>> be how we build against unreleased Storm versions, e.g. 2.0.0. I'm
>>>> wondering if anyone has ideas for this? It looks to me like both the
>>> Maven
>>>> plugins and Bahir are building against only released versions.
>>>>
>>>> @Hugo
>>>> I think at least a few of your points about storm-kafka-client's
>> implied
>>>> beta status would have been easier to handle if storm-kafka-client were
>>> not
>>>> coupled to the Storm release version. It was weird to have a new
>>>> connector's initial release be version 1.0.0, and following Storm's
>>> release
>>>> cycle has prevented us from releasing fixes as often as a new connector
>>>> will likely need. I think we could have also saved ourselves a bit of
>>> work
>>>> by releasing 0.x versions first, since we then wouldn't have had to
>> worry
>>>> about backward compatibility when doing major API changes like
>>>> https://issues.apache.org/jira/browse/STORM-2548.
>>>>
>>>> Regarding whether it is a concern when we make breaking changes in
>>>> storm-kafka-client if it is due to Kafka breaking something: Kafka was
>>> free
>>>> to include breaking changes, because they were still in the pre-1.0.0
>>>> version range which tells the user to expect breaking changes from time
>>> to
>>>> time. When we release breaking changes in a 1.0.y version, it defeats
>> the
>>>> purpose of semantic versioning, which is to tell the user upgrading
>>> whether
>>>> an upgrade can be done freely, or whether they should expect to have to
>>>> recompile and maybe reserve some time to upgrade.
>>>>
>>>> 2018-01-25 23:44 GMT+01:00 Jungtaek Lim <kabh...@gmail.com>:
>>>>
>>>>> Hugo,
>>>>>
>>>>> My idea is basically came from Apache bahir. (
>> http://bahir.apache.org/
>>> )
>>>> It
>>>>> was for Apache Spark, but Flink decided to migrate their connectors
>> to
>>>>> bahir so it is also for Apache Flink.
>>>>> I admit we may want to keep some connectors in core even we split out
>>>>> connectors, since moving to another repo. makes connectors hard to be
>>> in
>>>>> sync with core version, and even has a chance to being forgotten.
>> Kafka
>>>>> connector is first class connector, so maybe storm-kafka-client would
>>> not
>>>>> take this way even we have similar, but we could incubate it and
>> bring
>>> to
>>>>> core once it's stabilized (like storm-kafka-client in 1.2.0).
>>>>>
>>>>> I don't think storm-kafka-client has been technically beta. It might
>> be
>>>>> considered as beta implicitly in 1.0.x, but we had 1.1.0 in 10 months
>>>> ago.
>>>>> storm-kafka-client is introduced over a year being included as Storm
>>>> 1.0.0
>>>>> implicitly announcing we are no longer beta. Explicitly marking as
>> beta
>>>> is
>>>>> very important in practice and that's why InterfaceStability
>> annotation
>>>>> came in and widely used for big project. (We have started to apply it
>>> for
>>>>> streams API in 2.0.0: they're marked as @Unstable.)
>>>>>
>>>>> Thanks,
>>>>> Jungtaek Lim (HeartSaVioR)
>>>>>
>>>>> 2018년 1월 26일 (금) 오전 4:16, Hugo Da Cruz Louro <hlo...@hortonworks.com
>>>> 님이
>>>>> 작성:
>>>>>
>>>>>> I am in favor of not incubating storm-kafka-client but rather keep
>> it
>>>> as
>>>>>> part of the storm project. We can consider supporting a separate
>>>> branch,
>>>>>> but before we agree to go that route I would like to hear lessons
>>>> learned
>>>>>> from community members that have been part of similar transitions
>> in
>>>>> other
>>>>>> projects.
>>>>>>
>>>>>> As for not back-porting all the storm-kafka-client changes onto
>> 1.0.x
>>>> and
>>>>>> 1.1.x branches, I expressed my opinion in the JIRA when the this
>>>>> discussion
>>>>>> first came up. Basically, I don’t think it is feasible to back-port
>>>>>> everything. Furthermore, 1.2.0 will be a minor version for which it
>>> is
>>>>>> reasonable to expect users of earlier versions to upgrade to
>> without
>>>>> major
>>>>>> hassle. In any software it is expected that there will be
>> divergence
>>>>> across
>>>>>> minor versions. New versions become available because they have
>>>>>> improvements and and sometimes new, small, features in it. If the
>>> user
>>>>>> wants to benefit from those improvements, she/he will have to
>> upgrade
>>>> to
>>>>>> the most recent version. Compatibility should be accounted for
>> across
>>>>>> versions, but as Stig mentioned, for the most part we believe it
>> has
>>>>> been.
>>>>>>
>>>>>> Although it is possible to copy the 1.x storm-kafka-client changes
>> to
>>>>>> 1.1.x and 1.0.x, that same argument could be made for every other
>>>>> connector
>>>>>> or storm component, and I am not sure it’s a good principle. I just
>>>> think
>>>>>> that it’s natural and beneficial to the community and the product
>>> that
>>>>>> users keep upgrading (especially across backwards compatible
>>> versions),
>>>>>> rather that keeping on patching things up.
>>>>>>
>>>>>> A few more opinions on storm-kafka-client
>>>>>> - Can we identify the pros and cons of keeping Kafka 0.9
>>> dependencies,
>>>>> and
>>>>>> unless there are very strong reasons not to do so, simply upgrade
>> to
>>> a
>>>>> more
>>>>>> recent version.
>>>>>> - Although storm-kafka-client was not labeled as beta, technically
>> it
>>>> was
>>>>>> beta, and in practice it was used by users as beta. As far as I
>> know,
>>>>> users
>>>>>> that used it from the beginning were still exploring it, and I
>> don’t
>>>>> think
>>>>>> that until a lot of the improvements came in it made it to
>>> production.
>>>> So
>>>>>> Beta is a nice label to have behind a component that can cover for
>>> some
>>>>>> necessary non backwards compatible changes. However, I don’t think
>> it
>>>>> would
>>>>>> have made a world of difference in practice for storm-kafka-client.
>>>>>> - The Kaka community has at times made non backward compatible
>>> changes.
>>>>> If
>>>>>> Kafka itself makes such changes, is it that concerning if
>>>>>> storm-kafka-client has to make or or two such non backward
>> compatible
>>>>>> changes?
>>>>>>
>>>>>> Thanks,
>>>>>> Hugo
>>>>>>
>>>>>> On Jan 25, 2018, at 9:47 AM, P. Taylor Goetz <ptgo...@gmail.com
>>>> <mailto:
>>>>>> ptgo...@gmail.com>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Jan 25, 2018, at 12:32 PM, Bobby Evans <ev...@oath.com.INVALID<
>>>>> mailto:
>>>>>> ev...@oath.com.INVALID>> wrote:
>>>>>>
>>>>>> To make this happen though someone is going to need to step up and
>>> take
>>>>>> lead on this.
>>>>>>
>>>>>> We will need a plan on what to do (is it becoming a separate repo
>>> with
>>>> a
>>>>>> separate release cycle but still a part of the storm project?)
>>>>>> Do we plan to spin it off to be an incubator project on its own?
>>>>>>
>>>>>> I don’t think it’s necessary to spin it off as a separate incubator
>>>>>> project, that seems like overkill and would be weird from a
>> community
>>>>>> perspective.
>>>>>>
>>>>>> The path of least resistance might be a separate git repo, or just
>>>>> keeping
>>>>>> it in the current Storm repo and decoupling it from the main
>>>>> build/release
>>>>>> process so it can be independently released.
>>>>>>
>>>>>> -Taylor
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
