[jira] [Comment Edited] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1705#comment-1705 ] Till Rohrmann edited comment on FLINK-13417 at 12/19/19 12:06 PM: -- cc [~liyu2], [~GJL], escalated this issue to be a blocker for the 1.10.0 release. was (Author: till.rohrmann): cc [~liyu2][~GJL], escalated this issue to be a blocker for the 1.10.0 release. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Assignee: Till Rohrmann >Priority: Blocker > Labels: pull-request-available > Fix For: 1.10.0 > > Time Spent: 10m > Remaining Estimate: 0h > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16999279#comment-16999279 ] Zili Chen edited comment on FLINK-13417 at 12/18/19 3:40 PM: - Thanks for reviving this thread [~sewen] & [~trohrmann]. One thing I'd like to emphasize again is that the bump to zk 3.5 will force user update version of their zookeeper ensemble. See also [this comment|https://issues.apache.org/jira/browse/FLINK-13417?focusedCommentId=16921386=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16921386] was (Author: tison): Thanks for reviving this thread [~sewen][~trohrmann]. One thing I'd like to emphasize again is that the bump to zk 3.5 will force user update version of their zookeeper ensemble. See also [this comment|https://issues.apache.org/jira/browse/FLINK-13417?focusedCommentId=16921386=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16921386] > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.11.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-13417) Bump Zookeeper to 3.5.5
[ https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942853#comment-16942853 ] Zili Chen edited comment on FLINK-13417 at 10/2/19 2:20 PM: Run against master repo but it seems not trigger corn job. Could you help trigger corn job on zk-3,5-shaded branch [~trohrmann]? Although we should start a discussion about whether or not and if so, how to migrate from zk 3.4 to zk 3.5(since it enforces our users to bump their zk server version), we can concurrently verify that if there is any technical broker to do so. was (Author: tison): Run against master repo but it seems not trigger corn job. Could you help trigger corn job on zk-3,5-shaded branch [~trohrmann]? Although we should take several discussion about whether or not and if so, how to migrate from zk 3.4 to zk 3.5(since it enforces our users to bump their zk server version), we can concurrently verify that if there is any technical broker to do so. > Bump Zookeeper to 3.5.5 > --- > > Key: FLINK-13417 > URL: https://issues.apache.org/jira/browse/FLINK-13417 > Project: Flink > Issue Type: Improvement > Components: Runtime / Coordination >Affects Versions: 1.9.0 >Reporter: Konstantin Knauf >Priority: Major > Fix For: 1.10.0 > > > User might want to secure their Zookeeper connection via SSL. > This requires a Zookeeper version >= 3.5.1. We might as well try to bump it > to 3.5.5, which is the latest version. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931131#comment-16931131
]
TisonKun edited comment on FLINK-13417 at 9/17/19 6:47 AM:
---
Updates: Manually apply the diff and go through. Maybe there is something
non-deterministic in {{tools/releasing/collect_license_files.sh}}.
Travis deadlock on {{KafkaTestBase}} because ZK AuthFailed, full log is here
https://api.travis-ci.org/v3/job/585870154/log.txt
[~StephanEwen] yes I think it is reasonable to shade Flink's ZK as you
proposed. Otherwise we have to twist our tests quite a bit to let them work.
Even since Kafka and HBase don't really base on ZK 3.5 the twist is brittle.
I'd like to give it a try to see the effect. However, I'm not so familiar with
the procedure so here I'd like to describe what I want to do first.
1. create a sub-module {{flink-shaded-zookeeper}} as what we do for
{{flink-shaded-curator}}. Here I don't know how to generate the NOTICE file.
2. replace {{zookeeper}} dependencies in Flink scope with
{{flink-shaded-zookeeper}}. Here I don't know whether there is an automatic way
to list which dependencies should be replaced.
was (Author: tison):
Updates: Manually apply the diff and go through. Maybe there is something
non-deterministic in {{tools/releasing/collect_license_files.sh}}.
Travis deadlock on {{KafkaTestBase}} because ZK AuthFailed, full log is here
https://api.travis-ci.org/v3/job/585870154/log.txt
[~StephanEwen] yes I think it is reasonable to shade Flink's ZK as you
proposed. Otherwise we have to twist our tests quite a bit to let them work.
I'd like to give it a try to see the effect. However, I'm not so familiar with
the procedure so here I'd like to describe what I want to do first.
1. create a sub-module {{flink-shaded-zookeeper}} as what we do for
{{flink-shaded-curator}}. Here I don't know how to generate the NOTICE file.
2. replace {{zookeeper}} dependencies in Flink scope with
{{flink-shaded-zookeeper}}. Here I don't know whether there is an automatic way
to list which dependencies should be replaced.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Major
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931131#comment-16931131
]
TisonKun edited comment on FLINK-13417 at 9/17/19 6:45 AM:
---
Updates: Manually apply the diff and go through. Maybe there is something
non-deterministic in {{tools/releasing/collect_license_files.sh}}.
Travis deadlock on {{KafkaTestBase}} because ZK AuthFailed, full log is here
https://api.travis-ci.org/v3/job/585870154/log.txt
[~StephanEwen] yes I think it is reasonable to shade Flink's ZK as you
proposed. Otherwise we have to twist our tests quite a bit to let them work.
I'd like to give it a try to see the effect. However, I'm not so familiar with
the procedure so here I'd like to describe what I want to do first.
1. create a sub-module {{flink-shaded-zookeeper}} as what we do for
{{flink-shaded-curator}}. Here I don't know how to generate the NOTICE file.
2. replace {{zookeeper}} dependencies in Flink scope with
{{flink-shaded-zookeeper}}. Here I don't know whether there is an automatic way
to list which dependencies should be replaced.
was (Author: tison):
Updates: Manually apply the diff and go through. Maybe there is something
non-deterministic in {{tools.releasing/collect_license_files.sh}}.
Travis deadlock on {{KafkaTestBase}} because ZK AuthFailed, full log is here
https://api.travis-ci.org/v3/job/585870154/log.txt
[~StephanEwen] yes I think it is reasonable that shade Flink's ZK as you
proposed. Otherwise we have to twist our tests quite a bit to let them work.
I'd like to give it a try to see the effect. However, I'm not so familiar with
the procedure so here I'd like to describe what I want to do first.
1. create a sub-module {{flink-shaded-zookeeper}} as what we do for
{{flink-shaded-curator}}. Here I don't know how to generate the NOTICE file.
2. replace {{zookeeper}} dependencies in Flink scope with
{{flink-shaded-zookeeper}}. Here I don't know whether there is an automatic way
to list which dependencies should be replaced.
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Major
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (FLINK-13417) Bump Zookeeper to 3.5.5
[
https://issues.apache.org/jira/browse/FLINK-13417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928316#comment-16928316
]
TisonKun edited comment on FLINK-13417 at 9/12/19 7:43 AM:
---
[~StephanEwen] I figured out that it is because since 3.5.3 ZooKeeper default
disable four letter words which HBase uses for waiting for server up. We can
set the property in Flink test scope to enable four letter words and workaround
this issue. Have patched the fix in private branch and still digging whether
there are other issues.
references:
[1] https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_4lw
[2]
https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_clusterOptions
(look up {{zookeeper.4lw.commands.whitelist}})
was (Author: tison):
[~StephanEwen] I figured out that it is because since 3.5.3 ZooKeeper default
disable four letter words which HBase uses for waiting for server up. We can
set the property in Flink test scope to enable four letter words and workaround
this issue. Have sent the fix and still digging whether there are other issues.
references:
[1] https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_4lw
[2]
https://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_clusterOptions
(look up {{zookeeper.4lw.commands.whitelist}})
> Bump Zookeeper to 3.5.5
> ---
>
> Key: FLINK-13417
> URL: https://issues.apache.org/jira/browse/FLINK-13417
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.9.0
>Reporter: Konstantin Knauf
>Priority: Blocker
> Fix For: 1.10.0
>
>
> User might want to secure their Zookeeper connection via SSL.
> This requires a Zookeeper version >= 3.5.1. We might as well try to bump it
> to 3.5.5, which is the latest version.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
