[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[ https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721358#comment-17721358 ] ASF GitHub Bot commented on HADOOP-18709: - ferdelyi opened a new pull request, #5638: URL: https://github.com/apache/hadoop/pull/5638 … SSL/TLS into the common library ### Description of PR In order to enable SSL/TLS for the ZK Client to communicate with the secured ZK Server, the configuration parameters for the truststore and the keystore are introduced into the core-default.xml. These parameters then used in the ZKCuratorManager class to set up the ZK Client for SSL/TLS encryption when the caller sets the sslEnabled flag true. The secured ZK Client uses Netty, hence the dependency is included in the pom.xml. A separate unit test class was introduced (TestSecureZKCuratorManager), as the secured server needs a very different setup from the already implemented unit test (TestZKCuratorManager). ### How was this patch tested? This patch was tested with a unit test. ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Add curator based ZooKeeper communication support over SSL/TLS into the > common library > -- > > Key: HADOOP-18709 > URL: https://issues.apache.org/jira/browse/HADOOP-18709 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ferenc Erdelyi >Assignee: Ferenc Erdelyi >Priority: Major > > With HADOOP-16579 the ZooKeeper client is capable of securing communication > with SSL. > To follow the convention introduced in HADOOP-14741, proposing to add to the > core-default.xml the following configurations, as the groundwork for the > components to enable encrypted communication between the individual > components and ZooKeeper: > * hadoop.zk.ssl.keystore.location > * hadoop.zk.ssl.keystore.password > * hadoop.zk.ssl.truststore.location > * hadoop.zk.ssl.truststore.password > These parameters along with the component-specific ssl.client.enable option > (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the > ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a > new overloaded start() method to build the encrypted communication. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721461#comment-17721461
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1542486038
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 29s | | trunk passed |
| +1 :green_heart: | compile | 20m 26s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 22m 18s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 4s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 10s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 48s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 51s | | the patch passed |
| -1 :x: | compile | 7m 30s |
[/patch-compile-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/1/artifact/out/patch-compile-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt)
| root in the patch failed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1. |
| -1 :x: | javac | 7m 30s |
[/patch-compile-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/1/artifact/out/patch-compile-root-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt)
| root in the patch failed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1. |
| -1 :x: | compile | 6m 57s |
[/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/1/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt)
| root in the patch failed with JDK Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09. |
| -1 :x: | javac | 6m 57s |
[/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/1/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt)
| root in the patch failed with JDK Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 48s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 30 new + 70
unchanged - 0 fixed = 100 total (was 70) |
| +1 :green_heart: | mvnsite | 1m 14s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| -1 :x: | javadoc | 0m 47s |
[/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/1/artifact/out/results-javadoc-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt)
|
hadoop-common-project_hadoop-common-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 generated 8 new + 0
unchanged - 0 fixed = 8 total (was 0) |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK
Private Build-1.8.0
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721948#comment-17721948
]
ASF GitHub Bot commented on HADOOP-18709:
-
szilard-nemeth commented on code in PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#discussion_r1191770472
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java:
##
@@ -157,12 +175,44 @@ public void start(List authInfos) throws
IOException {
authInfos.add(new AuthInfo(zkAuth.getScheme(), zkAuth.getAuth()));
}
+/* Pre-check on SSL/TLS client connection requirements to emit the name of
the
+configuration missing. It improves supportability. */
+if(sslEnabled) {
+ if
(StringUtils.isEmpty(conf.get(CommonConfigurationKeys.ZK_SSL_KEYSTORE_LOCATION)))
{
+throw new ConfigurationException(
Review Comment:
Nit: Can you extract this to a helper method?
Only the config name should be passed, the rest of the method body (even the
exception message) can be the same.
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java:
##
@@ -478,10 +558,53 @@ public ZooKeeper newZooKeeper(String connectString, int
sessionTimeout,
if (zkClientConfig.isSaslClientEnabled() &&
!isJaasConfigurationSet(zkClientConfig)) {
setJaasConfiguration(zkClientConfig);
}
+ if (sslEnabled) {
+setSslConfiguration(zkClientConfig);
+ }
return new ZooKeeper(connectString, sessionTimeout, watcher,
canBeReadOnly, zkClientConfig);
}
+/**
+ * Configure ZooKeeper Client with SSL/TLS connection.
+ * @param zkClientConfig ZooKeeper Client configuration
+ * */
+private void setSslConfiguration(ZKClientConfig zkClientConfig) throws
ConfigurationException {
+ this.setSslConfiguration(zkClientConfig, new ClientX509Util());
+}
+public void setSslConfiguration(ZKClientConfig zkClientConfig,
ClientX509Util x509Util )
Review Comment:
Nit: Line break between methods.
Also it seems your formatter seems to be off, as method parentheses has an
additional space before it after the parameters.
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java:
##
@@ -478,10 +558,53 @@ public ZooKeeper newZooKeeper(String connectString, int
sessionTimeout,
if (zkClientConfig.isSaslClientEnabled() &&
!isJaasConfigurationSet(zkClientConfig)) {
setJaasConfiguration(zkClientConfig);
}
+ if (sslEnabled) {
+setSslConfiguration(zkClientConfig);
+ }
return new ZooKeeper(connectString, sessionTimeout, watcher,
canBeReadOnly, zkClientConfig);
}
+/**
+ * Configure ZooKeeper Client with SSL/TLS connection.
+ * @param zkClientConfig ZooKeeper Client configuration
+ * */
+private void setSslConfiguration(ZKClientConfig zkClientConfig) throws
ConfigurationException {
+ this.setSslConfiguration(zkClientConfig, new ClientX509Util());
+}
+public void setSslConfiguration(ZKClientConfig zkClientConfig,
ClientX509Util x509Util )
+throws ConfigurationException {
+ LOG.info("Configuring the ZooKeeper client to use SSL/TLS encryption for
connecting to the ZooKeeper server.");
+ if (StringUtils.isEmpty(this.keystoreLocation)) {
Review Comment:
Nit: Here you could also extract the isempty check + throwing the exception
to a method.
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/curator/TestSecureZKCuratorManager.java:
##
@@ -0,0 +1,157 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.util.curator;
+
+import org.apache.curator.test.InstanceSpec;
+import org.apache.curator.test.TestingServer;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.CommonConfigurationKeys;
+import org.apache.zookeeper.ZooKeeper;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.common.Cl
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[ https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17721950#comment-17721950 ] ASF GitHub Bot commented on HADOOP-18709: - szilard-nemeth commented on PR #5638: URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1544898512 Thanks for the patch @ferdelyi, good job. Added some comments / questions. Could you please add testing evidence to the Jira? To document how this was tested on a real cluster environment. Also, is there a ZK config page among Hadooop documentation that need to be updated? Thanks > Add curator based ZooKeeper communication support over SSL/TLS into the > common library > -- > > Key: HADOOP-18709 > URL: https://issues.apache.org/jira/browse/HADOOP-18709 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ferenc Erdelyi >Assignee: Ferenc Erdelyi >Priority: Major > Labels: pull-request-available > > With HADOOP-16579 the ZooKeeper client is capable of securing communication > with SSL. > To follow the convention introduced in HADOOP-14741, proposing to add to the > core-default.xml the following configurations, as the groundwork for the > components to enable encrypted communication between the individual > components and ZooKeeper: > * hadoop.zk.ssl.keystore.location > * hadoop.zk.ssl.keystore.password > * hadoop.zk.ssl.truststore.location > * hadoop.zk.ssl.truststore.password > These parameters along with the component-specific ssl.client.enable option > (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the > ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a > new overloaded start() method to build the encrypted communication. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724450#comment-17724450
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1555369000
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 10s | | trunk passed |
| +1 :green_heart: | compile | 15m 35s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 14m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 16s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 35s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 17s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | | the patch passed |
| +1 :green_heart: | compile | 15m 0s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 15m 0s | | the patch passed |
| +1 :green_heart: | compile | 14m 30s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 14m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 8s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 32s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 8s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 52s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 33s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 27s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 9s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 1m 0s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/13/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| | | 181m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux 13421f96a9da 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 521931a58471fe5da6a2fd792f7550f5b737ef46 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/13/testReport/ |
| Max. process+thread count | 1332 (vs. ulimit of 5500) |
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724464#comment-17724464
]
ASF GitHub Bot commented on HADOOP-18709:
-
szilard-nemeth commented on code in PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#discussion_r1199521127
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java:
##
@@ -157,12 +175,44 @@ public void start(List authInfos) throws
IOException {
authInfos.add(new AuthInfo(zkAuth.getScheme(), zkAuth.getAuth()));
}
+/* Pre-check on SSL/TLS client connection requirements to emit the name of
the
+configuration missing. It improves supportability. */
+if(sslEnabled) {
+ if
(StringUtils.isEmpty(conf.get(CommonConfigurationKeys.ZK_SSL_KEYSTORE_LOCATION)))
{
+throw new ConfigurationException(
Review Comment:
I meant a method called something like validateX(String confKey) that throws
the exception. The exception message is repeated 4 times, but it's not the end
of the world if we don't do this. It's okay how it is now :)
> Add curator based ZooKeeper communication support over SSL/TLS into the
> common library
> --
>
> Key: HADOOP-18709
> URL: https://issues.apache.org/jira/browse/HADOOP-18709
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ferenc Erdelyi
>Assignee: Ferenc Erdelyi
>Priority: Major
> Labels: pull-request-available
>
> With HADOOP-16579 the ZooKeeper client is capable of securing communication
> with SSL.
> To follow the convention introduced in HADOOP-14741, proposing to add to the
> core-default.xml the following configurations, as the groundwork for the
> components to enable encrypted communication between the individual
> components and ZooKeeper:
> * hadoop.zk.ssl.keystore.location
> * hadoop.zk.ssl.keystore.password
> * hadoop.zk.ssl.truststore.location
> * hadoop.zk.ssl.truststore.password
> These parameters along with the component-specific ssl.client.enable option
> (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the
> ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a
> new overloaded start() method to build the encrypted communication.
> * The secured ZK Client uses Netty, hence the dependency is included in the
> pom.xml. Added netty-handler and netty-transport-native-epoll dependency to
> the pom.xml based on ZOOKEEPER-3494 - "No need to depend on netty-all (SSL)".
> * The change was exclusively tested with the unit test, which is a kind of
> integration test, as a ZK Server was brought up and the communication tested
> between the client and the server.
> * This code change is in the common code base and there is no component
> calling it yet. Once YARN-11468 - "Zookeeper SSL/TLS support" is implemented,
> we can test it in a real cluster environment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724465#comment-17724465
]
ASF GitHub Bot commented on HADOOP-18709:
-
szilard-nemeth commented on code in PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#discussion_r1199521345
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java:
##
@@ -478,10 +558,53 @@ public ZooKeeper newZooKeeper(String connectString, int
sessionTimeout,
if (zkClientConfig.isSaslClientEnabled() &&
!isJaasConfigurationSet(zkClientConfig)) {
setJaasConfiguration(zkClientConfig);
}
+ if (sslEnabled) {
+setSslConfiguration(zkClientConfig);
+ }
return new ZooKeeper(connectString, sessionTimeout, watcher,
canBeReadOnly, zkClientConfig);
}
+/**
+ * Configure ZooKeeper Client with SSL/TLS connection.
+ * @param zkClientConfig ZooKeeper Client configuration
+ * */
+private void setSslConfiguration(ZKClientConfig zkClientConfig) throws
ConfigurationException {
+ this.setSslConfiguration(zkClientConfig, new ClientX509Util());
+}
+public void setSslConfiguration(ZKClientConfig zkClientConfig,
ClientX509Util x509Util )
Review Comment:
There was also a missing one before method: validateSslConfiguration
I fixed the javadoc as it was not starting with /** (but with: /*) before
committing the change. Something must be odd with your formatter.
> Add curator based ZooKeeper communication support over SSL/TLS into the
> common library
> --
>
> Key: HADOOP-18709
> URL: https://issues.apache.org/jira/browse/HADOOP-18709
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ferenc Erdelyi
>Assignee: Ferenc Erdelyi
>Priority: Major
> Labels: pull-request-available
>
> With HADOOP-16579 the ZooKeeper client is capable of securing communication
> with SSL.
> To follow the convention introduced in HADOOP-14741, proposing to add to the
> core-default.xml the following configurations, as the groundwork for the
> components to enable encrypted communication between the individual
> components and ZooKeeper:
> * hadoop.zk.ssl.keystore.location
> * hadoop.zk.ssl.keystore.password
> * hadoop.zk.ssl.truststore.location
> * hadoop.zk.ssl.truststore.password
> These parameters along with the component-specific ssl.client.enable option
> (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the
> ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a
> new overloaded start() method to build the encrypted communication.
> * The secured ZK Client uses Netty, hence the dependency is included in the
> pom.xml. Added netty-handler and netty-transport-native-epoll dependency to
> the pom.xml based on ZOOKEEPER-3494 - "No need to depend on netty-all (SSL)".
> * The change was exclusively tested with the unit test, which is a kind of
> integration test, as a ZK Server was brought up and the communication tested
> between the client and the server.
> * This code change is in the common code base and there is no component
> calling it yet. Once YARN-11468 - "Zookeeper SSL/TLS support" is implemented,
> we can test it in a real cluster environment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[ https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724466#comment-17724466 ] ASF GitHub Bot commented on HADOOP-18709: - szilard-nemeth commented on code in PR #5638: URL: https://github.com/apache/hadoop/pull/5638#discussion_r1199521447 ## hadoop-common-project/hadoop-common/pom.xml: ## @@ -342,6 +342,14 @@ + Review Comment: Cool, thanks :) > Add curator based ZooKeeper communication support over SSL/TLS into the > common library > -- > > Key: HADOOP-18709 > URL: https://issues.apache.org/jira/browse/HADOOP-18709 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ferenc Erdelyi >Assignee: Ferenc Erdelyi >Priority: Major > Labels: pull-request-available > > With HADOOP-16579 the ZooKeeper client is capable of securing communication > with SSL. > To follow the convention introduced in HADOOP-14741, proposing to add to the > core-default.xml the following configurations, as the groundwork for the > components to enable encrypted communication between the individual > components and ZooKeeper: > * hadoop.zk.ssl.keystore.location > * hadoop.zk.ssl.keystore.password > * hadoop.zk.ssl.truststore.location > * hadoop.zk.ssl.truststore.password > These parameters along with the component-specific ssl.client.enable option > (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the > ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a > new overloaded start() method to build the encrypted communication. > * The secured ZK Client uses Netty, hence the dependency is included in the > pom.xml. Added netty-handler and netty-transport-native-epoll dependency to > the pom.xml based on ZOOKEEPER-3494 - "No need to depend on netty-all (SSL)". > * The change was exclusively tested with the unit test, which is a kind of > integration test, as a ZK Server was brought up and the communication tested > between the client and the server. > * This code change is in the common code base and there is no component > calling it yet. Once YARN-11468 - "Zookeeper SSL/TLS support" is implemented, > we can test it in a real cluster environment. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724467#comment-17724467
]
ASF GitHub Bot commented on HADOOP-18709:
-
szilard-nemeth commented on code in PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#discussion_r1199521669
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/curator/TestSecureZKCuratorManager.java:
##
@@ -0,0 +1,157 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.util.curator;
+
+import org.apache.curator.test.InstanceSpec;
+import org.apache.curator.test.TestingServer;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.CommonConfigurationKeys;
+import org.apache.zookeeper.ZooKeeper;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.common.ClientX509Util;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.FileContext.LOG;
+import static org.junit.Assert.assertEquals;
+
+
+/**
+ * Test the manager for ZooKeeper Curator when SSL/TLS is enabled for the ZK
server-client connection.
+ */
+public class TestSecureZKCuratorManager {
+
+ private TestingServer server;
+ private ZKCuratorManager curator;
+ private Configuration hadoopConf;
+ private Integer secureClientPort = 2281;
+ private File zkDataDir = new File("testZkSSLClientConnectionDataDir");
+
+ @Before
+ public void setup() throws Exception {
+//set zkServer
+this.hadoopConf = setUpSecure();
+Map customConfiguration = new HashMap<>();
+
customConfiguration.put("secureClientPort",this.secureClientPort.toString());
+customConfiguration.put("audit.enable",true);
+
+InstanceSpec spec = new InstanceSpec(
+this.zkDataDir,
+this.secureClientPort,
+-1,
+-1,
+true,
+1,
+100,
+10,
+customConfiguration);
+this.server = new TestingServer(spec, true);
+hadoopConf.set(CommonConfigurationKeys.ZK_ADDRESS,
this.server.getConnectString());
+this.curator = new ZKCuratorManager(hadoopConf);
+this.curator.start(new ArrayList<>(), true);
+ }
+
+ public Configuration setUpSecure() throws Exception {
+Configuration hadoopConf = new Configuration();
+String testDataPath =
"src/test/java/org/apache/hadoop/util/curator/resources/data";
+System.setProperty("zookeeper.serverCnxnFactory",
"org.apache.zookeeper.server.NettyServerCnxnFactory");
+//System.setProperty("zookeeper.client.secure", "true");
+
+
+System.setProperty("zookeeper.ssl.keyStore.location", testDataPath +
"/ssl/keystore.jks");
+System.setProperty("zookeeper.ssl.keyStore.password", "password");
+System.setProperty("zookeeper.ssl.trustStore.location", testDataPath +
"/ssl/truststore.jks");
+System.setProperty("zookeeper.ssl.trustStore.password", "password");
+System.setProperty("zookeeper.request.timeout", "12345");
+
+System.setProperty("jute.maxbuffer", "469296129");
Review Comment:
I see. thanks
> Add curator based ZooKeeper communication support over SSL/TLS into the
> common library
> --
>
> Key: HADOOP-18709
> URL: https://issues.apache.org/jira/browse/HADOOP-18709
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ferenc Erdelyi
>Assignee: Ferenc Erdelyi
>Priority: Major
> Labels: pull-request-available
>
> With HADOOP-16579 the ZooKeeper client is capable of securing communication
> with SSL.
> To follow the convention introduced in HADOOP-14741, proposing to add to the
> core-default.xml the following configurations, as the groundwork for the
> components to enable encrypted communication between the individual
> components and ZooKeeper:
> * hadoop.zk.ssl.keystore.location
> * hadoop
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724468#comment-17724468
]
ASF GitHub Bot commented on HADOOP-18709:
-
szilard-nemeth commented on code in PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#discussion_r1199522108
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/curator/TestSecureZKCuratorManager.java:
##
@@ -0,0 +1,157 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.util.curator;
+
+import org.apache.curator.test.InstanceSpec;
+import org.apache.curator.test.TestingServer;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.CommonConfigurationKeys;
+import org.apache.zookeeper.ZooKeeper;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.common.ClientX509Util;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.FileContext.LOG;
+import static org.junit.Assert.assertEquals;
+
+
+/**
+ * Test the manager for ZooKeeper Curator when SSL/TLS is enabled for the ZK
server-client connection.
+ */
+public class TestSecureZKCuratorManager {
+
+ private TestingServer server;
+ private ZKCuratorManager curator;
+ private Configuration hadoopConf;
+ private Integer secureClientPort = 2281;
+ private File zkDataDir = new File("testZkSSLClientConnectionDataDir");
+
+ @Before
+ public void setup() throws Exception {
+//set zkServer
+this.hadoopConf = setUpSecure();
+Map customConfiguration = new HashMap<>();
+
customConfiguration.put("secureClientPort",this.secureClientPort.toString());
+customConfiguration.put("audit.enable",true);
+
+InstanceSpec spec = new InstanceSpec(
+this.zkDataDir,
+this.secureClientPort,
+-1,
+-1,
+true,
+1,
+100,
+10,
+customConfiguration);
+this.server = new TestingServer(spec, true);
+hadoopConf.set(CommonConfigurationKeys.ZK_ADDRESS,
this.server.getConnectString());
+this.curator = new ZKCuratorManager(hadoopConf);
+this.curator.start(new ArrayList<>(), true);
+ }
+
+ public Configuration setUpSecure() throws Exception {
+Configuration hadoopConf = new Configuration();
+String testDataPath =
"src/test/java/org/apache/hadoop/util/curator/resources/data";
+System.setProperty("zookeeper.serverCnxnFactory",
"org.apache.zookeeper.server.NettyServerCnxnFactory");
+//System.setProperty("zookeeper.client.secure", "true");
+
+
+System.setProperty("zookeeper.ssl.keyStore.location", testDataPath +
"/ssl/keystore.jks");
+System.setProperty("zookeeper.ssl.keyStore.password", "password");
+System.setProperty("zookeeper.ssl.trustStore.location", testDataPath +
"/ssl/truststore.jks");
+System.setProperty("zookeeper.ssl.trustStore.password", "password");
+System.setProperty("zookeeper.request.timeout", "12345");
+
+System.setProperty("jute.maxbuffer", "469296129");
+
+System.setProperty("javax.net.debug", "ssl");
+System.setProperty("zookeeper.authProvider.x509",
"org.apache.zookeeper.server.auth.X509AuthenticationProvider");
+
+
+hadoopConf.set(CommonConfigurationKeys.ZK_SSL_KEYSTORE_LOCATION,
testDataPath + "/ssl/keystore.jks");
+hadoopConf.set(CommonConfigurationKeys.ZK_SSL_KEYSTORE_PASSWORD,
"password");
+hadoopConf.set(CommonConfigurationKeys.ZK_SSL_TRUSTSTORE_LOCATION,
testDataPath + "/ssl/truststore.jks");
+hadoopConf.set(CommonConfigurationKeys.ZK_SSL_TRUSTSTORE_PASSWORD,
"password");
+return hadoopConf;
+ }
+
+ @After
+ public void teardown() throws Exception {
+this.curator.close();
+if (this.server != null) {
+ this.server.close();
+ this.server = null;
Review Comment:
As the setup is annotated with @Before, it always initializes this.server
with an Object. I don't think setting it to null makes anything better. Let's
leave it as
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724469#comment-17724469
]
ASF GitHub Bot commented on HADOOP-18709:
-
szilard-nemeth commented on code in PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#discussion_r1199521345
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java:
##
@@ -478,10 +558,53 @@ public ZooKeeper newZooKeeper(String connectString, int
sessionTimeout,
if (zkClientConfig.isSaslClientEnabled() &&
!isJaasConfigurationSet(zkClientConfig)) {
setJaasConfiguration(zkClientConfig);
}
+ if (sslEnabled) {
+setSslConfiguration(zkClientConfig);
+ }
return new ZooKeeper(connectString, sessionTimeout, watcher,
canBeReadOnly, zkClientConfig);
}
+/**
+ * Configure ZooKeeper Client with SSL/TLS connection.
+ * @param zkClientConfig ZooKeeper Client configuration
+ * */
+private void setSslConfiguration(ZKClientConfig zkClientConfig) throws
ConfigurationException {
+ this.setSslConfiguration(zkClientConfig, new ClientX509Util());
+}
+public void setSslConfiguration(ZKClientConfig zkClientConfig,
ClientX509Util x509Util )
Review Comment:
There was also a missing one before method: validateSslConfiguration
Can you fix the javadoc as it is not starting with /** (but with: /*) ?
Something must be odd with your formatter.
> Add curator based ZooKeeper communication support over SSL/TLS into the
> common library
> --
>
> Key: HADOOP-18709
> URL: https://issues.apache.org/jira/browse/HADOOP-18709
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ferenc Erdelyi
>Assignee: Ferenc Erdelyi
>Priority: Major
> Labels: pull-request-available
>
> With HADOOP-16579 the ZooKeeper client is capable of securing communication
> with SSL.
> To follow the convention introduced in HADOOP-14741, proposing to add to the
> core-default.xml the following configurations, as the groundwork for the
> components to enable encrypted communication between the individual
> components and ZooKeeper:
> * hadoop.zk.ssl.keystore.location
> * hadoop.zk.ssl.keystore.password
> * hadoop.zk.ssl.truststore.location
> * hadoop.zk.ssl.truststore.password
> These parameters along with the component-specific ssl.client.enable option
> (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the
> ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a
> new overloaded start() method to build the encrypted communication.
> * The secured ZK Client uses Netty, hence the dependency is included in the
> pom.xml. Added netty-handler and netty-transport-native-epoll dependency to
> the pom.xml based on ZOOKEEPER-3494 - "No need to depend on netty-all (SSL)".
> * The change was exclusively tested with the unit test, which is a kind of
> integration test, as a ZK Server was brought up and the communication tested
> between the client and the server.
> * This code change is in the common code base and there is no component
> calling it yet. Once YARN-11468 - "Zookeeper SSL/TLS support" is implemented,
> we can test it in a real cluster environment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724473#comment-17724473
]
ASF GitHub Bot commented on HADOOP-18709:
-
szilard-nemeth commented on code in PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#discussion_r1199527654
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java:
##
@@ -503,4 +644,50 @@ private void setJaasConfiguration(ZKClientConfig
zkClientConfig) throws IOExcept
zkClientConfig.setProperty(ZKClientConfig.LOGIN_CONTEXT_NAME_KEY,
JAAS_CLIENT_ENTRY);
}
}
-}
\ No newline at end of file
+
+ /**
+ * Helper class to contain the Truststore/Keystore paths for the ZK client
connection over
+ * SSL/TLS.
+ */
+ public static class TruststoreKeystore{
+private static String keystoreLocation;
+private static String keystorePassword;
+private static String truststoreLocation;
+private static String truststorePassword;
+/** Configuration for the ZooKeeper connection when SSL/TLS is enabled.
+ * When a value is not configured, ensure that empty string is set instead
of null.
+ * @param conf ZooKeeper Client configuration
+ */
+public TruststoreKeystore(Configuration conf){
+
+ keystoreLocation =
+
StringUtils.defaultString(conf.get(CommonConfigurationKeys.ZK_SSL_KEYSTORE_LOCATION,
Review Comment:
Why the StringUtils.defaultString is needed?
I mean, conf.get() will return an empty string if the config is not found,
given that you passed empty strings for all conf.get calls already.
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/curator/TestSecureZKCuratorManager.java:
##
@@ -0,0 +1,192 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.util.curator;
+
+import org.apache.curator.test.InstanceSpec;
+import org.apache.curator.test.TestingServer;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.CommonConfigurationKeys;
+import org.apache.zookeeper.ZooKeeper;
+import org.apache.zookeeper.client.ZKClientConfig;
+import org.apache.zookeeper.common.ClientX509Util;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.io.File;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.Map;
+
+import static org.apache.hadoop.fs.FileContext.LOG;
+import static org.junit.Assert.assertEquals;
+
+
+/**
+ * Test the manager for ZooKeeper Curator when SSL/TLS is enabled for the ZK
server-client
+ * connection.
+ */
+public class TestSecureZKCuratorManager {
+
+ private TestingServer server;
+ private ZKCuratorManager curator;
+ private Configuration hadoopConf;
+ static final Integer SECURE_CLIENT_PORT = 2281;
+ static final Integer JUTE_MAXBUFFER = 4;
+ static final File ZK_DATA_DIR = new File("testZkSSLClientConnectionDataDir");
+
+ @Before
+ public void setup() throws Exception {
+Integer defaultValue = -1;
+Map customConfiguration = new HashMap<>();
+customConfiguration.put("secureClientPort", SECURE_CLIENT_PORT.toString());
+customConfiguration.put("audit.enable", true);
+this.hadoopConf = setUpSecure();
+InstanceSpec spec = new InstanceSpec(ZK_DATA_DIR, SECURE_CLIENT_PORT,
+defaultValue,
+defaultValue,
+true,
Review Comment:
Extracting these (at least 1, 100, and 10) to static finals would make this
more readable and straightforward.
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java:
##
@@ -452,21 +502,50 @@ public static class HadoopZookeeperFactory implements
ZookeeperFactory {
private final String zkPrincipal;
private final String kerberosPrincipal;
private final String kerberosKeytab;
+private final Boolean sslEnabled;
+/**
+ * Constructor for the helper class to configure the ZooKeeper client
connection.
+ * @param zkPrincipal Optional.
+ */
public HadoopZookeeperFactory(String zkPrincipal) {
this(zkPr
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[ https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724474#comment-17724474 ] ASF GitHub Bot commented on HADOOP-18709: - szilard-nemeth commented on PR #5638: URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1555433823 > Thank you Szilard for the CR. > > The change was exclusively tested with the unit test, which is a kind of integration test, as a ZK Server was brought up and the communication tested between the client and the server. > > This code change is in the common code base and there is no component calling it yet. Once [YARN-11468](https://issues.apache.org/jira/browse/YARN-11468) [Zookeeper SSL/TLS support] is implemented, we can test it in a real cluster environment. > > Wondering if we should update the [hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs-rbf/dependency-analysis.html](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs-rbf/dependency-analysis.html) page with the Netty dependency? The parameter descriptions are added to the commit to the core-default.xml. I see, thanks for the info. Didn't know about the YARN jira. I don't think you need to update the dependency report, TBH I never updated it and I don't know how it's generated. Probably copied from the output of some script? Our codebase might have a reference to this somewhere, in markdown files. > Add curator based ZooKeeper communication support over SSL/TLS into the > common library > -- > > Key: HADOOP-18709 > URL: https://issues.apache.org/jira/browse/HADOOP-18709 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ferenc Erdelyi >Assignee: Ferenc Erdelyi >Priority: Major > Labels: pull-request-available > > With HADOOP-16579 the ZooKeeper client is capable of securing communication > with SSL. > To follow the convention introduced in HADOOP-14741, proposing to add to the > core-default.xml the following configurations, as the groundwork for the > components to enable encrypted communication between the individual > components and ZooKeeper: > * hadoop.zk.ssl.keystore.location > * hadoop.zk.ssl.keystore.password > * hadoop.zk.ssl.truststore.location > * hadoop.zk.ssl.truststore.password > These parameters along with the component-specific ssl.client.enable option > (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the > ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a > new overloaded start() method to build the encrypted communication. > * The secured ZK Client uses Netty, hence the dependency is included in the > pom.xml. Added netty-handler and netty-transport-native-epoll dependency to > the pom.xml based on ZOOKEEPER-3494 - "No need to depend on netty-all (SSL)". > * The change was exclusively tested with the unit test, which is a kind of > integration test, as a ZK Server was brought up and the communication tested > between the client and the server. > * This code change is in the common code base and there is no component > calling it yet. Once YARN-11468 - "Zookeeper SSL/TLS support" is implemented, > we can test it in a real cluster environment. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[ https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724475#comment-17724475 ] ASF GitHub Bot commented on HADOOP-18709: - szilard-nemeth commented on PR #5638: URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1555435928 Hi @ferdelyi , I added a couple of review comments. Could you please also add comments to the test class (e.g. in javadoc) about how the added certificate files, keystore and truststore files were generated, for example you may also add the commands that created those files. As a reader of the test class, I woulnd't have any idea how those files got there and if any issue comes up in the future, the javadoc would tell. Thanks. > Add curator based ZooKeeper communication support over SSL/TLS into the > common library > -- > > Key: HADOOP-18709 > URL: https://issues.apache.org/jira/browse/HADOOP-18709 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ferenc Erdelyi >Assignee: Ferenc Erdelyi >Priority: Major > Labels: pull-request-available > > With HADOOP-16579 the ZooKeeper client is capable of securing communication > with SSL. > To follow the convention introduced in HADOOP-14741, proposing to add to the > core-default.xml the following configurations, as the groundwork for the > components to enable encrypted communication between the individual > components and ZooKeeper: > * hadoop.zk.ssl.keystore.location > * hadoop.zk.ssl.keystore.password > * hadoop.zk.ssl.truststore.location > * hadoop.zk.ssl.truststore.password > These parameters along with the component-specific ssl.client.enable option > (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the > ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a > new overloaded start() method to build the encrypted communication. > * The secured ZK Client uses Netty, hence the dependency is included in the > pom.xml. Added netty-handler and netty-transport-native-epoll dependency to > the pom.xml based on ZOOKEEPER-3494 - "No need to depend on netty-all (SSL)". > * The change was exclusively tested with the unit test, which is a kind of > integration test, as a ZK Server was brought up and the communication tested > between the client and the server. > * This code change is in the common code base and there is no component > calling it yet. Once YARN-11468 - "Zookeeper SSL/TLS support" is implemented, > we can test it in a real cluster environment. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17724943#comment-17724943
]
ASF GitHub Bot commented on HADOOP-18709:
-
ferdelyi commented on code in PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#discussion_r1200468351
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/curator/ZKCuratorManager.java:
##
@@ -503,4 +644,50 @@ private void setJaasConfiguration(ZKClientConfig
zkClientConfig) throws IOExcept
zkClientConfig.setProperty(ZKClientConfig.LOGIN_CONTEXT_NAME_KEY,
JAAS_CLIENT_ENTRY);
}
}
-}
\ No newline at end of file
+
+ /**
+ * Helper class to contain the Truststore/Keystore paths for the ZK client
connection over
+ * SSL/TLS.
+ */
+ public static class TruststoreKeystore{
+private static String keystoreLocation;
+private static String keystorePassword;
+private static String truststoreLocation;
+private static String truststorePassword;
+/** Configuration for the ZooKeeper connection when SSL/TLS is enabled.
+ * When a value is not configured, ensure that empty string is set instead
of null.
+ * @param conf ZooKeeper Client configuration
+ */
+public TruststoreKeystore(Configuration conf){
+
+ keystoreLocation =
+
StringUtils.defaultString(conf.get(CommonConfigurationKeys.ZK_SSL_KEYSTORE_LOCATION,
Review Comment:
I was getting some NPE exception based on my recollection without this, but
can't reproduce it now, so removing it and will see during the build if it
comes up again.
> Add curator based ZooKeeper communication support over SSL/TLS into the
> common library
> --
>
> Key: HADOOP-18709
> URL: https://issues.apache.org/jira/browse/HADOOP-18709
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Ferenc Erdelyi
>Assignee: Ferenc Erdelyi
>Priority: Major
> Labels: pull-request-available
>
> With HADOOP-16579 the ZooKeeper client is capable of securing communication
> with SSL.
> To follow the convention introduced in HADOOP-14741, proposing to add to the
> core-default.xml the following configurations, as the groundwork for the
> components to enable encrypted communication between the individual
> components and ZooKeeper:
> * hadoop.zk.ssl.keystore.location
> * hadoop.zk.ssl.keystore.password
> * hadoop.zk.ssl.truststore.location
> * hadoop.zk.ssl.truststore.password
> These parameters along with the component-specific ssl.client.enable option
> (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the
> ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a
> new overloaded start() method to build the encrypted communication.
> * The secured ZK Client uses Netty, hence the dependency is included in the
> pom.xml. Added netty-handler and netty-transport-native-epoll dependency to
> the pom.xml based on ZOOKEEPER-3494 - "No need to depend on netty-all (SSL)".
> * The change was exclusively tested with the unit test, which is a kind of
> integration test, as a ZK Server was brought up and the communication tested
> between the client and the server.
> * This code change is in the common code base and there is no component
> calling it yet. Once YARN-11468 - "Zookeeper SSL/TLS support" is implemented,
> we can test it in a real cluster environment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725518#comment-17725518
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1559894584
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 34s | | trunk passed |
| +1 :green_heart: | compile | 16m 13s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 14m 23s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 13s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 29s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 13s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 43s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 24s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | | the patch passed |
| +1 :green_heart: | compile | 15m 47s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 15m 47s | | the patch passed |
| +1 :green_heart: | compile | 14m 57s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 14m 57s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 8s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 32s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| -1 :x: | javadoc | 1m 10s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/14/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt)
| hadoop-common in the patch failed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1. |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 36s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 16s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 1m 4s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/14/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| | | 177m 57s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux 1a0085c52c62 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1f841659394175924dec47eeec82335a13680d07 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725522#comment-17725522
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1559905475
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 50s | | trunk passed |
| +1 :green_heart: | compile | 17m 1s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 15m 57s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 8s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 17s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 46s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 28s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 53s | | the patch passed |
| +1 :green_heart: | compile | 16m 39s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 16m 39s | | the patch passed |
| +1 :green_heart: | compile | 16m 19s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 16m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 7s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 41s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| -1 :x: | javadoc | 1m 4s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/15/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt)
| hadoop-common in the patch failed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1. |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 55s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 0m 58s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/15/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| | | 186m 14s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/15/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux fec8731db6de 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1f841659394175924dec47eeec82335a13680d07 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725573#comment-17725573
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1560121784
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 46s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 2s | | trunk passed |
| +1 :green_heart: | compile | 15m 36s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 14m 22s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 13s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 17s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 52s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 16s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | | the patch passed |
| +1 :green_heart: | compile | 15m 3s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 15m 3s | | the patch passed |
| +1 :green_heart: | compile | 14m 16s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 14m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 6s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 33s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 5s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 27s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 1s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 1m 2s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/16/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| | | 183m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/16/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux f01ffa2966cd 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ef1756f41ac86ee8522ffab554e16a3237597eea |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/16/testReport/ |
| Max. process+thread count | 1403 (vs. ulimit of 5500) |
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725574#comment-17725574
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1560122458
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 23s | | trunk passed |
| +1 :green_heart: | compile | 16m 55s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 15m 34s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 9s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 52s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 59s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 50s | | the patch passed |
| +1 :green_heart: | compile | 16m 11s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 16m 11s | | the patch passed |
| +1 :green_heart: | compile | 15m 41s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 15m 41s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 6s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 33s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 21s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 0m 58s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/18/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| | | 182m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/18/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux dc1f2d4f3a2f 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ef1756f41ac86ee8522ffab554e16a3237597eea |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/18/testReport/ |
| Max. process+thread count | 1906 (vs. ulimit of 5500) |
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725575#comment-17725575
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1560123778
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 11s | | trunk passed |
| +1 :green_heart: | compile | 16m 50s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 15m 23s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 13s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 15s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 44s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 21s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 52s | | the patch passed |
| +1 :green_heart: | compile | 16m 16s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 16m 16s | | the patch passed |
| +1 :green_heart: | compile | 15m 45s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 15m 45s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 57s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 40s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 9s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 44s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 28s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 1m 0s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/17/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| | | 183m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/17/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux 1ec4dbc3c843 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ef1756f41ac86ee8522ffab554e16a3237597eea |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/17/testReport/ |
| Max. process+thread count | 1291 (vs. ulimit of 5500) |
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725577#comment-17725577
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1560135150
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 14s | | trunk passed |
| +1 :green_heart: | compile | 16m 15s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 14m 56s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 15s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 15s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 44s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 32s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 52s | | the patch passed |
| +1 :green_heart: | compile | 16m 38s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 16m 38s | | the patch passed |
| +1 :green_heart: | compile | 16m 32s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 16m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 4s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 32s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 6s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 53s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 3s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 22s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 1m 5s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/19/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| | | 194m 40s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/19/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux 0e55918adc25 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ef1756f41ac86ee8522ffab554e16a3237597eea |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/19/testReport/ |
| Max. process+thread count | 1288 (vs. ulimit of 5500) |
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725578#comment-17725578
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1560135690
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 36s | | trunk passed |
| +1 :green_heart: | compile | 17m 32s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 15m 50s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 31s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 52s | | the patch passed |
| +1 :green_heart: | compile | 16m 47s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 16m 47s | | the patch passed |
| +1 :green_heart: | compile | 15m 50s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 15m 50s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 27s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 0m 59s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 42s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 44s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 0m 53s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/20/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| | | 195m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/20/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux 9fa60a5d8292 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ef1756f41ac86ee8522ffab554e16a3237597eea |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/20/testReport/ |
| Max. process+thread count | 1239 (vs. ulimit of 5500) |
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17725579#comment-17725579
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1560145101
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 49m 37s | | trunk passed |
| +1 :green_heart: | compile | 17m 53s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 15m 26s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 16s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 47s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 48s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 28s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 54s | | the patch passed |
| +1 :green_heart: | compile | 17m 21s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 17m 21s | | the patch passed |
| +1 :green_heart: | compile | 16m 2s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 16m 2s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 6s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 38s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 1m 7s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 41s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 23s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 0m 54s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/21/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| | | 204m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/21/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux 09369248f268 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ef1756f41ac86ee8522ffab554e16a3237597eea |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/21/testReport/ |
| Max. process+thread count | 3149 (vs. ulimit of 5500) |
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17726288#comment-17726288
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1563135627
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| -1 :x: | mvninstall | 26m 19s |
[/branch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/25/artifact/out/branch-mvninstall-root.txt)
| root in trunk failed. |
| +1 :green_heart: | compile | 15m 36s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 14m 27s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 12s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 33s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 20s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 47s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 55s | | the patch passed |
| -1 :x: | compile | 6m 50s |
[/patch-compile-root-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/25/artifact/out/patch-compile-root-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt)
| root in the patch failed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1. |
| -1 :x: | javac | 6m 50s |
[/patch-compile-root-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/25/artifact/out/patch-compile-root-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt)
| root in the patch failed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1. |
| -1 :x: | compile | 6m 10s |
[/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/25/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09.txt)
| root in the patch failed with JDK Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09. |
| -1 :x: | javac | 6m 10s |
[/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/25/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09.txt)
| root in the patch failed with JDK Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 46s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 13s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 0m 48s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 12s | | the patch passed |
| -1 :x: | shadedclient | 8m 54s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 41s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 0m 36s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/25/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| |
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17726292#comment-17726292
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1563142275
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 8 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 18s | | trunk passed |
| +1 :green_heart: | compile | 15m 42s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 14m 19s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 16s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 14s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 49s | | the patch passed |
| -1 :x: | compile | 6m 38s |
[/patch-compile-root-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/24/artifact/out/patch-compile-root-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt)
| root in the patch failed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1. |
| -1 :x: | javac | 6m 38s |
[/patch-compile-root-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/24/artifact/out/patch-compile-root-jdkUbuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1.txt)
| root in the patch failed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1. |
| -1 :x: | compile | 6m 12s |
[/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/24/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09.txt)
| root in the patch failed with JDK Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09. |
| -1 :x: | javac | 6m 12s |
[/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/24/artifact/out/patch-compile-root-jdkPrivateBuild-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09.txt)
| root in the patch failed with JDK Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09. |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 48s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 15s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 15s | | the patch passed |
| -1 :x: | shadedclient | 8m 53s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 48s | | hadoop-common in the patch
passed. |
| -1 :x: | asflicense | 0m 36s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/24/artifact/out/results-asflicense.txt)
| The patch generated 5 ASF License warnings. |
| | | 141m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 ba
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[
https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17727938#comment-17727938
]
ASF GitHub Bot commented on HADOOP-18709:
-
hadoop-yetus commented on PR #5638:
URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1570164034
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 2s | | trunk passed |
| +1 :green_heart: | compile | 17m 39s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 16m 1s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 1m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 9s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 33s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 51s | | the patch passed |
| +1 :green_heart: | compile | 16m 44s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 16m 44s | | the patch passed |
| +1 :green_heart: | compile | 15m 57s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 15m 57s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 1s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 29s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 0s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 41s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 2m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 9s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 19s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 52s | | The patch does not
generate ASF License warnings. |
| | | 191m 21s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/29/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle |
| uname | Linux e7c99d323ae3 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7e9ab3d63e82a21973371c48512833b0cc7a7827 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/29/testReport/ |
| Max. process+thread count | 3144 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5638/29/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[ https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729101#comment-17729101 ] ASF GitHub Bot commented on HADOOP-18709: - szilard-nemeth commented on PR #5638: URL: https://github.com/apache/hadoop/pull/5638#issuecomment-1575670756 Thanks @ferdelyi for working on this. Latest patch LGTM, committed to trunk. Thaknks @goiri for the review. > Add curator based ZooKeeper communication support over SSL/TLS into the > common library > -- > > Key: HADOOP-18709 > URL: https://issues.apache.org/jira/browse/HADOOP-18709 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ferenc Erdelyi >Assignee: Ferenc Erdelyi >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > With HADOOP-16579 the ZooKeeper client is capable of securing communication > with SSL. > To follow the convention introduced in HADOOP-14741, proposing to add to the > core-default.xml the following configurations, as the groundwork for the > components to enable encrypted communication between the individual > components and ZooKeeper: > * hadoop.zk.ssl.keystore.location > * hadoop.zk.ssl.keystore.password > * hadoop.zk.ssl.truststore.location > * hadoop.zk.ssl.truststore.password > These parameters along with the component-specific ssl.client.enable option > (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the > ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a > new overloaded start() method to build the encrypted communication. > * The secured ZK Client uses Netty, hence the dependency is included in the > pom.xml. Added netty-handler and netty-transport-native-epoll dependency to > the pom.xml based on ZOOKEEPER-3494 - "No need to depend on netty-all (SSL)". > * The change was exclusively tested with the unit test, which is a kind of > integration test, as a ZK Server was brought up and the communication tested > between the client and the server. > * This code change is in the common code base and there is no component > calling it yet. Once YARN-11468 - "Zookeeper SSL/TLS support" is implemented, > we can test it in a real cluster environment. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18709) Add curator based ZooKeeper communication support over SSL/TLS into the common library
[ https://issues.apache.org/jira/browse/HADOOP-18709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17729100#comment-17729100 ] ASF GitHub Bot commented on HADOOP-18709: - szilard-nemeth closed pull request #5638: HADOOP-18709. Add curator based ZooKeeper communication support over… URL: https://github.com/apache/hadoop/pull/5638 > Add curator based ZooKeeper communication support over SSL/TLS into the > common library > -- > > Key: HADOOP-18709 > URL: https://issues.apache.org/jira/browse/HADOOP-18709 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ferenc Erdelyi >Assignee: Ferenc Erdelyi >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > > With HADOOP-16579 the ZooKeeper client is capable of securing communication > with SSL. > To follow the convention introduced in HADOOP-14741, proposing to add to the > core-default.xml the following configurations, as the groundwork for the > components to enable encrypted communication between the individual > components and ZooKeeper: > * hadoop.zk.ssl.keystore.location > * hadoop.zk.ssl.keystore.password > * hadoop.zk.ssl.truststore.location > * hadoop.zk.ssl.truststore.password > These parameters along with the component-specific ssl.client.enable option > (e.g. yarn.zookeeper.ssl.client.enable) should be passed to the > ZKCuratorManager to build the CuratorFramework. The ZKCuratorManager needs a > new overloaded start() method to build the encrypted communication. > * The secured ZK Client uses Netty, hence the dependency is included in the > pom.xml. Added netty-handler and netty-transport-native-epoll dependency to > the pom.xml based on ZOOKEEPER-3494 - "No need to depend on netty-all (SSL)". > * The change was exclusively tested with the unit test, which is a kind of > integration test, as a ZK Server was brought up and the communication tested > between the client and the server. > * This code change is in the common code base and there is no component > calling it yet. Once YARN-11468 - "Zookeeper SSL/TLS support" is implemented, > we can test it in a real cluster environment. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
