[jira] [Commented] (HADOOP-17151) upgrade jetty to 9.4.21
[
https://issues.apache.org/jira/browse/HADOOP-17151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17164762#comment-17164762
]
liusheng commented on HADOOP-17151:
---
Hi [~ste...@apache.org],
Yes, I have tested locally, doing this can fix the problem.
> upgrade jetty to 9.4.21
> ---
>
> Key: HADOOP-17151
> URL: https://issues.apache.org/jira/browse/HADOOP-17151
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.3.0
>Reporter: liusheng
>Priority: Major
> Attachments: HADOOP-17151.001.patch
>
>
> I have tried to configure and start Hadoop KMS service, it was failed to
> start the error log messages:
> {noformat}
> 2020-07-23 10:57:31,872 INFO Server - jetty-9.4.20.v20190813; built:
> 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:31,899 INFO session - DefaultSessionIdManager
> workerName=node0
> 2020-07-23 10:57:31,899 INFO session - No SessionScavenger set, using
> defaults
> 2020-07-23 10:57:31,901 INFO session - node0 Scavenging every 66ms
> 2020-07-23 10:57:31,912 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@5bf0d49{logs,/logs,file:///opt/hadoop-3.4.0-SNAPSHOT/logs/,AVAILABLE}
> 2020-07-23 10:57:31,913 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@7c7a06ec{static,/static,jar:file:/opt/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-kms-3.4.0-SNAPSHOT.jar!/webapps/static,AVAILABLE}
> 2020-07-23 10:57:31,986 INFO TypeUtil - JVM Runtime does not support Modules
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,015 INFO KMSWebApp - Java runtime version :
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:32,015 INFO KMSWebApp - User: hadoop
> 2020-07-23 10:57:32,015 INFO KMSWebApp - KMS Hadoop Version: 3.4.0-SNAPSHOT
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,023 INFO KMSACLs - 'CREATE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DELETE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'ROLLOVER' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_KEYS' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_METADATA' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'SET_KEY_MATERIAL' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GENERATE_EEK' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DECRYPT_EEK' ACL '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP 'READ' is
> set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'MANAGEMENT' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'GENERATE_EEK' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'DECRYPT_EEK' is set to '*'
> 2020-07-23 10:57:32,080 INFO KMSAudit - Initializing audit logger class
> org.apache.hadoop.crypto.key.kms.server.SimpleKMSAuditLogger
> 2020-07-23 10:57:32,537 INFO KMSWebServer - SHUTDOWN_MSG:
> /
> SHUTDOWN_MSG: Shutting down KMSWebServer at
> hadoop-benchmark/172.17.0.2{noformat}
> I have googled the error and found there is a simlar issue：
> [https://github.com/eclipse/jetty.project/issues/4064]
> It looks like a bug of jetty and has been fixed in jetty>=9.4.21, currently
> Hadoop use the jetty is version of 9.4.20, see hadoop-project/pom.xml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ajfabbri commented on a change in pull request #2069: HADOOP-16830. IOStatistics API.
ajfabbri commented on a change in pull request #2069:
URL: https://github.com/apache/hadoop/pull/2069#discussion_r456140577
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/statistics/IOStatistics.java
##
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.statistics;
+
+import java.util.Map;
+
+import org.apache.hadoop.classification.InterfaceAudience;
+import org.apache.hadoop.classification.InterfaceStability;
+
+/**
+ * IO Statistics.
+ *
+ * These are low-cost per-instance statistics provided by any Hadoop
+ * I/O class instance.
+ *
+ * Consult the filesystem specification document for the requirements
+ * of an implementation of this interface.
+ */
+@InterfaceAudience.Public
+@InterfaceStability.Unstable
+public interface IOStatistics {
+
+ /**
+ * Map of counters.
+ * @return the current map of counters.
+ */
+ Map counters();
+
+ /**
+ * Map of gauges.
+ * @return the current map of gauges.
+ */
+ Map gauges();
+
+ /**
+ * Map of minumums.
+ * @return the current map of minumums.
+ */
+ Map minimums();
+
+ /**
+ * Map of maximums.
+ * @return the current map of maximums.
+ */
+ Map maximums();
+
+ /**
+ * Map of meanStatistics.
+ * @return the current map of MeanStatistic statistics.
+ */
+ Map meanStatistics();
+
Review comment:
This interface seems reasonable to me.
##
File path:
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/statistics/impl/IOStatisticsStoreImpl.java
##
@@ -0,0 +1,424 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.statistics.impl;
+
+import javax.annotation.Nullable;
+import java.time.Duration;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicLong;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import org.apache.hadoop.fs.statistics.IOStatistics;
+import org.apache.hadoop.fs.statistics.MeanStatistic;
+
+import static java.util.Objects.requireNonNull;
+import static org.apache.hadoop.fs.statistics.StoreStatisticNames.SUFFIX_MAX;
+import static org.apache.hadoop.fs.statistics.StoreStatisticNames.SUFFIX_MEAN;
+import static org.apache.hadoop.fs.statistics.StoreStatisticNames.SUFFIX_MIN;
+import static
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.dynamicIOStatistics;
+import static
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.maybeUpdateMaximum;
+import static
org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding.maybeUpdateMinimum;
+
+/**
+ * Implementation of {@link IOStatisticsStore}.
+ */
+final class IOStatisticsStoreImpl extends WrappedIOStatistics
+implements IOStatisticsStore {
+
+ /**
+ * Log changes at debug.
+ * Noisy, but occasionally useful.
+ */
+ private static final Logger LOG =
+ LoggerFactory.getLogger(IOStatisticsStoreImpl.class);
+
+ private final Map counterMap = new HashMap<>();
+
Review comment:
How are you guarding against concurrent modification of this HashMap?
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractStreamIOStatisticsTest.java
##
@@ -0,0 +1,281 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * dis
[GitHub] [hadoop] hadoop-yetus commented on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
hadoop-yetus commented on pull request #2170: URL: https://github.com/apache/hadoop/pull/2170#issuecomment-663656386 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 12m 47s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | ||| _ branch-3.2 Compile Tests _ | | +0 :ok: | mvndep | 0m 23s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 21m 1s | branch-3.2 passed | | +1 :green_heart: | compile | 15m 25s | branch-3.2 passed | | +1 :green_heart: | checkstyle | 2m 26s | branch-3.2 passed | | +1 :green_heart: | mvnsite | 2m 3s | branch-3.2 passed | | +1 :green_heart: | shadedclient | 19m 3s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 1m 45s | branch-3.2 passed | | +0 :ok: | spotbugs | 1m 1s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 2m 56s | branch-3.2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 22s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 25s | the patch passed | | +1 :green_heart: | compile | 14m 35s | the patch passed | | +1 :green_heart: | javac | 14m 35s | the patch passed | | -0 :warning: | checkstyle | 2m 26s | root: The patch generated 19 new + 5 unchanged - 0 fixed = 24 total (was 5) | | +1 :green_heart: | mvnsite | 2m 4s | the patch passed | | -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | shadedclient | 12m 36s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 1m 45s | the patch passed | | +1 :green_heart: | findbugs | 3m 16s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 8m 59s | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 4m 45s | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 49s | The patch does not generate ASF License warnings. | | | | 131m 26s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2170 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 92f7e0490731 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | branch-3.2 / 0fb7c48 | | Default Java | Private Build-1.8.0_252-8u252-b09-1~16.04-b09 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/diff-checkstyle-root.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/artifact/out/whitespace-eol.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/testReport/ | | Max. process+thread count | 1385 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/2/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
hadoop-yetus commented on pull request #2170: URL: https://github.com/apache/hadoop/pull/2170#issuecomment-663644838 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 11m 5s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 4 new or modified test files. | ||| _ branch-3.2 Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 21m 8s | branch-3.2 passed | | +1 :green_heart: | compile | 15m 48s | branch-3.2 passed | | +1 :green_heart: | checkstyle | 2m 25s | branch-3.2 passed | | +1 :green_heart: | mvnsite | 2m 2s | branch-3.2 passed | | +1 :green_heart: | shadedclient | 17m 55s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 1m 41s | branch-3.2 passed | | +0 :ok: | spotbugs | 1m 3s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 0s | branch-3.2 passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 24s | the patch passed | | +1 :green_heart: | compile | 15m 7s | the patch passed | | +1 :green_heart: | javac | 15m 7s | the patch passed | | -0 :warning: | checkstyle | 2m 25s | root: The patch generated 19 new + 5 unchanged - 0 fixed = 24 total (was 5) | | +1 :green_heart: | mvnsite | 1m 59s | the patch passed | | -1 :x: | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply | | +1 :green_heart: | shadedclient | 13m 8s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 1m 34s | the patch passed | | +1 :green_heart: | findbugs | 3m 26s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 9m 26s | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 4m 38s | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 44s | The patch does not generate ASF License warnings. | | | | 130m 7s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2170 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux c4b6daf3e36e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | branch-3.2 / 0fb7c48 | | Default Java | Private Build-1.8.0_252-8u252-b09-1~16.04-b09 | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/artifact/out/diff-checkstyle-root.txt | | whitespace | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/artifact/out/whitespace-eol.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/testReport/ | | Max. process+thread count | 1370 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: . | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2170/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2169: HADOOP-17153. Add boost installation steps to build instruction on CentOS 8.
hadoop-yetus commented on pull request #2169: URL: https://github.com/apache/hadoop/pull/2169#issuecomment-663598610 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 31m 51s | Docker failed to build yetus/hadoop:cce5a6f6094. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/2169 | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2169/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran edited a comment on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
steveloughran edited a comment on pull request #2170: URL: https://github.com/apache/hadoop/pull/2170#issuecomment-663591666 failure on unguarded run ``` [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 36.712 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost [ERROR] testCostOfGetFileStatusOnNonEmptyDir(org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost) Time elapsed: 5.415 s <<< FAILURE! java.lang.AssertionError: Count of object_metadata_requests starting=5 current=6 diff=1: object_metadata_requests expected:<2> but was:<1> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:691) at org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:700) at org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost.testCostOfGetFileStatusOnNonEmptyDir(ITestS3AFileOperationCost.java:141) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
steveloughran commented on pull request #2170: URL: https://github.com/apache/hadoop/pull/2170#issuecomment-663591666 ``` [ERROR] Tests run: 8, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 36.712 s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost [ERROR] testCostOfGetFileStatusOnNonEmptyDir(org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost) Time elapsed: 5.415 s <<< FAILURE! java.lang.AssertionError: Count of object_metadata_requests starting=5 current=6 diff=1: object_metadata_requests expected:<2> but was:<1> at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.failNotEquals(Assert.java:743) at org.junit.Assert.assertEquals(Assert.java:118) at org.junit.Assert.assertEquals(Assert.java:555) at org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:691) at org.apache.hadoop.fs.s3a.S3ATestUtils$MetricDiff.assertDiffEquals(S3ATestUtils.java:700) at org.apache.hadoop.fs.s3a.ITestS3AFileOperationCost.testCostOfGetFileStatusOnNonEmptyDir(ITestS3AFileOperationCost.java:141) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2149: HADOOP-13230. S3A to optionally retain directory markers
steveloughran commented on pull request #2149: URL: https://github.com/apache/hadoop/pull/2149#issuecomment-663587120 see #2170 for the <= 3.2 subset of this patch, needed to not mistake paths with dir markers as empty. As well as not deleting markers on file creation, I'm going to add the option of not doing any re-creation of markers when files are deleted. This reduces IO on file deletion at the cost of a common assumption: "after you delete all files in a directory -the directory still exists". I don't know what that's going to break, so it is going to be explicitly optional, with the recovery policy to be true/false/authoritative This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran opened a new pull request #2170: HADOOP-1320. Dir Marker getFileStatus() changes backport
steveloughran opened a new pull request #2170: URL: https://github.com/apache/hadoop/pull/2170 This contains the getFileStatus changes of #2149 This does not change dir marker deletion, but it * Changes s3GetFileStatus to do HEAD + LIST only * add in StatusProbeEnum from the 404 work * Fix relevant tests to work -tweaking the FileOperationsCost rather than the big move to the new cost test framework These changes can actually go in ahead of the dir marker patch as they should marginally speed up directory listings everywhere. A variant of this patch will be needed for 3.0+ branches; this will be the basis for those. ( I should probably have a separate JIRA ID...) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17153) Add boost installation steps to build instruction on CentOS 8
[ https://issues.apache.org/jira/browse/HADOOP-17153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki updated HADOOP-17153: -- Status: Patch Available (was: Open) > Add boost installation steps to build instruction on CentOS 8 > - > > Key: HADOOP-17153 > URL: https://issues.apache.org/jira/browse/HADOOP-17153 > Project: Hadoop Common > Issue Type: Bug > Components: documentation >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > > After HDFS-15385, -Pnative build fails without boost 1.72 used in libhdfs++. > It must be installed from source since boost 1.66 packaged by the CentOS > distribution does not match. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] iwasakims opened a new pull request #2169: HADOOP-17153. Add boost installation steps to build instruction on CentOS 8.
iwasakims opened a new pull request #2169: URL: https://github.com/apache/hadoop/pull/2169 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-17153) Add boost installation steps to build instruction on CentOS 8
Masatake Iwasaki created HADOOP-17153: - Summary: Add boost installation steps to build instruction on CentOS 8 Key: HADOOP-17153 URL: https://issues.apache.org/jira/browse/HADOOP-17153 Project: Hadoop Common Issue Type: Bug Components: documentation Reporter: Masatake Iwasaki Assignee: Masatake Iwasaki After HDFS-15385, -Pnative build fails without boost 1.72 used in libhdfs++. It must be installed from source since boost 1.66 packaged by the CentOS distribution does not match. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2168: HADOOP-16202. Enhance openFile()
hadoop-yetus commented on pull request #2168: URL: https://github.com/apache/hadoop/pull/2168#issuecomment-663534484 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 0s | Docker mode activated. | | -1 :x: | docker | 25m 11s | Docker failed to build yetus/hadoop:cce5a6f6094. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/2168 | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-2168/1/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran opened a new pull request #2168: HADOOP-16202. Enhance openFile()
steveloughran opened a new pull request #2168: URL: https://github.com/apache/hadoop/pull/2168 Roll-up of the previous PR #2046 Change-Id: Ib0aec173afcd8aae33f52da3f99ac813bd38c32f This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran closed pull request #2046: HADOOP-16202 Enhance S3A openFile()
steveloughran closed pull request #2046: URL: https://github.com/apache/hadoop/pull/2046 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2046: HADOOP-16202 Enhance S3A openFile()
steveloughran commented on pull request #2046: URL: https://github.com/apache/hadoop/pull/2046#issuecomment-663516981 things aren't complaining as the patch chain is confused; a file which was added and then renamed is still present. I'm going to have to close this PR and create a new one This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2046: HADOOP-16202 Enhance S3A openFile()
hadoop-yetus removed a comment on pull request #2046: URL: https://github.com/apache/hadoop/pull/2046#issuecomment-662961938 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 9s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 1s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 0s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 21m 47s | trunk passed | | +1 :green_heart: | compile | 21m 0s | trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | compile | 17m 53s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | checkstyle | 2m 53s | trunk passed | | +1 :green_heart: | mvnsite | 2m 10s | trunk passed | | +1 :green_heart: | shadedclient | 21m 1s | branch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 35s | hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 36s | hadoop-aws in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 1m 29s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +0 :ok: | spotbugs | 1m 10s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 23s | trunk passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 21s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 1m 24s | the patch passed | | +1 :green_heart: | compile | 21m 19s | the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 | | +1 :green_heart: | javac | 21m 19s | the patch passed | | +1 :green_heart: | compile | 18m 22s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | javac | 18m 22s | the patch passed | | +1 :green_heart: | checkstyle | 3m 8s | the patch passed | | +1 :green_heart: | mvnsite | 2m 15s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 15m 34s | patch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 36s | hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | -1 :x: | javadoc | 0m 37s | hadoop-aws in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04. | | +1 :green_heart: | javadoc | 1m 30s | the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | findbugs | 3m 36s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 9m 20s | hadoop-common in the patch passed. | | +1 :green_heart: | unit | 1m 29s | hadoop-aws in the patch passed. | | +1 :green_heart: | asflicense | 0m 47s | The patch does not generate ASF License warnings. | | | | 174m 50s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.security.TestRaceWhenRelogin | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2046/4/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2046 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle markdownlint | | uname | Linux 0bdcb509b029 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28 11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 2d12496643b | | Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2046/4/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt | | javadoc | https://builds.apache.org/job/hadoop-multibranch/job/PR-2046/4/artifact/out/branch-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-11.0.7+10-post-U
[jira] [Commented] (HADOOP-17151) upgrade jetty to 9.4.21
[
https://issues.apache.org/jira/browse/HADOOP-17151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17164409#comment-17164409
]
Steve Loughran commented on HADOOP-17151:
-
that github issue you link to is about a null pointer exception -not clear it
is related.
does doing this locally fix everything for you?
> upgrade jetty to 9.4.21
> ---
>
> Key: HADOOP-17151
> URL: https://issues.apache.org/jira/browse/HADOOP-17151
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.3.0
>Reporter: liusheng
>Priority: Major
> Attachments: HADOOP-17151.001.patch
>
>
> I have tried to configure and start Hadoop KMS service, it was failed to
> start the error log messages:
> {noformat}
> 2020-07-23 10:57:31,872 INFO Server - jetty-9.4.20.v20190813; built:
> 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:31,899 INFO session - DefaultSessionIdManager
> workerName=node0
> 2020-07-23 10:57:31,899 INFO session - No SessionScavenger set, using
> defaults
> 2020-07-23 10:57:31,901 INFO session - node0 Scavenging every 66ms
> 2020-07-23 10:57:31,912 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@5bf0d49{logs,/logs,file:///opt/hadoop-3.4.0-SNAPSHOT/logs/,AVAILABLE}
> 2020-07-23 10:57:31,913 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@7c7a06ec{static,/static,jar:file:/opt/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-kms-3.4.0-SNAPSHOT.jar!/webapps/static,AVAILABLE}
> 2020-07-23 10:57:31,986 INFO TypeUtil - JVM Runtime does not support Modules
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,015 INFO KMSWebApp - Java runtime version :
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:32,015 INFO KMSWebApp - User: hadoop
> 2020-07-23 10:57:32,015 INFO KMSWebApp - KMS Hadoop Version: 3.4.0-SNAPSHOT
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,023 INFO KMSACLs - 'CREATE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DELETE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'ROLLOVER' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_KEYS' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_METADATA' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'SET_KEY_MATERIAL' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GENERATE_EEK' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DECRYPT_EEK' ACL '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP 'READ' is
> set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'MANAGEMENT' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'GENERATE_EEK' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'DECRYPT_EEK' is set to '*'
> 2020-07-23 10:57:32,080 INFO KMSAudit - Initializing audit logger class
> org.apache.hadoop.crypto.key.kms.server.SimpleKMSAuditLogger
> 2020-07-23 10:57:32,537 INFO KMSWebServer - SHUTDOWN_MSG:
> /
> SHUTDOWN_MSG: Shutting down KMSWebServer at
> hadoop-benchmark/172.17.0.2{noformat}
> I have googled the error and found there is a simlar issue：
> [https://github.com/eclipse/jetty.project/issues/4064]
> It looks like a bug of jetty and has been fixed in jetty>=9.4.21, currently
> Hadoop use the jetty is version of 9.4.20, see hadoop-project/pom.xml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17151) upgrade jetty to 9.4.21
[
https://issues.apache.org/jira/browse/HADOOP-17151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-17151:
Summary: upgrade jetty to 9.4.21 (was: upgrade jetty)
> upgrade jetty to 9.4.21
> ---
>
> Key: HADOOP-17151
> URL: https://issues.apache.org/jira/browse/HADOOP-17151
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.3.0
>Reporter: liusheng
>Priority: Major
> Attachments: HADOOP-17151.001.patch
>
>
> I have tried to configure and start Hadoop KMS service, it was failed to
> start the error log messages:
> {noformat}
> 2020-07-23 10:57:31,872 INFO Server - jetty-9.4.20.v20190813; built:
> 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:31,899 INFO session - DefaultSessionIdManager
> workerName=node0
> 2020-07-23 10:57:31,899 INFO session - No SessionScavenger set, using
> defaults
> 2020-07-23 10:57:31,901 INFO session - node0 Scavenging every 66ms
> 2020-07-23 10:57:31,912 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@5bf0d49{logs,/logs,file:///opt/hadoop-3.4.0-SNAPSHOT/logs/,AVAILABLE}
> 2020-07-23 10:57:31,913 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@7c7a06ec{static,/static,jar:file:/opt/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-kms-3.4.0-SNAPSHOT.jar!/webapps/static,AVAILABLE}
> 2020-07-23 10:57:31,986 INFO TypeUtil - JVM Runtime does not support Modules
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,015 INFO KMSWebApp - Java runtime version :
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:32,015 INFO KMSWebApp - User: hadoop
> 2020-07-23 10:57:32,015 INFO KMSWebApp - KMS Hadoop Version: 3.4.0-SNAPSHOT
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,023 INFO KMSACLs - 'CREATE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DELETE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'ROLLOVER' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_KEYS' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_METADATA' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'SET_KEY_MATERIAL' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GENERATE_EEK' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DECRYPT_EEK' ACL '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP 'READ' is
> set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'MANAGEMENT' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'GENERATE_EEK' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'DECRYPT_EEK' is set to '*'
> 2020-07-23 10:57:32,080 INFO KMSAudit - Initializing audit logger class
> org.apache.hadoop.crypto.key.kms.server.SimpleKMSAuditLogger
> 2020-07-23 10:57:32,537 INFO KMSWebServer - SHUTDOWN_MSG:
> /
> SHUTDOWN_MSG: Shutting down KMSWebServer at
> hadoop-benchmark/172.17.0.2{noformat}
> I have googled the error and found there is a simlar issue：
> [https://github.com/eclipse/jetty.project/issues/4064]
> It looks like a bug of jetty and has been fixed in jetty>=9.4.21, currently
> Hadoop use the jetty is version of 9.4.20, see hadoop-project/pom.xml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17151) upgrade jetty
[
https://issues.apache.org/jira/browse/HADOOP-17151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-17151:
Summary: upgrade jetty (was: Need to upgrad the version of jetty)
> upgrade jetty
> -
>
> Key: HADOOP-17151
> URL: https://issues.apache.org/jira/browse/HADOOP-17151
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.3.0
>Reporter: liusheng
>Priority: Major
> Attachments: HADOOP-17151.001.patch
>
>
> I have tried to configure and start Hadoop KMS service, it was failed to
> start the error log messages:
> {noformat}
> 2020-07-23 10:57:31,872 INFO Server - jetty-9.4.20.v20190813; built:
> 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:31,899 INFO session - DefaultSessionIdManager
> workerName=node0
> 2020-07-23 10:57:31,899 INFO session - No SessionScavenger set, using
> defaults
> 2020-07-23 10:57:31,901 INFO session - node0 Scavenging every 66ms
> 2020-07-23 10:57:31,912 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@5bf0d49{logs,/logs,file:///opt/hadoop-3.4.0-SNAPSHOT/logs/,AVAILABLE}
> 2020-07-23 10:57:31,913 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@7c7a06ec{static,/static,jar:file:/opt/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-kms-3.4.0-SNAPSHOT.jar!/webapps/static,AVAILABLE}
> 2020-07-23 10:57:31,986 INFO TypeUtil - JVM Runtime does not support Modules
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,015 INFO KMSWebApp - Java runtime version :
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:32,015 INFO KMSWebApp - User: hadoop
> 2020-07-23 10:57:32,015 INFO KMSWebApp - KMS Hadoop Version: 3.4.0-SNAPSHOT
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,023 INFO KMSACLs - 'CREATE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DELETE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'ROLLOVER' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_KEYS' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_METADATA' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'SET_KEY_MATERIAL' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GENERATE_EEK' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DECRYPT_EEK' ACL '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP 'READ' is
> set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'MANAGEMENT' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'GENERATE_EEK' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'DECRYPT_EEK' is set to '*'
> 2020-07-23 10:57:32,080 INFO KMSAudit - Initializing audit logger class
> org.apache.hadoop.crypto.key.kms.server.SimpleKMSAuditLogger
> 2020-07-23 10:57:32,537 INFO KMSWebServer - SHUTDOWN_MSG:
> /
> SHUTDOWN_MSG: Shutting down KMSWebServer at
> hadoop-benchmark/172.17.0.2{noformat}
> I have googled the error and found there is a simlar issue：
> [https://github.com/eclipse/jetty.project/issues/4064]
> It looks like a bug of jetty and has been fixed in jetty>=9.4.21, currently
> Hadoop use the jetty is version of 9.4.20, see hadoop-project/pom.xml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17151) Need to upgrad the version of jetty
[
https://issues.apache.org/jira/browse/HADOOP-17151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-17151:
Component/s: kms
> Need to upgrad the version of jetty
> ---
>
> Key: HADOOP-17151
> URL: https://issues.apache.org/jira/browse/HADOOP-17151
> Project: Hadoop Common
> Issue Type: Bug
> Components: kms
>Affects Versions: 3.3.0
>Reporter: liusheng
>Priority: Major
> Attachments: HADOOP-17151.001.patch
>
>
> I have tried to configure and start Hadoop KMS service, it was failed to
> start the error log messages:
> {noformat}
> 2020-07-23 10:57:31,872 INFO Server - jetty-9.4.20.v20190813; built:
> 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:31,899 INFO session - DefaultSessionIdManager
> workerName=node0
> 2020-07-23 10:57:31,899 INFO session - No SessionScavenger set, using
> defaults
> 2020-07-23 10:57:31,901 INFO session - node0 Scavenging every 66ms
> 2020-07-23 10:57:31,912 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@5bf0d49{logs,/logs,file:///opt/hadoop-3.4.0-SNAPSHOT/logs/,AVAILABLE}
> 2020-07-23 10:57:31,913 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@7c7a06ec{static,/static,jar:file:/opt/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-kms-3.4.0-SNAPSHOT.jar!/webapps/static,AVAILABLE}
> 2020-07-23 10:57:31,986 INFO TypeUtil - JVM Runtime does not support Modules
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,015 INFO KMSWebApp - Java runtime version :
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:32,015 INFO KMSWebApp - User: hadoop
> 2020-07-23 10:57:32,015 INFO KMSWebApp - KMS Hadoop Version: 3.4.0-SNAPSHOT
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,023 INFO KMSACLs - 'CREATE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DELETE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'ROLLOVER' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_KEYS' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_METADATA' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'SET_KEY_MATERIAL' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GENERATE_EEK' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DECRYPT_EEK' ACL '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP 'READ' is
> set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'MANAGEMENT' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'GENERATE_EEK' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'DECRYPT_EEK' is set to '*'
> 2020-07-23 10:57:32,080 INFO KMSAudit - Initializing audit logger class
> org.apache.hadoop.crypto.key.kms.server.SimpleKMSAuditLogger
> 2020-07-23 10:57:32,537 INFO KMSWebServer - SHUTDOWN_MSG:
> /
> SHUTDOWN_MSG: Shutting down KMSWebServer at
> hadoop-benchmark/172.17.0.2{noformat}
> I have googled the error and found there is a simlar issue：
> [https://github.com/eclipse/jetty.project/issues/4064]
> It looks like a bug of jetty and has been fixed in jetty>=9.4.21, currently
> Hadoop use the jetty is version of 9.4.20, see hadoop-project/pom.xml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17151) Need to upgrad the version of jetty
[
https://issues.apache.org/jira/browse/HADOOP-17151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-17151:
Affects Version/s: 3.3.0
> Need to upgrad the version of jetty
> ---
>
> Key: HADOOP-17151
> URL: https://issues.apache.org/jira/browse/HADOOP-17151
> Project: Hadoop Common
> Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: liusheng
>Priority: Major
> Attachments: HADOOP-17151.001.patch
>
>
> I have tried to configure and start Hadoop KMS service, it was failed to
> start the error log messages:
> {noformat}
> 2020-07-23 10:57:31,872 INFO Server - jetty-9.4.20.v20190813; built:
> 2019-08-13T21:28:18.144Z; git: 84700530e645e812b336747464d6fbbf370c9a20; jvm
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:31,899 INFO session - DefaultSessionIdManager
> workerName=node0
> 2020-07-23 10:57:31,899 INFO session - No SessionScavenger set, using
> defaults
> 2020-07-23 10:57:31,901 INFO session - node0 Scavenging every 66ms
> 2020-07-23 10:57:31,912 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@5bf0d49{logs,/logs,file:///opt/hadoop-3.4.0-SNAPSHOT/logs/,AVAILABLE}
> 2020-07-23 10:57:31,913 INFO ContextHandler - Started
> o.e.j.s.ServletContextHandler@7c7a06ec{static,/static,jar:file:/opt/hadoop-3.4.0-SNAPSHOT/share/hadoop/common/hadoop-kms-3.4.0-SNAPSHOT.jar!/webapps/static,AVAILABLE}
> 2020-07-23 10:57:31,986 INFO TypeUtil - JVM Runtime does not support Modules
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,015 INFO KMSWebApp - Java runtime version :
> 1.8.0_252-8u252-b09-1~18.04-b09
> 2020-07-23 10:57:32,015 INFO KMSWebApp - User: hadoop
> 2020-07-23 10:57:32,015 INFO KMSWebApp - KMS Hadoop Version: 3.4.0-SNAPSHOT
> 2020-07-23 10:57:32,015 INFO KMSWebApp -
> -
> 2020-07-23 10:57:32,023 INFO KMSACLs - 'CREATE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DELETE' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'ROLLOVER' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_KEYS' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GET_METADATA' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'SET_KEY_MATERIAL' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'GENERATE_EEK' ACL '*'
> 2020-07-23 10:57:32,024 INFO KMSACLs - 'DECRYPT_EEK' ACL '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP 'READ' is
> set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'MANAGEMENT' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'GENERATE_EEK' is set to '*'
> 2020-07-23 10:57:32,025 INFO KMSACLs - default.key.acl. for KEY_OP
> 'DECRYPT_EEK' is set to '*'
> 2020-07-23 10:57:32,080 INFO KMSAudit - Initializing audit logger class
> org.apache.hadoop.crypto.key.kms.server.SimpleKMSAuditLogger
> 2020-07-23 10:57:32,537 INFO KMSWebServer - SHUTDOWN_MSG:
> /
> SHUTDOWN_MSG: Shutting down KMSWebServer at
> hadoop-benchmark/172.17.0.2{noformat}
> I have googled the error and found there is a simlar issue：
> [https://github.com/eclipse/jetty.project/issues/4064]
> It looks like a bug of jetty and has been fixed in jetty>=9.4.21, currently
> Hadoop use the jetty is version of 9.4.20, see hadoop-project/pom.xml.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17141) Add Capability To Get Text Length
[
https://issues.apache.org/jira/browse/HADOOP-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17164325#comment-17164325
]
Hudson commented on HADOOP-17141:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18469 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/18469/])
HADOOP-17141. Add Capability To Get Text Length (#2157) (github: rev
e60096c377d8a3cb5bed3992352779195be95bb4)
* (edit)
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java
* (edit)
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/TestText.java
> Add Capability To Get Text Length
> -
>
> Key: HADOOP-17141
> URL: https://issues.apache.org/jira/browse/HADOOP-17141
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.4.0
>
>
> The Hadoop {{Text}} class contains an array of byte which contain a UTF-8
> encoded string. However, there is no way to quickly get the length of that
> string. One can get the number of bytes in the byte array, but to figure out
> the length of the String, it needs to be decoded first. In this simple
> example, sorting the {{Text}} objects by String length, the String needs to
> be decoded from the byte array repeatedly. This was brought to my attention
> based on [HIVE-23870].
> {code:java}
> public static void main(String[] args) {
> List list = Arrays.asList(new Text("1"), new Text("22"), new
> Text("333"));
> list.sort((Text t1, Text t2) -> t1.toString().length() -
> t2.toString().length());
> }
> {code}
> Also helpful if I want to check the last letter in the {{Text}} object
> repeatedly:
> {code:java}
> Text t = new Text("");
> System.out.println(t.charAt(t.toString().length() - 1));
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17145) Unauthenticated users are not authorized to access this page message is misleading in HttpServer2.java
[
https://issues.apache.org/jira/browse/HADOOP-17145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17164318#comment-17164318
]
Hadoop QA commented on HADOOP-17145:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 35m
17s{color} | {color:red} Docker failed to build yetus/hadoop:cce5a6f6094.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HADOOP-17145 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13008338/HADOOP-17145.002.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/17064/console |
| versions | git=2.17.1 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
> Unauthenticated users are not authorized to access this page message is
> misleading in HttpServer2.java
> --
>
> Key: HADOOP-17145
> URL: https://issues.apache.org/jira/browse/HADOOP-17145
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Andras Bokor
>Assignee: Andras Bokor
>Priority: Major
> Attachments: HADOOP-17145.001.patch, HADOOP-17145.002.patch
>
>
> Recently one of the users were misled by the message "Unauthenticated users
> are not authorized to access this page" when the user was not an admin user.
> At that point the user is authenticated but has no admin access, so it's
> actually not an authentication issue but an authorization issue.
> Also, 401 as error code would be better.
> Something like "User is unauthorized to access the page" would help to users
> to find out what is the problem during access an http endpoint.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2157: HADOOP-17141: Add Capability To Get Text Length
steveloughran commented on pull request #2157: URL: https://github.com/apache/hadoop/pull/2157#issuecomment-663453790 merged to trunk. Does not merge into branch-3.3. If you want it there (and I think it should), do a PR with the backport against that branch or, even better, get the change which causes the conflict (commit #eca05917d60) into branch-3.3 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17141) Add Capability To Get Text Length
[
https://issues.apache.org/jira/browse/HADOOP-17141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-17141.
-
Fix Version/s: 3.4.0
Resolution: Fixed
> Add Capability To Get Text Length
> -
>
> Key: HADOOP-17141
> URL: https://issues.apache.org/jira/browse/HADOOP-17141
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 3.4.0
>
>
> The Hadoop {{Text}} class contains an array of byte which contain a UTF-8
> encoded string. However, there is no way to quickly get the length of that
> string. One can get the number of bytes in the byte array, but to figure out
> the length of the String, it needs to be decoded first. In this simple
> example, sorting the {{Text}} objects by String length, the String needs to
> be decoded from the byte array repeatedly. This was brought to my attention
> based on [HIVE-23870].
> {code:java}
> public static void main(String[] args) {
> List list = Arrays.asList(new Text("1"), new Text("22"), new
> Text("333"));
> list.sort((Text t1, Text t2) -> t1.toString().length() -
> t2.toString().length());
> }
> {code}
> Also helpful if I want to check the last letter in the {{Text}} object
> repeatedly:
> {code:java}
> Text t = new Text("");
> System.out.println(t.charAt(t.toString().length() - 1));
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran merged pull request #2157: HADOOP-17141: Add Capability To Get Text Length
steveloughran merged pull request #2157: URL: https://github.com/apache/hadoop/pull/2157 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17145) Unauthenticated users are not authorized to access this page message is misleading in HttpServer2.java
[ https://issues.apache.org/jira/browse/HADOOP-17145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor updated HADOOP-17145: -- Attachment: HADOOP-17145.002.patch > Unauthenticated users are not authorized to access this page message is > misleading in HttpServer2.java > -- > > Key: HADOOP-17145 > URL: https://issues.apache.org/jira/browse/HADOOP-17145 > Project: Hadoop Common > Issue Type: Bug >Reporter: Andras Bokor >Assignee: Andras Bokor >Priority: Major > Attachments: HADOOP-17145.001.patch, HADOOP-17145.002.patch > > > Recently one of the users were misled by the message "Unauthenticated users > are not authorized to access this page" when the user was not an admin user. > At that point the user is authenticated but has no admin access, so it's > actually not an authentication issue but an authorization issue. > Also, 401 as error code would be better. > Something like "User is unauthorized to access the page" would help to users > to find out what is the problem during access an http endpoint. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] sunchao commented on a change in pull request #2138: HDFS-15469. Dynamically configure the size of PacketReceiver#MAX_PACKET_SIZE.
sunchao commented on a change in pull request #2138: URL: https://github.com/apache/hadoop/pull/2138#discussion_r459899397 ## File path: hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/client/HdfsClientConfigKeys.java ## @@ -221,6 +221,12 @@ "dfs.encrypt.data.transfer.cipher.key.bitlength"; intDFS_ENCRYPT_DATA_TRANSFER_CIPHER_KEY_BITLENGTH_DEFAULT = 128; + public static final String Review comment: nit: fix the checkstyle complain This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17149) ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run with SharedKey
[ https://issues.apache.org/jira/browse/HADOOP-17149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sneha Vijayarajan updated HADOOP-17149: --- Description: When authentication is set to SharedKey, below test fails. [ERROR] ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 Expecting org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException with text "Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.", 403 but got : "void" This test fails when the newly introduced config "fs.azure.account.hns.enabled" is set. This config will avoid network call to check if namespace is enabled, whereas the test expects thsi call to be made. The assert in test to 403 needs check too. Should ideally be 401. was: When authentication is set to SharedKey, below test fails. [ERROR] ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 Expecting org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException with text "Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.", 403 but got : "void" 2 problems: # This test should probably be disabled for SharedKey # Assert is wrong. Expected Http Status code should 401. > ABFS: Test failure: testFailedRequestWhenCredentialsNotCorrect fails when run > with SharedKey > > > Key: HADOOP-17149 > URL: https://issues.apache.org/jira/browse/HADOOP-17149 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Sneha Vijayarajan >Assignee: Sneha Vijayarajan >Priority: Minor > Labels: abfsactive > Fix For: 3.4.0 > > > When authentication is set to SharedKey, below test fails. > > [ERROR] > ITestGetNameSpaceEnabled.testFailedRequestWhenCredentialsNotCorrect:161 > Expecting > org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException > with text "Server failed to authenticate the request. Make sure the value of > Authorization header is formed correctly including the signature.", 403 but > got : "void" > > This test fails when the newly introduced config > "fs.azure.account.hns.enabled" is set. This config will avoid network call to > check if namespace is enabled, whereas the test expects thsi call to be made. > > The assert in test to 403 needs check too. Should ideally be 401. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
