[GitHub] [hadoop] virajjasani commented on pull request #5523: HDFS-16967. RBF: File based state stores should allow concurrent access to the records
virajjasani commented on PR #5523: URL: https://github.com/apache/hadoop/pull/5523#issuecomment-1491325194 Default mode: https://user-images.githubusercontent.com/34790606/229032556-3603807e-94de-48f8-87bc-831f4519ed11.png";> https://user-images.githubusercontent.com/34790606/229032560-24d586f9-4814-4192-afc3-09bb524846ea.png";> Concurrent access mode: https://user-images.githubusercontent.com/34790606/229032661-9f131979-d329-4bf3-8933-ae2009d94301.png";> https://user-images.githubusercontent.com/34790606/229032665-f846e183-bf89-45a6-a68b-55d57fb8a6e6.png";> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] rdingankar commented on pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…
rdingankar commented on PR #5495: URL: https://github.com/apache/hadoop/pull/5495#issuecomment-1491253601 The failed UTs seem flaky and unrelated to the change in this PR. They pass locally. @goiri Can you please help merging the PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18680) Insufficient heap during full test runs in Docker container.
[
https://issues.apache.org/jira/browse/HADOOP-18680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707100#comment-17707100
]
ASF GitHub Bot commented on HADOOP-18680:
-
hadoop-yetus commented on PR #5522:
URL: https://github.com/apache/hadoop/pull/5522#issuecomment-1491249469
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 21m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | hadolint | 0m 2s | | No new issues. |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | shadedclient | 22m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 28s | | The patch does not
generate ASF License warnings. |
| | | 48m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5522 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint
shellcheck shelldocs |
| uname | Linux f829e031f7f0 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f845594b5903ee290f5986fd23abac4f50ddfa10 |
| Max. process+thread count | 571 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/console |
| versions | git=2.25.1 maven=3.6.3 hadolint=1.11.1-0-g0e692dd
shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Insufficient heap during full test runs in Docker container.
>
>
> Key: HADOOP-18680
> URL: https://issues.apache.org/jira/browse/HADOOP-18680
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Labels: pull-request-available
>
> During verification of releases on the 3.3 line, I often run out of heap
> during full test runs inside the Docker container. Let's increase the default
> in {{MAVEN_OPTS}} to match trunk.
> Additionally, on trunk, the settings are different in Dockerfile vs.
> Dockerfile_aarch64. We can align those.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5522: HADOOP-18680: Insufficient heap during full test runs in Docker container.
hadoop-yetus commented on PR #5522:
URL: https://github.com/apache/hadoop/pull/5522#issuecomment-1491249469
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 21m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | hadolint | 0m 2s | | No new issues. |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | shadedclient | 22m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 28s | | The patch does not
generate ASF License warnings. |
| | | 48m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5522 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint
shellcheck shelldocs |
| uname | Linux f829e031f7f0 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f845594b5903ee290f5986fd23abac4f50ddfa10 |
| Max. process+thread count | 571 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/console |
| versions | git=2.25.1 maven=3.6.3 hadolint=1.11.1-0-g0e692dd
shellcheck=0.7.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] cnauroth commented on a diff in pull request #5520: HDFS-16965. Add switch to decide whether to enable native codec.
cnauroth commented on code in PR #5520:
URL: https://github.com/apache/hadoop/pull/5520#discussion_r1153980763
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/CodecUtil.java:
##
@@ -170,8 +174,14 @@ private static String[] getRawCoderNames(
private static RawErasureEncoder createRawEncoderWithFallback(
Configuration conf, String codecName, ErasureCoderOptions coderOptions) {
+boolean ISALEnabled =
conf.getBoolean(IO_ERASURECODE_CODEC_NATIVE_ENABLED_KEY,
+IO_ERASURECODE_CODEC_NATIVE_ENABLED_DEFAULT);
String[] rawCoderNames = getRawCoderNames(conf, codecName);
for (String rawCoderName : rawCoderNames) {
+ if (!ISALEnabled && rawCoderName.split("_")[1].equals("native")) {
Review Comment:
Ideally, this would be made more resilient to unexpected input (e.g. a
configured coder name without an "_", resulting in an
`ArrayIndexOutOfBoundsException`.).
##
hadoop-common-project/hadoop-common/src/main/resources/core-default.xml:
##
@@ -920,6 +920,14 @@
+
+ io.erasurecode.codec.native.enabled
+ true
+
+Used to decide whether to enable native codec.
Review Comment:
This needs more explanation. In what cases is it a good idea to disable
ISA-L support?
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCodecRawCoderMapping.java:
##
@@ -30,17 +30,22 @@
import org.apache.hadoop.io.erasurecode.rawcoder.RawErasureEncoder;
import org.apache.hadoop.io.erasurecode.rawcoder.XORRawDecoder;
import org.apache.hadoop.io.erasurecode.rawcoder.XORRawEncoder;
+import org.apache.hadoop.io.erasurecode.rawcoder.NativeXORRawEncoder;
+import org.apache.hadoop.io.erasurecode.rawcoder.NativeXORRawDecoder;
import org.apache.hadoop.io.erasurecode.rawcoder.XORRawErasureCoderFactory;
import org.apache.hadoop.test.GenericTestUtils;
import org.junit.Assert;
import org.junit.Before;
import org.junit.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
/**
* Test the codec to raw coder mapping.
*/
public class TestCodecRawCoderMapping {
-
+ public static final Logger LOG =
Review Comment:
Nitpick: Switch to `private`.
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCodecRawCoderMapping.java:
##
@@ -150,4 +155,44 @@ public void testIgnoreInvalidCodec() {
conf, ErasureCodeConstants.XOR_CODEC_NAME, coderOptions);
Assert.assertTrue(decoder instanceof XORRawDecoder);
}
+
+ @Test
+ public void testCodecNativeEnabled() {
+if (!ErasureCodeNative.isNativeCodeLoaded()) {
+ LOG.warn("ISA-L support is not available in your platform.");
+ return;
+}
+ErasureCoderOptions coderOptions = new ErasureCoderOptions(
+numDataUnit, numParityUnit);
+
+conf.setBoolean(CodecUtil.IO_ERASURECODE_CODEC_NATIVE_ENABLED_KEY,
+CodecUtil.IO_ERASURECODE_CODEC_NATIVE_ENABLED_DEFAULT);
+RawErasureEncoder rsEncoder = CodecUtil.createRawEncoder(
+conf, ErasureCodeConstants.RS_CODEC_NAME, coderOptions);
+RawErasureDecoder rsDecoder = CodecUtil.createRawDecoder(
+conf, ErasureCodeConstants.RS_CODEC_NAME, coderOptions);
+RawErasureEncoder xorEncoder = CodecUtil.createRawEncoder(
+conf, ErasureCodeConstants.XOR_CODEC_NAME, coderOptions);
+RawErasureDecoder xorDecoder = CodecUtil.createRawDecoder(
+conf, ErasureCodeConstants.XOR_CODEC_NAME, coderOptions);
+Assert.assertTrue(rsEncoder instanceof NativeRSRawEncoder);
Review Comment:
Not entirely related to your change, but the whole file would probably be
more readable with static imports of the assert methods.
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCodecRawCoderMapping.java:
##
@@ -150,4 +155,44 @@ public void testIgnoreInvalidCodec() {
conf, ErasureCodeConstants.XOR_CODEC_NAME, coderOptions);
Assert.assertTrue(decoder instanceof XORRawDecoder);
}
+
+ @Test
+ public void testCodecNativeEnabled() {
+if (!ErasureCodeNative.isNativeCodeLoaded()) {
Review Comment:
This looks more like a use case for JUnit
[`Assume#assumeTrue`](https://junit.org/junit4/javadoc/4.13/org/junit/Assume.html#assumeTrue(boolean)).
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/io/erasurecode/TestCodecRawCoderMapping.java:
##
@@ -150,4 +155,44 @@ public void testIgnoreInvalidCodec() {
conf, ErasureCodeConstants.XOR_CODEC_NAME, coderOptions);
Assert.assertTrue(decoder instanceof XORRawDecoder);
}
+
+ @Test
+ public void testCodecNativeEnabled() {
+if (!ErasureCodeNative.isNativeCodeLoaded()) {
+ LOG.warn("ISA-L support is not available in your platform.");
+ return;
+}
+ErasureCoderOptions coderOptions = new ErasureCoderOptions(
+numDataUnit, numParityUnit);
+
+conf.
[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[
https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707098#comment-17707098
]
ASF GitHub Bot commented on HADOOP-18684:
-
cnauroth commented on code in PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#discussion_r1153974579
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/InternalConstants.java:
##
@@ -182,4 +182,9 @@ private InternalConstants() {
Collections.unmodifiableSet(
new HashSet<>(Arrays.asList(Constants.FS_S3A_CREATE_PERFORMANCE)));
+ /**
+ * The default scheme for S3A filesystem.
+ */
+ public static final String S3A_SCHEME_DEFAULT = "s3a";
Review Comment:
Instead of defining a new constant, can you reuse
`org.apache.hadoop.fs.s3a.Constans#FS_S3A`?
> Fix S3A filesystem such that the scheme matches the URI scheme
> --
>
> Key: HADOOP-18684
> URL: https://issues.apache.org/jira/browse/HADOOP-18684
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.3.5
>Reporter: Harshit Gupta
>Priority: Major
> Labels: pull-request-available
>
> Certain codepaths use the FileContext API's to perform FS based operations
> such as yarn log aggregations. While trying to reuse the S3A connector for
> GCS based workloads the yarn log aggregation was not happening. Upon further
> investigation it was observed that FileContext API have hardcoded URI scheme
> checks that need to disabled/updated to make S3A compatible with non AWS
> stores.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] cnauroth commented on a diff in pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme
cnauroth commented on code in PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#discussion_r1153974579
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/InternalConstants.java:
##
@@ -182,4 +182,9 @@ private InternalConstants() {
Collections.unmodifiableSet(
new HashSet<>(Arrays.asList(Constants.FS_S3A_CREATE_PERFORMANCE)));
+ /**
+ * The default scheme for S3A filesystem.
+ */
+ public static final String S3A_SCHEME_DEFAULT = "s3a";
Review Comment:
Instead of defining a new constant, can you reuse
`org.apache.hadoop.fs.s3a.Constans#FS_S3A`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18680) Insufficient heap during full test runs in Docker container.
[
https://issues.apache.org/jira/browse/HADOOP-18680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707093#comment-17707093
]
ASF GitHub Bot commented on HADOOP-18680:
-
hadoop-yetus commented on PR #5522:
URL: https://github.com/apache/hadoop/pull/5522#issuecomment-1491227005
(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/console in
case of problems.
> Insufficient heap during full test runs in Docker container.
>
>
> Key: HADOOP-18680
> URL: https://issues.apache.org/jira/browse/HADOOP-18680
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Labels: pull-request-available
>
> During verification of releases on the 3.3 line, I often run out of heap
> during full test runs inside the Docker container. Let's increase the default
> in {{MAVEN_OPTS}} to match trunk.
> Additionally, on trunk, the settings are different in Dockerfile vs.
> Dockerfile_aarch64. We can align those.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5522: HADOOP-18680: Insufficient heap during full test runs in Docker container.
hadoop-yetus commented on PR #5522: URL: https://github.com/apache/hadoop/pull/5522#issuecomment-1491227005 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/console in case of problems. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18680) Insufficient heap during full test runs in Docker container.
[
https://issues.apache.org/jira/browse/HADOOP-18680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707092#comment-17707092
]
ASF GitHub Bot commented on HADOOP-18680:
-
hadoop-yetus commented on PR #5522:
URL: https://github.com/apache/hadoop/pull/5522#issuecomment-1491225676
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 26m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 31m 9s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | hadolint | 0m 2s | | No new issues. |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | shadedclient | 31m 53s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 92m 43s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5522 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint
shellcheck shelldocs |
| uname | Linux b67f2816b39f 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f845594b5903ee290f5986fd23abac4f50ddfa10 |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/console |
| versions | git=2.20.1 maven=3.6.0 hadolint=1.11.1-0-g0e692dd
shellcheck=0.5.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Insufficient heap during full test runs in Docker container.
>
>
> Key: HADOOP-18680
> URL: https://issues.apache.org/jira/browse/HADOOP-18680
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Labels: pull-request-available
>
> During verification of releases on the 3.3 line, I often run out of heap
> during full test runs inside the Docker container. Let's increase the default
> in {{MAVEN_OPTS}} to match trunk.
> Additionally, on trunk, the settings are different in Dockerfile vs.
> Dockerfile_aarch64. We can align those.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5522: HADOOP-18680: Insufficient heap during full test runs in Docker container.
hadoop-yetus commented on PR #5522:
URL: https://github.com/apache/hadoop/pull/5522#issuecomment-1491225676
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 26m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 31m 9s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | hadolint | 0m 2s | | No new issues. |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | shadedclient | 31m 53s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 92m 43s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5522 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint
shellcheck shelldocs |
| uname | Linux b67f2816b39f 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f845594b5903ee290f5986fd23abac4f50ddfa10 |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/console |
| versions | git=2.20.1 maven=3.6.0 hadolint=1.11.1-0-g0e692dd
shellcheck=0.5.0 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18680) Insufficient heap during full test runs in Docker container.
[
https://issues.apache.org/jira/browse/HADOOP-18680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707081#comment-17707081
]
ASF GitHub Bot commented on HADOOP-18680:
-
hadoop-yetus commented on PR #5522:
URL: https://github.com/apache/hadoop/pull/5522#issuecomment-1491166976
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 25m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | hadolint | 0m 0s | | hadolint was not available. |
| +0 :ok: | shellcheck | 0m 0s | | Shellcheck was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 1m 4s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 23m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | shadedclient | 21m 55s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 38s | | The patch does not
generate ASF License warnings. |
| | | 73m 29s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5522 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint
shellcheck shelldocs |
| uname | Linux 0b85b98009ec 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f845594b5903ee290f5986fd23abac4f50ddfa10 |
| Max. process+thread count | 538 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/console |
| versions | git=2.27.0 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Insufficient heap during full test runs in Docker container.
>
>
> Key: HADOOP-18680
> URL: https://issues.apache.org/jira/browse/HADOOP-18680
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Labels: pull-request-available
>
> During verification of releases on the 3.3 line, I often run out of heap
> during full test runs inside the Docker container. Let's increase the default
> in {{MAVEN_OPTS}} to match trunk.
> Additionally, on trunk, the settings are different in Dockerfile vs.
> Dockerfile_aarch64. We can align those.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5522: HADOOP-18680: Insufficient heap during full test runs in Docker container.
hadoop-yetus commented on PR #5522:
URL: https://github.com/apache/hadoop/pull/5522#issuecomment-1491166976
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 25m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | hadolint | 0m 0s | | hadolint was not available. |
| +0 :ok: | shellcheck | 0m 0s | | Shellcheck was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 1m 4s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 23m 22s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | shadedclient | 21m 55s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 38s | | The patch does not
generate ASF License warnings. |
| | | 73m 29s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5522 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint
shellcheck shelldocs |
| uname | Linux 0b85b98009ec 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f845594b5903ee290f5986fd23abac4f50ddfa10 |
| Max. process+thread count | 538 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/console |
| versions | git=2.27.0 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…
hadoop-yetus commented on PR #5495:
URL: https://github.com/apache/hadoop/pull/5495#issuecomment-1491146475
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 32s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 53s | | trunk passed |
| +1 :green_heart: | compile | 25m 9s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 45s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 4m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 19s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 14s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 22s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 14s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 26s | | the patch passed |
| +1 :green_heart: | compile | 27m 10s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 27m 10s | | the patch passed |
| +1 :green_heart: | compile | 24m 11s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 24m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 17s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 7s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 28s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 230m 53s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/14/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not
generate ASF License warnings. |
| | | 484m 35s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.server.namenode.ha.TestObserverNode |
| | hadoop.hdfs.TestRollingUpgrade |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/14/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5495 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4ea18ab044b1 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6868c3a0f25012d771c1dfe8e7fe44a55b8bc4ef |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/14/testReport/ |
| Max. process+thread count | 2644 (vs. ulimit
[GitHub] [hadoop] hadoop-yetus commented on pull request #5523: HDFS-16967. RBF: File based state stores should allow concurrent access to the records
hadoop-yetus commented on PR #5523:
URL: https://github.com/apache/hadoop/pull/5523#issuecomment-1491141117
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 2s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 2s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 2s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 49s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 48s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 57s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 51s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 22s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 22m 11s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 125m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5523/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5523 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 1a3d234f15b4 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0c41ead9dc12fca491f65604bea088c9e19b4a5a |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5523/1/testReport/ |
| Max. process+thread count | 2917 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5523/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To uns
[jira] [Commented] (HADOOP-18680) Insufficient heap during full test runs in Docker container.
[
https://issues.apache.org/jira/browse/HADOOP-18680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707073#comment-17707073
]
ASF GitHub Bot commented on HADOOP-18680:
-
hadoop-yetus commented on PR #5522:
URL: https://github.com/apache/hadoop/pull/5522#issuecomment-1491125289
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 43m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | hadolint | 0m 0s | | hadolint was not available. |
| +0 :ok: | shellcheck | 0m 0s | | Shellcheck was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 58s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 40m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | shadedclient | 21m 44s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 109m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5522 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint
shellcheck shelldocs |
| uname | Linux 744e0c36e772 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f845594b5903ee290f5986fd23abac4f50ddfa10 |
| Max. process+thread count | 600 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/console |
| versions | git=2.9.5 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Insufficient heap during full test runs in Docker container.
>
>
> Key: HADOOP-18680
> URL: https://issues.apache.org/jira/browse/HADOOP-18680
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Labels: pull-request-available
>
> During verification of releases on the 3.3 line, I often run out of heap
> during full test runs inside the Docker container. Let's increase the default
> in {{MAVEN_OPTS}} to match trunk.
> Additionally, on trunk, the settings are different in Dockerfile vs.
> Dockerfile_aarch64. We can align those.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5522: HADOOP-18680: Insufficient heap during full test runs in Docker container.
hadoop-yetus commented on PR #5522:
URL: https://github.com/apache/hadoop/pull/5522#issuecomment-1491125289
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 43m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | hadolint | 0m 0s | | hadolint was not available. |
| +0 :ok: | shellcheck | 0m 0s | | Shellcheck was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 58s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 40m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 24s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | shadedclient | 21m 44s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 109m 11s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5522 |
| Optional Tests | dupname asflicense codespell detsecrets hadolint
shellcheck shelldocs |
| uname | Linux 744e0c36e772 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f845594b5903ee290f5986fd23abac4f50ddfa10 |
| Max. process+thread count | 600 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5522/1/console |
| versions | git=2.9.5 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] cnauroth commented on a diff in pull request #5519: MAPREDUCE-7435. Manifest Committer OOM on abfs
cnauroth commented on code in PR #5519:
URL: https://github.com/apache/hadoop/pull/5519#discussion_r1153857994
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/impl/ManifestCommitterSupport.java:
##
@@ -224,6 +231,23 @@ public static ManifestSuccessData createManifestOutcome(
return outcome;
}
+ /**
+ * Add heap information to IOStatisticSetters gauges, with a stage in front
of every key.
+ * @param ioStatisticsSetters map to update
+ * @param stage stage
+ */
+ public static void addHeapInformation(IOStatisticsSetters
ioStatisticsSetters,
+ String stage) {
+// force a gc. bit of bad form but it makes for better numbers
+System.gc();
Review Comment:
This triggered a Spotbugs warning. Do think the forced GC should go behind a
config flag, default off, and turned on in the tests?
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/statistics/IOStatisticsSnapshot.java:
##
@@ -222,6 +223,33 @@ public synchronized Map
meanStatistics() {
return meanStatistics;
}
+ @Override
+ public synchronized void setCounter(final String key, final long value) {
+counters().put(key, value);
+ }
+
+ @Override
+ public synchronized void setGauge(final String key, final long value) {
+gauges().put(key, value);
+
+ }
+
+ @Override
+ public synchronized void setMaximum(final String key, final long value) {
+maximums().put(key, value);
+
+ }
+
+ @Override
+ public synchronized void setMinimum(final String key, final long value) {
+minimums().put(key, value);
+ }
+
+ @Override
+ public void setMeanStatistic(final String key, final MeanStatistic value) {
+
Review Comment:
`meanStatistics().put(key, value);`?
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/TestLoadManifestsStage.java:
##
@@ -63,6 +81,10 @@ public void setup() throws Exception {
.isGreaterThan(0);
}
+ public long heapSize() {
+ return Runtime.getRuntime().totalMemory() -
Runtime.getRuntime().freeMemory();
Review Comment:
Nitpick: some indentation issues here.
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/TestLoadManifestsStage.java:
##
@@ -134,8 +169,26 @@ public void testSaveThenLoadManyManifests() throws
Throwable {
// and skipping the rename stage (which is going to fail),
// go straight to cleanup
-new CleanupJobStage(getJobStageConfig()).apply(
+new CleanupJobStage(stageConfig).apply(
new CleanupJobStage.Arguments("", true, true, false));
+addHeapInformation(heapInfo, "cleanup");
+
+ManifestSuccessData success = createManifestOutcome(stageConfig,
OP_STAGE_JOB_COMMIT);
+success.snapshotIOStatistics(getStageStatistics());
+success.getIOStatistics().aggregate(heapInfo);
+
+Configuration conf = getConfiguration();
+enableManifestCommitter(conf);
+String reportDir = conf.getTrimmed(OPT_SUMMARY_REPORT_DIR, "");
+Path reportDirPath = new Path(reportDir);
+Path path = new Path(reportDirPath,
+createJobSummaryFilename("TestLoadManifestsStage"));
+final FileSystem summaryFS = path.getFileSystem(conf);
+success.save(summaryFS, path, true);
+LOG.info("Saved summary to {}", path);
+ManifestPrinter showManifest = new ManifestPrinter();
+ManifestSuccessData manifestSuccessData =
Review Comment:
Nitpick: some indentation issues here.
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestFileOutputCommitter.java:
##
@@ -756,66 +756,75 @@ private void testConcurrentCommitTaskWithSubDir(int
version)
conf.setInt(FileOutputCommitter.FILEOUTPUTCOMMITTER_ALGORITHM_VERSION,
version);
-conf.setClass("fs.file.impl", RLFS.class, FileSystem.class);
+final String fileImpl = "fs.file.impl";
+final String fileImplClassname = "org.apache.hadoop.fs.LocalFileSystem";
+conf.setClass(fileImpl, RLFS.class, FileSystem.class);
FileSystem.closeAll();
-final JobContext jContext = new JobContextImpl(conf, taskID.getJobID());
-final FileOutputCommitter amCommitter =
-new FileOutputCommitter(outDir, jContext);
-amCommitter.setupJob(jContext);
-
-final TaskAttemptContext[] taCtx = new TaskAttemptContextImpl[2];
-taCtx[0] = new TaskAttemptContextImpl(conf, taskID);
-taCtx[1] = new TaskAttemptContextImpl(conf, taskID1);
-
-final TextOutputFormat[] tof = new TextOutputFormat[2];
-for (int i = 0; i < tof.length; i++) {
- tof[i] = new TextOutputFormat() {
-@Override
-public Path getDefaultWorkFil
[GitHub] [hadoop] virajjasani commented on pull request #5523: HDFS-16967. RBF: File based state stores should allow concurrent access to the records
virajjasani commented on PR #5523: URL: https://github.com/apache/hadoop/pull/5523#issuecomment-1491057858 Based on one of the testing data points, for the same num of mount table records to be loaded in the cache, avg time taken by default is ~1500 ms whereas with concurrent mode, it goes down to ~130 ms. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #5523: HDFS-16967. RBF: File based state stores should allow concurrent access to the records
virajjasani commented on PR #5523: URL: https://github.com/apache/hadoop/pull/5523#issuecomment-1491056033 @goiri @ZanderXu @tasanuma could you please review this PR? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani opened a new pull request, #5523: HDFS-16967. RBF: File based state stores should allow concurrent access to the records
virajjasani opened a new pull request, #5523: URL: https://github.com/apache/hadoop/pull/5523 File based state store implementations (StateStoreFileImpl and StateStoreFileSystemImpl) should allow updating as well as reading of the state store records concurrently rather than serially. Concurrent access to the record files on the hdfs based store seems to be improving the state store cache loading performance by more than 10x. For instance, in order to maintain data integrity, when any mount table record(s) is updated, the cache is reloaded. This reload operation seems to be able to gain significant performance improvement by the concurrent access of the mount table records. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] rdingankar commented on a diff in pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…
rdingankar commented on code in PR #5495:
URL: https://github.com/apache/hadoop/pull/5495#discussion_r1153853239
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/util/TestSampleQuantiles.java:
##
@@ -37,8 +37,10 @@ public class TestSampleQuantiles {
new Quantile(0.95, 0.005), new Quantile(0.99, 0.001) };
SampleQuantiles estimator;
+ Random rnd = new Random(0xDEADDEAD);
+ final static int NUM_REPEATS = 10;
- @Before
+@Before
Review Comment:
In one of my commits I had unintentionally added extra space. Removed and
fixed it now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] rdingankar commented on a diff in pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…
rdingankar commented on code in PR #5495:
URL: https://github.com/apache/hadoop/pull/5495#discussion_r1153852005
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/util/TestSampleQuantiles.java:
##
@@ -92,22 +94,21 @@ public void testClear() throws IOException {
@Test
public void testQuantileError() throws IOException {
final int count = 10;
-Random r = new Random(0xDEADDEAD);
int[] values = new int[count];
for (int i = 0; i < count; i++) {
values[i] = i + 1;
}
-// Do 10 shuffle/insert/check cycles
-for (int i = 0; i < 10; i++) {
+// Repeat shuffle/insert/check cycles 10 times
+for (int i = 0; i < NUM_REPEATS; i++) {
// Shuffle
- Collections.shuffle(Arrays.asList(values), r);
+ Collections.shuffle(Arrays.asList(values), rnd);
Review Comment:
fixed the empty space after comment on L105
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18680) Insufficient heap during full test runs in Docker container.
[
https://issues.apache.org/jira/browse/HADOOP-18680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707055#comment-17707055
]
ASF GitHub Bot commented on HADOOP-18680:
-
cnauroth commented on code in PR #5511:
URL: https://github.com/apache/hadoop/pull/5511#discussion_r1153849194
##
dev-support/docker/Dockerfile_aarch64:
##
@@ -144,7 +144,7 @@ RUN npm install -g bower@1.8.8
###
# Avoid out of memory errors in builds
###
-ENV MAVEN_OPTS -Xms256m -Xmx1536m
+ENV MAVEN_OPTS -Xms256m -Xmx3072m
Review Comment:
Thanks for the review. Good idea. I requested your review on #5522 for this.
> Insufficient heap during full test runs in Docker container.
>
>
> Key: HADOOP-18680
> URL: https://issues.apache.org/jira/browse/HADOOP-18680
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Labels: pull-request-available
>
> During verification of releases on the 3.3 line, I often run out of heap
> during full test runs inside the Docker container. Let's increase the default
> in {{MAVEN_OPTS}} to match trunk.
> Additionally, on trunk, the settings are different in Dockerfile vs.
> Dockerfile_aarch64. We can align those.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri commented on a diff in pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…
goiri commented on code in PR #5495:
URL: https://github.com/apache/hadoop/pull/5495#discussion_r1153848968
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/util/TestSampleQuantiles.java:
##
@@ -37,8 +37,10 @@ public class TestSampleQuantiles {
new Quantile(0.95, 0.005), new Quantile(0.99, 0.001) };
SampleQuantiles estimator;
+ Random rnd = new Random(0xDEADDEAD);
+ final static int NUM_REPEATS = 10;
- @Before
+@Before
Review Comment:
Spacing is wrong?
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/util/TestSampleQuantiles.java:
##
@@ -92,22 +94,21 @@ public void testClear() throws IOException {
@Test
public void testQuantileError() throws IOException {
final int count = 10;
-Random r = new Random(0xDEADDEAD);
int[] values = new int[count];
for (int i = 0; i < count; i++) {
values[i] = i + 1;
}
-// Do 10 shuffle/insert/check cycles
-for (int i = 0; i < 10; i++) {
+// Repeat shuffle/insert/check cycles 10 times
+for (int i = 0; i < NUM_REPEATS; i++) {
// Shuffle
- Collections.shuffle(Arrays.asList(values), r);
+ Collections.shuffle(Arrays.asList(values), rnd);
Review Comment:
Check the checkstyle and empty line warnings.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] cnauroth commented on a diff in pull request #5511: HADOOP-18680: Insufficient heap during full test runs in Docker container.
cnauroth commented on code in PR #5511: URL: https://github.com/apache/hadoop/pull/5511#discussion_r1153849194 ## dev-support/docker/Dockerfile_aarch64: ## @@ -144,7 +144,7 @@ RUN npm install -g bower@1.8.8 ### # Avoid out of memory errors in builds ### -ENV MAVEN_OPTS -Xms256m -Xmx1536m +ENV MAVEN_OPTS -Xms256m -Xmx3072m Review Comment: Thanks for the review. Good idea. I requested your review on #5522 for this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…
hadoop-yetus commented on PR #5495:
URL: https://github.com/apache/hadoop/pull/5495#issuecomment-1491048324
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 1s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 57s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 45s | | trunk passed |
| +1 :green_heart: | compile | 23m 5s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 20m 28s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 47s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 28s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 38s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 19s | | the patch passed |
| +1 :green_heart: | compile | 22m 27s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 22m 27s | | the patch passed |
| +1 :green_heart: | compile | 20m 26s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 20m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 35s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 18s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 39s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 20s | | the patch passed |
| -1 :x: | shadedclient | 22m 55s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 18m 5s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/13/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| -1 :x: | unit | 217m 56s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/13/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 11s | | The patch does not
generate ASF License warnings. |
| | | 451m 24s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.metrics2.lib.TestMetricsRegistry |
| | hadoop.util.TestReadWriteDiskValidator |
| | hadoop.metrics2.lib.TestMutableMetrics |
| | hadoop.security.TestUserGroupInformation |
| | hadoop.ipc.TestRPC |
| | hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.hdfs.TestStateAlignmentContextWithHA |
| | hadoop.hdfs.TestFileChecksum |
| | hadoop.hdfs.server.namenode.ha.TestUpdateBlockTailing |
| | hadoop.hdfs.server.namenode.ha.TestMultiObserverNode |
| | hadoop.hdfs.qjournal.server.TestJournal |
| | hadoop.hdfs.server.namenode.ha.TestStandbyInProgressTail |
| | hadoop.hdfs.server.namenode.ha.TestConsistentReadsObserver |
| | hadoop.hdfs.TestRollingUpgrade |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl |
| | hadoop.hdfs.TestRollingUpgradeDowngrade |
| | hadoop.hdfs.tools.TestDFSAdminWithHA |
| | hadoop.hdfs.qjournal.client.TestEpochsAreUnique |
| | hadoop.hdfs.qjournal.server.TestJournalNode |
| | hadoop.hdfs.Te
[jira] [Commented] (HADOOP-18680) Insufficient heap during full test runs in Docker container.
[
https://issues.apache.org/jira/browse/HADOOP-18680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707053#comment-17707053
]
ASF GitHub Bot commented on HADOOP-18680:
-
cnauroth opened a new pull request, #5522:
URL: https://github.com/apache/hadoop/pull/5522
### Description of PR
The `MAVEN_OPTS` max heap size setting is different between Dockerfile vs.
Dockerfile_aarch64 on trunk. The Dockerfile setting is larger to avoid running
out of memory during full test runs. Update Dockerfile_aarch64 to match.
### How was this patch tested?
```
./start-build-env.sh
mvn --fail-never clean test -Pnative -Dparallel-tests -Drequire.snappy
-Drequire.zstd -Drequire.openssl -Dsurefire.rerunFailingTestsCount=3
-DtestsThreadCount=8
```
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> Insufficient heap during full test runs in Docker container.
>
>
> Key: HADOOP-18680
> URL: https://issues.apache.org/jira/browse/HADOOP-18680
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Labels: pull-request-available
>
> During verification of releases on the 3.3 line, I often run out of heap
> during full test runs inside the Docker container. Let's increase the default
> in {{MAVEN_OPTS}} to match trunk.
> Additionally, on trunk, the settings are different in Dockerfile vs.
> Dockerfile_aarch64. We can align those.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] cnauroth opened a new pull request, #5522: HADOOP-18680: Insufficient heap during full test runs in Docker container.
cnauroth opened a new pull request, #5522: URL: https://github.com/apache/hadoop/pull/5522 ### Description of PR The `MAVEN_OPTS` max heap size setting is different between Dockerfile vs. Dockerfile_aarch64 on trunk. The Dockerfile setting is larger to avoid running out of memory during full test runs. Update Dockerfile_aarch64 to match. ### How was this patch tested? ``` ./start-build-env.sh mvn --fail-never clean test -Pnative -Dparallel-tests -Drequire.snappy -Drequire.zstd -Drequire.openssl -Dsurefire.rerunFailingTestsCount=3 -DtestsThreadCount=8 ``` ### For code changes: - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18680) Insufficient heap during full test runs in Docker container.
[
https://issues.apache.org/jira/browse/HADOOP-18680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-18680:
---
Description:
During verification of releases on the 3.3 line, I often run out of heap during
full test runs inside the Docker container. Let's increase the default in
{{MAVEN_OPTS}} to match trunk.
Additionally, on trunk, the settings are different in Dockerfile vs.
Dockerfile_aarch64. We can align those.
was:During verification of releases on the 3.3 line, I often run out of heap
during full test runs inside the Docker container. Let's increase the default
in {{MAVEN_OPTS}} to match trunk.
Summary: Insufficient heap during full test runs in Docker container.
(was: Insufficient heap during full test runs in Docker container on
branch-3.3.)
> Insufficient heap during full test runs in Docker container.
>
>
> Key: HADOOP-18680
> URL: https://issues.apache.org/jira/browse/HADOOP-18680
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Labels: pull-request-available
>
> During verification of releases on the 3.3 line, I often run out of heap
> during full test runs inside the Docker container. Let's increase the default
> in {{MAVEN_OPTS}} to match trunk.
> Additionally, on trunk, the settings are different in Dockerfile vs.
> Dockerfile_aarch64. We can align those.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18680) Insufficient heap during full test runs in Docker container on branch-3.3.
[
https://issues.apache.org/jira/browse/HADOOP-18680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706989#comment-17706989
]
ASF GitHub Bot commented on HADOOP-18680:
-
ayushtkn commented on code in PR #5511:
URL: https://github.com/apache/hadoop/pull/5511#discussion_r1153646744
##
dev-support/docker/Dockerfile_aarch64:
##
@@ -144,7 +144,7 @@ RUN npm install -g bower@1.8.8
###
# Avoid out of memory errors in builds
###
-ENV MAVEN_OPTS -Xms256m -Xmx1536m
+ENV MAVEN_OPTS -Xms256m -Xmx3072m
Review Comment:
For aarch64 trunk doesn't have this change, worth pushing there as well.
https://github.com/apache/hadoop/blob/trunk/dev-support/docker/Dockerfile_aarch64#L77
> Insufficient heap during full test runs in Docker container on branch-3.3.
> --
>
> Key: HADOOP-18680
> URL: https://issues.apache.org/jira/browse/HADOOP-18680
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Labels: pull-request-available
>
> During verification of releases on the 3.3 line, I often run out of heap
> during full test runs inside the Docker container. Let's increase the default
> in {{MAVEN_OPTS}} to match trunk.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5511: HADOOP-18680: Insufficient heap during full test runs in Docker container on branch-3.3.
ayushtkn commented on code in PR #5511: URL: https://github.com/apache/hadoop/pull/5511#discussion_r1153646744 ## dev-support/docker/Dockerfile_aarch64: ## @@ -144,7 +144,7 @@ RUN npm install -g bower@1.8.8 ### # Avoid out of memory errors in builds ### -ENV MAVEN_OPTS -Xms256m -Xmx1536m +ENV MAVEN_OPTS -Xms256m -Xmx3072m Review Comment: For aarch64 trunk doesn't have this change, worth pushing there as well. https://github.com/apache/hadoop/blob/trunk/dev-support/docker/Dockerfile_aarch64#L77 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18680) Insufficient heap during full test runs in Docker container on branch-3.3.
[
https://issues.apache.org/jira/browse/HADOOP-18680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706988#comment-17706988
]
ASF GitHub Bot commented on HADOOP-18680:
-
ayushtkn commented on code in PR #5511:
URL: https://github.com/apache/hadoop/pull/5511#discussion_r1153646744
##
dev-support/docker/Dockerfile_aarch64:
##
@@ -144,7 +144,7 @@ RUN npm install -g bower@1.8.8
###
# Avoid out of memory errors in builds
###
-ENV MAVEN_OPTS -Xms256m -Xmx1536m
+ENV MAVEN_OPTS -Xms256m -Xmx3072m
Review Comment:
For aarch64 trunk doesn't have this change, worth pushing there as well.
> Insufficient heap during full test runs in Docker container on branch-3.3.
> --
>
> Key: HADOOP-18680
> URL: https://issues.apache.org/jira/browse/HADOOP-18680
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Reporter: Chris Nauroth
>Assignee: Chris Nauroth
>Priority: Minor
> Labels: pull-request-available
>
> During verification of releases on the 3.3 line, I often run out of heap
> during full test runs inside the Docker container. Let's increase the default
> in {{MAVEN_OPTS}} to match trunk.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ayushtkn commented on a diff in pull request #5511: HADOOP-18680: Insufficient heap during full test runs in Docker container on branch-3.3.
ayushtkn commented on code in PR #5511: URL: https://github.com/apache/hadoop/pull/5511#discussion_r1153646744 ## dev-support/docker/Dockerfile_aarch64: ## @@ -144,7 +144,7 @@ RUN npm install -g bower@1.8.8 ### # Avoid out of memory errors in builds ### -ENV MAVEN_OPTS -Xms256m -Xmx1536m +ENV MAVEN_OPTS -Xms256m -Xmx3072m Review Comment: For aarch64 trunk doesn't have this change, worth pushing there as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5363: YARN-11424. [Federation] Router Supports DeregisterSubCluster.
hadoop-yetus commented on PR #5363:
URL: https://github.com/apache/hadoop/pull/5363#issuecomment-1490667779
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 27s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 41s | | trunk passed |
| +1 :green_heart: | compile | 10m 24s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 8m 49s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 47s | | trunk passed |
| +1 :green_heart: | mvnsite | 5m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 11s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 44s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 11m 5s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 54s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 14s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 22s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 5s | | the patch passed |
| +1 :green_heart: | compile | 9m 50s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | cc | 9m 50s | | the patch passed |
| -1 :x: | javac | 9m 50s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5363/14/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 generated 1 new + 730
unchanged - 0 fixed = 731 total (was 730) |
| +1 :green_heart: | compile | 8m 45s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | cc | 8m 45s | | the patch passed |
| -1 :x: | javac | 8m 45s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5363/14/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09
with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 generated 2 new
+ 643 unchanged - 4 fixed = 645 total (was 647) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 38s | |
hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 230 unchanged - 1
fixed = 230 total (was 231) |
| +1 :green_heart: | mvnsite | 5m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 39s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 4m 22s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 12m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 5s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 22s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 13s | | hadoop-yarn-se
[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[
https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706975#comment-17706975
]
ASF GitHub Bot commented on HADOOP-18684:
-
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490651740
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 55m 6s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 5s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/4/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 17s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 27s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 116m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux abea41c84f72 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4bddca798790f88d98579ad40e041591e3bf97db |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/4/testReport/ |
| Max. process+thread count | 530 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job
[GitHub] [hadoop] hadoop-yetus commented on pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490651740
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 55m 6s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 5s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/4/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 17s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 27s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 116m 47s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux abea41c84f72 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4bddca798790f88d98579ad40e041591e3bf97db |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/4/testReport/ |
| Max. process+thread count | 530 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message
[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[
https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706971#comment-17706971
]
ASF GitHub Bot commented on HADOOP-18684:
-
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490635342
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 34s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 37s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 35s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/6/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 15s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 34s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 102m 0s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 0adcfc773b96 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4bddca798790f88d98579ad40e041591e3bf97db |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/6/testReport/ |
| Max. process+thread count | 640 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job
[GitHub] [hadoop] hadoop-yetus commented on pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490635342
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 34s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 37s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 35s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/6/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 15s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 34s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 102m 0s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 0adcfc773b96 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4bddca798790f88d98579ad40e041591e3bf97db |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/6/testReport/ |
| Max. process+thread count | 640 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message
[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[
https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706970#comment-17706970
]
ASF GitHub Bot commented on HADOOP-18684:
-
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490632200
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 1s | | trunk passed |
| +1 :green_heart: | compile | 0m 43s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 35s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 48s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/5/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 15s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 12s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 25s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 102m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 9862111fd6ff 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4bddca798790f88d98579ad40e041591e3bf97db |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/5/testReport/ |
| Max. process+thread count | 555 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job
[GitHub] [hadoop] hadoop-yetus commented on pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490632200
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 1s | | trunk passed |
| +1 :green_heart: | compile | 0m 43s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 35s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 48s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/5/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 15s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 12s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 25s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 102m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 9862111fd6ff 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4bddca798790f88d98579ad40e041591e3bf97db |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/5/testReport/ |
| Max. process+thread count | 555 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message
[GitHub] [hadoop] hadoop-yetus commented on pull request #5520: HDFS-16965. Add switch to decide whether to enable native codec.
hadoop-yetus commented on PR #5520:
URL: https://github.com/apache/hadoop/pull/5520#issuecomment-1490629400
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 51m 15s | | trunk passed |
| +1 :green_heart: | compile | 25m 12s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 21m 31s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 45s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 9s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 0s | | the patch passed |
| +1 :green_heart: | compile | 24m 33s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 24m 33s | | the patch passed |
| +1 :green_heart: | compile | 21m 41s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 21m 41s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 58s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5520/2/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 3 new + 2
unchanged - 0 fixed = 5 total (was 2) |
| +1 :green_heart: | mvnsite | 1m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 59s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 42s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 42s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 15s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 8s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 229m 15s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5520/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5520 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 2b49cc2fb4a5 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ed64df4dc94f859501002007e2364c06a8fa4828 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5520/2/testReport/ |
| Max. process+thread count | 1239 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5520/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by
[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[
https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706964#comment-17706964
]
ASF GitHub Bot commented on HADOOP-18684:
-
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490620563
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 25s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 21s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 15s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 18s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/3/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 2s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 32s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 106m 51s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux add2e8f11b7e 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7a4848a19fcc83939d19177a93c868c1a729c748 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/3/testReport/ |
| Max. process+thread count | 529 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job
[GitHub] [hadoop] hadoop-yetus commented on pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490620563
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 44m 25s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 21s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 15s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 18s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/3/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 9s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 2s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 32s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 106m 51s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux add2e8f11b7e 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7a4848a19fcc83939d19177a93c868c1a729c748 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/3/testReport/ |
| Max. process+thread count | 529 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message
[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[
https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706958#comment-17706958
]
ASF GitHub Bot commented on HADOOP-18684:
-
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490603596
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 44s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 20s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/2/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 12s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 33s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 96m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 19c7c5611f35 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7a4848a19fcc83939d19177a93c868c1a729c748 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/2/testReport/ |
| Max. process+thread count | 556 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job
[GitHub] [hadoop] hadoop-yetus commented on pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490603596
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 44s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 20s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/2/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 12s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 33s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 96m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 19c7c5611f35 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7a4848a19fcc83939d19177a93c868c1a729c748 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/2/testReport/ |
| Max. process+thread count | 556 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message
[jira] [Commented] (HADOOP-18671) Add recoverLease(), setSafeMode(), isFileClosed() APIs to FileSystem
[ https://issues.apache.org/jira/browse/HADOOP-18671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706942#comment-17706942 ] Tak-Lon (Stephen) Wu commented on HADOOP-18671: --- I'm going to give a try, will create the PR and come back on this. > Add recoverLease(), setSafeMode(), isFileClosed() APIs to FileSystem > > > Key: HADOOP-18671 > URL: https://issues.apache.org/jira/browse/HADOOP-18671 > Project: Hadoop Common > Issue Type: New Feature > Components: fs >Reporter: Wei-Chiu Chuang >Priority: Major > > We are in the midst of enabling HBase and Solr to run on Ozone. > An obstacle is that HBase relies heavily on HDFS APIs and semantics for its > Write Ahead Log (WAL) file (similarly, for Solr's transaction log). We > propose to push up these HDFS APIs, i.e. recoverLease(), setSafeMode(), > isFileClosed() to FileSystem abstraction so that HBase and other applications > do not need to take on Ozone dependency at compile time. This work will > (hopefully) enable HBase to run on other storage system implementations in > the future. > There are other HDFS features that HBase uses, including hedged read and > favored nodes. Those are FS-specific optimizations and are not critical to > enable HBase on Ozone. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4963: YARN-11326. [Federation] Add RM FederationStateStoreService Metrics.
hadoop-yetus commented on PR #4963:
URL: https://github.com/apache/hadoop/pull/4963#issuecomment-1490521782
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 0s | | trunk passed |
| +1 :green_heart: | compile | 1m 5s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 55s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 50s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 3s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 11s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 16s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 33s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 52s | | the patch passed |
| +1 :green_heart: | compile | 0m 59s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| -1 :x: | javac | 0m 59s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4963/30/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 generated 26 new + 226
unchanged - 0 fixed = 252 total (was 226) |
| +1 :green_heart: | compile | 0m 51s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| -1 :x: | javac | 0m 51s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4963/30/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09
with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 generated 26
new + 183 unchanged - 0 fixed = 209 total (was 183) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 39s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4963/30/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
|
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 54s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 36s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 2s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 1s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 116m 51s | |
hadoop-yarn-se
[jira] [Commented] (HADOOP-18565) AWS SDK V2 - Complete outstanding items
[ https://issues.apache.org/jira/browse/HADOOP-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706902#comment-17706902 ] ASF GitHub Bot commented on HADOOP-18565: - ahmarsuhail commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1490355464 Add the amazon.cn check where? haven't added `Endpoint is not set || Endpoint is set && ends in amazonaws.com || ARN is set` in this PR (if that is where you meant) > AWS SDK V2 - Complete outstanding items > --- > > Key: HADOOP-18565 > URL: https://issues.apache.org/jira/browse/HADOOP-18565 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Priority: Major > Labels: pull-request-available > > The following work remains to complete the SDK upgrade work: > * S3A allows users configure to custom signers, add in support for this. > * Remove SDK V1 bundle dependency > * Update `getRegion()` logic to use retries. > * Add in progress listeners for `S3ABlockOutputStream` > * Fix any failing tests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ahmarsuhail commented on pull request #5421: HADOOP-18565. Completes outstanding items for the SDK V2 upgrade.
ahmarsuhail commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1490355464 Add the amazon.cn check where? haven't added `Endpoint is not set || Endpoint is set && ends in amazonaws.com || ARN is set` in this PR (if that is where you meant) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18565) AWS SDK V2 - Complete outstanding items
[ https://issues.apache.org/jira/browse/HADOOP-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706899#comment-17706899 ] ASF GitHub Bot commented on HADOOP-18565: - steveloughran commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1490348322 w.r.t merging, add the amazon.cn check so it doesn't get forgotten about. then we should be good to mergel > AWS SDK V2 - Complete outstanding items > --- > > Key: HADOOP-18565 > URL: https://issues.apache.org/jira/browse/HADOOP-18565 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Priority: Major > Labels: pull-request-available > > The following work remains to complete the SDK upgrade work: > * S3A allows users configure to custom signers, add in support for this. > * Remove SDK V1 bundle dependency > * Update `getRegion()` logic to use retries. > * Add in progress listeners for `S3ABlockOutputStream` > * Fix any failing tests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5421: HADOOP-18565. Completes outstanding items for the SDK V2 upgrade.
steveloughran commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1490348322 w.r.t merging, add the amazon.cn check so it doesn't get forgotten about. then we should be good to mergel -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18565) AWS SDK V2 - Complete outstanding items
[ https://issues.apache.org/jira/browse/HADOOP-18565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706897#comment-17706897 ] ASF GitHub Bot commented on HADOOP-18565: - steveloughran commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1490346831 i don't know about those third party stores; someone in your sdk team probably knows better there. I think generally it is just account + secret, unless something like kerberos/active directory is used > AWS SDK V2 - Complete outstanding items > --- > > Key: HADOOP-18565 > URL: https://issues.apache.org/jira/browse/HADOOP-18565 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Ahmar Suhail >Priority: Major > Labels: pull-request-available > > The following work remains to complete the SDK upgrade work: > * S3A allows users configure to custom signers, add in support for this. > * Remove SDK V1 bundle dependency > * Update `getRegion()` logic to use retries. > * Add in progress listeners for `S3ABlockOutputStream` > * Fix any failing tests. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5421: HADOOP-18565. Completes outstanding items for the SDK V2 upgrade.
steveloughran commented on PR #5421: URL: https://github.com/apache/hadoop/pull/5421#issuecomment-1490346831 i don't know about those third party stores; someone in your sdk team probably knows better there. I think generally it is just account + secret, unless something like kerberos/active directory is used -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on pull request #5491: HDFS-16951. Add description of GETSERVERDEFAULTS to WebHDFS doc
slfan1989 commented on PR #5491: URL: https://github.com/apache/hadoop/pull/5491#issuecomment-1490331247 LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18647) x-ms-client-request-id to have some way that identifies retry of an API.
[ https://issues.apache.org/jira/browse/HADOOP-18647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18647: Fix Version/s: 3.3.9 > x-ms-client-request-id to have some way that identifies retry of an API. > > > Key: HADOOP-18647 > URL: https://issues.apache.org/jira/browse/HADOOP-18647 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.5 >Reporter: Pranav Saxena >Assignee: Pranav Saxena >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0, 3.3.9 > > > In case primaryRequestId in x-ms-client-request-id is empty-string, the > retry's primaryRequestId has to contain last part of clientRequestId UUID. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18647) x-ms-client-request-id to have some way that identifies retry of an API.
[ https://issues.apache.org/jira/browse/HADOOP-18647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706893#comment-17706893 ] ASF GitHub Bot commented on HADOOP-18647: - steveloughran commented on PR #5498: URL: https://github.com/apache/hadoop/pull/5498#issuecomment-1490303264 +1, merged > x-ms-client-request-id to have some way that identifies retry of an API. > > > Key: HADOOP-18647 > URL: https://issues.apache.org/jira/browse/HADOOP-18647 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.5 >Reporter: Pranav Saxena >Assignee: Pranav Saxena >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > In case primaryRequestId in x-ms-client-request-id is empty-string, the > retry's primaryRequestId has to contain last part of clientRequestId UUID. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18647) x-ms-client-request-id to have some way that identifies retry of an API.
[ https://issues.apache.org/jira/browse/HADOOP-18647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706892#comment-17706892 ] ASF GitHub Bot commented on HADOOP-18647: - steveloughran merged PR #5498: URL: https://github.com/apache/hadoop/pull/5498 > x-ms-client-request-id to have some way that identifies retry of an API. > > > Key: HADOOP-18647 > URL: https://issues.apache.org/jira/browse/HADOOP-18647 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.5 >Reporter: Pranav Saxena >Assignee: Pranav Saxena >Priority: Minor > Labels: pull-request-available > Fix For: 3.4.0 > > > In case primaryRequestId in x-ms-client-request-id is empty-string, the > retry's primaryRequestId has to contain last part of clientRequestId UUID. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5498: HADOOP-18647. x-ms-client-request-id to have some way that identifies retry of an API. (#5437)
steveloughran commented on PR #5498: URL: https://github.com/apache/hadoop/pull/5498#issuecomment-1490303264 +1, merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran merged pull request #5498: HADOOP-18647. x-ms-client-request-id to have some way that identifies retry of an API. (#5437)
steveloughran merged PR #5498: URL: https://github.com/apache/hadoop/pull/5498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency
[
https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706887#comment-17706887
]
ASF GitHub Bot commented on HADOOP-18012:
-
steveloughran commented on PR #5488:
URL: https://github.com/apache/hadoop/pull/5488#issuecomment-1490292881
+1 pending the merge and checkstyle issues
```
./hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientRenameResult.java:64:
return "AbfsClientRenameResult{" +:38: '+' should be on a new line.
[OperatorWrap]
./hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientRenameResult.java:65:
"op=" + op +:20: '+' should be on a new line. [OperatorWrap]
./hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientRenameResult.java:66:
", renameRecovered=" + renameRecovered +:48: '+' should be on a new
line. [OperatorWrap]
./hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientRenameResult.java:67:
", isIncompleteMetadataState=" + isIncompleteMetadataState +:68: '+'
should be on a new line. [OperatorWrap]
./hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java:343:
switch(client.getAuthType()) {: switch without "default" clause.
[MissingSwitchDefault]
./hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemDelegationSAS.java:29:import
org.apache.hadoop.fs.azurebfs.utils.TracingContext;:8: Unused import -
org.apache.hadoop.fs.azurebfs.utils.TracingContext. [UnusedImports]
```
can you change this from draft to "ready for review" so I can merge this
once you've done those
> ABFS: Enable config controlled ETag check for Rename idempotency
>
>
> Key: HADOOP-18012
> URL: https://issues.apache.org/jira/browse/HADOOP-18012
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.2
>Reporter: Sneha Vijayarajan
>Assignee: Sree Bhattacharyya
>Priority: Major
> Labels: pull-request-available
>
> ABFS driver has a handling for rename idempotency which relies on LMT of the
> destination file to conclude if the rename was successful or not when source
> file is absent and if the rename request had entered retry loop.
> This handling is incorrect as LMT of the destination does not change on
> rename.
> This Jira will track the change to undo the current implementation and add a
> new one where for an incoming rename operation, source file eTag is fetched
> first and then rename is done only if eTag matches for the source file.
> As this is going to be a costly operation given an extra HEAD request is
> added to each rename, this implementation will be guarded over a config and
> can enabled by customers who have workloads that do multiple renames.
> Long term plan to handle rename idempotency without HEAD request is being
> discussed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks
steveloughran commented on PR #5488:
URL: https://github.com/apache/hadoop/pull/5488#issuecomment-1490292881
+1 pending the merge and checkstyle issues
```
./hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientRenameResult.java:64:
return "AbfsClientRenameResult{" +:38: '+' should be on a new line.
[OperatorWrap]
./hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientRenameResult.java:65:
"op=" + op +:20: '+' should be on a new line. [OperatorWrap]
./hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientRenameResult.java:66:
", renameRecovered=" + renameRecovered +:48: '+' should be on a new
line. [OperatorWrap]
./hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClientRenameResult.java:67:
", isIncompleteMetadataState=" + isIncompleteMetadataState +:68: '+'
should be on a new line. [OperatorWrap]
./hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java:343:
switch(client.getAuthType()) {: switch without "default" clause.
[MissingSwitchDefault]
./hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemDelegationSAS.java:29:import
org.apache.hadoop.fs.azurebfs.utils.TracingContext;:8: Unused import -
org.apache.hadoop.fs.azurebfs.utils.TracingContext. [UnusedImports]
```
can you change this from draft to "ready for review" so I can merge this
once you've done those
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18012) ABFS: Enable config controlled ETag check for Rename idempotency
[
https://issues.apache.org/jira/browse/HADOOP-18012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706885#comment-17706885
]
ASF GitHub Bot commented on HADOOP-18012:
-
steveloughran commented on code in PR #5488:
URL: https://github.com/apache/hadoop/pull/5488#discussion_r1153240354
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##
@@ -503,27 +512,50 @@ public AbfsRestOperation breakLease(final String path,
* took place.
* As rename recovery is only attempted if the source etag is non-empty,
* in normal rename operations rename recovery will never happen.
- * @param source path to source file
- * @param destination destination of rename.
- * @param continuation continuation.
- * @param tracingContext trace context
- * @param sourceEtag etag of source file. may be null or empty
+ *
+ * @param sourcepath to source file
+ * @param destination destination of rename.
+ * @param continuation continuation.
+ * @param tracingContexttrace context
+ * @param sourceEtagetag of source file. may be null or empty
* @param isMetadataIncompleteState was there a rename failure due to
* incomplete metadata state?
* @return AbfsClientRenameResult result of rename operation indicating the
* AbfsRest operation, rename recovery and incomplete metadata state failure.
* @throws AzureBlobFileSystemException failure, excluding any recovery from
overload failures.
*/
public AbfsClientRenameResult renamePath(
- final String source,
- final String destination,
- final String continuation,
- final TracingContext tracingContext,
- final String sourceEtag,
- boolean isMetadataIncompleteState)
- throws AzureBlobFileSystemException {
+ final String source,
+ final String destination,
+ final String continuation,
+ final TracingContext tracingContext,
+ String sourceEtag,
+ boolean isMetadataIncompleteState,
+ boolean isNamespaceEnabled)
Review Comment:
@saxenapranav got to disagree there; having it as a field gives a lifecycle
of
1. create client not knowing state
2. use client to work out state
3. update client
4. invoke calls which require this knowledge to work properly (which is
rename)
passing it in as the param to rename() makes clear that it must be known at
the time of invocation
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##
@@ -578,26 +613,49 @@ public AbfsClientRenameResult renamePath(
// Extract the sourceEtag, using the status Op, and set it
// for future rename recovery.
AbfsHttpOperation sourceStatusResult = sourceStatusOp.getResult();
- String sourceEtagAfterFailure =
extractEtagHeader(sourceStatusResult);
- renamePath(source, destination, continuation, tracingContext,
- sourceEtagAfterFailure, isMetadataIncompleteState);
-}
-// if we get out of the condition without a successful rename, then
-// it isn't metadata incomplete state issue.
-isMetadataIncompleteState = false;
-
-boolean etagCheckSucceeded = renameIdempotencyCheckOp(
-source,
-sourceEtag, op, destination, tracingContext);
-if (!etagCheckSucceeded) {
- // idempotency did not return different result
- // throw back the exception
- throw e;
+ sourceEtagAfterFailure = extractEtagHeader(sourceStatusResult);
Review Comment:
good point. either it recovers or it doesn't, we may as well try at this
point
> ABFS: Enable config controlled ETag check for Rename idempotency
>
>
> Key: HADOOP-18012
> URL: https://issues.apache.org/jira/browse/HADOOP-18012
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.2
>Reporter: Sneha Vijayarajan
>Assignee: Sree Bhattacharyya
>Priority: Major
> Labels: pull-request-available
>
> ABFS driver has a handling for rename idempotency which relies on LMT of the
> destination file to conclude if the rename was successful or not when source
> file is absent and if the rename request had entered retry loop.
> This handling is incorrect as LMT of the destination does not change on
> rename.
> This Jira will track the change to undo the current implementation and add a
> new one where for an incoming rename operation, source file eTag is fetched
> first and then rename is done only if eTag matches for the source file.
>
[GitHub] [hadoop] hadoop-yetus commented on pull request #5420: YARN-11442. Refactor FederationInterceptorREST Code.
hadoop-yetus commented on PR #5420:
URL: https://github.com/apache/hadoop/pull/5420#issuecomment-1490289178
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 44s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 27s | | trunk passed |
| +1 :green_heart: | compile | 4m 20s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 3m 31s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 15s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 16s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 10s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 59s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 32s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 57s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 16s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 0m 59s | | the patch passed |
| +1 :green_heart: | compile | 3m 59s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 3m 59s | | the patch passed |
| +1 :green_heart: | compile | 3m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 3m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 7s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 6s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 31s | | hadoop-yarn-server-common in
the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1. |
| +1 :green_heart: | javadoc | 0m 24s | |
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 generated 0 new + 1
unchanged - 4 fixed = 1 total (was 5) |
| +1 :green_heart: | javadoc | 0m 29s | | hadoop-yarn-server-common in
the patch passed with JDK Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09. |
| +1 :green_heart: | javadoc | 0m 21s | |
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09
with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 generated 0 new
+ 1 unchanged - 4 fixed = 1 total (was 5) |
| +1 :green_heart: | spotbugs | 2m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 23s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 12s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 0m 31s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 122m 17s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5420/19/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5420 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d0e0fd1fdfd5 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision
[GitHub] [hadoop] steveloughran commented on a diff in pull request #5488: HADOOP-18012. ABFS: Using Source Path eTags for Rename Idemptonency checks
steveloughran commented on code in PR #5488:
URL: https://github.com/apache/hadoop/pull/5488#discussion_r1153240354
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##
@@ -503,27 +512,50 @@ public AbfsRestOperation breakLease(final String path,
* took place.
* As rename recovery is only attempted if the source etag is non-empty,
* in normal rename operations rename recovery will never happen.
- * @param source path to source file
- * @param destination destination of rename.
- * @param continuation continuation.
- * @param tracingContext trace context
- * @param sourceEtag etag of source file. may be null or empty
+ *
+ * @param sourcepath to source file
+ * @param destination destination of rename.
+ * @param continuation continuation.
+ * @param tracingContexttrace context
+ * @param sourceEtagetag of source file. may be null or empty
* @param isMetadataIncompleteState was there a rename failure due to
* incomplete metadata state?
* @return AbfsClientRenameResult result of rename operation indicating the
* AbfsRest operation, rename recovery and incomplete metadata state failure.
* @throws AzureBlobFileSystemException failure, excluding any recovery from
overload failures.
*/
public AbfsClientRenameResult renamePath(
- final String source,
- final String destination,
- final String continuation,
- final TracingContext tracingContext,
- final String sourceEtag,
- boolean isMetadataIncompleteState)
- throws AzureBlobFileSystemException {
+ final String source,
+ final String destination,
+ final String continuation,
+ final TracingContext tracingContext,
+ String sourceEtag,
+ boolean isMetadataIncompleteState,
+ boolean isNamespaceEnabled)
Review Comment:
@saxenapranav got to disagree there; having it as a field gives a lifecycle
of
1. create client not knowing state
2. use client to work out state
3. update client
4. invoke calls which require this knowledge to work properly (which is
rename)
passing it in as the param to rename() makes clear that it must be known at
the time of invocation
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsClient.java:
##
@@ -578,26 +613,49 @@ public AbfsClientRenameResult renamePath(
// Extract the sourceEtag, using the status Op, and set it
// for future rename recovery.
AbfsHttpOperation sourceStatusResult = sourceStatusOp.getResult();
- String sourceEtagAfterFailure =
extractEtagHeader(sourceStatusResult);
- renamePath(source, destination, continuation, tracingContext,
- sourceEtagAfterFailure, isMetadataIncompleteState);
-}
-// if we get out of the condition without a successful rename, then
-// it isn't metadata incomplete state issue.
-isMetadataIncompleteState = false;
-
-boolean etagCheckSucceeded = renameIdempotencyCheckOp(
-source,
-sourceEtag, op, destination, tracingContext);
-if (!etagCheckSucceeded) {
- // idempotency did not return different result
- // throw back the exception
- throw e;
+ sourceEtagAfterFailure = extractEtagHeader(sourceStatusResult);
Review Comment:
good point. either it recovers or it doesn't, we may as well try at this
point
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18685) Amazon S3 disabling ACLs on all new buckets
[ https://issues.apache.org/jira/browse/HADOOP-18685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Carl Jones updated HADOOP-18685: --- Description: In April 2023, Amazon S3 will be disabling ACLs by default on *all* new buckets. [https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/.|https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/] Note, buckets created using the AWS Console are already created with ACLs disabled by default. In S3A, we have tests that rely on ACLs being enabled. We should either update test documentation to tell developers to enable ACLs on buckets, or we should make the ACL tests opt-in if they are used infrequently by S3A users. was: In April 2023, Amazon S3 will be disabling ACLs by default on all new buckets. [https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/] In S3A, we have tests that rely on ACLs being enabled. We should either update test documentation to tell developers to enable ACLs on buckets, or we should make the ACL tests opt-in if they are used infrequently by S3A users. > Amazon S3 disabling ACLs on all new buckets > --- > > Key: HADOOP-18685 > URL: https://issues.apache.org/jira/browse/HADOOP-18685 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Daniel Carl Jones >Priority: Minor > > In April 2023, Amazon S3 will be disabling ACLs by default on *all* new > buckets. > [https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/.|https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/] > Note, buckets created using the AWS Console are already created with ACLs > disabled by default. > In S3A, we have tests that rely on ACLs being enabled. We should either > update test documentation to tell developers to enable ACLs on buckets, or we > should make the ACL tests opt-in if they are used infrequently by S3A users. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18685) Amazon S3 disabling ACLs on all new buckets
[ https://issues.apache.org/jira/browse/HADOOP-18685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706883#comment-17706883 ] Daniel Carl Jones commented on HADOOP-18685: This would already be tripping up developers, as AWS Console has been defaulting to disabled ACLs for some time now. > Amazon S3 disabling ACLs on all new buckets > --- > > Key: HADOOP-18685 > URL: https://issues.apache.org/jira/browse/HADOOP-18685 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Daniel Carl Jones >Priority: Minor > > In April 2023, Amazon S3 will be disabling ACLs by default on *all* new > buckets. > [https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/.|https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/] > Note, buckets created using the AWS Console are already created with ACLs > disabled by default. > In S3A, we have tests that rely on ACLs being enabled. We should either > update test documentation to tell developers to enable ACLs on buckets, or we > should make the ACL tests opt-in if they are used infrequently by S3A users. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706870#comment-17706870 ] ASF GitHub Bot commented on HADOOP-18197: - steveloughran commented on PR #4418: URL: https://github.com/apache/hadoop/pull/4418#issuecomment-1490261364 @xizhu-mstr @tooptoop4 i'm not actively working on this; too many pressing issues and after getting 3.3.5 out the door I'm catching up with the internal stuff. Either of you two want to take it on? I'd also like to get #4996 in; if anyone wants to run with that, I'd be very happy. We shouldn't need protobuf 2.5 on the CP given we aren't using it > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #4418: HADOOP-18197. Upgrade protobuf to 3.21.7 (through upgraded hadoop-shaded-protobuf jar)
steveloughran commented on PR #4418: URL: https://github.com/apache/hadoop/pull/4418#issuecomment-1490261364 @xizhu-mstr @tooptoop4 i'm not actively working on this; too many pressing issues and after getting 3.3.5 out the door I'm catching up with the internal stuff. Either of you two want to take it on? I'd also like to get #4996 in; if anyone wants to run with that, I'd be very happy. We shouldn't need protobuf 2.5 on the CP given we aren't using it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706867#comment-17706867 ] ASF GitHub Bot commented on HADOOP-18197: - steveloughran commented on code in PR #4418: URL: https://github.com/apache/hadoop/pull/4418#discussion_r1153223024 ## BUILDING.txt: ## @@ -403,10 +403,10 @@ Installing required dependencies for clean install of macOS 10.14: * Install native libraries, only openssl is required to compile native code, you may optionally install zlib, lz4, etc. $ brew install openssl -* Protocol Buffers 3.7.1 (required to compile native code) - $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz - $ mkdir -p protobuf-3.7 && tar zxvf protobuf-java-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7 - $ cd protobuf-3.7 +* Protocol Buffers 3.21.1 (required to compile native code) + $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.21.1/protobuf-java-3.21.1.tar.gz + $ mkdir -p protobuf-3.21 && tar zxvf protobuf-java-3.21.1.tar.gz --strip-components 1 -C protobuf-3.21 + $ cd protobuf-3.721 Review Comment: yeah > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a diff in pull request #4418: HADOOP-18197. Upgrade protobuf to 3.21.7 (through upgraded hadoop-shaded-protobuf jar)
steveloughran commented on code in PR #4418: URL: https://github.com/apache/hadoop/pull/4418#discussion_r1153223024 ## BUILDING.txt: ## @@ -403,10 +403,10 @@ Installing required dependencies for clean install of macOS 10.14: * Install native libraries, only openssl is required to compile native code, you may optionally install zlib, lz4, etc. $ brew install openssl -* Protocol Buffers 3.7.1 (required to compile native code) - $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz - $ mkdir -p protobuf-3.7 && tar zxvf protobuf-java-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7 - $ cd protobuf-3.7 +* Protocol Buffers 3.21.1 (required to compile native code) + $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.21.1/protobuf-java-3.21.1.tar.gz + $ mkdir -p protobuf-3.21 && tar zxvf protobuf-java-3.21.1.tar.gz --strip-components 1 -C protobuf-3.21 + $ cd protobuf-3.721 Review Comment: yeah -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18685) Amazon S3 changing ACL defaults
Daniel Carl Jones created HADOOP-18685: -- Summary: Amazon S3 changing ACL defaults Key: HADOOP-18685 URL: https://issues.apache.org/jira/browse/HADOOP-18685 Project: Hadoop Common Issue Type: Sub-task Components: fs/s3 Reporter: Daniel Carl Jones In April 2023, Amazon S3 will be disabling ACLs by default on all new buckets. [https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/] In S3A, we have tests that rely on ACLs being enabled. We should either update test documentation to tell developers to enable ACLs on buckets, or we should make the ACL tests opt-in if they are used infrequently by S3A users. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18685) Amazon S3 disabling ACLs on all new buckets
[ https://issues.apache.org/jira/browse/HADOOP-18685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Carl Jones updated HADOOP-18685: --- Summary: Amazon S3 disabling ACLs on all new buckets (was: Amazon S3 changing ACL defaults) > Amazon S3 disabling ACLs on all new buckets > --- > > Key: HADOOP-18685 > URL: https://issues.apache.org/jira/browse/HADOOP-18685 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Daniel Carl Jones >Priority: Minor > > In April 2023, Amazon S3 will be disabling ACLs by default on all new > buckets. > [https://aws.amazon.com/blogs/aws/heads-up-amazon-s3-security-changes-are-coming-in-april-of-2023/] > > In S3A, we have tests that rely on ACLs being enabled. We should either > update test documentation to tell developers to enable ACLs on buckets, or we > should make the ACL tests opt-in if they are used infrequently by S3A users. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…
hadoop-yetus commented on PR #5495:
URL: https://github.com/apache/hadoop/pull/5495#issuecomment-1490184577
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 30s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 33m 10s | | trunk passed |
| +1 :green_heart: | compile | 31m 3s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 24m 17s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 4m 11s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 25s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 27m 10s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 24s | | the patch passed |
| +1 :green_heart: | compile | 29m 53s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 29m 53s | | the patch passed |
| +1 :green_heart: | compile | 24m 35s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 24m 35s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/12/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 4m 2s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/12/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 1 new + 155 unchanged - 0 fixed = 156 total (was
155) |
| +1 :green_heart: | mvnsite | 3m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 16s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 45s | | the patch passed |
| -1 :x: | shadedclient | 27m 16s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 19m 29s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/12/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| -1 :x: | unit | 253m 2s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/12/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 14s | | The patch does not
generate ASF License warnings. |
| | | 528m 41s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.metrics2.lib.TestMutableMetrics |
| | hadoop.ipc.TestRPC |
| | hadoop.security.TestUserGroupInformation |
| | hadoop.metrics2.lib.TestMetricsRegistry |
| | hadoop.util.TestReadWriteDiskValidator |
| | hadoop.hdfs.qjournal.server.TestJournalNodeMXBean |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | hadoop.hdfs.TestDFSInotifyEventInputStream |
| | hadoop.hdfs.server.namenode.ha.TestStandbyInProgressTail |
| | hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes |
| | hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy |
| | hadoop.hdfs.qjournal.client.Te
[GitHub] [hadoop] hadoop-yetus commented on pull request #5520: HDFS-16965. Add switch to decide whether to enable native codec.
hadoop-yetus commented on PR #5520:
URL: https://github.com/apache/hadoop/pull/5520#issuecomment-1490165934
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 19s | | trunk passed |
| +1 :green_heart: | compile | 26m 47s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 22m 49s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 49s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 15s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 48s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 8s | | the patch passed |
| +1 :green_heart: | compile | 26m 7s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 26m 7s | | the patch passed |
| +1 :green_heart: | compile | 21m 6s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 21m 6s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5520/1/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 1m 6s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5520/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 3 new + 2
unchanged - 0 fixed = 5 total (was 2) |
| +1 :green_heart: | mvnsite | 1m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 5s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 38s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 15s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 1s | | The patch does not
generate ASF License warnings. |
| | | 222m 39s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5520/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5520 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 104e8fd7be1f 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / efc9a6fdc8de9fa8b9e75a96f1e748e2178fcde2 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5520/1/testReport/ |
| Max. process+thread count | 2778 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-proj
[GitHub] [hadoop] hadoop-yetus commented on pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…
hadoop-yetus commented on PR #5495:
URL: https://github.com/apache/hadoop/pull/5495#issuecomment-1490153112
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 29s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 28s | | trunk passed |
| +1 :green_heart: | compile | 29m 32s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 25m 59s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 4m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 16s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 33s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 49s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 36s | | the patch passed |
| +1 :green_heart: | compile | 29m 25s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 29m 25s | | the patch passed |
| +1 :green_heart: | compile | 26m 17s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 26m 17s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/11/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 4m 4s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/11/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 155 unchanged - 0 fixed = 157 total (was
155) |
| +1 :green_heart: | mvnsite | 3m 29s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 8s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 7m 2s | | the patch passed |
| -1 :x: | shadedclient | 27m 22s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 18m 59s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/11/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| -1 :x: | unit | 237m 29s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/11/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 23s | | The patch does not
generate ASF License warnings. |
| | | 509m 44s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.metrics2.lib.TestMutableMetrics |
| | hadoop.ipc.TestRPC |
| | hadoop.security.TestUserGroupInformation |
| | hadoop.metrics2.lib.TestMetricsRegistry |
| | hadoop.util.TestReadWriteDiskValidator |
| | hadoop.hdfs.qjournal.server.TestJournalNodeMXBean |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | hadoop.hdfs.TestDFSInotifyEventInputStream |
| | hadoop.hdfs.server.namenode.ha.TestStandbyInProgressTail |
| | hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes |
| | hadoop.hdfs.qjournal.client.TestQuorumJournalManager |
| | hadoop.hdfs.server.datanode.fsdataset.impl.
[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[
https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706843#comment-17706843
]
ASF GitHub Bot commented on HADOOP-18684:
-
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490137673
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 44s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 31s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 32s | | hadoop-aws in the patch passed.
|
| -1 :x: | asflicense | 0m 37s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/1/artifact/out/results-asflicense.txt)
| The patch generated 1 ASF License warnings. |
| | | 94m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 7a4703377f7b 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 15a1551f1ee368e6fb0bca6e744cc22b34162449 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/1/testReport/ |
| Max. process+thread count | 692 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 h
[GitHub] [hadoop] hadoop-yetus commented on pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme
hadoop-yetus commented on PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#issuecomment-1490137673
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 44s | | trunk passed |
| +1 :green_heart: | compile | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 31s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 1m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 32s | | hadoop-aws in the patch passed.
|
| -1 :x: | asflicense | 0m 37s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/1/artifact/out/results-asflicense.txt)
| The patch generated 1 ASF License warnings. |
| | | 94m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5521 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 7a4703377f7b 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 15a1551f1ee368e6fb0bca6e744cc22b34162449 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/1/testReport/ |
| Max. process+thread count | 692 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5521/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To un
[GitHub] [hadoop] hadoop-yetus commented on pull request #5420: YARN-11442. Refactor FederationInterceptorREST Code.
hadoop-yetus commented on PR #5420:
URL: https://github.com/apache/hadoop/pull/5420#issuecomment-1490106434
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 36s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 14s | | trunk passed |
| +1 :green_heart: | compile | 4m 14s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 3m 33s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 17s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 16s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 11s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 2m 31s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 8s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 27s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 4s | | the patch passed |
| +1 :green_heart: | compile | 4m 15s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 4m 15s | | the patch passed |
| +1 :green_heart: | compile | 3m 30s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 3m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 5s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 5s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 30s | | hadoop-yarn-server-common in
the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1. |
| +1 :green_heart: | javadoc | 0m 23s | |
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 generated 0 new + 1
unchanged - 4 fixed = 1 total (was 5) |
| +1 :green_heart: | javadoc | 0m 29s | | hadoop-yarn-server-common in
the patch passed with JDK Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09. |
| +1 :green_heart: | javadoc | 0m 23s | |
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-router-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09
with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 generated 0 new
+ 1 unchanged - 4 fixed = 1 total (was 5) |
| +1 :green_heart: | spotbugs | 2m 34s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 24s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 13s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 0m 30s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 123m 33s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5420/18/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5420 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 970adf175a2e 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision
[GitHub] [hadoop] zhtttylz commented on pull request #5517: HDFS-16952. Support getLinkTarget API in WebHDFS
zhtttylz commented on PR #5517: URL: https://github.com/apache/hadoop/pull/5517#issuecomment-1490101718 @slfan1989 I've updated the PR based on your suggestion, could you review this please. thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5517: HDFS-16952. Support getLinkTarget API in WebHDFS
hadoop-yetus commented on PR #5517:
URL: https://github.com/apache/hadoop/pull/5517#issuecomment-1490079978
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 48s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 24s | | trunk passed |
| +1 :green_heart: | compile | 5m 56s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 5m 47s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 11s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 29s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 3m 15s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 7m 10s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 52s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 40s | | the patch passed |
| +1 :green_heart: | compile | 5m 51s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 5m 51s | | the patch passed |
| +1 :green_heart: | compile | 5m 32s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 5m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 5s |
[/results-checkstyle-hadoop-hdfs-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5517/5/artifact/out/results-checkstyle-hadoop-hdfs-project.txt)
| hadoop-hdfs-project: The patch generated 10 new + 146 unchanged - 0 fixed =
156 total (was 146) |
| +1 :green_heart: | mvnsite | 2m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 59s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 56s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 7m 15s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 45s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 28s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 206m 3s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | unit | 21m 3s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 44s | | The patch does not
generate ASF License warnings. |
| | | 376m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.42 ServerAPI=1.42 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5517/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5517 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 6f1ae6ee5f7f 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 2a28640c03a1ebaa5b90d355bd4a8f353be0a439 |
| Default Java | Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5517/5/testReport/ |
| Max. pr
[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[
https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706817#comment-17706817
]
ASF GitHub Bot commented on HADOOP-18684:
-
steveloughran commented on code in PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#discussion_r1153027000
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java:
##
@@ -0,0 +1,31 @@
+package org.apache.hadoop.fs.s3a;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.junit.Test;
+
+import java.io.IOException;
+
+public class ITestS3AUrlScheme extends AbstractS3ATestBase{
+
+ @Override
+ public void setup() throws Exception {
+super.setup();
+ }
+
+ @Override
+ protected Configuration createConfiguration() {
+Configuration conf = super.createConfiguration();
+conf.set("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem");
Review Comment:
this is the default. why are you setting it again?
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java:
##
@@ -0,0 +1,31 @@
+package org.apache.hadoop.fs.s3a;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.junit.Test;
+
+import java.io.IOException;
+
+public class ITestS3AUrlScheme extends AbstractS3ATestBase{
+
+ @Override
Review Comment:
not needed
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java:
##
@@ -0,0 +1,31 @@
+package org.apache.hadoop.fs.s3a;
Review Comment:
yetus will reject this without the asf copyright header
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java:
##
@@ -0,0 +1,31 @@
+package org.apache.hadoop.fs.s3a;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.junit.Test;
+
+import java.io.IOException;
+
+public class ITestS3AUrlScheme extends AbstractS3ATestBase{
+
+ @Override
+ public void setup() throws Exception {
+super.setup();
+ }
+
+ @Override
+ protected Configuration createConfiguration() {
+Configuration conf = super.createConfiguration();
+conf.set("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem");
+return conf;
+ }
+
+ @Test
+ public void testFSScheme() throws IOException {
+FileSystem fs = getFileSystem();
+assertEquals(fs.getScheme(), "s3a");
Review Comment:
1. junit assert equals has the order (expected, actual) for the error
messages to work
2. use AssertJ.assertThat() for new tests please -it's far more powerful.
while it takes a while to learn, smaller tests are the place to do it.
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java:
##
@@ -0,0 +1,31 @@
+package org.apache.hadoop.fs.s3a;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.junit.Test;
+
+import java.io.IOException;
Review Comment:
check your import rules. they MUST be
```
java.*
java.*
--
not-asf.*
--
org.apace.*
--
statics
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3A.java:
##
@@ -37,7 +37,8 @@ public class S3A extends DelegateToFileSystem {
public S3A(URI theUri, Configuration conf)
throws IOException, URISyntaxException {
-super(theUri, new S3AFileSystem(), conf, "s3a", false);
+super(theUri, new S3AFileSystem(), conf,
+theUri.getScheme().isEmpty() ? "s3a" : theUri.getScheme(), false);
Review Comment:
move this to a constant in `org.apache.hadoop.fs.s3a.impl.InternalConstants`
and then you can reference it in tests.
> Fix S3A filesystem such that the scheme matches the URI scheme
> --
>
> Key: HADOOP-18684
> URL: https://issues.apache.org/jira/browse/HADOOP-18684
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 3.3.5
>Reporter: Harshit Gupta
>Priority: Major
> Labels: pull-request-available
>
> Certain codepaths use the FileContext API's to perform FS based operations
> such as yarn log aggregations. While trying to reuse the S3A connector for
> GCS based workloads the yarn log aggregation was not happening. Upon further
> investigation it was observed that FileContext API have hardcoded URI scheme
> checks that need to disabled/updated to make S3A compatible with non AWS
> stores.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsu
[GitHub] [hadoop] steveloughran commented on a diff in pull request #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme
steveloughran commented on code in PR #5521:
URL: https://github.com/apache/hadoop/pull/5521#discussion_r1153027000
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java:
##
@@ -0,0 +1,31 @@
+package org.apache.hadoop.fs.s3a;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.junit.Test;
+
+import java.io.IOException;
+
+public class ITestS3AUrlScheme extends AbstractS3ATestBase{
+
+ @Override
+ public void setup() throws Exception {
+super.setup();
+ }
+
+ @Override
+ protected Configuration createConfiguration() {
+Configuration conf = super.createConfiguration();
+conf.set("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem");
Review Comment:
this is the default. why are you setting it again?
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java:
##
@@ -0,0 +1,31 @@
+package org.apache.hadoop.fs.s3a;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.junit.Test;
+
+import java.io.IOException;
+
+public class ITestS3AUrlScheme extends AbstractS3ATestBase{
+
+ @Override
Review Comment:
not needed
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java:
##
@@ -0,0 +1,31 @@
+package org.apache.hadoop.fs.s3a;
Review Comment:
yetus will reject this without the asf copyright header
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java:
##
@@ -0,0 +1,31 @@
+package org.apache.hadoop.fs.s3a;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.junit.Test;
+
+import java.io.IOException;
+
+public class ITestS3AUrlScheme extends AbstractS3ATestBase{
+
+ @Override
+ public void setup() throws Exception {
+super.setup();
+ }
+
+ @Override
+ protected Configuration createConfiguration() {
+Configuration conf = super.createConfiguration();
+conf.set("fs.s3a.impl", "org.apache.hadoop.fs.s3a.S3AFileSystem");
+return conf;
+ }
+
+ @Test
+ public void testFSScheme() throws IOException {
+FileSystem fs = getFileSystem();
+assertEquals(fs.getScheme(), "s3a");
Review Comment:
1. junit assert equals has the order (expected, actual) for the error
messages to work
2. use AssertJ.assertThat() for new tests please -it's far more powerful.
while it takes a while to learn, smaller tests are the place to do it.
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AUrlScheme.java:
##
@@ -0,0 +1,31 @@
+package org.apache.hadoop.fs.s3a;
+
+import org.apache.hadoop.conf.Configuration;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.junit.Test;
+
+import java.io.IOException;
Review Comment:
check your import rules. they MUST be
```
java.*
java.*
--
not-asf.*
--
org.apace.*
--
statics
##
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3A.java:
##
@@ -37,7 +37,8 @@ public class S3A extends DelegateToFileSystem {
public S3A(URI theUri, Configuration conf)
throws IOException, URISyntaxException {
-super(theUri, new S3AFileSystem(), conf, "s3a", false);
+super(theUri, new S3AFileSystem(), conf,
+theUri.getScheme().isEmpty() ? "s3a" : theUri.getScheme(), false);
Review Comment:
move this to a constant in `org.apache.hadoop.fs.s3a.impl.InternalConstants`
and then you can reference it in tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[ https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harshit Gupta updated HADOOP-18684: --- Affects Version/s: 3.3.5 > Fix S3A filesystem such that the scheme matches the URI scheme > -- > > Key: HADOOP-18684 > URL: https://issues.apache.org/jira/browse/HADOOP-18684 > Project: Hadoop Common > Issue Type: Improvement >Affects Versions: 3.3.5 >Reporter: Harshit Gupta >Priority: Major > Labels: pull-request-available > > Certain codepaths use the FileContext API's to perform FS based operations > such as yarn log aggregations. While trying to reuse the S3A connector for > GCS based workloads the yarn log aggregation was not happening. Upon further > investigation it was observed that FileContext API have hardcoded URI scheme > checks that need to disabled/updated to make S3A compatible with non AWS > stores. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[ https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706809#comment-17706809 ] ASF GitHub Bot commented on HADOOP-18684: - HarshitGupta11 opened a new pull request, #5521: URL: https://github.com/apache/hadoop/pull/5521 ### Description of PR Certain codepaths use the FileContext API's to perform FS based operations such as yarn log aggregations. While trying to reuse the S3A connector for GCS based workloads the yarn log aggregation was not happening. Upon further investigation it was observed that FileContext API have hardcoded URI scheme checks that need to disabled/updated to make S3A compatible with non AWS stores. ### How was this patch tested? The patch was tested against us-west-2 ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [x] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Fix S3A filesystem such that the scheme matches the URI scheme > -- > > Key: HADOOP-18684 > URL: https://issues.apache.org/jira/browse/HADOOP-18684 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Harshit Gupta >Priority: Major > > Certain codepaths use the FileContext API's to perform FS based operations > such as yarn log aggregations. While trying to reuse the S3A connector for > GCS based workloads the yarn log aggregation was not happening. Upon further > investigation it was observed that FileContext API have hardcoded URI scheme > checks that need to disabled/updated to make S3A compatible with non AWS > stores. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18684) Fix S3A filesystem such that the scheme matches the URI scheme
[ https://issues.apache.org/jira/browse/HADOOP-18684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18684: Labels: pull-request-available (was: ) > Fix S3A filesystem such that the scheme matches the URI scheme > -- > > Key: HADOOP-18684 > URL: https://issues.apache.org/jira/browse/HADOOP-18684 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Harshit Gupta >Priority: Major > Labels: pull-request-available > > Certain codepaths use the FileContext API's to perform FS based operations > such as yarn log aggregations. While trying to reuse the S3A connector for > GCS based workloads the yarn log aggregation was not happening. Upon further > investigation it was observed that FileContext API have hardcoded URI scheme > checks that need to disabled/updated to make S3A compatible with non AWS > stores. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] HarshitGupta11 opened a new pull request, #5521: HADOOP-18684: Fix S3A filesystem such that the scheme matches the URI scheme
HarshitGupta11 opened a new pull request, #5521: URL: https://github.com/apache/hadoop/pull/5521 ### Description of PR Certain codepaths use the FileContext API's to perform FS based operations such as yarn log aggregations. While trying to reuse the S3A connector for GCS based workloads the yarn log aggregation was not happening. Upon further investigation it was observed that FileContext API have hardcoded URI scheme checks that need to disabled/updated to make S3A compatible with non AWS stores. ### How was this patch tested? The patch was tested against us-west-2 ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [x] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5495: HDFS-16949 Introduce inverse quantiles for metrics where higher numer…
hadoop-yetus commented on PR #5495:
URL: https://github.com/apache/hadoop/pull/5495#issuecomment-1489978632
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 34s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 13s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 1s | | trunk passed |
| +1 :green_heart: | compile | 23m 5s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 20m 31s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 3m 45s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 22s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 40s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 8s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 14s | | the patch passed |
| +1 :green_heart: | compile | 22m 22s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 22m 22s | | the patch passed |
| +1 :green_heart: | compile | 20m 30s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | javac | 20m 30s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/10/artifact/out/blanks-eol.txt)
| The patch has 2 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 3m 39s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/10/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 2 new + 155 unchanged - 0 fixed = 157 total (was
155) |
| +1 :green_heart: | mvnsite | 3m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 18s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 2m 38s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 6m 17s | | the patch passed |
| -1 :x: | shadedclient | 23m 10s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 18m 5s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/10/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| -1 :x: | unit | 217m 34s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5495/10/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 13s | | The patch does not
generate ASF License warnings. |
| | | 451m 46s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.metrics2.lib.TestMetricsRegistry |
| | hadoop.util.TestReadWriteDiskValidator |
| | hadoop.metrics2.lib.TestMutableMetrics |
| | hadoop.security.TestUserGroupInformation |
| | hadoop.ipc.TestRPC |
| | hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized |
| | hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA |
| | hadoop.hdfs.TestStateAlignmentContextWithHA |
| | hadoop.hdfs.server.namenode.ha.TestUpdateBlockTailing |
| | hadoop.hdfs.server.namenode.ha.TestMultiObserverNode |
| | hadoop.hdfs.qjournal.server.TestJournal |
| | hadoop.hdfs.server.namenode.ha.TestStandbyInProgressTail |
| | hadoop.hdfs.server.namenode.ha.TestConsistentReadsObserve
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706779#comment-17706779 ] ASF GitHub Bot commented on HADOOP-18197: - xizhu-mstr commented on code in PR #4418: URL: https://github.com/apache/hadoop/pull/4418#discussion_r1152934268 ## BUILDING.txt: ## @@ -403,10 +403,10 @@ Installing required dependencies for clean install of macOS 10.14: * Install native libraries, only openssl is required to compile native code, you may optionally install zlib, lz4, etc. $ brew install openssl -* Protocol Buffers 3.7.1 (required to compile native code) - $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz - $ mkdir -p protobuf-3.7 && tar zxvf protobuf-java-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7 - $ cd protobuf-3.7 +* Protocol Buffers 3.21.1 (required to compile native code) + $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.21.1/protobuf-java-3.21.1.tar.gz + $ mkdir -p protobuf-3.21 && tar zxvf protobuf-java-3.21.1.tar.gz --strip-components 1 -C protobuf-3.21 + $ cd protobuf-3.721 Review Comment: Typo. Should be protobuf-3.21 > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xizhu-mstr commented on a diff in pull request #4418: HADOOP-18197. Upgrade protobuf to 3.21.7 (through upgraded hadoop-shaded-protobuf jar)
xizhu-mstr commented on code in PR #4418: URL: https://github.com/apache/hadoop/pull/4418#discussion_r1152934268 ## BUILDING.txt: ## @@ -403,10 +403,10 @@ Installing required dependencies for clean install of macOS 10.14: * Install native libraries, only openssl is required to compile native code, you may optionally install zlib, lz4, etc. $ brew install openssl -* Protocol Buffers 3.7.1 (required to compile native code) - $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz - $ mkdir -p protobuf-3.7 && tar zxvf protobuf-java-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7 - $ cd protobuf-3.7 +* Protocol Buffers 3.21.1 (required to compile native code) + $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.21.1/protobuf-java-3.21.1.tar.gz + $ mkdir -p protobuf-3.21 && tar zxvf protobuf-java-3.21.1.tar.gz --strip-components 1 -C protobuf-3.21 + $ cd protobuf-3.721 Review Comment: Typo. Should be protobuf-3.21 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18197) Update protobuf 3.7.1 to a version without CVE-2021-22569
[ https://issues.apache.org/jira/browse/HADOOP-18197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706777#comment-17706777 ] ASF GitHub Bot commented on HADOOP-18197: - xizhu-mstr commented on code in PR #4418: URL: https://github.com/apache/hadoop/pull/4418#discussion_r1152934268 ## BUILDING.txt: ## @@ -403,10 +403,10 @@ Installing required dependencies for clean install of macOS 10.14: * Install native libraries, only openssl is required to compile native code, you may optionally install zlib, lz4, etc. $ brew install openssl -* Protocol Buffers 3.7.1 (required to compile native code) - $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz - $ mkdir -p protobuf-3.7 && tar zxvf protobuf-java-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7 - $ cd protobuf-3.7 +* Protocol Buffers 3.21.1 (required to compile native code) + $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.21.1/protobuf-java-3.21.1.tar.gz + $ mkdir -p protobuf-3.21 && tar zxvf protobuf-java-3.21.1.tar.gz --strip-components 1 -C protobuf-3.21 + $ cd protobuf-3.721 Review Comment: Typo > Update protobuf 3.7.1 to a version without CVE-2021-22569 > - > > Key: HADOOP-18197 > URL: https://issues.apache.org/jira/browse/HADOOP-18197 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ivan Viaznikov >Priority: Major > Labels: pull-request-available, security > Time Spent: 1.5h > Remaining Estimate: 0h > > The artifact `org.apache.hadoop:hadoop-common` brings in a dependency > `com.google.protobuf:protobuf-java:2.5.0`, which is an outdated version > released in 2013 and it contains a vulnerability > [CVE-2021-22569|https://nvd.nist.gov/vuln/detail/CVE-2021-22569]. > Therefore, requesting you to clarify if this library version is going to be > updated in the following releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xizhu-mstr commented on a diff in pull request #4418: HADOOP-18197. Upgrade protobuf to 3.21.7 (through upgraded hadoop-shaded-protobuf jar)
xizhu-mstr commented on code in PR #4418: URL: https://github.com/apache/hadoop/pull/4418#discussion_r1152934268 ## BUILDING.txt: ## @@ -403,10 +403,10 @@ Installing required dependencies for clean install of macOS 10.14: * Install native libraries, only openssl is required to compile native code, you may optionally install zlib, lz4, etc. $ brew install openssl -* Protocol Buffers 3.7.1 (required to compile native code) - $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-java-3.7.1.tar.gz - $ mkdir -p protobuf-3.7 && tar zxvf protobuf-java-3.7.1.tar.gz --strip-components 1 -C protobuf-3.7 - $ cd protobuf-3.7 +* Protocol Buffers 3.21.1 (required to compile native code) + $ wget https://github.com/protocolbuffers/protobuf/releases/download/v3.21.1/protobuf-java-3.21.1.tar.gz + $ mkdir -p protobuf-3.21 && tar zxvf protobuf-java-3.21.1.tar.gz --strip-components 1 -C protobuf-3.21 + $ cd protobuf-3.721 Review Comment: Typo -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] WYB489341394 commented on pull request #5520: HDFS-16965. Add switch to decide whether to enable native codec.
WYB489341394 commented on PR #5520: URL: https://github.com/apache/hadoop/pull/5520#issuecomment-1489885854 JIRA: [HDFS-16965](https://issues.apache.org/jira/browse/HDFS-16965). Add switch to decide whether to enable native codec. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] WYB489341394 opened a new pull request, #5520: HDFS-16965. Add switch to decide whether to enable native codec.
WYB489341394 opened a new pull request, #5520: URL: https://github.com/apache/hadoop/pull/5520 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #5363: YARN-11424. [Federation] Router Supports DeregisterSubCluster.
hadoop-yetus commented on PR #5363:
URL: https://github.com/apache/hadoop/pull/5363#issuecomment-1489864292
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 18s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 52s | | trunk passed |
| +1 :green_heart: | compile | 9m 46s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 8m 34s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | checkstyle | 1m 48s | | trunk passed |
| +1 :green_heart: | mvnsite | 6m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 7s | | trunk passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 5m 47s | | trunk passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 11m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 47s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 8s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 17s | | the patch passed |
| +1 :green_heart: | compile | 9m 4s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | cc | 9m 4s | | the patch passed |
| -1 :x: | javac | 9m 4s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5363/13/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1
with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 generated 1 new + 730
unchanged - 0 fixed = 731 total (was 730) |
| +1 :green_heart: | compile | 8m 23s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | cc | 8m 23s | | the patch passed |
| -1 :x: | javac | 8m 23s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5363/13/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09
with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 generated 2 new
+ 643 unchanged - 4 fixed = 645 total (was 647) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5363/13/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 1m 39s | |
hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 228 unchanged - 1
fixed = 228 total (was 229) |
| +1 :green_heart: | mvnsite | 6m 8s | | the patch passed |
| +1 :green_heart: | javadoc | 5m 27s | | the patch passed with JDK
Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 5m 11s | | the patch passed with JDK
Private Build-1.8.0_362-8u362-ga-0ubuntu1~20.04.1-b09 |
| +1 :green_heart: | spotbugs | 12m 13s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 49s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 13s | | ha
