[jira] [Commented] (HADOOP-18257) Analyzing S3A Audit Logs
[ https://issues.apache.org/jira/browse/HADOOP-18257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17787075#comment-17787075 ] ASF GitHub Bot commented on HADOOP-18257: - mehakmeet commented on PR #6000: URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1815897072 [/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/4/artifact/out/patch-unit-root.txt) says the Build was success but still gave a -1. Going to put an empty patch up for yetus again CC: @mukund-thakur @steveloughran > Analyzing S3A Audit Logs > - > > Key: HADOOP-18257 > URL: https://issues.apache.org/jira/browse/HADOOP-18257 > Project: Hadoop Common > Issue Type: Task > Components: fs/s3 >Reporter: Sravani Gadey >Assignee: Mehakmeet Singh >Priority: Major > Labels: pull-request-available > > The main aim is to analyze S3A Audit logs to give better insights in Hive and > Spark jobs. > Steps involved are: > * Merging audit log files containing huge number of audit logs collected > from a job containing various S3 requests. > * Parsing audit logs using regular expressions i.e., dividing them into key > value pairs. > * Converting the key value pairs into CSV file and AVRO file formats. > * Querying on data which would give better insights for different jobs. > * Visualizing the audit logs on Zeppelin or Jupyter notebook with graphs. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18257. Merging and Parsing S3A audit logs into Avro format for analysis. [hadoop]
mehakmeet commented on PR #6000: URL: https://github.com/apache/hadoop/pull/6000#issuecomment-1815897072 [/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6000/4/artifact/out/patch-unit-root.txt) says the Build was success but still gave a -1. Going to put an empty patch up for yetus again CC: @mukund-thakur @steveloughran -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] Hadoop 18860 [hadoop]
hadoop-yetus commented on PR #6275:
URL: https://github.com/apache/hadoop/pull/6275#issuecomment-1815734871
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 17m 55s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 15 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 59s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 30m 26s | | trunk passed |
| +1 :green_heart: | compile | 16m 16s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 14m 46s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 4m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 10m 49s | | trunk passed |
| +1 :green_heart: | javadoc | 8m 20s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 7m 54s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 22s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 41m 49s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 42m 11s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 36s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 18s | | the patch passed |
| +1 :green_heart: | compile | 15m 29s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 15m 29s | | the patch passed |
| +1 :green_heart: | compile | 14m 38s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 14m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 4s | | the patch passed |
| +1 :green_heart: | mvnsite | 10m 36s | | the patch passed |
| +1 :green_heart: | javadoc | 8m 14s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 7m 57s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +0 :ok: | spotbugs | 0m 21s | | hadoop-project has no data from
spotbugs |
| -1 :x: | shadedclient | 41m 11s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 24s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | unit | 19m 14s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 263m 50s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6275/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | unit | 105m 20s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 24m 37s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | unit | 23m 27s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | unit | 22m 17s | | hadoop-yarn-services-core in
the patch passed. |
| -1 :x: | unit | 247m 2s |
[/patch-unit-hadoop-yarn-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6275/1/artifact/out/patch-unit-hadoop-yarn-project.txt)
| hadoop-yarn-project in the patch passed. |
| +1 :green_heart: | asflicense | 1m 15s | | The patch does not
generate ASF License warnings. |
| | | 1015m 27s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.TestRollingUpgrade |
| |
hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2 |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-
Re: [PR] HDFS-17063. Support to configure different capacity reserved for each disk of DataNode. [hadoop]
Hexiaoqiao commented on PR #5793: URL: https://github.com/apache/hadoop/pull/5793#issuecomment-1815712416 Committed to trunk. Thanks @qijiale76 and @tomscut ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17063. Support to configure different capacity reserved for each disk of DataNode. [hadoop]
Hexiaoqiao merged PR #5793: URL: https://github.com/apache/hadoop/pull/5793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17172. Support FSNamesystemLock Parameters reconfigurable [hadoop]
Hexiaoqiao commented on PR #6002: URL: https://github.com/apache/hadoop/pull/6002#issuecomment-1815706727 The failed unit test seems unrelated to this PR. Committed to trunk. Thanks @haiyang1987 and @ZanderXu ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17172. Support FSNamesystemLock Parameters reconfigurable [hadoop]
Hexiaoqiao merged PR #6002: URL: https://github.com/apache/hadoop/pull/6002 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-15413. add dfs.client.read.striped.datanode.max.attempts to fix read ecfile timeout [hadoop]
Hexiaoqiao commented on PR #5829: URL: https://github.com/apache/hadoop/pull/5829#issuecomment-1815704414 Hi @Neilxzn Any progress here? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]
haiyang1987 commented on PR #6176: URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1815653206 The failed unit test seems unrelated to the change, local run test is normal. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Created] (HADOOP-18976) Enable service specific keystores and truststores for ZK SSL setup
István Fajth created HADOOP-18976: - Summary: Enable service specific keystores and truststores for ZK SSL setup Key: HADOOP-18976 URL: https://issues.apache.org/jira/browse/HADOOP-18976 Project: Hadoop Common Issue Type: Improvement Reporter: István Fajth Currently we have the common config properties hadoop.zk.(key|trust)store.(location|password) configuration options. In HADOOP-18956 a ZKDelegationTokenSecretManager specific option was provided for these configurations, so with that ZKDelegationTokenSecretManager's ZK access can be set in a centralized fashion along with enabling it within ResourceManager, and DFSZKFailoverController. On the other hand with DTSecretMgr we introduce specific options to be able to specify a separate keystore and truststore to be used. A good improvement would be to add the truststore/keystore related options to all the components, so that even if the common hadoop.zk.* properties are set, and SSL is enabled, an individual component can have its own separate keystore and truststore set via specific configs, however if there are no specific config specified it can fall back to the common config values. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18956) Zookeeper SSL/TLS support in ZKDelegationTokenSecretManager and ZKSignerSecretProvider
[ https://issues.apache.org/jira/browse/HADOOP-18956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786958#comment-17786958 ] ASF GitHub Bot commented on HADOOP-18956: - jojochuang commented on PR #6263: URL: https://github.com/apache/hadoop/pull/6263#issuecomment-1815432647 +1 thanks @fapifta > Zookeeper SSL/TLS support in ZKDelegationTokenSecretManager and > ZKSignerSecretProvider > -- > > Key: HADOOP-18956 > URL: https://issues.apache.org/jira/browse/HADOOP-18956 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Zita Dombi >Assignee: István Fajth >Priority: Major > Labels: pull-request-available > > HADOOP-18709 added support for Zookeeper to communicate with SSL/TLS enabled > in hadoop-common. With those changes we have the necessary parameters, that > we need to set to enable SSL/TLS in a ZK Client. That change also did changes > in ZKCuratorManager, so with that it is easy to set the SSL/TLS, for Yarn it > was done in YARN-11468. > In DelegationTokenAuthenticationFilter currently we are using > CuratorFrameworkFactory, it'd be good to change it to use ZKCuratorManager > and with that we should support SSL/TLS enablement. > *UPDATE* > So as I investigated this a bit more, it wouldn't be so easy to move to using > ZKCuratorManager. > DelegationTokenAuthenticationFilter uses ZK from two places: in > ZKDelegationTokenSecretManager and in ZKSignerSecretProvider. In both places > it uses CuratorFrameworkFactory, but the attributes and creation > differentiates from ZKCuratorManager. > In ZKDelegationTokenSecretManager it would be easy to add the new config and > based on that create ZK with CuratorFrameworkFactory. But > ZKSignerSecretProvider is in hadoop-auth module and with my change it would > need hadoop-common, so it would introduce circular dependency between modules > 'hadoop-auth' and 'hadoop-common'. I'm still working on a straightforward > solution. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18956. Zookeeper SSL/TLS support in ZKDelegationTokenSecretManager and ZKSignerSecretProvider [hadoop]
jojochuang commented on PR #6263: URL: https://github.com/apache/hadoop/pull/6263#issuecomment-1815432647 +1 thanks @fapifta -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18956) Zookeeper SSL/TLS support in ZKDelegationTokenSecretManager and ZKSignerSecretProvider
[ https://issues.apache.org/jira/browse/HADOOP-18956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786957#comment-17786957 ] ASF GitHub Bot commented on HADOOP-18956: - fapifta commented on PR #6263: URL: https://github.com/apache/hadoop/pull/6263#issuecomment-1815427936 Thank you for the review @sodonnel and @jojochuang, much appreciated. I have added extra information about the precedence order of ssl.enabled configuration options. I am glad that you like the testing approach, thank you for the feedback! > Zookeeper SSL/TLS support in ZKDelegationTokenSecretManager and > ZKSignerSecretProvider > -- > > Key: HADOOP-18956 > URL: https://issues.apache.org/jira/browse/HADOOP-18956 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Zita Dombi >Assignee: István Fajth >Priority: Major > Labels: pull-request-available > > HADOOP-18709 added support for Zookeeper to communicate with SSL/TLS enabled > in hadoop-common. With those changes we have the necessary parameters, that > we need to set to enable SSL/TLS in a ZK Client. That change also did changes > in ZKCuratorManager, so with that it is easy to set the SSL/TLS, for Yarn it > was done in YARN-11468. > In DelegationTokenAuthenticationFilter currently we are using > CuratorFrameworkFactory, it'd be good to change it to use ZKCuratorManager > and with that we should support SSL/TLS enablement. > *UPDATE* > So as I investigated this a bit more, it wouldn't be so easy to move to using > ZKCuratorManager. > DelegationTokenAuthenticationFilter uses ZK from two places: in > ZKDelegationTokenSecretManager and in ZKSignerSecretProvider. In both places > it uses CuratorFrameworkFactory, but the attributes and creation > differentiates from ZKCuratorManager. > In ZKDelegationTokenSecretManager it would be easy to add the new config and > based on that create ZK with CuratorFrameworkFactory. But > ZKSignerSecretProvider is in hadoop-auth module and with my change it would > need hadoop-common, so it would introduce circular dependency between modules > 'hadoop-auth' and 'hadoop-common'. I'm still working on a straightforward > solution. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HADOOP-18956. Zookeeper SSL/TLS support in ZKDelegationTokenSecretManager and ZKSignerSecretProvider [hadoop]
fapifta commented on PR #6263: URL: https://github.com/apache/hadoop/pull/6263#issuecomment-1815427936 Thank you for the review @sodonnel and @jojochuang, much appreciated. I have added extra information about the precedence order of ssl.enabled configuration options. I am glad that you like the testing approach, thank you for the feedback! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11483. [Federation] Router AdminCLI Supports Clean Finish Apps. [hadoop]
hadoop-yetus commented on PR #6251:
URL: https://github.com/apache/hadoop/pull/6251#issuecomment-1815297882
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 14m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 34m 42s | | trunk passed |
| +1 :green_heart: | compile | 7m 56s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 7m 13s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 55s | | trunk passed |
| +1 :green_heart: | mvnsite | 5m 10s | | trunk passed |
| +1 :green_heart: | javadoc | 5m 2s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 45s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 9m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 38m 41s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 17s | | the patch passed |
| +1 :green_heart: | compile | 7m 20s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | cc | 7m 20s | | the patch passed |
| +1 :green_heart: | javac | 7m 20s | | the patch passed |
| +1 :green_heart: | compile | 6m 57s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | cc | 6m 57s | | the patch passed |
| +1 :green_heart: | javac | 6m 57s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 54s | | the patch passed |
| +1 :green_heart: | mvnsite | 4m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 36s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 4m 15s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 10m 22s | | the patch passed |
| +1 :green_heart: | shadedclient | 39m 0s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 8s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 5m 37s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 49s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 104m 23s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 28m 9s | | hadoop-yarn-client in the patch
passed. |
| +1 :green_heart: | unit | 0m 42s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 56s | | The patch does not
generate ASF License warnings. |
| | | 363m 53s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6251/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6251 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint
bufcompat |
| uname | Linux 9ad6ad747887 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6e9b8e1c6964c100e2178616bfd6eb30f82d2ba2 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120
[jira] [Commented] (HADOOP-18975) AWS SDK v2: extend support for FIPS endpoints
[
https://issues.apache.org/jira/browse/HADOOP-18975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786939#comment-17786939
]
ASF GitHub Bot commented on HADOOP-18975:
-
hadoop-yetus commented on PR #6277:
URL: https://github.com/apache/hadoop/pull/6277#issuecomment-1815290371
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 24s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 18s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 19s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 10s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 19s | | the patch passed |
| +1 :green_heart: | compile | 0m 20s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 20s | | the patch passed |
| +1 :green_heart: | compile | 0m 15s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 15s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6277/1/artifact/out/blanks-eol.txt)
| The patch has 8 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 0m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 9s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 45s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 21s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 5s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 23s | | The patch does not
generate ASF License warnings. |
| | | 85m 31s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6277/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6277 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
xmllint |
| uname | Linux 4c05ea0df9a4 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0a58d48856f5c7e01903192575bea39363709209 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6277/1/testReport/ |
| Max. process+thread count | 559 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/
Re: [PR] HADOOP-18975. AWS SDK v2: extend support for FIPS endpoints [hadoop]
hadoop-yetus commented on PR #6277:
URL: https://github.com/apache/hadoop/pull/6277#issuecomment-1815290371
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 23s | | trunk passed |
| +1 :green_heart: | compile | 0m 24s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 18s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 27s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 19s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 23s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 42s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 10s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 19s | | the patch passed |
| +1 :green_heart: | compile | 0m 20s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 20s | | the patch passed |
| +1 :green_heart: | compile | 0m 15s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 15s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6277/1/artifact/out/blanks-eol.txt)
| The patch has 8 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 0m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 9s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 45s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 21s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 5s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 23s | | The patch does not
generate ASF License warnings. |
| | | 85m 31s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6277/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6277 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
xmllint |
| uname | Linux 4c05ea0df9a4 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0a58d48856f5c7e01903192575bea39363709209 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6277/1/testReport/ |
| Max. process+thread count | 559 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6277/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was
[jira] [Updated] (HADOOP-18975) AWS SDK v2: extend support for FIPS endpoints
[ https://issues.apache.org/jira/browse/HADOOP-18975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18975: Summary: AWS SDK v2: extend support for FIPS endpoints (was: AWS SDK v2: support FIPS endpoints) > AWS SDK v2: extend support for FIPS endpoints > -- > > Key: HADOOP-18975 > URL: https://issues.apache.org/jira/browse/HADOOP-18975 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Priority: Major > Labels: pull-request-available > > v1 SDK supported FIPS just by changing the endpoint. > Now we have a new builder setting to use. > * add new fs.s3a.endpoint.fips option > * pass it down > * test -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18975) AWS SDK v2: extend support for FIPS endpoints
[ https://issues.apache.org/jira/browse/HADOOP-18975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reassigned HADOOP-18975: --- Assignee: Steve Loughran > AWS SDK v2: extend support for FIPS endpoints > -- > > Key: HADOOP-18975 > URL: https://issues.apache.org/jira/browse/HADOOP-18975 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > > v1 SDK supported FIPS just by changing the endpoint. > Now we have a new builder setting to use. > * add new fs.s3a.endpoint.fips option > * pass it down > * test -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18975) AWS SDK v2: support FIPS endpoints
[ https://issues.apache.org/jira/browse/HADOOP-18975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17786912#comment-17786912 ] ASF GitHub Bot commented on HADOOP-18975: - steveloughran opened a new pull request, #6277: URL: https://github.com/apache/hadoop/pull/6277 Adds a new option `fs.s3a.endpoint.fips` to switch the SDK client to use FIPS endpoints, as an alternative to explicitly declaring them. * Also provides it as a path capability. * This is not a blocker for FIPS support. * SDK itself doesn't know that some places (eu) don't have FIPs endpoints * SDK only fails with endpoint + fips flag as a retried exceptionl; this PR fails fast. * Adds a new "connecting.md" doc; moves existing docs there and restructures. ### How was this patch tested? Testing in progress through a bucket with the option set. ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > AWS SDK v2: support FIPS endpoints > -- > > Key: HADOOP-18975 > URL: https://issues.apache.org/jira/browse/HADOOP-18975 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Priority: Major > > v1 SDK supported FIPS just by changing the endpoint. > Now we have a new builder setting to use. > * add new fs.s3a.endpoint.fips option > * pass it down > * test -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18975) AWS SDK v2: support FIPS endpoints
[ https://issues.apache.org/jira/browse/HADOOP-18975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18975: Labels: pull-request-available (was: ) > AWS SDK v2: support FIPS endpoints > -- > > Key: HADOOP-18975 > URL: https://issues.apache.org/jira/browse/HADOOP-18975 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Priority: Major > Labels: pull-request-available > > v1 SDK supported FIPS just by changing the endpoint. > Now we have a new builder setting to use. > * add new fs.s3a.endpoint.fips option > * pass it down > * test -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[PR] HADOOP-18975 AWS SDK v2: extend support for FIPS endpoints [hadoop]
steveloughran opened a new pull request, #6277: URL: https://github.com/apache/hadoop/pull/6277 Adds a new option `fs.s3a.endpoint.fips` to switch the SDK client to use FIPS endpoints, as an alternative to explicitly declaring them. * Also provides it as a path capability. * This is not a blocker for FIPS support. * SDK itself doesn't know that some places (eu) don't have FIPs endpoints * SDK only fails with endpoint + fips flag as a retried exceptionl; this PR fails fast. * Adds a new "connecting.md" doc; moves existing docs there and restructures. ### How was this patch tested? Testing in progress through a bucket with the option set. ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11614. [Federation] Add Federation PolicyManager Validation Rules. [hadoop]
hadoop-yetus commented on PR #6271:
URL: https://github.com/apache/hadoop/pull/6271#issuecomment-1815074665
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 59s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 35m 56s | | trunk passed |
| +1 :green_heart: | compile | 8m 8s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 7m 17s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 2m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 20s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 13s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 2m 0s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 4m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 37m 51s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 27s | | the patch passed |
| +1 :green_heart: | compile | 7m 21s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 7m 21s | | the patch passed |
| +1 :green_heart: | compile | 7m 16s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 7m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 52s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 7s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 57s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 49s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 4m 43s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 50s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 8s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | unit | 28m 10s | | hadoop-yarn-client in the patch
passed. |
| +1 :green_heart: | unit | 0m 41s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 56s | | The patch does not
generate ASF License warnings. |
| | | 221m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6271/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6271 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 2c264ecf5a72 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1d6c0703a68d0fe703497bbcc6cebde1f2abd804 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6271/2/testReport/ |
| Max. process+thread count | 577 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:
hado
[jira] [Assigned] (HADOOP-18974) Replace commons-lang imports with commons-lang3
[ https://issues.apache.org/jira/browse/HADOOP-18974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Gupta reassigned HADOOP-18974: --- Assignee: Ashutosh Gupta > Replace commons-lang imports with commons-lang3 > --- > > Key: HADOOP-18974 > URL: https://issues.apache.org/jira/browse/HADOOP-18974 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ashutosh Gupta >Priority: Major > > we have a direct dependency on commons-lang3 but still we are using > commons-lang at a couple of places, which is a transitive dependency coming > from hbase, solr, and couple of other places. > > good to replace those with lang3, rather than relying on older transitive > dependency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11610. [Federation] Add WeightedHomePolicyManager. [hadoop]
hadoop-yetus commented on PR #6256:
URL: https://github.com/apache/hadoop/pull/6256#issuecomment-1814909705
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 18m 7s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 45s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 38m 23s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 35s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 33s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 29s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 0s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 50s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 159m 26s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6256/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6256 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux cde24f6f02c8 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 9dbdadf8c8ff9c442512fe5037c5cdce0f8f2286 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6256/4/testReport/ |
| Max. process+thread count | 536 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6256/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsub
Re: [PR] YARN-11485. [Federation] Router Supports Yarn Admin CLI Cmds. [hadoop]
hadoop-yetus commented on PR #6265:
URL: https://github.com/apache/hadoop/pull/6265#issuecomment-1814887057
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 46m 6s | | trunk passed |
| +1 :green_heart: | compile | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 33s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 56s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 24s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | compile | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6265/3/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 6
new + 70 unchanged - 1 fixed = 76 total (was 71) |
| +1 :green_heart: | mvnsite | 0m 26s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 57s | | the patch passed |
| +1 :green_heart: | shadedclient | 33m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 28m 33s | | hadoop-yarn-client in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 155m 48s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6265/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6265 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux ae8ec5260be1 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6eb94fbbeae341827e987b1648a8e52d55c8b519 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6265/3/testReport/ |
| Max. process+thread count | 699 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6265/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org
Re: [PR] HDFS-17232. RBF: Fix NoNamenodesAvailableException for a long time, when use observer. [hadoop]
simbadzina commented on PR #6208: URL: https://github.com/apache/hadoop/pull/6208#issuecomment-1814884016 Merged the change. Thanks for the contribution @KeeProMise -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17232. RBF: Fix NoNamenodesAvailableException for a long time, when use observer. [hadoop]
simbadzina merged PR #6208: URL: https://github.com/apache/hadoop/pull/6208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11485. [Federation] Router Supports Yarn Admin CLI Cmds. [hadoop]
hadoop-yetus commented on PR #6265:
URL: https://github.com/apache/hadoop/pull/6265#issuecomment-1814812274
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 8m 25s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 51s | | trunk passed |
| +1 :green_heart: | compile | 0m 19s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 18s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 19s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 24s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 19s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 8s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 17s | | the patch passed |
| +1 :green_heart: | compile | 0m 17s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 17s | | the patch passed |
| +1 :green_heart: | compile | 0m 15s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 15s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 12s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6265/4/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-client.txt)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client: The patch generated 6
new + 70 unchanged - 1 fixed = 76 total (was 71) |
| +1 :green_heart: | mvnsite | 0m 18s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 15s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 39s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 25m 12s | | hadoop-yarn-client in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 26s | | The patch does not
generate ASF License warnings. |
| | | 114m 22s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6265/4/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6265 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4fc527f2d8ff 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6eb94fbbeae341827e987b1648a8e52d55c8b519 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6265/4/testReport/ |
| Max. process+thread count | 578 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6265/4/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org
Re: [PR] YARN-11577. Improve FederationInterceptorREST Method Result. [hadoop]
hadoop-yetus commented on PR #6190:
URL: https://github.com/apache/hadoop/pull/6190#issuecomment-1814775580
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 23s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 17s | | trunk passed |
| +1 :green_heart: | compile | 0m 19s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 17s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 18s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 23s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 24s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 19s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 1s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 16s | | the patch passed |
| +1 :green_heart: | compile | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 16s | | the patch passed |
| +1 :green_heart: | compile | 0m 13s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 13s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 11s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 17s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 16s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 40s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 56s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 18s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 24s | | The patch does not
generate ASF License warnings. |
| | | 80m 35s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6190/23/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6190 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux 157023a255c0 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b110add0fecdf043d82d8bbdc3624a67274784cf |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6190/23/testReport/ |
| Max. process+thread count | 554 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6190/23/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
UR
Re: [PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
saxenapranav commented on code in PR #6276:
URL: https://github.com/apache/hadoop/pull/6276#discussion_r1395841543
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java:
##
@@ -337,32 +356,31 @@ private boolean executeHttpOperation(final int retryCount,
return false;
} finally {
- int status = httpOperation.getStatusCode();
/*
-A status less than 300 (2xx range) or greater than or equal
-to 500 (5xx range) should contribute to throttling metrics being
updated.
-Less than 200 or greater than or equal to 500 show failed operations.
2xx
-range contributes to successful operations. 3xx range is for redirects
-and 4xx range is for user errors. These should not be a part of
-throttling backoff computation.
+Updating Client Side Throttling Metrics for relevant response status
codes.
+1. Status code in 2xx range: Successful Operations should contribute
+2. Status code in 3xx range: Redirection Operations should not
contribute
+3. Status code in 4xx range: User Errors should not contribute
+4. Status code in 503 range: Server Error should contribute as
following:
+ a. 503, Ingress Over Account Limit: Should Contribute
+ b. 503, Egress Over Account Limit: Should Contribute
+ c. 503, TPS Over Account Limit: Should Contribute
+ d. 503, Other Server Throttling: Should not contribute
+5. Status code in 5xx range other than 503: Should not contribute
+6. IOException and UnknownHostExceptions: Should not contribute
*/
- boolean updateMetricsResponseCode = (status <
HttpURLConnection.HTTP_MULT_CHOICE
- || status >= HttpURLConnection.HTTP_INTERNAL_ERROR);
- if (updateMetricsResponseCode) {
+ int statusCode = httpOperation.getStatusCode();
+ boolean shouldUpdateCSTMetrics = (statusCode <
HttpURLConnection.HTTP_MULT_CHOICE // Case 1
Review Comment:
should we do
`(statusCode < MULT_CHOICE || (status > INTERNAL_ERROR &&
!OTHER_SERVER_THROTTLING_ABBREVIATION.equals(failureReason))) &&
!wasExceptionThrown`
reason being: in `AbfsClientThrottlingIntercept`, `isFailedOperation`
depends on `status >= INTERNAL_ERROR`.
+in case its the last possible retry of retryLoop, in that case,
failureReason will not get updated.
If we use above heuristic, we can remove dependency on failureReason for
checking condition. in that case we would have to do:
```
(statusCode < MULT_CHOICE || (status > INTERNAL_ERROR &&
!OTHER_SERVER_THROTTLING.getErrorMessage().equals(httpOperation.getStorageErrorMessage(
&& !wasExceptionThrown
```
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java:
##
@@ -337,32 +356,31 @@ private boolean executeHttpOperation(final int retryCount,
return false;
} finally {
- int status = httpOperation.getStatusCode();
/*
-A status less than 300 (2xx range) or greater than or equal
-to 500 (5xx range) should contribute to throttling metrics being
updated.
-Less than 200 or greater than or equal to 500 show failed operations.
2xx
-range contributes to successful operations. 3xx range is for redirects
-and 4xx range is for user errors. These should not be a part of
-throttling backoff computation.
+Updating Client Side Throttling Metrics for relevant response status
codes.
+1. Status code in 2xx range: Successful Operations should contribute
+2. Status code in 3xx range: Redirection Operations should not
contribute
+3. Status code in 4xx range: User Errors should not contribute
+4. Status code in 503 range: Server Error should contribute as
following:
+ a. 503, Ingress Over Account Limit: Should Contribute
+ b. 503, Egress Over Account Limit: Should Contribute
+ c. 503, TPS Over Account Limit: Should Contribute
+ d. 503, Other Server Throttling: Should not contribute
+5. Status code in 5xx range other than 503: Should not contribute
+6. IOException and UnknownHostExceptions: Should not contribute
*/
- boolean updateMetricsResponseCode = (status <
HttpURLConnection.HTTP_MULT_CHOICE
- || status >= HttpURLConnection.HTTP_INTERNAL_ERROR);
- if (updateMetricsResponseCode) {
+ int statusCode = httpOperation.getStatusCode();
+ boolean shouldUpdateCSTMetrics = (statusCode <
HttpURLConnection.HTTP_MULT_CHOICE // Case 1
Review Comment:
Also, was thinking about `wasExceptionThown`. Because apart from read
timeout, JDK would be setting -1 as status for all exception (-1 is default).
Was Looking in SocketInputStream of java.net, setting -1 is not there.
In case of readTimeout happens, setting of -1 will not happen if we are
tryi
Re: [PR] YARN-11614. [Federation] Add Federation PolicyManager Validation Rules. [hadoop]
slfan1989 commented on code in PR #6271:
URL: https://github.com/apache/hadoop/pull/6271#discussion_r1395856958
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/FederationQueueWeight.java:
##
@@ -170,6 +170,26 @@ public static void checkHeadRoomAlphaValid(String
headRoomAlpha) throws YarnExce
}
}
+ public static void checkPolicyManagerValid(String policyManager) throws
YarnException {
+switch (policyManager) {
+ case
"org.apache.hadoop.yarn.server.federation.policies.manager.HashBroadcastPolicyManager":
+throw new YarnException("HashBroadcastPolicyManager does not support
the use of queue weights.");
+ case
"org.apache.hadoop.yarn.server.federation.policies.manager.HomePolicyManager":
+throw new YarnException("HomePolicyManager does not support the use of
queue weights.");
Review Comment:
I will improve this part of the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11614. [Federation] Add Federation PolicyManager Validation Rules. [hadoop]
slfan1989 commented on code in PR #6271:
URL: https://github.com/apache/hadoop/pull/6271#discussion_r1395856311
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/server/api/protocolrecords/FederationQueueWeight.java:
##
@@ -170,6 +170,26 @@ public static void checkHeadRoomAlphaValid(String
headRoomAlpha) throws YarnExce
}
}
+ public static void checkPolicyManagerValid(String policyManager) throws
YarnException {
+switch (policyManager) {
+ case
"org.apache.hadoop.yarn.server.federation.policies.manager.HashBroadcastPolicyManager":
Review Comment:
This is a good idea! I will improve this part of the code. I will add
verification on the router side.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11577. Improve FederationInterceptorREST Method Result. [hadoop]
slfan1989 commented on code in PR #6190:
URL: https://github.com/apache/hadoop/pull/6190#discussion_r1395847402
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/subcluster/capacity/TestYarnFederationWithCapacityScheduler.java:
##
@@ -73,4 +147,469 @@ public void testGetClusterInfo() throws
InterruptedException, IOException {
assertTrue(subClusters.contains(clusterInfo.getSubClusterId()));
}
}
+
+ @Test
+ public void testInfo() throws InterruptedException, IOException {
+FederationClusterInfo federationClusterInfo =
+TestFederationSubCluster.performGetCalls(ROUTER_WEB_ADDRESS,
RM_WEB_SERVICE_PATH + INFO,
+FederationClusterInfo.class, null, null);
+List clusterInfos = federationClusterInfo.getList();
+assertNotNull(clusterInfos);
+assertEquals(2, clusterInfos.size());
+for (ClusterInfo clusterInfo : clusterInfos) {
+ assertNotNull(clusterInfo);
+ assertTrue(subClusters.contains(clusterInfo.getSubClusterId()));
+}
+ }
+
+ @Test
+ public void testClusterUserInfo() throws Exception {
+FederationClusterUserInfo federationClusterUserInfo =
+TestFederationSubCluster.performGetCalls(ROUTER_WEB_ADDRESS,
+RM_WEB_SERVICE_PATH + CLUSTER_USER_INFO,
+FederationClusterUserInfo.class, null, null);
+List clusterUserInfos =
federationClusterUserInfo.getList();
+assertNotNull(clusterUserInfos);
+assertEquals(2, clusterUserInfos.size());
+for (ClusterUserInfo clusterUserInfo : clusterUserInfos) {
+ assertNotNull(clusterUserInfo);
+ assertTrue(subClusters.contains(clusterUserInfo.getSubClusterId()));
+}
+ }
+
+ @Test
+ public void testMetricsInfo() throws Exception {
+// It takes time to start the sub-cluster.
+// We need to wait for the sub-cluster to be completely started,
+// so we need to set the waiting time.
+// The resources of the two sub-clusters we registered are 24C and 12G,
+// so the resources that the Router should collect are 48C and 24G.
+GenericTestUtils.waitFor(() -> {
+ try {
+ClusterMetricsInfo clusterMetricsInfo =
+TestFederationSubCluster.performGetCalls(ROUTER_WEB_ADDRESS,
+RM_WEB_SERVICE_PATH + METRICS, ClusterMetricsInfo.class, null,
null);
+assertNotNull(clusterMetricsInfo);
+return (48 == clusterMetricsInfo.getTotalVirtualCores() &&
+24576 == clusterMetricsInfo.getTotalMB());
+ } catch (Exception e) {
+return false;
+ }
+}, 5000, 50 * 5000);
+ }
+
+ @Test
+ public void testSchedulerInfo() throws Exception {
+FederationSchedulerTypeInfo schedulerTypeInfo =
+TestFederationSubCluster.performGetCalls(ROUTER_WEB_ADDRESS,
+RM_WEB_SERVICE_PATH + SCHEDULER, FederationSchedulerTypeInfo.class,
null, null);
+assertNotNull(schedulerTypeInfo);
+List schedulerTypeInfos = schedulerTypeInfo.getList();
+assertNotNull(schedulerTypeInfos);
+assertEquals(2, schedulerTypeInfos.size());
+for (SchedulerTypeInfo schedulerTypeInfoItem : schedulerTypeInfos) {
+ assertNotNull(schedulerTypeInfoItem);
+
assertTrue(subClusters.contains(schedulerTypeInfoItem.getSubClusterId()));
+ CapacitySchedulerInfo schedulerInfo =
+ (CapacitySchedulerInfo) schedulerTypeInfoItem.getSchedulerInfo();
+ assertNotNull(schedulerInfo);
+ assertEquals(3, schedulerInfo.getQueues().getQueueInfoList().size());
+}
+ }
+
+ @Test
+ public void testNodesEmpty() throws Exception {
+// We are in 2 sub-clusters, each with 3 nodes, so our Router should
correspond to 6 nodes.
+GenericTestUtils.waitFor(() -> {
+ try {
+NodesInfo nodesInfo =
+TestFederationSubCluster.performGetCalls(ROUTER_WEB_ADDRESS,
+RM_WEB_SERVICE_PATH + NODES, NodesInfo.class, null, null);
+assertNotNull(nodesInfo);
+ArrayList nodes = nodesInfo.getNodes();
+assertNotNull(nodes);
+return (6 == nodes.size());
+ } catch (Exception e) {
+return false;
+ }
+}, 5000, 50 * 5000);
+ }
+
+ @Test
+ public void testNodesLost() throws Exception {
+GenericTestUtils.waitFor(() -> {
+ try {
+NodesInfo nodesInfo =
+TestFederationSubCluster.performGetCalls(ROUTER_WEB_ADDRESS,
+RM_WEB_SERVICE_PATH + NODES, NodesInfo.class, STATES, "LOST");
+assertNotNull(nodesInfo);
+ArrayList nodes = nodesInfo.getNodes();
+assertNotNull(nodes);
+return (0 == nodes.size());
Review Comment:
Thanks for your suggestion! I will improve it.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/subcluster/fair/TestYarnFederationWithFairScheduler.java:
##
@@ -72,4 +152,488 @@ public void testGetClusterInfo() throws
InterruptedException, I
Re: [PR] YARN-11483. [Federation] Router AdminCLI Supports Clean Finish Apps. [hadoop]
slfan1989 commented on code in PR #6251: URL: https://github.com/apache/hadoop/pull/6251#discussion_r1395841725 ## hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/MockResourceManagerFacade.java: ## @@ -141,46 +141,11 @@ import org.apache.hadoop.yarn.exceptions.YarnException; import org.apache.hadoop.yarn.security.AMRMTokenIdentifier; import org.apache.hadoop.yarn.server.api.ResourceManagerAdministrationProtocol; -import org.apache.hadoop.yarn.server.api.protocolrecords.AddToClusterNodeLabelsRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.AddToClusterNodeLabelsResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.CheckForDecommissioningNodesRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.CheckForDecommissioningNodesResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshAdminAclsRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshAdminAclsResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshClusterMaxPriorityRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshClusterMaxPriorityResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesResourcesRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesResourcesResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshNodesResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshQueuesRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshQueuesResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshServiceAclsRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshServiceAclsResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshSuperUserGroupsConfigurationRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshSuperUserGroupsConfigurationResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshUserToGroupsMappingsRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.RefreshUserToGroupsMappingsResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.RemoveFromClusterNodeLabelsRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.RemoveFromClusterNodeLabelsResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.ReplaceLabelsOnNodeRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.ReplaceLabelsOnNodeResponse; -import org.apache.hadoop.yarn.server.api.protocolrecords.UpdateNodeResourceRequest; -import org.apache.hadoop.yarn.server.api.protocolrecords.UpdateNodeResourceResponse; +import org.apache.hadoop.yarn.server.api.protocolrecords.*; Review Comment: Thank you very much for your help in reviewing the code! I will fix it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11610. [Federation] Add WeightedHomePolicyManager. [hadoop]
slfan1989 commented on code in PR #6256:
URL: https://github.com/apache/hadoop/pull/6256#discussion_r1395822378
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/policies/manager/TestWeightedHomePolicyManager.java:
##
@@ -0,0 +1,62 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with this
+ * work for additional information regarding copyright ownership. The ASF
+ * licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ * License for the specific language governing permissions and limitations
under
+ * the License.
+ */
+package org.apache.hadoop.yarn.server.federation.policies.manager;
+
+import
org.apache.hadoop.yarn.server.federation.policies.amrmproxy.HomeAMRMProxyPolicy;
+import
org.apache.hadoop.yarn.server.federation.policies.dao.WeightedPolicyInfo;
+import
org.apache.hadoop.yarn.server.federation.policies.router.WeightedRandomRouterPolicy;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterId;
+import org.apache.hadoop.yarn.server.federation.store.records.SubClusterIdInfo;
+import org.junit.Assert;
+import org.junit.Before;
+import org.junit.Test;
+
+import java.util.HashMap;
+import java.util.Map;
+
+public class TestWeightedHomePolicyManager extends BasePolicyManagerTest {
+ private WeightedPolicyInfo policyInfo;
+
+ @Before
+ public void setup() {
+// configure a policy
+wfp = new WeightedHomePolicyManager();
+wfp.setQueue("queue1");
+SubClusterId sc1 = SubClusterId.newInstance("sc1");
+policyInfo = new WeightedPolicyInfo();
+
+Map routerWeights = new HashMap<>();
+routerWeights.put(new SubClusterIdInfo(sc1), 0.2f);
+policyInfo.setRouterPolicyWeights(routerWeights);
+
+((WeightedHomePolicyManager) wfp).setWeightedPolicyInfo(policyInfo);
Review Comment:
Thank you very much for your help in reviewing the code! I will improve this
part of the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] YARN-11485. [Federation] Router Supports Yarn Admin CLI Cmds. [hadoop]
slfan1989 commented on code in PR #6265:
URL: https://github.com/apache/hadoop/pull/6265#discussion_r1395798501
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/cli/RMAdminCLI.java:
##
@@ -1039,6 +1114,54 @@ protected String getUsageString() {
return "Usage: rmadmin";
}
+ /**
+ * Parse subClusterId.
+ * This method will only parse subClusterId when Yarn Federation mode is
enabled.
+ *
+ * @param cliParser CommandLine.
+ * @return subClusterId.
+ */
+ protected String parseSubClusterId(CommandLine cliParser) {
+// If YARN Federation mode is not enabled, return empty.
+if (!isYarnFederationEnabled(getConf())) {
+ return StringUtils.EMPTY;
+}
+
+String subClusterId = cliParser.getOptionValue(OPTION_SUBCLUSTERID);
+if(StringUtils.isBlank(subClusterId)) {
Review Comment:
Thank you very much for your help in reviewing the code! I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17172. Support FSNamesystemLock Parameters reconfigurable [hadoop]
hadoop-yetus commented on PR #6002:
URL: https://github.com/apache/hadoop/pull/6002#issuecomment-1814509059
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 11m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 43m 19s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 17s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 8s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 6s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 42s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 35m 28s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 1m 14s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 14s | | the patch passed |
| +1 :green_heart: | compile | 1m 12s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 1m 12s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 58s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 16s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 19s | | the patch passed |
| +1 :green_heart: | shadedclient | 34m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 208m 49s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6002/7/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 357m 54s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6002/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6002 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 7fb0d6c9f1c7 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 200643ea65cd415775a34ed66c5eb0cf7f661fd6 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6002/7/testReport/ |
| Max. process+thread count | 3834 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6002/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an auto
Re: [PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
hadoop-yetus commented on PR #6276:
URL: https://github.com/apache/hadoop/pull/6276#issuecomment-1814400743
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 23s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 51s | | trunk passed |
| +1 :green_heart: | compile | 0m 21s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 20s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 26s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 21s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 44s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 19s | | the patch passed |
| +1 :green_heart: | compile | 0m 21s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 21s | | the patch passed |
| +1 :green_heart: | compile | 0m 16s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 13s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6276/1/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 6 new + 1 unchanged - 0
fixed = 7 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 17s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 0m 43s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 1s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 44s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 24s | | The patch does not
generate ASF License warnings. |
| | | 82m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6276/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6276 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux d09ec31f9437 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / dc3938b6a431bb0abfea3f56e85c69448a78683d |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6276/1/testReport/ |
| Max. process+thread count | 706 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6276/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respo
Re: [PR] HDFS-17121. BPServiceActor to provide new thread to handle FBR [hadoop]
hadoop-yetus commented on PR #5888:
URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1814358688
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 29s | | trunk passed |
| +1 :green_heart: | compile | 1m 19s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 13s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 25s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 37s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 48s | | trunk passed |
| +1 :green_heart: | shadedclient | 38m 51s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 17s | | the patch passed |
| +1 :green_heart: | compile | 1m 20s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5888/33/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -1 :x: | blanks | 0m 0s |
[/blanks-tabs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5888/33/artifact/out/blanks-tabs.txt)
| The patch 260 line(s) with tabs. |
| -0 :warning: | checkstyle | 1m 6s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5888/33/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 264 unchanged
- 0 fixed = 268 total (was 264) |
| +1 :green_heart: | mvnsite | 1m 17s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 54s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 34s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 52s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 23s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 217m 7s | | hadoop-hdfs in the patch
passed. |
| -1 :x: | asflicense | 0m 38s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5888/33/artifact/out/results-asflicense.txt)
| The patch generated 1 ASF License warnings. |
| | | 368m 6s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5888/33/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5888 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint |
| uname | Linux 30a4f6049ebb 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 227feac76ec559384f31a47817fbe5f048c43ab4 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~
Re: [PR] HDFS-17218. NameNode should process time out excess redundancy blocks [hadoop]
hadoop-yetus commented on PR #6176:
URL: https://github.com/apache/hadoop/pull/6176#issuecomment-1814326717
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 17m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 14s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 1m 14s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 1m 14s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 24s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 38s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 40m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 1m 15s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 1m 15s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 5s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 15s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 55s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 29s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 3m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 40m 0s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 253m 43s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6176/12/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not
generate ASF License warnings. |
| | | 423m 55s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.TestRollingUpgrade |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6176/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6176 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint |
| uname | Linux f4711f33a855 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 09eed20fc0df5c71731d279a2b516a4b8db4e6b6 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6176/12/testReport/ |
| Max. process+thread count | 2835 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6176/12/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apa
Re: [PR] HDFS-17121. BPServiceActor to provide new thread to handle FBR [hadoop]
hadoop-yetus commented on PR #5888:
URL: https://github.com/apache/hadoop/pull/5888#issuecomment-1814310186
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 20s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 40s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 37s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 1m 5s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 25s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5888/34/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -1 :x: | blanks | 0m 0s |
[/blanks-tabs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5888/34/artifact/out/blanks-tabs.txt)
| The patch 260 line(s) with tabs. |
| -0 :warning: | checkstyle | 0m 30s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5888/34/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 264 unchanged
- 0 fixed = 268 total (was 264) |
| +1 :green_heart: | mvnsite | 0m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 41s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 188m 43s | | hadoop-hdfs in the patch
passed. |
| -1 :x: | asflicense | 0m 27s |
[/results-asflicense.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5888/34/artifact/out/results-asflicense.txt)
| The patch generated 1 ASF License warnings. |
| | | 274m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5888/34/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5888 |
| Optional Tests | dupname asflicense codespell detsecrets compile javac
javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle xmllint |
| uname | Linux 0d24d0bd382c 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0909b493f8604e51e0a9797065906b034d710718 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~
Re: [PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
anmolanmol1234 commented on code in PR #6276: URL: https://github.com/apache/hadoop/pull/6276#discussion_r1395589847 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestRetryReason.java: ## @@ -29,16 +29,14 @@ import static java.net.HttpURLConnection.HTTP_FORBIDDEN; import static java.net.HttpURLConnection.HTTP_INTERNAL_ERROR; import static java.net.HttpURLConnection.HTTP_UNAVAILABLE; -import static org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.EGRESS_OVER_ACCOUNT_LIMIT; -import static org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.INGRESS_OVER_ACCOUNT_LIMIT; +import static org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.*; Review Comment: same as above -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
anmolanmol1234 commented on code in PR #6276:
URL: https://github.com/apache/hadoop/pull/6276#discussion_r1395589475
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsRestOperationMockFailures.java:
##
@@ -35,35 +35,25 @@
import static java.net.HttpURLConnection.HTTP_INTERNAL_ERROR;
import static java.net.HttpURLConnection.HTTP_OK;
import static java.net.HttpURLConnection.HTTP_UNAVAILABLE;
-import static
org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.EGRESS_OVER_ACCOUNT_LIMIT;
-import static
org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.INGRESS_OVER_ACCOUNT_LIMIT;
+import static
org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.*;
import static
org.apache.hadoop.fs.azurebfs.services.AbfsClientTestUtil.addGeneralMockBehaviourToAbfsClient;
import static
org.apache.hadoop.fs.azurebfs.services.AbfsClientTestUtil.addGeneralMockBehaviourToRestOpAndHttpOp;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.CONNECTION_RESET_ABBREVIATION;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.CONNECTION_RESET_MESSAGE;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.CONNECTION_TIMEOUT_ABBREVIATION;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.CONNECTION_TIMEOUT_JDK_MESSAGE;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.EGRESS_LIMIT_BREACH_ABBREVIATION;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.INGRESS_LIMIT_BREACH_ABBREVIATION;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.IO_EXCEPTION_ABBREVIATION;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.OPERATION_BREACH_MESSAGE;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.OPERATION_LIMIT_BREACH_ABBREVIATION;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.READ_TIMEOUT_ABBREVIATION;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.READ_TIMEOUT_JDK_MESSAGE;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.SOCKET_EXCEPTION_ABBREVIATION;
-import static
org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.UNKNOWN_HOST_EXCEPTION_ABBREVIATION;
+import static org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.*;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.nullable;
public class TestAbfsRestOperationMockFailures {
+ // In these tests a request first fails with given exceptions and then
succeed on retry.
Review Comment:
Add a test where number of time metrics are updated for say 500 kind of
error code is 0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
anmolanmol1234 commented on code in PR #6276: URL: https://github.com/apache/hadoop/pull/6276#discussion_r1395585890 ## hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/services/TestAbfsRestOperationMockFailures.java: ## @@ -35,35 +35,25 @@ import static java.net.HttpURLConnection.HTTP_INTERNAL_ERROR; import static java.net.HttpURLConnection.HTTP_OK; import static java.net.HttpURLConnection.HTTP_UNAVAILABLE; -import static org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.EGRESS_OVER_ACCOUNT_LIMIT; -import static org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.INGRESS_OVER_ACCOUNT_LIMIT; +import static org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.*; Review Comment: same as above -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
anmolanmol1234 commented on code in PR #6276: URL: https://github.com/apache/hadoop/pull/6276#discussion_r1395583913 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/retryReasonCategories/ServerErrorRetryReason.java: ## @@ -20,12 +20,8 @@ import static java.net.HttpURLConnection.HTTP_UNAVAILABLE; import static org.apache.hadoop.fs.azurebfs.constants.AbfsHttpConstants.HTTP_STATUS_CATEGORY_QUOTIENT; -import static org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.EGRESS_OVER_ACCOUNT_LIMIT; -import static org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.INGRESS_OVER_ACCOUNT_LIMIT; -import static org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.EGRESS_LIMIT_BREACH_ABBREVIATION; -import static org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.INGRESS_LIMIT_BREACH_ABBREVIATION; -import static org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.OPERATION_BREACH_MESSAGE; -import static org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.OPERATION_LIMIT_BREACH_ABBREVIATION; +import static org.apache.hadoop.fs.azurebfs.contracts.services.AzureServiceErrorCode.*; +import static org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.*; Review Comment: same as above -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
anmolanmol1234 commented on code in PR #6276:
URL: https://github.com/apache/hadoop/pull/6276#discussion_r1395579164
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java:
##
@@ -337,32 +356,31 @@ private boolean executeHttpOperation(final int retryCount,
return false;
} finally {
- int status = httpOperation.getStatusCode();
/*
-A status less than 300 (2xx range) or greater than or equal
-to 500 (5xx range) should contribute to throttling metrics being
updated.
-Less than 200 or greater than or equal to 500 show failed operations.
2xx
-range contributes to successful operations. 3xx range is for redirects
-and 4xx range is for user errors. These should not be a part of
-throttling backoff computation.
+Updating Client Side Throttling Metrics for relevant response status
codes.
+1. Status code in 2xx range: Successful Operations should contribute
+2. Status code in 3xx range: Redirection Operations should not
contribute
+3. Status code in 4xx range: User Errors should not contribute
+4. Status code in 503 range: Server Error should contribute as
following:
+ a. 503, Ingress Over Account Limit: Should Contribute
+ b. 503, Egress Over Account Limit: Should Contribute
+ c. 503, TPS Over Account Limit: Should Contribute
+ d. 503, Other Server Throttling: Should not contribute
+5. Status code in 5xx range other than 503: Should not contribute
+6. IOException and UnknownHostExceptions: Should not contribute
*/
- boolean updateMetricsResponseCode = (status <
HttpURLConnection.HTTP_MULT_CHOICE
- || status >= HttpURLConnection.HTTP_INTERNAL_ERROR);
- if (updateMetricsResponseCode) {
+ int statusCode = httpOperation.getStatusCode();
+ boolean shouldUpdateCSTMetrics = (statusCode <
HttpURLConnection.HTTP_MULT_CHOICE // Case 1
+ || INGRESS_LIMIT_BREACH_ABBREVIATION.equals(failureReason) //
Case 4.a
+ || EGRESS_LIMIT_BREACH_ABBREVIATION.equals(failureReason) //
Case 4.b
+ || OPERATION_LIMIT_BREACH_ABBREVIATION.equals(failureReason)) //
Case 4.c
+ && !wasExceptionThrown; // Case 6
+
+ if (shouldUpdateCSTMetrics) {
intercept.updateMetrics(operationType, httpOperation);
}
}
-LOG.debug("HttpRequest: {}: {}", operationType, httpOperation);
-
-if (client.getRetryPolicy().shouldRetry(retryCount,
httpOperation.getStatusCode())) {
- int status = httpOperation.getStatusCode();
- failureReason = RetryReason.getAbbreviation(null, status,
httpOperation.getStorageErrorMessage());
- return false;
-}
-
-result = httpOperation;
-
return true;
Review Comment:
The return statement should not be a part of finally right ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
anmolanmol1234 commented on code in PR #6276:
URL: https://github.com/apache/hadoop/pull/6276#discussion_r1395579164
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java:
##
@@ -337,32 +356,31 @@ private boolean executeHttpOperation(final int retryCount,
return false;
} finally {
- int status = httpOperation.getStatusCode();
/*
-A status less than 300 (2xx range) or greater than or equal
-to 500 (5xx range) should contribute to throttling metrics being
updated.
-Less than 200 or greater than or equal to 500 show failed operations.
2xx
-range contributes to successful operations. 3xx range is for redirects
-and 4xx range is for user errors. These should not be a part of
-throttling backoff computation.
+Updating Client Side Throttling Metrics for relevant response status
codes.
+1. Status code in 2xx range: Successful Operations should contribute
+2. Status code in 3xx range: Redirection Operations should not
contribute
+3. Status code in 4xx range: User Errors should not contribute
+4. Status code in 503 range: Server Error should contribute as
following:
+ a. 503, Ingress Over Account Limit: Should Contribute
+ b. 503, Egress Over Account Limit: Should Contribute
+ c. 503, TPS Over Account Limit: Should Contribute
+ d. 503, Other Server Throttling: Should not contribute
+5. Status code in 5xx range other than 503: Should not contribute
+6. IOException and UnknownHostExceptions: Should not contribute
*/
- boolean updateMetricsResponseCode = (status <
HttpURLConnection.HTTP_MULT_CHOICE
- || status >= HttpURLConnection.HTTP_INTERNAL_ERROR);
- if (updateMetricsResponseCode) {
+ int statusCode = httpOperation.getStatusCode();
+ boolean shouldUpdateCSTMetrics = (statusCode <
HttpURLConnection.HTTP_MULT_CHOICE // Case 1
+ || INGRESS_LIMIT_BREACH_ABBREVIATION.equals(failureReason) //
Case 4.a
+ || EGRESS_LIMIT_BREACH_ABBREVIATION.equals(failureReason) //
Case 4.b
+ || OPERATION_LIMIT_BREACH_ABBREVIATION.equals(failureReason)) //
Case 4.c
+ && !wasExceptionThrown; // Case 6
+
+ if (shouldUpdateCSTMetrics) {
intercept.updateMetrics(operationType, httpOperation);
}
}
-LOG.debug("HttpRequest: {}: {}", operationType, httpOperation);
-
-if (client.getRetryPolicy().shouldRetry(retryCount,
httpOperation.getStatusCode())) {
- int status = httpOperation.getStatusCode();
- failureReason = RetryReason.getAbbreviation(null, status,
httpOperation.getStorageErrorMessage());
- return false;
-}
-
-result = httpOperation;
-
return true;
Review Comment:
The return statement should not be a part of finally right ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
anmolanmol1234 commented on code in PR #6276:
URL: https://github.com/apache/hadoop/pull/6276#discussion_r1395575791
##
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java:
##
@@ -314,7 +316,23 @@ private boolean executeHttpOperation(final int retryCount,
} else if (httpOperation.getStatusCode() ==
HttpURLConnection.HTTP_UNAVAILABLE) {
incrementCounter(AbfsStatistic.SERVER_UNAVAILABLE, 1);
}
+
+ // If no exception occurred here it means http operation was
successfully complete and
+ // a response from server has been received which might be failure or
success.
+ LOG.debug("HttpRequest: {}: {}", operationType, httpOperation);
+
+ // If request failed at server and should be retried, we will determine
failure reason and retry policy here
Review Comment:
nit: . at end
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
anmolanmol1234 commented on code in PR #6276: URL: https://github.com/apache/hadoop/pull/6276#discussion_r1395575058 ## hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java: ## @@ -39,6 +39,7 @@ import org.apache.hadoop.fs.statistics.impl.IOStatisticsBinding; import static org.apache.hadoop.fs.azurebfs.constants.AbfsHttpConstants.HTTP_CONTINUE; +import static org.apache.hadoop.fs.azurebfs.services.RetryReasonConstants.*; Review Comment: * should be avoided -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[PR] [ABFS][CST Optimization] Enhancing Client-Side Metrics Updating Logic [hadoop]
anujmodi2021 opened a new pull request, #6276: URL: https://github.com/apache/hadoop/pull/6276 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[PR] Hadoop 18860 [hadoop]
anmolanmol1234 opened a new pull request, #6275: URL: https://github.com/apache/hadoop/pull/6275 [HADOOP-18860](https://issues.apache.org/jira/browse/HADOOP-18860): Upgrade mockito version to 4.11.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17232. RBF: Fix NoNamenodesAvailableException for a long time, when use observer. [hadoop]
KeeProMise commented on PR #6208: URL: https://github.com/apache/hadoop/pull/6208#issuecomment-1814177154 @goiri @simbadzina If no more comments here, please help merge it, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] Hadoop-18759: [ABFS][Backoff-Optimization] Have a Static retry policy for connection timeout. [hadoop]
anujmodi2021 commented on PR #5881: URL: https://github.com/apache/hadoop/pull/5881#issuecomment-1814068639 Hi @steveloughran I request you to please merge this PR if it looks all good to you. We are looking forward to this from quite a while now. Thanks a lot. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17063. Support to configure different capacity reserved for each disk of DataNode. [hadoop]
qijiale76 commented on PR #5793: URL: https://github.com/apache/hadoop/pull/5793#issuecomment-1814032280 > LGTM. +1 from my side. Try to trigger CI manually, let's wait what it will say. If no more other comments and CI run fine. Will commit it. Thanks. Thank you. It seems that the failed test `TestDirectoryScanner` has nothing to do with this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
Re: [PR] HDFS-17232. RBF: Fix NoNamenodesAvailableException for a long time, when use observer. [hadoop]
hadoop-yetus commented on PR #6208:
URL: https://github.com/apache/hadoop/pull/6208#issuecomment-1814024514
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 47m 41s | | trunk passed |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 37s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 44s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 38m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | |
hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 0 new + 0 unchanged -
1 fixed = 0 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 56s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 24m 57s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 167m 13s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6208/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6208 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 395f1b30500a 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / bd1d203aaf0a45a13e8b4099e23b07841aaa2945 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6208/6/testReport/ |
| Max. process+thread count | 2415 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6208/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubsc
Re: [PR] HDFS-17232. RBF: Fix NoNamenodesAvailableException for a long time, when use observer. [hadoop]
hadoop-yetus commented on PR #6208:
URL: https://github.com/apache/hadoop/pull/6208#issuecomment-1814023485
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 19m 4s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 48m 15s | | trunk passed |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 46s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 45s | | trunk passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 39m 14s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | |
hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 0 new + 0 unchanged -
1 fixed = 0 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| +1 :green_heart: | spotbugs | 1m 42s | | the patch passed |
| +1 :green_heart: | shadedclient | 38m 50s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 25m 5s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 185m 34s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6208/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/6208 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 42def640a01a 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2
15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / bd1d203aaf0a45a13e8b4099e23b07841aaa2945 |
| Default Java | Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_382-8u382-ga-1~20.04.1-b05 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6208/5/testReport/ |
| Max. process+thread count | 2433 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6208/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubsc
