[jira] [Work logged] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client
[
https://issues.apache.org/jira/browse/HADOOP-18069?focusedWorklogId=763301=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763301
]
ASF GitHub Bot logged work on HADOOP-18069:
---
Author: ASF GitHub Bot
Created on: 28/Apr/22 05:47
Start Date: 28/Apr/22 05:47
Worklog Time Spent: 10m
Work Description: aajisaka commented on code in PR #4229:
URL: https://github.com/apache/hadoop/pull/4229#discussion_r860506970
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java:
##
@@ -103,36 +104,39 @@ public synchronized String getAccessToken() throws
IOException {
void refresh() throws IOException {
try {
- OkHttpClient client = new OkHttpClient();
- client.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
- TimeUnit.MILLISECONDS);
- client.setReadTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
-TimeUnit.MILLISECONDS);
+ OkHttpClient client =
+ new
OkHttpClient.Builder().connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
+ TimeUnit.MILLISECONDS)
+ .readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
+ .build();
Review Comment:
I meant starting with
```
try (OkHttpClient client =
new
OkHttpClient.Builder().connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
.build()) {
```
Issue Time Tracking
---
Worklog Id: (was: 763301)
Time Spent: 3h (was: 2h 50m)
> CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client
> ---
>
> Key: HADOOP-18069
> URL: https://issues.apache.org/jira/browse/HADOOP-18069
> Project: Hadoop Common
> Issue Type: Bug
> Components: hdfs-client
>Affects Versions: 3.3.1
>Reporter: Eugene Shinn (Truveta)
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> Our static vulnerability scanner (Fortify On Demand) detected [NVD -
> CVE-2021-0341
> (nist.gov)|https://nvd.nist.gov/vuln/detail/CVE-2021-0341#VulnChangeHistorySection]
> in our application. We traced the vulnerability to a transitive dependency
> coming from hadoop-hdfs-client, which depends on okhttp@2.7.5
> ([hadoop/pom.xml at trunk · apache/hadoop
> (github.com)|https://github.com/apache/hadoop/blob/trunk/hadoop-project/pom.xml#L137]).
> To resolve this issue, okhttp should be upgraded to 4.9.2+ (ref:
> [CVE-2021-0341 · Issue #6724 · square/okhttp
> (github.com)|https://github.com/square/okhttp/issues/6724]).
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a diff in pull request #4229: HADOOP-18069. okhttp@2.7.5 to 4.9.3
aajisaka commented on code in PR #4229:
URL: https://github.com/apache/hadoop/pull/4229#discussion_r860506970
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/web/oauth2/ConfRefreshTokenBasedAccessTokenProvider.java:
##
@@ -103,36 +104,39 @@ public synchronized String getAccessToken() throws
IOException {
void refresh() throws IOException {
try {
- OkHttpClient client = new OkHttpClient();
- client.setConnectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
- TimeUnit.MILLISECONDS);
- client.setReadTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
-TimeUnit.MILLISECONDS);
+ OkHttpClient client =
+ new
OkHttpClient.Builder().connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
+ TimeUnit.MILLISECONDS)
+ .readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
+ .build();
Review Comment:
I meant starting with
```
try (OkHttpClient client =
new
OkHttpClient.Builder().connectTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
.readTimeout(URLConnectionFactory.DEFAULT_SOCKET_TIMEOUT,
TimeUnit.MILLISECONDS)
.build()) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client
[ https://issues.apache.org/jira/browse/HADOOP-18069?focusedWorklogId=763300=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763300 ] ASF GitHub Bot logged work on HADOOP-18069: --- Author: ASF GitHub Bot Created on: 28/Apr/22 05:46 Start Date: 28/Apr/22 05:46 Worklog Time Spent: 10m Work Description: aajisaka commented on code in PR #4229: URL: https://github.com/apache/hadoop/pull/4229#discussion_r860506282 ## LICENSE-binary: ## @@ -242,6 +242,7 @@ com.google.guava:listenablefuture:.0-empty-to-avoid-conflict-with-guava com.microsoft.azure:azure-storage:7.0.0 com.nimbusds:nimbus-jose-jwt:9.8.1 com.squareup.okhttp:okhttp:2.7.5 Review Comment: Would you remove this line? Issue Time Tracking --- Worklog Id: (was: 763300) Time Spent: 2h 50m (was: 2h 40m) > CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client > --- > > Key: HADOOP-18069 > URL: https://issues.apache.org/jira/browse/HADOOP-18069 > Project: Hadoop Common > Issue Type: Bug > Components: hdfs-client >Affects Versions: 3.3.1 >Reporter: Eugene Shinn (Truveta) >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > Our static vulnerability scanner (Fortify On Demand) detected [NVD - > CVE-2021-0341 > (nist.gov)|https://nvd.nist.gov/vuln/detail/CVE-2021-0341#VulnChangeHistorySection] > in our application. We traced the vulnerability to a transitive dependency > coming from hadoop-hdfs-client, which depends on okhttp@2.7.5 > ([hadoop/pom.xml at trunk · apache/hadoop > (github.com)|https://github.com/apache/hadoop/blob/trunk/hadoop-project/pom.xml#L137]). > To resolve this issue, okhttp should be upgraded to 4.9.2+ (ref: > [CVE-2021-0341 · Issue #6724 · square/okhttp > (github.com)|https://github.com/square/okhttp/issues/6724]). -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a diff in pull request #4229: HADOOP-18069. okhttp@2.7.5 to 4.9.3
aajisaka commented on code in PR #4229: URL: https://github.com/apache/hadoop/pull/4229#discussion_r860506282 ## LICENSE-binary: ## @@ -242,6 +242,7 @@ com.google.guava:listenablefuture:.0-empty-to-avoid-conflict-with-guava com.microsoft.azure:azure-storage:7.0.0 com.nimbusds:nimbus-jose-jwt:9.8.1 com.squareup.okhttp:okhttp:2.7.5 Review Comment: Would you remove this line? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on a diff in pull request #4242: YARN-11116. Migrate Times util from SimpleDateFormat to thread-safe D…
aajisaka commented on code in PR #4242:
URL: https://github.com/apache/hadoop/pull/4242#discussion_r860503837
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/util/TestTimes.java:
##
@@ -61,4 +67,15 @@ public void testFinishTimesAheadOfStartTimes() {
elapsed = Times.elapsed(Long.MAX_VALUE, 0, true);
Assert.assertEquals("Elapsed time is not -1", -1, elapsed);
}
+
+ @Test
+ public void validateISO() throws IOException {
+SimpleDateFormat isoFormat = new SimpleDateFormat(ISO8601_DATE_FORMAT);
+for (int i = 0; i < 1000; i++) {
+ long now = System.currentTimeMillis();
+ String instant = Times.formatISO8601(now);
+ String date = isoFormat.format(new Date(now));
+ Assert.assertEquals(date, instant);
Review Comment:
The order of the arguments are (expected, actual). I think it should be
(instant, date).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-13332) Remove jackson 1.9.13 and switch all jackson code to 2.x code line
[ https://issues.apache.org/jira/browse/HADOOP-13332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-13332. Assignee: (was: Akira Ajisaka) Resolution: Done All the sub-tasks completed. Closing this umbrella JIRA. > Remove jackson 1.9.13 and switch all jackson code to 2.x code line > -- > > Key: HADOOP-13332 > URL: https://issues.apache.org/jira/browse/HADOOP-13332 > Project: Hadoop Common > Issue Type: Improvement > Components: build >Affects Versions: 2.8.0 >Reporter: PJ Fanning >Priority: Major > Attachments: HADOOP-13332.00.patch, HADOOP-13332.01.patch, > HADOOP-13332.02.patch, HADOOP-13332.03.patch > > > This jackson 1.9 code line is no longer maintained. Upgrade > Most changes from jackson 1.9 to 2.x just involve changing the package name. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18195) make jackson v1 a runtime scope dependency
[ https://issues.apache.org/jira/browse/HADOOP-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-18195. Resolution: Duplicate The jackson v1 dependency has been removed by HADOOP-15983. Closing. > make jackson v1 a runtime scope dependency > -- > > Key: HADOOP-18195 > URL: https://issues.apache.org/jira/browse/HADOOP-18195 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: PJ Fanning >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > In trunk, jackson v1 is only needed as a transitive dependency of jersey-json > (1.19). > The jackson v1 jars still appear in hadoop pom so it would be useful to make > them 'runtime' scope so that no hadoop code can compile with links to this > old code. > > [https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-project/3.4.0-SNAPSHOT/hadoop-project-3.4.0-20220407.091529-1274.pom] > > the linked PR is not needed if [https://github.com/apache/hadoop/pull/3988] > is merged -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-15983) Use jersey-json that is built to use jackson2
[ https://issues.apache.org/jira/browse/HADOOP-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HADOOP-15983. Fix Version/s: 3.4.0 Hadoop Flags: Incompatible change Release Note: Use modified jersey-json 1.20 in https://github.com/pjfanning/jersey-1.x/tree/v1.20 that uses Jackson 2.x. By this change, Jackson 1.x dependency has been removed from Hadoop. Resolution: Fixed Merged the PR into trunk. Thank you [~pj.fanning] for your contribution! I really appreciate this work. > Use jersey-json that is built to use jackson2 > - > > Key: HADOOP-15983 > URL: https://issues.apache.org/jira/browse/HADOOP-15983 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15983) Use jersey-json that is built to use jackson2
[ https://issues.apache.org/jira/browse/HADOOP-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HADOOP-15983: --- Summary: Use jersey-json that is built to use jackson2 (was: Remove the usage of jersey-json to remove jackson 1.x dependency.) > Use jersey-json that is built to use jackson2 > - > > Key: HADOOP-15983 > URL: https://issues.apache.org/jira/browse/HADOOP-15983 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-15983) Use jersey-json that is built to use jackson2
[ https://issues.apache.org/jira/browse/HADOOP-15983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HADOOP-15983: -- Assignee: PJ Fanning > Use jersey-json that is built to use jackson2 > - > > Key: HADOOP-15983 > URL: https://issues.apache.org/jira/browse/HADOOP-15983 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: PJ Fanning >Priority: Major > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-15983) Remove the usage of jersey-json to remove jackson 1.x dependency.
[ https://issues.apache.org/jira/browse/HADOOP-15983?focusedWorklogId=763295=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763295 ] ASF GitHub Bot logged work on HADOOP-15983: --- Author: ASF GitHub Bot Created on: 28/Apr/22 05:18 Start Date: 28/Apr/22 05:18 Worklog Time Spent: 10m Work Description: aajisaka commented on PR #3988: URL: https://github.com/apache/hadoop/pull/3988#issuecomment-756150 Merged into trunk. Thank you @pjfanning and @steveloughran Issue Time Tracking --- Worklog Id: (was: 763295) Time Spent: 3h 50m (was: 3h 40m) > Remove the usage of jersey-json to remove jackson 1.x dependency. > - > > Key: HADOOP-15983 > URL: https://issues.apache.org/jira/browse/HADOOP-15983 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 3h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-15983) Remove the usage of jersey-json to remove jackson 1.x dependency.
[ https://issues.apache.org/jira/browse/HADOOP-15983?focusedWorklogId=763294=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763294 ] ASF GitHub Bot logged work on HADOOP-15983: --- Author: ASF GitHub Bot Created on: 28/Apr/22 05:18 Start Date: 28/Apr/22 05:18 Worklog Time Spent: 10m Work Description: aajisaka merged PR #3988: URL: https://github.com/apache/hadoop/pull/3988 Issue Time Tracking --- Worklog Id: (was: 763294) Time Spent: 3h 40m (was: 3.5h) > Remove the usage of jersey-json to remove jackson 1.x dependency. > - > > Key: HADOOP-15983 > URL: https://issues.apache.org/jira/browse/HADOOP-15983 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka commented on pull request #3988: [HADOOP-15983] use jersey-json that is built to use jackson2
aajisaka commented on PR #3988: URL: https://github.com/apache/hadoop/pull/3988#issuecomment-756150 Merged into trunk. Thank you @pjfanning and @steveloughran -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] aajisaka merged pull request #3988: [HADOOP-15983] use jersey-json that is built to use jackson2
aajisaka merged PR #3988: URL: https://github.com/apache/hadoop/pull/3988 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao commented on pull request #4241: HDFS-16563. Namenode WebUI prints sensitive information on Token expiry
Hexiaoqiao commented on PR #4241: URL: https://github.com/apache/hadoop/pull/4241#issuecomment-754159 @prasad-acit Thanks for involving me here. IMO, the key and sensitive information is DelegationKey/Password for DelegationToken, the output message here does not include this information right? So I don't think it it security issue. Do you mind to more information about this output or stack demo? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4228: HDFS-16468. Define ssize_t for Windows
hadoop-yetus commented on PR #4228:
URL: https://github.com/apache/hadoop/pull/4228#issuecomment-750932
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 24m 48s | | trunk passed |
| +1 :green_heart: | compile | 4m 21s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 4m 28s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | mvnsite | 0m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 57m 7s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 57m 30s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 19s | | the patch passed |
| +1 :green_heart: | compile | 4m 9s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | cc | 4m 9s | | the patch passed |
| +1 :green_heart: | golang | 4m 9s | | the patch passed |
| +1 :green_heart: | javac | 4m 9s | | the patch passed |
| +1 :green_heart: | compile | 4m 11s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 4m 11s | | the patch passed |
| +1 :green_heart: | golang | 4m 11s | | the patch passed |
| +1 :green_heart: | javac | 4m 11s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 53s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 104m 12s | | hadoop-hdfs-native-client in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 197m 18s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4228 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux 55deec886ef7 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 93590cc528eab92c2e0c3da9b04f812c9bcc4ee5 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/testReport/ |
| Max. process+thread count | 522 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hemanthboyina commented on pull request #4241: HDFS-16563. Namenode WebUI prints sensitive information on Token expiry
hemanthboyina commented on PR #4241: URL: https://github.com/apache/hadoop/pull/4241#issuecomment-746697 thank you @prasad-acit for the PR, looks the test case failures are related, can you have a look once -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4242: YARN-11116. Migrate Times util from SimpleDateFormat to thread-safe D…
hadoop-yetus commented on PR #4242:
URL: https://github.com/apache/hadoop/pull/4242#issuecomment-739842
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 28s | | trunk passed |
| +1 :green_heart: | compile | 1m 5s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 7s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 2s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 45s | | the patch passed |
| +1 :green_heart: | compile | 0m 48s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 48s | | the patch passed |
| +1 :green_heart: | compile | 0m 44s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 44s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 34s | |
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common: The patch generated 0 new +
1 unchanged - 2 fixed = 1 total (was 3) |
| +1 :green_heart: | mvnsite | 0m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 44s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 46s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 6s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 4m 59s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 103m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4242/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4242 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 6e562b66e22e 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e194e3fae5ca1fb7e807de78255b0a8f162a4914 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4242/1/testReport/ |
| Max. process+thread count | 550 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4242/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail:
[GitHub] [hadoop] hadoop-yetus commented on pull request #4241: HDFS-16563. Namenode WebUI prints sensitive information on Token expiry
hadoop-yetus commented on PR #4241:
URL: https://github.com/apache/hadoop/pull/4241#issuecomment-739690
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 44s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 37s | | trunk passed |
| +1 :green_heart: | compile | 25m 21s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 21m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 32s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 0s | | trunk passed |
| -1 :x: | javadoc | 1m 39s |
[/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/1/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| hadoop-common in trunk failed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 4m 4s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 27m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 32s | | the patch passed |
| +1 :green_heart: | compile | 24m 20s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 24m 20s | | the patch passed |
| +1 :green_heart: | compile | 21m 51s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 51s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 22s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 58s | | the patch passed |
| -1 :x: | javadoc | 1m 29s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/1/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| hadoop-common in the patch failed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 4m 3s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 27m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 32s | | hadoop-common in the patch
passed. |
| -1 :x: | unit | 384m 14s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 37s | | The patch does not
generate ASF License warnings. |
| | | 645m 16s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.web.TestWebHdfsTokens |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4241/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4241 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 3902be4f191d 4.15.0-175-generic #184-Ubuntu SMP Thu Mar 24
17:48:36 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk /
[GitHub] [hadoop] skysiders opened a new pull request, #4243: YARN-11117 check permission for LeveldbRMStateStore
skysiders opened a new pull request, #4243: URL: https://github.com/apache/hadoop/pull/4243 YARN-7 check permission for LeveldbRMStateStore ### Description of PR LeveldbRMStateStore use fs.mkdirs(root,new FsPermission((short)0700)); to create root directory with permission 700.BUT if umask is too strict such as 0777, this directory will have wrong permission with 000. So it should check if umask can affect permission, if true , use setPermission to fix it ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16965) Introduce StreamContext for Abfs Input and Output streams.
[ https://issues.apache.org/jira/browse/HADOOP-16965?focusedWorklogId=763273=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763273 ] ASF GitHub Bot logged work on HADOOP-16965: --- Author: ASF GitHub Bot Created on: 28/Apr/22 03:34 Start Date: 28/Apr/22 03:34 Worklog Time Spent: 10m Work Description: arjun4084346 commented on PR #4171: URL: https://github.com/apache/hadoop/pull/4171#issuecomment-706441 > Yetus still failing with unit tests. Please fix those https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4171/3/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt Hi @mukund-thakur , those unit tests fails in branch-2.10 . This PR does not introduce any new failing tests. e.g. take this PR https://github.com/apache/hadoop/pull/4151. It is merged into branch-2.10 and it also has some failing unit tests. Issue Time Tracking --- Worklog Id: (was: 763273) Time Spent: 3h 50m (was: 3h 40m) > Introduce StreamContext for Abfs Input and Output streams. > -- > > Key: HADOOP-16965 > URL: https://issues.apache.org/jira/browse/HADOOP-16965 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/azure >Reporter: Mukund Thakur >Assignee: Mukund Thakur >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > The number of configuration keeps growing in AbfsOutputStream and > AbfsInputStream as we keep on adding new features. It is time to refactor the > configurations in a separate class like StreamContext and pass them around. > This is will improve the readability of code and reduce cherry-pick-backport > pain. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] arjun4084346 commented on pull request #4171: HADOOP-16965. Refactor abfs stream configuration. (#1956)
arjun4084346 commented on PR #4171: URL: https://github.com/apache/hadoop/pull/4171#issuecomment-706441 > Yetus still failing with unit tests. Please fix those https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4171/3/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt Hi @mukund-thakur , those unit tests fails in branch-2.10 . This PR does not introduce any new failing tests. e.g. take this PR https://github.com/apache/hadoop/pull/4151. It is merged into branch-2.10 and it also has some failing unit tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jteagles opened a new pull request, #4242: YARN-11116. Migrate Times util from SimpleDateFormat to thread-safe D…
jteagles opened a new pull request, #4242: URL: https://github.com/apache/hadoop/pull/4242 …ateTimeFormatter class ### Description of PR Upgrading to the recommended thread safe java.time libraries for formatting and parsing with a little better performance ### How was this patch tested? Attached a performance test patch to the jira and added unit test to ensure compatible behavior ### For code changes: - [ x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ x] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ x] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on a diff in pull request #4107: HDFS-16521. DFS API to retrieve slow datanodes
virajjasani commented on code in PR #4107:
URL: https://github.com/apache/hadoop/pull/4107#discussion_r860405718
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodePeerMetrics.java:
##
@@ -142,14 +144,28 @@ public void collectThreadLocalStates() {
* than their peers.
*/
public Map getOutliers() {
-// This maps the metric name to the aggregate latency.
-// The metric name is the datanode ID.
-final Map stats =
-sendPacketDownstreamRollingAverages.getStats(
-minOutlierDetectionSamples);
-LOG.trace("DataNodePeerMetrics: Got stats: {}", stats);
-
-return slowNodeDetector.getOutliers(stats);
+// outlier must be null for source code.
+if (testOutlier == null) {
+ // This maps the metric name to the aggregate latency.
+ // The metric name is the datanode ID.
+ final Map stats =
+
sendPacketDownstreamRollingAverages.getStats(minOutlierDetectionSamples);
+ LOG.trace("DataNodePeerMetrics: Got stats: {}", stats);
+ return slowNodeDetector.getOutliers(stats);
+} else {
+ // this happens only for test code.
+ return testOutlier;
+}
+ }
+
+ /**
+ * Strictly to be used by test code only. Source code is not supposed to use
this. This method
+ * directly sets outlier mapping so that aggregate latency metrics are not
calculated for tests.
+ *
+ * @param outlier outlier directly set by tests.
+ */
+ public void setTestOutliers(Map outlier) {
Review Comment:
Yeah it's very difficult to reproduce the actual slow node in UT, hence had
to do this way. Sure, added comment on `testOutlier` member as well (in
addition to this setter method Javadoc).
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java:
##
@@ -632,6 +638,20 @@ private static void
printDataNodeReports(DistributedFileSystem dfs,
}
}
+ private static void printSlowDataNodeReports(DistributedFileSystem dfs,
boolean listNodes,
Review Comment:
> One comment on the slow datanode report is that it seems to say nothing
about why the NN thinks it slow;
It's a datanode that determines whether it's peer datanodes are slower, NN
just aggregates all DN reports.
> For example, say something about how in excess a DNs latency is? (Perhaps
this could be added later)
Sure, this can be added as an additional info. Will create a follow-up Jira.
Thanks @saintstack
##
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md:
##
@@ -394,7 +394,7 @@ Usage:
| COMMAND\_OPTION | Description |
|: |: |
-| `-report` `[-live]` `[-dead]` `[-decommissioning]` `[-enteringmaintenance]`
`[-inmaintenance]` | Reports basic filesystem information and statistics, The
dfs usage can be different from "du" usage, because it measures raw space used
by replication, checksums, snapshots and etc. on all the DNs. Optional flags
may be used to filter the list of displayed DataNodes. |
+| `-report` `[-live]` `[-dead]` `[-decommissioning]` `[-enteringmaintenance]`
`[-inmaintenance]` `[-slownodes]` | Reports basic filesystem information and
statistics, The dfs usage can be different from "du" usage, because it measures
raw space used by replication, checksums, snapshots and etc. on all the DNs.
Optional flags may be used to filter the list of displayed DataNodes. Filters
are either based on the DN state (e.g. live, dead, decommissioning) or the
nature of the DN (e.g. slow nodes). |
Review Comment:
So far, some good explanation can be found on
[OutlierDetector](https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/OutlierDetector.java#L35-L53)
class itself, but I get your point, we should have this documented on the site
as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] yuanboliu commented on pull request #4230: HDFS-16558 Consider changing the lock of delegation token from write …
yuanboliu commented on PR #4230: URL: https://github.com/apache/hadoop/pull/4230#issuecomment-680424 @Hexiaoqiao Thanks for your comments. We will apply this patch to our test env, and give feedback here. This is the code from trunk to prove that read lock is good enough: https://user-images.githubusercontent.com/5643344/165666072-023f7b39-3499-405e-bfd5-1ff1412ab9e1.png;> edit sync is protected by synchronized (this), and renew/cancel/get are in-order requests, there is no way that all the requests for one uniq token will be handled at the same time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4228: HDFS-16468. Define ssize_t for Windows
hadoop-yetus commented on PR #4228:
URL: https://github.com/apache/hadoop/pull/4228#issuecomment-652665
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 53s | | trunk passed |
| +1 :green_heart: | compile | 3m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 68m 9s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 68m 33s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 20s | | the patch passed |
| +1 :green_heart: | compile | 3m 31s | | the patch passed |
| +1 :green_heart: | cc | 3m 31s | | the patch passed |
| +1 :green_heart: | golang | 3m 31s | | the patch passed |
| +1 :green_heart: | javac | 3m 31s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 23s | | the patch passed |
| +1 :green_heart: | shadedclient | 30m 53s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 104m 51s | | hadoop-hdfs-native-client in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 212m 19s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4228 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux 3a23aa764c40 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 93590cc528eab92c2e0c3da9b04f812c9bcc4ee5 |
| Default Java | Debian-11.0.14+9-post-Debian-1deb10u1 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/testReport/ |
| Max. process+thread count | 610 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/console |
| versions | git=2.20.1 maven=3.6.0 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18079) Upgrade Netty to 4.1.74
[ https://issues.apache.org/jira/browse/HADOOP-18079?focusedWorklogId=763219=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763219 ] ASF GitHub Bot logged work on HADOOP-18079: --- Author: ASF GitHub Bot Created on: 28/Apr/22 00:49 Start Date: 28/Apr/22 00:49 Worklog Time Spent: 10m Work Description: jojochuang commented on PR #3977: URL: https://github.com/apache/hadoop/pull/3977#issuecomment-619185 Ok pushed a change to update to netty 4.1.75. Basically other than version change, the only difference is that tcnative is no longer in the classpath. Issue Time Tracking --- Worklog Id: (was: 763219) Time Spent: 1h 50m (was: 1h 40m) > Upgrade Netty to 4.1.74 > --- > > Key: HADOOP-18079 > URL: https://issues.apache.org/jira/browse/HADOOP-18079 > Project: Hadoop Common > Issue Type: Bug >Reporter: Renukaprasad C >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > h4. Netty version - 4.1.71 has fix some CVEs. We can upgrade the netty to > 4.1.7.1.Final or latest stable version - 4.1.7.2.Final. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on pull request #3977: HADOOP-18079. Upgrade Netty to 4.1.74.
jojochuang commented on PR #3977: URL: https://github.com/apache/hadoop/pull/3977#issuecomment-619185 Ok pushed a change to update to netty 4.1.75. Basically other than version change, the only difference is that tcnative is no longer in the classpath. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4228: HDFS-16468. Define ssize_t for Windows
hadoop-yetus commented on PR #4228:
URL: https://github.com/apache/hadoop/pull/4228#issuecomment-535508
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 24m 57s | | trunk passed |
| +1 :green_heart: | compile | 4m 12s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 52s | | trunk passed |
| +1 :green_heart: | shadedclient | 52m 57s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 53m 26s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 24s | | the patch passed |
| +1 :green_heart: | compile | 3m 50s | | the patch passed |
| +1 :green_heart: | cc | 3m 50s | | the patch passed |
| +1 :green_heart: | golang | 3m 50s | | the patch passed |
| +1 :green_heart: | javac | 3m 50s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 29s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 33s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 106m 14s | | hadoop-hdfs-native-client in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 190m 43s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4228 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux 3cd3e6424901 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 93590cc528eab92c2e0c3da9b04f812c9bcc4ee5 |
| Default Java | Red Hat, Inc.-1.8.0_312-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/testReport/ |
| Max. process+thread count | 587 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/console |
| versions | git=2.27.0 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17833) Improve Magic Committer Performance
[
https://issues.apache.org/jira/browse/HADOOP-17833?focusedWorklogId=763168=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763168
]
ASF GitHub Bot logged work on HADOOP-17833:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 21:56
Start Date: 27/Apr/22 21:56
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #3289:
URL: https://github.com/apache/hadoop/pull/3289#issuecomment-522287
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 29 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 54s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 0s | | trunk passed |
| +1 :green_heart: | compile | 24m 27s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 21m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 54s | | trunk passed |
| -1 :x: | javadoc | 1m 45s |
[/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| hadoop-common in trunk failed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 4m 50s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 59s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 29s | | the patch passed |
| +1 :green_heart: | compile | 24m 8s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 24m 8s | | the patch passed |
| +1 :green_heart: | compile | 21m 38s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 20s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 10 new + 20 unchanged - 2 fixed = 30 total (was
22) |
| +1 :green_heart: | mvnsite | 5m 17s | | the patch passed |
| -1 :x: | javadoc | 1m 33s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| hadoop-common in the patch failed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| -1 :x: | javadoc | 1m 9s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new +
38 unchanged - 0 fixed = 43 total (was 38) |
| -1 :x: | spotbugs | 2m 3s |
[/new-spotbugs-hadoop-tools_hadoop-aws.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/new-spotbugs-hadoop-tools_hadoop-aws.html)
| hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 23m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1
[GitHub] [hadoop] hadoop-yetus commented on pull request #3289: HADOOP-17833. Improve Magic Committer performance
hadoop-yetus commented on PR #3289:
URL: https://github.com/apache/hadoop/pull/3289#issuecomment-522287
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 29 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 54s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 0s | | trunk passed |
| +1 :green_heart: | compile | 24m 27s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 21m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 54s | | trunk passed |
| -1 :x: | javadoc | 1m 45s |
[/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| hadoop-common in trunk failed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 4m 50s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 6m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 59s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 29s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 29s | | the patch passed |
| +1 :green_heart: | compile | 24m 8s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 24m 8s | | the patch passed |
| +1 :green_heart: | compile | 21m 38s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 38s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 20s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 10 new + 20 unchanged - 2 fixed = 30 total (was
22) |
| +1 :green_heart: | mvnsite | 5m 17s | | the patch passed |
| -1 :x: | javadoc | 1m 33s |
[/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| hadoop-common in the patch failed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| -1 :x: | javadoc | 1m 9s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 5 new +
38 unchanged - 0 fixed = 43 total (was 38) |
| -1 :x: | spotbugs | 2m 3s |
[/new-spotbugs-hadoop-tools_hadoop-aws.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3289/12/artifact/out/new-spotbugs-hadoop-tools_hadoop-aws.html)
| hadoop-tools/hadoop-aws generated 1 new + 0 unchanged - 0 fixed = 1 total
(was 0) |
| +1 :green_heart: | shadedclient | 23m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 47s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 6m 45s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 3m 11s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 38s | | The patch does not
generate ASF License warnings. |
| | | 270m 3s | | |
| Reason |
[jira] [Work logged] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client
[
https://issues.apache.org/jira/browse/HADOOP-18069?focusedWorklogId=763139=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763139
]
ASF GitHub Bot logged work on HADOOP-18069:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 20:21
Start Date: 27/Apr/22 20:21
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4229:
URL: https://github.com/apache/hadoop/pull/4229#issuecomment-443765
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 40s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 36s | | trunk passed |
| +1 :green_heart: | compile | 24m 57s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 21m 35s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 19m 50s | | trunk passed |
| -1 :x: | javadoc | 1m 36s |
[/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 8m 31s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 27s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 57m 53s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 39s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 25m 9s |
[/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/patch-mvninstall-root.txt)
| root in the patch failed. |
| +1 :green_heart: | compile | 24m 23s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 24m 23s | | the patch passed |
| +1 :green_heart: | compile | 21m 41s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 41s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 30s | | the patch passed |
| +1 :green_heart: | mvnsite | 19m 37s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | xml | 0m 4s | | The patch has no ill-formed XML
file. |
| -1 :x: | javadoc | 1m 28s |
[/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root in the patch failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.
|
| +1 :green_heart: | javadoc | 8m 22s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 26s | | hadoop-project has no data from
spotbugs |
| -1 :x: | spotbugs | 2m 55s |
[/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html)
| hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0
fixed = 2 total (was 0) |
| -1 :x: | spotbugs | 35m 47s |
[/new-spotbugs-root.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/new-spotbugs-root.html)
| root generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| -1 :x: | shadedclient | 58m 40s | | patch has errors when building
and testing
[GitHub] [hadoop] hadoop-yetus commented on pull request #4229: HADOOP-18069. okhttp@2.7.5 to 4.9.3
hadoop-yetus commented on PR #4229:
URL: https://github.com/apache/hadoop/pull/4229#issuecomment-443765
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 40s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 36s | | trunk passed |
| +1 :green_heart: | compile | 24m 57s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 21m 35s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 19m 50s | | trunk passed |
| -1 :x: | javadoc | 1m 36s |
[/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 8m 31s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 27s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 57m 53s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 39s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 25m 9s |
[/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/patch-mvninstall-root.txt)
| root in the patch failed. |
| +1 :green_heart: | compile | 24m 23s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 24m 23s | | the patch passed |
| +1 :green_heart: | compile | 21m 41s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 41s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 30s | | the patch passed |
| +1 :green_heart: | mvnsite | 19m 37s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | xml | 0m 4s | | The patch has no ill-formed XML
file. |
| -1 :x: | javadoc | 1m 28s |
[/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root in the patch failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.
|
| +1 :green_heart: | javadoc | 8m 22s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 26s | | hadoop-project has no data from
spotbugs |
| -1 :x: | spotbugs | 2m 55s |
[/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html)
| hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0 unchanged - 0
fixed = 2 total (was 0) |
| -1 :x: | spotbugs | 35m 47s |
[/new-spotbugs-root.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/new-spotbugs-root.html)
| root generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| -1 :x: | shadedclient | 58m 40s | | patch has errors when building
and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 1051m 27s |
[/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/8/artifact/out/patch-unit-root.txt)
| root in the patch passed. |
| +1 :green_heart: | asflicense | 2m 17s | | The patch does not
generate ASF License warnings. |
| | | 1417m 43s | | |
| Reason | Tests |
[jira] [Work logged] (HADOOP-16965) Introduce StreamContext for Abfs Input and Output streams.
[ https://issues.apache.org/jira/browse/HADOOP-16965?focusedWorklogId=763122=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763122 ] ASF GitHub Bot logged work on HADOOP-16965: --- Author: ASF GitHub Bot Created on: 27/Apr/22 19:17 Start Date: 27/Apr/22 19:17 Worklog Time Spent: 10m Work Description: mukund-thakur commented on PR #4171: URL: https://github.com/apache/hadoop/pull/4171#issuecomment-386850 Yetus still failing with unit tests. Please fix those https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4171/3/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt Issue Time Tracking --- Worklog Id: (was: 763122) Time Spent: 3h 40m (was: 3.5h) > Introduce StreamContext for Abfs Input and Output streams. > -- > > Key: HADOOP-16965 > URL: https://issues.apache.org/jira/browse/HADOOP-16965 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/azure >Reporter: Mukund Thakur >Assignee: Mukund Thakur >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3h 40m > Remaining Estimate: 0h > > The number of configuration keeps growing in AbfsOutputStream and > AbfsInputStream as we keep on adding new features. It is time to refactor the > configurations in a separate class like StreamContext and pass them around. > This is will improve the readability of code and reduce cherry-pick-backport > pain. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on pull request #4171: HADOOP-16965. Refactor abfs stream configuration. (#1956)
mukund-thakur commented on PR #4171: URL: https://github.com/apache/hadoop/pull/4171#issuecomment-386850 Yetus still failing with unit tests. Please fix those https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4171/3/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4228: HDFS-16468. Define ssize_t for Windows
hadoop-yetus commented on PR #4228:
URL: https://github.com/apache/hadoop/pull/4228#issuecomment-377475
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 36s | | trunk passed |
| +1 :green_heart: | compile | 4m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 69m 13s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 69m 36s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 19s | | the patch passed |
| +1 :green_heart: | compile | 3m 45s | | the patch passed |
| +1 :green_heart: | cc | 3m 45s | | the patch passed |
| +1 :green_heart: | golang | 3m 45s | | the patch passed |
| +1 :green_heart: | javac | 3m 45s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 22s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 108m 11s | | hadoop-hdfs-native-client in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not
generate ASF License warnings. |
| | | 208m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4228 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux 02f10f6bd5d4 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 93590cc528eab92c2e0c3da9b04f812c9bcc4ee5 |
| Default Java | Red Hat, Inc.-1.8.0_322-b06 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/testReport/ |
| Max. process+thread count | 618 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/6/console |
| versions | git=2.9.5 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16965) Introduce StreamContext for Abfs Input and Output streams.
[ https://issues.apache.org/jira/browse/HADOOP-16965?focusedWorklogId=763118=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763118 ] ASF GitHub Bot logged work on HADOOP-16965: --- Author: ASF GitHub Bot Created on: 27/Apr/22 19:00 Start Date: 27/Apr/22 19:00 Worklog Time Spent: 10m Work Description: virajith commented on PR #4171: URL: https://github.com/apache/hadoop/pull/4171#issuecomment-371455 Thanks for checking the skipped tests and confirming @arjun4084346! @mukund-thakur / @steveloughran unless you have any objections, I'd like to commit this. Issue Time Tracking --- Worklog Id: (was: 763118) Time Spent: 3.5h (was: 3h 20m) > Introduce StreamContext for Abfs Input and Output streams. > -- > > Key: HADOOP-16965 > URL: https://issues.apache.org/jira/browse/HADOOP-16965 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/azure >Reporter: Mukund Thakur >Assignee: Mukund Thakur >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > The number of configuration keeps growing in AbfsOutputStream and > AbfsInputStream as we keep on adding new features. It is time to refactor the > configurations in a separate class like StreamContext and pass them around. > This is will improve the readability of code and reduce cherry-pick-backport > pain. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajith commented on pull request #4171: HADOOP-16965. Refactor abfs stream configuration. (#1956)
virajith commented on PR #4171: URL: https://github.com/apache/hadoop/pull/4171#issuecomment-371455 Thanks for checking the skipped tests and confirming @arjun4084346! @mukund-thakur / @steveloughran unless you have any objections, I'd like to commit this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4240: HDFS-16562. Upgrade moment.min.js to 2.29.2
hadoop-yetus commented on PR #4240:
URL: https://github.com/apache/hadoop/pull/4240#issuecomment-350767
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | jshint | 0m 0s | | jshint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 46s | | trunk passed |
| +1 :green_heart: | shadedclient | 58m 34s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | shadedclient | 19m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 53s | | The patch does not
generate ASF License warnings. |
| | | 82m 9s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4240/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4240 |
| Optional Tests | dupname asflicense shadedclient codespell jshint |
| uname | Linux 1a91cf2a8f95 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / be860c71a2cdeae5df17ca7ecab0e1f223e1b406 |
| Max. process+thread count | 547 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4240/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16202) Enhance openFile() for better read performance against object stores
[
https://issues.apache.org/jira/browse/HADOOP-16202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528961#comment-17528961
]
Steve Loughran commented on HADOOP-16202:
-
this is in branch&3.3.
going to reopen and pull out the distcp subtask, as this pr doesn't yet fix
distcp to use the passed in file status when opening files or set the read
policy. doing that would slightly speed up s3a file opening, and by fixing the
read policy, ensure distcp was performant even if the cluster wide settings was
for random io
> Enhance openFile() for better read performance against object stores
> -
>
> Key: HADOOP-16202
> URL: https://issues.apache.org/jira/browse/HADOOP-16202
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, fs/s3, tools/distcp
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.4
>
> Time Spent: 22.5h
> Remaining Estimate: 0h
>
> The {{openFile()}} builder API lets us add new options when reading a file
> Add an option {{"fs.s3a.open.option.length"}} which takes a long and allows
> the length of the file to be declared. If set, *no check for the existence of
> the file is issued when opening the file*
> Also: withFileStatus() to take any FileStatus implementation, rather than
> only S3AFileStatus -and not check that the path matches the path being
> opened. Needed to support viewFS-style wrapping and mounting.
> and Adopt where appropriate to stop clusters with S3A reads switched to
> random IO from killing download/localization
> * fs shell copyToLocal
> * distcp
> * IOUtils.copy
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16202) Enhance openFile() for better read performance against object stores
[
https://issues.apache.org/jira/browse/HADOOP-16202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-16202:
Fix Version/s: 3.3.4
> Enhance openFile() for better read performance against object stores
> -
>
> Key: HADOOP-16202
> URL: https://issues.apache.org/jira/browse/HADOOP-16202
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, fs/s3, tools/distcp
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.4
>
> Time Spent: 22.5h
> Remaining Estimate: 0h
>
> The {{openFile()}} builder API lets us add new options when reading a file
> Add an option {{"fs.s3a.open.option.length"}} which takes a long and allows
> the length of the file to be declared. If set, *no check for the existence of
> the file is issued when opening the file*
> Also: withFileStatus() to take any FileStatus implementation, rather than
> only S3AFileStatus -and not check that the path matches the path being
> opened. Needed to support viewFS-style wrapping and mounting.
> and Adopt where appropriate to stop clusters with S3A reads switched to
> random IO from killing download/localization
> * fs shell copyToLocal
> * distcp
> * IOUtils.copy
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16202) Enhance openFile() for better read performance against object stores
[
https://issues.apache.org/jira/browse/HADOOP-16202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-16202:
Issue Type: Improvement (was: Bug)
> Enhance openFile() for better read performance against object stores
> -
>
> Key: HADOOP-16202
> URL: https://issues.apache.org/jira/browse/HADOOP-16202
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, fs/s3, tools/distcp
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.4
>
> Time Spent: 22.5h
> Remaining Estimate: 0h
>
> The {{openFile()}} builder API lets us add new options when reading a file
> Add an option {{"fs.s3a.open.option.length"}} which takes a long and allows
> the length of the file to be declared. If set, *no check for the existence of
> the file is issued when opening the file*
> Also: withFileStatus() to take any FileStatus implementation, rather than
> only S3AFileStatus -and not check that the path matches the path being
> opened. Needed to support viewFS-style wrapping and mounting.
> and Adopt where appropriate to stop clusters with S3A reads switched to
> random IO from killing download/localization
> * fs shell copyToLocal
> * distcp
> * IOUtils.copy
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters
[ https://issues.apache.org/jira/browse/HADOOP-17682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17682. - Resolution: Fixed > ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters > -- > > Key: HADOOP-17682 > URL: https://issues.apache.org/jira/browse/HADOOP-17682 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Time Spent: 8h 10m > Remaining Estimate: 0h > > ABFS open methods require certain information (contentLength, eTag, etc) to > to create an InputStream for the file at the given path. This information is > retrieved via a GetFileStatus request to backend. > However, client applications may often have access to the FileStatus prior to > invoking the open API. Providing this FileStatus to the driver through the > OpenFileParameters argument of openFileWithOptions() can help avoid the call > to Store for FileStatus. > This PR adds handling for the FileStatus instance (if any) provided via the > OpenFileParameters argument. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters
[ https://issues.apache.org/jira/browse/HADOOP-17682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17682: Fix Version/s: 3.3.4 > ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters > -- > > Key: HADOOP-17682 > URL: https://issues.apache.org/jira/browse/HADOOP-17682 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Time Spent: 8h 10m > Remaining Estimate: 0h > > ABFS open methods require certain information (contentLength, eTag, etc) to > to create an InputStream for the file at the given path. This information is > retrieved via a GetFileStatus request to backend. > However, client applications may often have access to the FileStatus prior to > invoking the open API. Providing this FileStatus to the driver through the > OpenFileParameters argument of openFileWithOptions() can help avoid the call > to Store for FileStatus. > This PR adds handling for the FileStatus instance (if any) provided via the > OpenFileParameters argument. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jclarysse commented on pull request #4049: Hadoop 18154 branch 2.10.1
jclarysse commented on PR #4049: URL: https://github.com/apache/hadoop/pull/4049#issuecomment-336256 PR replaced by https://github.com/apache/hadoop/pull/4070 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jclarysse closed pull request #4049: Hadoop 18154 branch 2.10.1
jclarysse closed pull request #4049: Hadoop 18154 branch 2.10.1 URL: https://github.com/apache/hadoop/pull/4049 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] prasad-acit commented on pull request #4241: HDFS-16563. Namenode WebUI prints sensitive information on Token expiry
prasad-acit commented on PR #4241: URL: https://github.com/apache/hadoop/pull/4241#issuecomment-316742 @hemanthboyina @Hexiaoqiao Can you please review the PR when you get time? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] prasad-acit opened a new pull request, #4241: HDFS-16563. Namenode WebUI prints sensitive information on Token expiry
prasad-acit opened a new pull request, #4241: URL: https://github.com/apache/hadoop/pull/4241 ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dmmkr opened a new pull request, #4240: HDFS-16562. Upgrade moment.min.js to 2.29.2
dmmkr opened a new pull request, #4240: URL: https://github.com/apache/hadoop/pull/4240 ### Description of PR Upgrade moment.min.js to 2.29.2 to resolve https://nvd.nist.gov/vuln/detail/CVE-2022-24785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16202) Enhance openFile() for better read performance against object stores
[
https://issues.apache.org/jira/browse/HADOOP-16202?focusedWorklogId=763044=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763044
]
ASF GitHub Bot logged work on HADOOP-16202:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 16:50
Start Date: 27/Apr/22 16:50
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4238:
URL: https://github.com/apache/hadoop/pull/4238#issuecomment-234099
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 10m 15s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 23 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +0 :ok: | mvndep | 14m 55s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 52s | | branch-3.3 passed |
| +1 :green_heart: | compile | 18m 45s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 3m 21s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 10m 43s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 9m 48s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 15m 21s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 27m 13s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 11s | | the patch passed |
| +1 :green_heart: | compile | 18m 34s | | the patch passed |
| -1 :x: | javac | 18m 34s |
[/results-compile-javac-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4238/1/artifact/out/results-compile-javac-root.txt)
| root generated 1 new + 1926 unchanged - 0 fixed = 1927 total (was 1926) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 12s | | root: The patch generated
0 new + 696 unchanged - 2 fixed = 696 total (was 698) |
| +1 :green_heart: | mvnsite | 10m 49s | | the patch passed |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 1m 57s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | javadoc | 1m 21s | | hadoop-yarn-common in the
patch passed. |
| +1 :green_heart: | javadoc | 0m 51s | | hadoop-mapreduce-client-core
in the patch passed. |
| +1 :green_heart: | javadoc | 0m 55s | | hadoop-mapreduce-client-app
in the patch passed. |
| +1 :green_heart: | javadoc | 0m 53s | | hadoop-mapreduce-examples in
the patch passed. |
| +1 :green_heart: | javadoc | 0m 52s | | hadoop-streaming in the
patch passed. |
| +1 :green_heart: | javadoc | 0m 51s | | hadoop-distcp in the patch
passed. |
| +1 :green_heart: | javadoc | 0m 57s | | hadoop-tools_hadoop-aws
generated 0 new + 38 unchanged - 1 fixed = 38 total (was 39) |
| +1 :green_heart: | javadoc | 0m 55s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | spotbugs | 17m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 28m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 46s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 5m 3s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 6m 30s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 8m 47s | | hadoop-mapreduce-client-app in
the patch passed. |
| +1 :green_heart: | unit | 1m 13s | | hadoop-mapreduce-examples in
the patch passed. |
| +1 :green_heart: | unit | 6m 53s | | hadoop-streaming in the patch
passed. |
| +1 :green_heart: | unit | 15m 40s | | hadoop-distcp in the patch
passed. |
| +1 :green_heart: | unit | 2m 39s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 38s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 18s | | The patch does not
generate ASF License warnings. |
| | | 304m 3s | | |
| Subsystem | Report/Notes |
[GitHub] [hadoop] hadoop-yetus commented on pull request #4238: HADOOP-16202. Enhanced openFile() -branch-3.3 backport
hadoop-yetus commented on PR #4238:
URL: https://github.com/apache/hadoop/pull/4238#issuecomment-234099
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 10m 15s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 23 new or modified test files. |
_ branch-3.3 Compile Tests _ |
| +0 :ok: | mvndep | 14m 55s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 52s | | branch-3.3 passed |
| +1 :green_heart: | compile | 18m 45s | | branch-3.3 passed |
| +1 :green_heart: | checkstyle | 3m 21s | | branch-3.3 passed |
| +1 :green_heart: | mvnsite | 10m 43s | | branch-3.3 passed |
| +1 :green_heart: | javadoc | 9m 48s | | branch-3.3 passed |
| +1 :green_heart: | spotbugs | 15m 21s | | branch-3.3 passed |
| +1 :green_heart: | shadedclient | 27m 13s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 6m 11s | | the patch passed |
| +1 :green_heart: | compile | 18m 34s | | the patch passed |
| -1 :x: | javac | 18m 34s |
[/results-compile-javac-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4238/1/artifact/out/results-compile-javac-root.txt)
| root generated 1 new + 1926 unchanged - 0 fixed = 1927 total (was 1926) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 12s | | root: The patch generated
0 new + 696 unchanged - 2 fixed = 696 total (was 698) |
| +1 :green_heart: | mvnsite | 10m 49s | | the patch passed |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 1m 57s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | javadoc | 1m 21s | | hadoop-yarn-common in the
patch passed. |
| +1 :green_heart: | javadoc | 0m 51s | | hadoop-mapreduce-client-core
in the patch passed. |
| +1 :green_heart: | javadoc | 0m 55s | | hadoop-mapreduce-client-app
in the patch passed. |
| +1 :green_heart: | javadoc | 0m 53s | | hadoop-mapreduce-examples in
the patch passed. |
| +1 :green_heart: | javadoc | 0m 52s | | hadoop-streaming in the
patch passed. |
| +1 :green_heart: | javadoc | 0m 51s | | hadoop-distcp in the patch
passed. |
| +1 :green_heart: | javadoc | 0m 57s | | hadoop-tools_hadoop-aws
generated 0 new + 38 unchanged - 1 fixed = 38 total (was 39) |
| +1 :green_heart: | javadoc | 0m 55s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | spotbugs | 17m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 28m 7s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 46s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 5m 3s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 6m 30s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 8m 47s | | hadoop-mapreduce-client-app in
the patch passed. |
| +1 :green_heart: | unit | 1m 13s | | hadoop-mapreduce-examples in
the patch passed. |
| +1 :green_heart: | unit | 6m 53s | | hadoop-streaming in the patch
passed. |
| +1 :green_heart: | unit | 15m 40s | | hadoop-distcp in the patch
passed. |
| +1 :green_heart: | unit | 2m 39s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 38s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 18s | | The patch does not
generate ASF License warnings. |
| | | 304m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4238/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4238 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint xml |
| uname | Linux 8bcdd5b97190
[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged
[
https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=763040=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763040
]
ASF GitHub Bot logged work on HADOOP-18168:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 16:43
Start Date: 27/Apr/22 16:43
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4140:
URL: https://github.com/apache/hadoop/pull/4140#issuecomment-223043
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 56s | | trunk passed |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 1s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 20s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 36s | | the patch passed |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 23s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 94m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4140 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell xml |
| uname | Linux 593ad08f3277 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1692c71204dcbb340d35ed561991cf680864885a |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/7/testReport/ |
| Max. process+thread count | 546 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U:
[GitHub] [hadoop] hadoop-yetus commented on pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dependency on purged public bucket
hadoop-yetus commented on PR #4140:
URL: https://github.com/apache/hadoop/pull/4140#issuecomment-223043
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 56s | | trunk passed |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 34s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 18s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 1s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 20s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 38s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 38s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 20s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 36s | | the patch passed |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 10s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 23s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 40s | | The patch does not
generate ASF License warnings. |
| | | 94m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4140 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell xml |
| uname | Linux 593ad08f3277 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1692c71204dcbb340d35ed561991cf680864885a |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/7/testReport/ |
| Max. process+thread count | 546 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/7/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use
[GitHub] [hadoop] saintstack commented on a diff in pull request #4107: HDFS-16521. DFS API to retrieve slow datanodes
saintstack commented on code in PR #4107:
URL: https://github.com/apache/hadoop/pull/4107#discussion_r860008767
##
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md:
##
@@ -394,7 +394,7 @@ Usage:
| COMMAND\_OPTION | Description |
|: |: |
-| `-report` `[-live]` `[-dead]` `[-decommissioning]` `[-enteringmaintenance]`
`[-inmaintenance]` | Reports basic filesystem information and statistics, The
dfs usage can be different from "du" usage, because it measures raw space used
by replication, checksums, snapshots and etc. on all the DNs. Optional flags
may be used to filter the list of displayed DataNodes. |
+| `-report` `[-live]` `[-dead]` `[-decommissioning]` `[-enteringmaintenance]`
`[-inmaintenance]` `[-slownodes]` | Reports basic filesystem information and
statistics, The dfs usage can be different from "du" usage, because it measures
raw space used by replication, checksums, snapshots and etc. on all the DNs.
Optional flags may be used to filter the list of displayed DataNodes. Filters
are either based on the DN state (e.g. live, dead, decommissioning) or the
nature of the DN (e.g. slow nodes). |
Review Comment:
Where do we explain a 'slownode' is?
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodePeerMetrics.java:
##
@@ -142,14 +144,28 @@ public void collectThreadLocalStates() {
* than their peers.
*/
public Map getOutliers() {
-// This maps the metric name to the aggregate latency.
-// The metric name is the datanode ID.
-final Map stats =
-sendPacketDownstreamRollingAverages.getStats(
-minOutlierDetectionSamples);
-LOG.trace("DataNodePeerMetrics: Got stats: {}", stats);
-
-return slowNodeDetector.getOutliers(stats);
+// outlier must be null for source code.
+if (testOutlier == null) {
+ // This maps the metric name to the aggregate latency.
+ // The metric name is the datanode ID.
+ final Map stats =
+
sendPacketDownstreamRollingAverages.getStats(minOutlierDetectionSamples);
+ LOG.trace("DataNodePeerMetrics: Got stats: {}", stats);
+ return slowNodeDetector.getOutliers(stats);
+} else {
+ // this happens only for test code.
+ return testOutlier;
+}
+ }
+
+ /**
+ * Strictly to be used by test code only. Source code is not supposed to use
this. This method
+ * directly sets outlier mapping so that aggregate latency metrics are not
calculated for tests.
+ *
+ * @param outlier outlier directly set by tests.
+ */
+ public void setTestOutliers(Map outlier) {
Review Comment:
This is a little awkward? Add comment on the testOutlier data member that it
is for test only?
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java:
##
@@ -632,6 +638,20 @@ private static void
printDataNodeReports(DistributedFileSystem dfs,
}
}
+ private static void printSlowDataNodeReports(DistributedFileSystem dfs,
boolean listNodes,
Review Comment:
One comment on the slow datanode report is that it seems to say nothing
about why the NN thinks it slow; it does not note the attributes that are in
excess of normal. Should it? (Hard to do when this info is not part of
DatanodeInfo) For example, say something about how in excess a DNs latency is?
(Perhaps this could be added later)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18198) Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
[ https://issues.apache.org/jira/browse/HADOOP-18198?focusedWorklogId=763032=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763032 ] ASF GitHub Bot logged work on HADOOP-18198: --- Author: ASF GitHub Bot Created on: 27/Apr/22 16:31 Start Date: 27/Apr/22 16:31 Worklog Time Spent: 10m Work Description: steveloughran merged PR #4239: URL: https://github.com/apache/hadoop/pull/4239 Issue Time Tracking --- Worklog Id: (was: 763032) Time Spent: 2h 10m (was: 2h) > Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes > -- > > Key: HADOOP-18198 > URL: https://issues.apache.org/jira/browse/HADOOP-18198 > Project: Hadoop Common > Issue Type: Task > Components: build >Affects Versions: 3.3.2 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > Hadoop 3.3.3 is a minor followup release to Hadoop 3.3.2 with all the > incremental changes which went in to the 3.2.4 release > * minor CVE fixes in Hadoop source > * CVE fixes in dependencies we know of (protobuf unmarshalling leading to > DoS, jackson stack overflow,...) > * replacement of log4j 1.2.17 to reload4j > * node.js update > This is not a release off branch-3.3, it is a fork of 3.3.2 with the changes. > The next release of branch-3.3 will be numbered hadoop-3.3.4; updating maven > versions and JIRA fix versions is part of this release process. > The changes here are already in branch 3.2.4; this completes the set > CVEs fixed > * CVE-2022-26612: Apache Hadoop: Arbitrary file write in > FileUtil#unpackEntries on Windows (HADOOP-18155) -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran merged pull request #4239: HADOOP-18198. add -mvnargs option to create-release command line
steveloughran merged PR #4239: URL: https://github.com/apache/hadoop/pull/4239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18198) Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
[ https://issues.apache.org/jira/browse/HADOOP-18198?focusedWorklogId=763028=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763028 ] ASF GitHub Bot logged work on HADOOP-18198: --- Author: ASF GitHub Bot Created on: 27/Apr/22 16:30 Start Date: 27/Apr/22 16:30 Worklog Time Spent: 10m Work Description: steveloughran commented on PR #4239: URL: https://github.com/apache/hadoop/pull/4239#issuecomment-210333 this works! ``` Hadoop Release Creator Version to create : 3.3.3 Release Candidate Label: Source Version : 3.3.3 Executed in 76.75 mins fish external usr time 363.10 millis0.00 micros 363.10 millis sys time 82.97 millis 471.00 micros 82.50 millis ``` Issue Time Tracking --- Worklog Id: (was: 763028) Time Spent: 2h (was: 1h 50m) > Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes > -- > > Key: HADOOP-18198 > URL: https://issues.apache.org/jira/browse/HADOOP-18198 > Project: Hadoop Common > Issue Type: Task > Components: build >Affects Versions: 3.3.2 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > Hadoop 3.3.3 is a minor followup release to Hadoop 3.3.2 with all the > incremental changes which went in to the 3.2.4 release > * minor CVE fixes in Hadoop source > * CVE fixes in dependencies we know of (protobuf unmarshalling leading to > DoS, jackson stack overflow,...) > * replacement of log4j 1.2.17 to reload4j > * node.js update > This is not a release off branch-3.3, it is a fork of 3.3.2 with the changes. > The next release of branch-3.3 will be numbered hadoop-3.3.4; updating maven > versions and JIRA fix versions is part of this release process. > The changes here are already in branch 3.2.4; this completes the set > CVEs fixed > * CVE-2022-26612: Apache Hadoop: Arbitrary file write in > FileUtil#unpackEntries on Windows (HADOOP-18155) -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #4239: HADOOP-18198. add -mvnargs option to create-release command line
steveloughran commented on PR #4239: URL: https://github.com/apache/hadoop/pull/4239#issuecomment-210333 this works! ``` Hadoop Release Creator Version to create : 3.3.3 Release Candidate Label: Source Version : 3.3.3 Executed in 76.75 mins fish external usr time 363.10 millis0.00 micros 363.10 millis sys time 82.97 millis 471.00 micros 82.50 millis ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged
[
https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=763023=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-763023
]
ASF GitHub Bot logged work on HADOOP-18168:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 16:20
Start Date: 27/Apr/22 16:20
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4140:
URL: https://github.com/apache/hadoop/pull/4140#issuecomment-199931
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 45s | | trunk passed |
| +1 :green_heart: | compile | 1m 0s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 55s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 2s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 29s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 42s | | the patch passed |
| +1 :green_heart: | compile | 0m 44s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 44s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 27s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/6/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 1 unchanged - 0 fixed
= 2 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 43s | | the patch passed |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 13s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 24s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 42s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 98m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4140 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell xml |
| uname | Linux 0e78edaed65a 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 79710a1d5e3946c9092d1826c72a13a1dbf4 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
[GitHub] [hadoop] hadoop-yetus commented on pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dependency on purged public bucket
hadoop-yetus commented on PR #4140:
URL: https://github.com/apache/hadoop/pull/4140#issuecomment-199931
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 45s | | trunk passed |
| +1 :green_heart: | compile | 1m 0s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 55s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 52s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 35s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 2s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 29s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 42s | | the patch passed |
| +1 :green_heart: | compile | 0m 44s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 44s | | the patch passed |
| +1 :green_heart: | compile | 0m 39s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 39s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 27s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/6/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 1 unchanged - 0 fixed
= 2 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 43s | | the patch passed |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 13s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 24s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 42s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 98m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4140 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell xml |
| uname | Linux 0e78edaed65a 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 79710a1d5e3946c9092d1826c72a13a1dbf4 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/6/testReport/ |
| Max. process+thread count | 673 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
|
[GitHub] [hadoop] GauthamBanasandra commented on a diff in pull request #4228: HDFS-16468. Define ssize_t for Windows
GauthamBanasandra commented on code in PR #4228: URL: https://github.com/apache/hadoop/pull/4228#discussion_r859962108 ## hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/x-platform/types.h: ## @@ -0,0 +1,39 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#ifndef NATIVE_LIBHDFSPP_LIB_CROSS_PLATFORM_TYPES +#define NATIVE_LIBHDFSPP_LIB_CROSS_PLATFORM_TYPES + +#if _WIN32 || _WIN64 Review Comment: Done. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18198) Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
[
https://issues.apache.org/jira/browse/HADOOP-18198?focusedWorklogId=762980=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762980
]
ASF GitHub Bot logged work on HADOOP-18198:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 15:33
Start Date: 27/Apr/22 15:33
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4239:
URL: https://github.com/apache/hadoop/pull/4239#issuecomment-148667
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 11m 3s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ branch-3.3.3 Compile Tests _ |
| +0 :ok: | mvndep | 5m 19s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 27m 51s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 20s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | shellcheck | 0m 2s | | No new issues. |
| +1 :green_heart: | shadedclient | 22m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 69m 59s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4239/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4239 |
| Optional Tests | dupname asflicense codespell shellcheck shelldocs |
| uname | Linux 9c187fadd132 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3.3 / efb097357e0070cf261fec6521fc2613145bb5ff |
| Max. process+thread count | 695 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4239/1/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
Issue Time Tracking
---
Worklog Id: (was: 762980)
Time Spent: 1h 50m (was: 1h 40m)
> Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
> --
>
> Key: HADOOP-18198
> URL: https://issues.apache.org/jira/browse/HADOOP-18198
> Project: Hadoop Common
> Issue Type: Task
> Components: build
>Affects Versions: 3.3.2
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Hadoop 3.3.3 is a minor followup release to Hadoop 3.3.2 with all the
> incremental changes which went in to the 3.2.4 release
> * minor CVE fixes in Hadoop source
> * CVE fixes in dependencies we know of (protobuf unmarshalling leading to
> DoS, jackson stack overflow,...)
> * replacement of log4j 1.2.17 to reload4j
> * node.js update
> This is not a release off branch-3.3, it is a fork of 3.3.2 with the changes.
> The next release of branch-3.3 will be numbered hadoop-3.3.4; updating maven
> versions and JIRA fix versions is part of this release process.
> The changes here are already in branch 3.2.4; this completes the set
> CVEs fixed
> * CVE-2022-26612: Apache Hadoop: Arbitrary file write in
> FileUtil#unpackEntries on Windows (HADOOP-18155)
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4239: HADOOP-18198. add -mvnargs option to create-release command line
hadoop-yetus commented on PR #4239:
URL: https://github.com/apache/hadoop/pull/4239#issuecomment-148667
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 11m 3s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
_ branch-3.3.3 Compile Tests _ |
| +0 :ok: | mvndep | 5m 19s | | Maven dependency ordering for branch |
| +1 :green_heart: | shadedclient | 27m 51s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 20s | | Maven dependency ordering for patch |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | shellcheck | 0m 2s | | No new issues. |
| +1 :green_heart: | shadedclient | 22m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 69m 59s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4239/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4239 |
| Optional Tests | dupname asflicense codespell shellcheck shelldocs |
| uname | Linux 9c187fadd132 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.3.3 / efb097357e0070cf261fec6521fc2613145bb5ff |
| Max. process+thread count | 695 (vs. ulimit of 5500) |
| modules | C: U: |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4239/1/console |
| versions | git=2.17.1 maven=3.6.0 shellcheck=0.4.6 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18187) Convert s3a prefetching to use JavaDoc for fields and enums
[ https://issues.apache.org/jira/browse/HADOOP-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528853#comment-17528853 ] Bhalchandra Pandit commented on HADOOP-18187: - Thanks for the links. I will take care of the comments. > Convert s3a prefetching to use JavaDoc for fields and enums > --- > > Key: HADOOP-18187 > URL: https://issues.apache.org/jira/browse/HADOOP-18187 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Daniel Carl Jones >Assignee: Bhalchandra Pandit >Priority: Minor > > There's lots of good comments for fields and enum values in the current code. > Let's expose these to your IDE with JavaDoc instead. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] saintstack commented on pull request #4170: HDFS-16540 Data locality is lost when DataNode pod restarts in kubern…
saintstack commented on PR #4170: URL: https://github.com/apache/hadoop/pull/4170#issuecomment-106375 I'll merge later today (unless someone else beats me to it). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged
[
https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=762940=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762940
]
ASF GitHub Bot logged work on HADOOP-18168:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 14:55
Start Date: 27/Apr/22 14:55
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4140:
URL: https://github.com/apache/hadoop/pull/4140#issuecomment-104577
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 20s | | trunk passed |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 15s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 35s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/5/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 1 unchanged - 0 fixed
= 2 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 35s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 24s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 94m 45s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4140 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell xml |
| uname | Linux 3f7569ed9f48 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e734caabaaa94a387891d721f08f54a342f7b250 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
[GitHub] [hadoop] hadoop-yetus commented on pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dependency on purged public bucket
hadoop-yetus commented on PR #4140:
URL: https://github.com/apache/hadoop/pull/4140#issuecomment-104577
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 20s | | trunk passed |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 45s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 15s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 20m 35s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 36s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/5/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 1 unchanged - 0 fixed
= 2 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 35s | | the patch passed |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 0m 18s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 35s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 24s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 94m 45s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4140 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell xml |
| uname | Linux 3f7569ed9f48 4.15.0-169-generic #177-Ubuntu SMP Thu Feb 3
10:50:38 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e734caabaaa94a387891d721f08f54a342f7b250 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/5/testReport/ |
| Max. process+thread count | 693 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4140/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
|
[GitHub] [hadoop] skysiders commented on pull request #4237: MAPREDUCE-7375 JobSubmissionFiles don't set right permission after mkdirs
skysiders commented on PR #4237: URL: https://github.com/apache/hadoop/pull/4237#issuecomment-088682 Hi,@eddyxu @phunt ,could you please take a look? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18216) Document "io.file.buffer.size" must be greater than zero
[
https://issues.apache.org/jira/browse/HADOOP-18216?focusedWorklogId=762919=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762919
]
ASF GitHub Bot logged work on HADOOP-18216:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 14:37
Start Date: 27/Apr/22 14:37
Worklog Time Spent: 10m
Work Description: steveloughran commented on PR #4220:
URL: https://github.com/apache/hadoop/pull/4220#issuecomment-084200
oh, and the ioutil code would be easy to test, wouldn't it, pass in null
stream values and a negative offset and expect the precondition failures, not
any null pointer exception
Issue Time Tracking
---
Worklog Id: (was: 762919)
Time Spent: 1h 50m (was: 1h 40m)
> Document "io.file.buffer.size" must be greater than zero
>
>
> Key: HADOOP-18216
> URL: https://issues.apache.org/jira/browse/HADOOP-18216
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Jingxuan Fu
>Assignee: Jingxuan Fu
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> when the configuration file in the "io.file.buffer.size" field is set to a
> value less than or equal to zero, hdfs can start normally, but read and write
> data will have problems.
> When the value is less than zero, the shell will throw the following
> exception:
> {code:java}
> hadoop@ljq1:~/hadoop-3.1.3-work/bin$ ./hdfs dfs -cat mapred
> -cat: Fatal internal error
> java.lang.NegativeArraySizeException: -4096
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:93)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:68)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:129)
> at
> org.apache.hadoop.fs.shell.Display$Cat.printToStdout(Display.java:101)
> at org.apache.hadoop.fs.shell.Display$Cat.processPath(Display.java:96)
> at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:331)
> at
> org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:303)
> at
> org.apache.hadoop.fs.shell.Command.processArgument(Command.java:285)
> at
> org.apache.hadoop.fs.shell.Command.processArguments(Command.java:269)
> at
> org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
> at org.apache.hadoop.fs.shell.Command.run(Command.java:176)
> at org.apache.hadoop.fs.FsShell.run(FsShell.java:328)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:391){code}
> When the value is equal to zero, the shell command will always block
> {code:java}
> hadoop@ljq1:~/hadoop-3.1.3-work/bin$ ./hdfs dfs -cat mapred
> ^Z
> [2]+ Stopped ./hdfs dfs -cat mapred{code}
> The description of the configuration file is not clear enough, it may make
> people think that set to 0 to enter the non-blocking mode.
>
> {code:java}
>
> io.file.buffer.size
> 4096
> The size of buffer for use in sequence files.
> The size of this buffer should probably be a multiple of hardware
> page size (4096 on Intel x86), and it determines how much data is
> buffered during read and write operations.
> {code}
>
> Considering that this value is uesd by hdfs and mapreduce frequently, we
> should make this value must be a number greater than zero.
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #4220: HADOOP-18216. io.file.buffer.size must be greater than zero
steveloughran commented on PR #4220: URL: https://github.com/apache/hadoop/pull/4220#issuecomment-084200 oh, and the ioutil code would be easy to test, wouldn't it, pass in null stream values and a negative offset and expect the precondition failures, not any null pointer exception -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18216) Document "io.file.buffer.size" must be greater than zero
[
https://issues.apache.org/jira/browse/HADOOP-18216?focusedWorklogId=762918=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762918
]
ASF GitHub Bot logged work on HADOOP-18216:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 14:36
Start Date: 27/Apr/22 14:36
Worklog Time Spent: 10m
Work Description: steveloughran commented on PR #4220:
URL: https://github.com/apache/hadoop/pull/4220#issuecomment-082405
could have some private method in ioutils to get and validate the value, use
from both places it is called. filesystem could have its own copy of the check
Issue Time Tracking
---
Worklog Id: (was: 762918)
Time Spent: 1h 40m (was: 1.5h)
> Document "io.file.buffer.size" must be greater than zero
>
>
> Key: HADOOP-18216
> URL: https://issues.apache.org/jira/browse/HADOOP-18216
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Jingxuan Fu
>Assignee: Jingxuan Fu
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> when the configuration file in the "io.file.buffer.size" field is set to a
> value less than or equal to zero, hdfs can start normally, but read and write
> data will have problems.
> When the value is less than zero, the shell will throw the following
> exception:
> {code:java}
> hadoop@ljq1:~/hadoop-3.1.3-work/bin$ ./hdfs dfs -cat mapred
> -cat: Fatal internal error
> java.lang.NegativeArraySizeException: -4096
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:93)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:68)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:129)
> at
> org.apache.hadoop.fs.shell.Display$Cat.printToStdout(Display.java:101)
> at org.apache.hadoop.fs.shell.Display$Cat.processPath(Display.java:96)
> at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:331)
> at
> org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:303)
> at
> org.apache.hadoop.fs.shell.Command.processArgument(Command.java:285)
> at
> org.apache.hadoop.fs.shell.Command.processArguments(Command.java:269)
> at
> org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
> at org.apache.hadoop.fs.shell.Command.run(Command.java:176)
> at org.apache.hadoop.fs.FsShell.run(FsShell.java:328)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:391){code}
> When the value is equal to zero, the shell command will always block
> {code:java}
> hadoop@ljq1:~/hadoop-3.1.3-work/bin$ ./hdfs dfs -cat mapred
> ^Z
> [2]+ Stopped ./hdfs dfs -cat mapred{code}
> The description of the configuration file is not clear enough, it may make
> people think that set to 0 to enter the non-blocking mode.
>
> {code:java}
>
> io.file.buffer.size
> 4096
> The size of buffer for use in sequence files.
> The size of this buffer should probably be a multiple of hardware
> page size (4096 on Intel x86), and it determines how much data is
> buffered during read and write operations.
> {code}
>
> Considering that this value is uesd by hdfs and mapreduce frequently, we
> should make this value must be a number greater than zero.
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #4220: HADOOP-18216. io.file.buffer.size must be greater than zero
steveloughran commented on PR #4220: URL: https://github.com/apache/hadoop/pull/4220#issuecomment-082405 could have some private method in ioutils to get and validate the value, use from both places it is called. filesystem could have its own copy of the check -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18198) Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
[ https://issues.apache.org/jira/browse/HADOOP-18198?focusedWorklogId=762907=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762907 ] ASF GitHub Bot logged work on HADOOP-18198: --- Author: ASF GitHub Bot Created on: 27/Apr/22 14:22 Start Date: 27/Apr/22 14:22 Worklog Time Spent: 10m Work Description: steveloughran opened a new pull request, #4239: URL: https://github.com/apache/hadoop/pull/4239 This allows for release builds to be run with options like --mvnargs="-Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false" Contributed by Ayush Saxena. Issue Time Tracking --- Worklog Id: (was: 762907) Time Spent: 1h 40m (was: 1.5h) > Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes > -- > > Key: HADOOP-18198 > URL: https://issues.apache.org/jira/browse/HADOOP-18198 > Project: Hadoop Common > Issue Type: Task > Components: build >Affects Versions: 3.3.2 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Hadoop 3.3.3 is a minor followup release to Hadoop 3.3.2 with all the > incremental changes which went in to the 3.2.4 release > * minor CVE fixes in Hadoop source > * CVE fixes in dependencies we know of (protobuf unmarshalling leading to > DoS, jackson stack overflow,...) > * replacement of log4j 1.2.17 to reload4j > * node.js update > This is not a release off branch-3.3, it is a fork of 3.3.2 with the changes. > The next release of branch-3.3 will be numbered hadoop-3.3.4; updating maven > versions and JIRA fix versions is part of this release process. > The changes here are already in branch 3.2.4; this completes the set > CVEs fixed > * CVE-2022-26612: Apache Hadoop: Arbitrary file write in > FileUtil#unpackEntries on Windows (HADOOP-18155) -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran opened a new pull request, #4239: HADOOP-18198. add -mvnargs option to create-release command line
steveloughran opened a new pull request, #4239: URL: https://github.com/apache/hadoop/pull/4239 This allows for release builds to be run with options like --mvnargs="-Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false" Contributed by Ayush Saxena. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18198) Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
[ https://issues.apache.org/jira/browse/HADOOP-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528825#comment-17528825 ] Steve Loughran commented on HADOOP-18198: - i'm going with your script patch; it is better for repeatability and docs. testing it in a docker build to see how it goes. > Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes > -- > > Key: HADOOP-18198 > URL: https://issues.apache.org/jira/browse/HADOOP-18198 > Project: Hadoop Common > Issue Type: Task > Components: build >Affects Versions: 3.3.2 >Reporter: Steve Loughran >Assignee: Steve Loughran >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > Hadoop 3.3.3 is a minor followup release to Hadoop 3.3.2 with all the > incremental changes which went in to the 3.2.4 release > * minor CVE fixes in Hadoop source > * CVE fixes in dependencies we know of (protobuf unmarshalling leading to > DoS, jackson stack overflow,...) > * replacement of log4j 1.2.17 to reload4j > * node.js update > This is not a release off branch-3.3, it is a fork of 3.3.2 with the changes. > The next release of branch-3.3 will be numbered hadoop-3.3.4; updating maven > versions and JIRA fix versions is part of this release process. > The changes here are already in branch 3.2.4; this completes the set > CVEs fixed > * CVE-2022-26612: Apache Hadoop: Arbitrary file write in > FileUtil#unpackEntries on Windows (HADOOP-18155) -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged
[
https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=762867=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762867
]
ASF GitHub Bot logged work on HADOOP-18168:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 13:18
Start Date: 27/Apr/22 13:18
Worklog Time Spent: 10m
Work Description: dannycjones commented on PR #4140:
URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1110990869
Tests passing against `eu-west-1` bucket.
Issue Time Tracking
---
Worklog Id: (was: 762867)
Time Spent: 1h 50m (was: 1h 40m)
> ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket
> content purged
> ---
>
> Key: HADOOP-18168
> URL: https://issues.apache.org/jira/browse/HADOOP-18168
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 3.3.4
>Reporter: Steve Loughran
>Assignee: Daniel Carl Jones
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> {{ITestMarkerTool.testRunLimitedLandsatAudit}} is failing -a scan which was
> meant to stop after the first page of results is finishing because there
> aren't so many objects there.
>
> first visible sign of the landsat-pds cleanup
> now we have requester pays, we could do this against another store with
> stability promises, e.g common crawl.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dannycjones commented on pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dependency on purged public bucket
dannycjones commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1110990869 Tests passing against `eu-west-1` bucket. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18216) Document "io.file.buffer.size" must be greater than zero
[
https://issues.apache.org/jira/browse/HADOOP-18216?focusedWorklogId=762839=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762839
]
ASF GitHub Bot logged work on HADOOP-18216:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 12:36
Start Date: 27/Apr/22 12:36
Worklog Time Spent: 10m
Work Description: liever18 commented on PR #4220:
URL: https://github.com/apache/hadoop/pull/4220#issuecomment-1110948706
> Hi @steveloughran and @liever18 Sorry for late comments. I found that this
pr just update comments, right? Is it better to improve the logic to guard the
illegal configuration? Thanks.
Do you think prepending precondition check to this code
[IOUtils.java](https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOUtils.java#L93)
would solve the problem?
Issue Time Tracking
---
Worklog Id: (was: 762839)
Time Spent: 1.5h (was: 1h 20m)
> Document "io.file.buffer.size" must be greater than zero
>
>
> Key: HADOOP-18216
> URL: https://issues.apache.org/jira/browse/HADOOP-18216
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Jingxuan Fu
>Assignee: Jingxuan Fu
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> when the configuration file in the "io.file.buffer.size" field is set to a
> value less than or equal to zero, hdfs can start normally, but read and write
> data will have problems.
> When the value is less than zero, the shell will throw the following
> exception:
> {code:java}
> hadoop@ljq1:~/hadoop-3.1.3-work/bin$ ./hdfs dfs -cat mapred
> -cat: Fatal internal error
> java.lang.NegativeArraySizeException: -4096
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:93)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:68)
> at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:129)
> at
> org.apache.hadoop.fs.shell.Display$Cat.printToStdout(Display.java:101)
> at org.apache.hadoop.fs.shell.Display$Cat.processPath(Display.java:96)
> at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:331)
> at
> org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:303)
> at
> org.apache.hadoop.fs.shell.Command.processArgument(Command.java:285)
> at
> org.apache.hadoop.fs.shell.Command.processArguments(Command.java:269)
> at
> org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:120)
> at org.apache.hadoop.fs.shell.Command.run(Command.java:176)
> at org.apache.hadoop.fs.FsShell.run(FsShell.java:328)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.fs.FsShell.main(FsShell.java:391){code}
> When the value is equal to zero, the shell command will always block
> {code:java}
> hadoop@ljq1:~/hadoop-3.1.3-work/bin$ ./hdfs dfs -cat mapred
> ^Z
> [2]+ Stopped ./hdfs dfs -cat mapred{code}
> The description of the configuration file is not clear enough, it may make
> people think that set to 0 to enter the non-blocking mode.
>
> {code:java}
>
> io.file.buffer.size
> 4096
> The size of buffer for use in sequence files.
> The size of this buffer should probably be a multiple of hardware
> page size (4096 on Intel x86), and it determines how much data is
> buffered during read and write operations.
> {code}
>
> Considering that this value is uesd by hdfs and mapreduce frequently, we
> should make this value must be a number greater than zero.
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] liever18 commented on pull request #4220: HADOOP-18216. io.file.buffer.size must be greater than zero
liever18 commented on PR #4220: URL: https://github.com/apache/hadoop/pull/4220#issuecomment-1110948706 > Hi @steveloughran and @liever18 Sorry for late comments. I found that this pr just update comments, right? Is it better to improve the logic to guard the illegal configuration? Thanks. Do you think prepending precondition check to this code [IOUtils.java](https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/IOUtils.java#L93) would solve the problem? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16202) Enhance openFile() for better read performance against object stores
[
https://issues.apache.org/jira/browse/HADOOP-16202?focusedWorklogId=762814=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762814
]
ASF GitHub Bot logged work on HADOOP-16202:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 11:45
Start Date: 27/Apr/22 11:45
Worklog Time Spent: 10m
Work Description: steveloughran opened a new pull request, #4238:
URL: https://github.com/apache/hadoop/pull/4238
### Description of PR
backport of HADOOP-16202. Enhanced openFile() to branch-3.3, plus a couple
of other cherrypicks from trunk to ease the backporting.
if yetus is happy i wil merge the entire sequence in as the ordered chain of
commits
### How was this patch tested?
cloud store testing in progress against aws london and azure cardiff.
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [X] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
Issue Time Tracking
---
Worklog Id: (was: 762814)
Time Spent: 22h 20m (was: 22h 10m)
> Enhance openFile() for better read performance against object stores
> -
>
> Key: HADOOP-16202
> URL: https://issues.apache.org/jira/browse/HADOOP-16202
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs, fs/s3, tools/distcp
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 22h 20m
> Remaining Estimate: 0h
>
> The {{openFile()}} builder API lets us add new options when reading a file
> Add an option {{"fs.s3a.open.option.length"}} which takes a long and allows
> the length of the file to be declared. If set, *no check for the existence of
> the file is issued when opening the file*
> Also: withFileStatus() to take any FileStatus implementation, rather than
> only S3AFileStatus -and not check that the path matches the path being
> opened. Needed to support viewFS-style wrapping and mounting.
> and Adopt where appropriate to stop clusters with S3A reads switched to
> random IO from killing download/localization
> * fs shell copyToLocal
> * distcp
> * IOUtils.copy
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran opened a new pull request, #4238: HADOOP-16202. Enhanced openFile() -branch-3.3 backport
steveloughran opened a new pull request, #4238: URL: https://github.com/apache/hadoop/pull/4238 ### Description of PR backport of HADOOP-16202. Enhanced openFile() to branch-3.3, plus a couple of other cherrypicks from trunk to ease the backporting. if yetus is happy i wil merge the entire sequence in as the ordered chain of commits ### How was this patch tested? cloud store testing in progress against aws london and azure cardiff. ### For code changes: - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [X] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4237: MAPREDUCE-7375 JobSubmissionFiles don't set right permission after mkdirs
hadoop-yetus commented on PR #4237:
URL: https://github.com/apache/hadoop/pull/4237#issuecomment-1110873442
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 14s | | trunk passed |
| +1 :green_heart: | compile | 1m 3s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 58s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 59s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 48s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 49s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 10s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 42s | | the patch passed |
| +1 :green_heart: | compile | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 45s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 35s |
[/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4237/1/artifact/out/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt)
|
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core:
The patch generated 12 new + 1 unchanged - 0 fixed = 13 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 44s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 32s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 44s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 6m 28s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 102m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4237/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4237 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 7484c0f5ccc6 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a2721170216fc3853a2a584d0768d29fcc0a1ec9 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4237/1/testReport/ |
| Max. process+thread count | 1286 (vs. ulimit of 5500) |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4237/1/console |
| versions | git=2.25.1 maven=3.6.3
[jira] [Comment Edited] (HADOOP-18187) Convert s3a prefetching to use JavaDoc for fields and enums
[ https://issues.apache.org/jira/browse/HADOOP-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528611#comment-17528611 ] Daniel Carl Jones edited comment on HADOOP-18187 at 4/27/22 9:52 AM: - Here's an example of a file with good JavaDoc. [https://github.com/apache/hadoop/blob/60cb6b5743c67dedc3f6575621d6a150ebc0761f/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/OpenFileSupport.java] The main goal for this task is to move a lot of non-JavaDoc comments to JavaDoc. There's many instances of good documentation but it won't be picked up by the JavaDoc pages or IDE hints. A few examples... * Class fields like this: [https://github.com/apache/hadoop/commit/342e1cd3fcbd2e7d00ab6a620c8ae4919bbc4201#diff-fa939ca467a01f13835de546620f926f3acd0f0f8730ae8c0960a7238e05fc4aR33-R40] * Enum values like this: [https://github.com/apache/hadoop/commit/342e1cd3fcbd2e7d00ab6a620c8ae4919bbc4201#diff-13d9d8e446f8fc6c5a1f2842ef9629dcd8310519cd6d40cdc26607b91ba609a8R45-R63] * Good comments about class invariants but should be part of the class docs: [https://github.com/apache/hadoop/commit/342e1cd3fcbd2e7d00ab6a620c8ae4919bbc4201#diff-05c34609715bb2a5a20e67e6f70d876f4ebe23ba275dad29cec4b184b552a1adR49-R52] was (Author: JIRAUSER284792): Here's an example of a file with good JavaDoc. [https://github.com/apache/hadoop/blob/60cb6b5743c67dedc3f6575621d6a150ebc0761f/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/OpenFileSupport.java] One of the main goals for this task is to move a lot of non-JavaDoc comments to JavaDoc. There's many instances of good documentation but it won't be picked up by the JavaDoc pages or IDE hints. A few examples... * Class fields like this: [https://github.com/apache/hadoop/commit/342e1cd3fcbd2e7d00ab6a620c8ae4919bbc4201#diff-fa939ca467a01f13835de546620f926f3acd0f0f8730ae8c0960a7238e05fc4aR33-R40] * Enum values like this: [https://github.com/apache/hadoop/commit/342e1cd3fcbd2e7d00ab6a620c8ae4919bbc4201#diff-13d9d8e446f8fc6c5a1f2842ef9629dcd8310519cd6d40cdc26607b91ba609a8R45-R63] * Good comments about class invariants but should be part of the class docs: [https://github.com/apache/hadoop/commit/342e1cd3fcbd2e7d00ab6a620c8ae4919bbc4201#diff-05c34609715bb2a5a20e67e6f70d876f4ebe23ba275dad29cec4b184b552a1adR49-R52] > Convert s3a prefetching to use JavaDoc for fields and enums > --- > > Key: HADOOP-18187 > URL: https://issues.apache.org/jira/browse/HADOOP-18187 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Daniel Carl Jones >Assignee: Bhalchandra Pandit >Priority: Minor > > There's lots of good comments for fields and enum values in the current code. > Let's expose these to your IDE with JavaDoc instead. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18182) S3File to store reference to active S3Object in a field.
[
https://issues.apache.org/jira/browse/HADOOP-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528695#comment-17528695
]
Ahmar Suhail commented on HADOOP-18182:
---
I'm still a bit confused about this one and am maybe missing something/getting
something wrong, but I feel like we're already doing this. `S3Reader` calls
`S3File` which opens the input stream to S3. `S3File` stores a reference to the
`S3Object` in its map
[here|https://github.com/apache/hadoop/blob/feature-HADOOP-18028-s3a-prefetch/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/read/S3File.java#L65]
which should prevent it from being GC'd? `S3Reader` reads the entire block
from the input stream in chunks of 64KB, and then it's finally method closes
the input stream, so in this case the lifespan of two objects should be in
sync. I've been looking at implementing `unbuffer()`..when you say we would
want to clean all this up, is there something we need to do other than calling
the `close()` methods which may cause the lifespans to go out of sync?
> S3File to store reference to active S3Object in a field.
>
>
> Key: HADOOP-18182
> URL: https://issues.apache.org/jira/browse/HADOOP-18182
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Steve Loughran
>Assignee: Bhalchandra Pandit
>Priority: Major
>
> HADOOP-17338 showed us how recent {{S3Object.finalize()}} can call
> stream.close() and so close an active stream if a GC happens during a read.
> replicate the same fix here.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4107: HDFS-16521. DFS API to retrieve slow datanodes
hadoop-yetus commented on PR #4107:
URL: https://github.com/apache/hadoop/pull/4107#issuecomment-1110789635
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 34s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 2s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 8s | | trunk passed |
| +1 :green_heart: | compile | 6m 52s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 6m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 41s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 26s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 4m 8s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 8m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 31s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 58s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 30s | | the patch passed |
| +1 :green_heart: | compile | 7m 33s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | cc | 7m 33s | | the patch passed |
| -1 :x: | javac | 7m 33s |
[/results-compile-javac-hadoop-hdfs-project-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4107/8/artifact/out/results-compile-javac-hadoop-hdfs-project-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| hadoop-hdfs-project-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 1 new + 651 unchanged - 0
fixed = 652 total (was 651) |
| +1 :green_heart: | compile | 7m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 7m 18s | | the patch passed |
| -1 :x: | javac | 7m 18s |
[/results-compile-javac-hadoop-hdfs-project-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4107/8/artifact/out/results-compile-javac-hadoop-hdfs-project-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| hadoop-hdfs-project-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new +
629 unchanged - 0 fixed = 630 total (was 629) |
| +1 :green_heart: | blanks | 0m 1s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 32s | | hadoop-hdfs-project: The
patch generated 0 new + 455 unchanged - 1 fixed = 455 total (was 456) |
| +1 :green_heart: | mvnsite | 3m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 51s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 3m 39s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 10m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 26m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 0m 43s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4107/8/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client.txt)
| hadoop-hdfs-client in the patch failed. |
| -1 :x: | unit | 0m 42s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4107/8/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch
[jira] [Commented] (HADOOP-17956) Replace all default Charset usage with UTF-8
[ https://issues.apache.org/jira/browse/HADOOP-17956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528688#comment-17528688 ] Steve Loughran commented on HADOOP-17956: - backported to branch 3 to avoid conflicts with other patches > Replace all default Charset usage with UTF-8 > > > Key: HADOOP-17956 > URL: https://issues.apache.org/jira/browse/HADOOP-17956 > Project: Hadoop Common > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > As discussed on PR#3515, creating this sub-task to replace all default > charset with UTF-8 as default charset has some potential problems (e.g. > HADOOP-11379, HADOOP-11389). > FYI [~aajisaka] -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17956) Replace all default Charset usage with UTF-8
[ https://issues.apache.org/jira/browse/HADOOP-17956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-17956: Fix Version/s: 3.3.4 > Replace all default Charset usage with UTF-8 > > > Key: HADOOP-17956 > URL: https://issues.apache.org/jira/browse/HADOOP-17956 > Project: Hadoop Common > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.4 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > As discussed on PR#3515, creating this sub-task to replace all default > charset with UTF-8 as default charset has some potential problems (e.g. > HADOOP-11379, HADOOP-11389). > FYI [~aajisaka] -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] skysiders opened a new pull request, #4237: Mapreduce 7375 JobSubmissionFiles don't set right permission after mkdirs
skysiders opened a new pull request, #4237: URL: https://github.com/apache/hadoop/pull/4237 Mapreduce 7375 JobSubmissionFiles don't set right permission after mkdirs ### Description of PR JobSubmissionFiles provide getStagingDir to get Staging Directory.If stagingArea missing, method will create new directory with this. fs.mkdirs(stagingArea, new FsPermission(JOB_DIR_PERMISSION)); It seems create new directory with JOB_DIR_PERMISSION,but this permission will be apply by umask.If umask too strict , this permission may be 000(if umask is 700).So we should change permission after create. ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] skysiders closed pull request #4236: MAPREDUCE-7375 set permission for JobSubmissionFiles
skysiders closed pull request #4236: MAPREDUCE-7375 set permission for JobSubmissionFiles URL: https://github.com/apache/hadoop/pull/4236 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18182) S3File to store reference to active S3Object in a field.
[
https://issues.apache.org/jira/browse/HADOOP-18182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528664#comment-17528664
]
Steve Loughran commented on HADOOP-18182:
-
before HADOOP-17338 the code was something like
{code}
GetObjectRequest request = client.newGetRequest(key)
S3Object object = client.getObject(request));
wrappedStream = object.getObjectContent();
{code}
where wrappedStream was a field of S3AInputStream...we would keep it and then
close/abort as needed.
this worked well until an aws sdk update added a finalizer method to S3Object
which closed the http stream when it was GC'd, even though the wrappedStream
was still actively using the stream. as a result, if a stream was kept open
long enough for a GC to happen, things would break.
we need to keep the lifespan of the two objects in sync, retaining a ref to the
outer S3Object for as long as the inner stream is used.
> S3File to store reference to active S3Object in a field.
>
>
> Key: HADOOP-18182
> URL: https://issues.apache.org/jira/browse/HADOOP-18182
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Affects Versions: 3.4.0
>Reporter: Steve Loughran
>Assignee: Bhalchandra Pandit
>Priority: Major
>
> HADOOP-17338 showed us how recent {{S3Object.finalize()}} can call
> stream.close() and so close an active stream if a GC happens during a read.
> replicate the same fix here.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18168) ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket content purged
[
https://issues.apache.org/jira/browse/HADOOP-18168?focusedWorklogId=762757=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762757
]
ASF GitHub Bot logged work on HADOOP-18168:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 08:49
Start Date: 27/Apr/22 08:49
Worklog Time Spent: 10m
Work Description: dannycjones commented on PR #4140:
URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1110735715
Integration tests not currently passing with this commit, diving deeper here.
Issue Time Tracking
---
Worklog Id: (was: 762757)
Time Spent: 1h 40m (was: 1.5h)
> ITestMarkerTool.testRunLimitedLandsatAudit failing due to most of bucket
> content purged
> ---
>
> Key: HADOOP-18168
> URL: https://issues.apache.org/jira/browse/HADOOP-18168
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 3.3.4
>Reporter: Steve Loughran
>Assignee: Daniel Carl Jones
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> {{ITestMarkerTool.testRunLimitedLandsatAudit}} is failing -a scan which was
> meant to stop after the first page of results is finishing because there
> aren't so many objects there.
>
> first visible sign of the landsat-pds cleanup
> now we have requester pays, we could do this against another store with
> stability promises, e.g common crawl.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] dannycjones commented on pull request #4140: HADOOP-18168. Fix S3A ITestMarkerTool dependency on purged public bucket
dannycjones commented on PR #4140: URL: https://github.com/apache/hadoop/pull/4140#issuecomment-1110735715 Integration tests not currently passing with this commit, diving deeper here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18104) Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads
[ https://issues.apache.org/jira/browse/HADOOP-18104?focusedWorklogId=762754=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762754 ] ASF GitHub Bot logged work on HADOOP-18104: --- Author: ASF GitHub Bot Created on: 27/Apr/22 08:38 Start Date: 27/Apr/22 08:38 Worklog Time Spent: 10m Work Description: steveloughran commented on PR #3964: URL: https://github.com/apache/hadoop/pull/3964#issuecomment-1110718632 +1 merge at your leisure. you will also need to rebase the branch. you don't need to do a pr for that, just rebase/fix/retest locally and then do a force push with the update. that is allowed on this branch (anything which doesn't match the pattern branch-*) Issue Time Tracking --- Worklog Id: (was: 762754) Time Spent: 3.5h (was: 3h 20m) > Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads > > > Key: HADOOP-18104 > URL: https://issues.apache.org/jira/browse/HADOOP-18104 > Project: Hadoop Common > Issue Type: Sub-task > Components: common, fs >Reporter: Mukund Thakur >Assignee: Mukund Thakur >Priority: Major > Labels: pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #3964: HADOOP-18104: S3A: Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads
steveloughran commented on PR #3964: URL: https://github.com/apache/hadoop/pull/3964#issuecomment-1110718632 +1 merge at your leisure. you will also need to rebase the branch. you don't need to do a pr for that, just rebase/fix/retest locally and then do a force push with the update. that is allowed on this branch (anything which doesn't match the pattern branch-*) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18184) s3a prefetching stream to support unbuffer()
[ https://issues.apache.org/jira/browse/HADOOP-18184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmar Suhail reassigned HADOOP-18184: - Assignee: Ahmar Suhail > s3a prefetching stream to support unbuffer() > > > Key: HADOOP-18184 > URL: https://issues.apache.org/jira/browse/HADOOP-18184 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Steve Loughran >Assignee: Ahmar Suhail >Priority: Minor > > Apache Impala uses unbuffer() to free up all client side resources held by a > stream, so allowing it to have a map of available (path -> stream) objects, > retained across queries. > This saves on having to reopen the files, with the cost of HEAD checks etc. > S3AInputStream just closes its http connection. here there is a lot more > state to discard, but all memory and file storage must be freed. > until this done, ITestS3AContractUnbuffer must skip when the prefetch stream > is used. > its notable that the other tests don't fail, even though the stream doesn't > implement the interface; the graceful degradation handles that. it should > fail if the test xml resource says the stream does it, but that the stream > capabilities say it doesn't. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18187) Convert s3a prefetching to use JavaDoc for fields and enums
[ https://issues.apache.org/jira/browse/HADOOP-18187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17528611#comment-17528611 ] Daniel Carl Jones commented on HADOOP-18187: Here's an example of a file with good JavaDoc. [https://github.com/apache/hadoop/blob/60cb6b5743c67dedc3f6575621d6a150ebc0761f/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/OpenFileSupport.java] One of the main goals for this task is to move a lot of non-JavaDoc comments to JavaDoc. There's many instances of good documentation but it won't be picked up by the JavaDoc pages or IDE hints. A few examples... * Class fields like this: [https://github.com/apache/hadoop/commit/342e1cd3fcbd2e7d00ab6a620c8ae4919bbc4201#diff-fa939ca467a01f13835de546620f926f3acd0f0f8730ae8c0960a7238e05fc4aR33-R40] * Enum values like this: [https://github.com/apache/hadoop/commit/342e1cd3fcbd2e7d00ab6a620c8ae4919bbc4201#diff-13d9d8e446f8fc6c5a1f2842ef9629dcd8310519cd6d40cdc26607b91ba609a8R45-R63] * Good comments about class invariants but should be part of the class docs: [https://github.com/apache/hadoop/commit/342e1cd3fcbd2e7d00ab6a620c8ae4919bbc4201#diff-05c34609715bb2a5a20e67e6f70d876f4ebe23ba275dad29cec4b184b552a1adR49-R52] > Convert s3a prefetching to use JavaDoc for fields and enums > --- > > Key: HADOOP-18187 > URL: https://issues.apache.org/jira/browse/HADOOP-18187 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.4.0 >Reporter: Daniel Carl Jones >Assignee: Bhalchandra Pandit >Priority: Minor > > There's lots of good comments for fields and enum values in the current code. > Let's expose these to your IDE with JavaDoc instead. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-18069) CVE-2021-0341 in okhttp@2.7.5 detected in hdfs-client
[
https://issues.apache.org/jira/browse/HADOOP-18069?focusedWorklogId=762745=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-762745
]
ASF GitHub Bot logged work on HADOOP-18069:
---
Author: ASF GitHub Bot
Created on: 27/Apr/22 07:47
Start Date: 27/Apr/22 07:47
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4229:
URL: https://github.com/apache/hadoop/pull/4229#issuecomment-1110663481
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 16s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 26s | | trunk passed |
| +1 :green_heart: | compile | 23m 13s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 20m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 19m 19s | | trunk passed |
| -1 :x: | javadoc | 1m 42s |
[/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 8m 25s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 34s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 52m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 47s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 22m 3s |
[/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/patch-mvninstall-root.txt)
| root in the patch failed. |
| +1 :green_heart: | compile | 22m 39s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 22m 39s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 2 new + 1810 unchanged - 0
fixed = 1812 total (was 1810) |
| +1 :green_heart: | compile | 20m 49s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 20m 49s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 2 new + 1686
unchanged - 0 fixed = 1688 total (was 1686) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 59s | | the patch passed |
| +1 :green_heart: | mvnsite | 18m 54s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | xml | 0m 4s | | The patch has no ill-formed XML
file. |
| -1 :x: | javadoc | 1m 33s |
[/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root in the patch failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.
|
| +1 :green_heart: | javadoc | 8m 25s | | the patch passed with JDK
Private
[GitHub] [hadoop] hadoop-yetus commented on pull request #4229: HADOOP-18069. okhttp@2.7.5 to 4.9.3
hadoop-yetus commented on PR #4229:
URL: https://github.com/apache/hadoop/pull/4229#issuecomment-1110663481
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 16m 16s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 26s | | trunk passed |
| +1 :green_heart: | compile | 23m 13s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 20m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 21s | | trunk passed |
| +1 :green_heart: | mvnsite | 19m 19s | | trunk passed |
| -1 :x: | javadoc | 1m 42s |
[/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/branch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root in trunk failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04. |
| +1 :green_heart: | javadoc | 8m 25s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 34s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 52m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 47s | | Maven dependency ordering for patch |
| -1 :x: | mvninstall | 22m 3s |
[/patch-mvninstall-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/patch-mvninstall-root.txt)
| root in the patch failed. |
| +1 :green_heart: | compile | 22m 39s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 22m 39s |
[/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 generated 2 new + 1810 unchanged - 0
fixed = 1812 total (was 1810) |
| +1 :green_heart: | compile | 20m 49s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 20m 49s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 2 new + 1686
unchanged - 0 fixed = 1688 total (was 1686) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 59s | | the patch passed |
| +1 :green_heart: | mvnsite | 18m 54s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | xml | 0m 4s | | The patch has no ill-formed XML
file. |
| -1 :x: | javadoc | 1m 33s |
[/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/patch-javadoc-root-jdkUbuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.txt)
| root in the patch failed with JDK Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04.
|
| +1 :green_heart: | javadoc | 8m 25s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 34s | | hadoop-project has no data from
spotbugs |
| -1 :x: | spotbugs | 2m 52s |
[/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4229/7/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html)
| hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 0
[GitHub] [hadoop] hadoop-yetus commented on pull request #4228: HDFS-16468. Define ssize_t for Windows
hadoop-yetus commented on PR #4228:
URL: https://github.com/apache/hadoop/pull/4228#issuecomment-1110627735
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 24m 47s | | trunk passed |
| +1 :green_heart: | compile | 4m 26s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 4m 25s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 57m 14s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 57m 36s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 20s | | the patch passed |
| +1 :green_heart: | compile | 4m 8s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | cc | 4m 8s | | the patch passed |
| +1 :green_heart: | golang | 4m 8s | | the patch passed |
| +1 :green_heart: | javac | 4m 8s | | the patch passed |
| +1 :green_heart: | compile | 4m 5s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 4m 5s | | the patch passed |
| +1 :green_heart: | golang | 4m 5s | | the patch passed |
| +1 :green_heart: | javac | 4m 5s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 52s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 111m 43s | | hadoop-hdfs-native-client in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 39s | | The patch does not
generate ASF License warnings. |
| | | 204m 52s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4228 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux ce2a51f2bdeb 4.15.0-166-generic #174-Ubuntu SMP Wed Dec 8
19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 7b26a9a96131cc3112107e471ed0b13e3a29dffd |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/5/testReport/ |
| Max. process+thread count | 523 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4228/5/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4236: MAPREDUCE-7375 set permission for JobSubmissionFiles
hadoop-yetus commented on PR #4236:
URL: https://github.com/apache/hadoop/pull/4236#issuecomment-1110622337
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 13m 24s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 43s | | trunk passed |
| +1 :green_heart: | compile | 1m 2s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 59s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 0s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 44s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 51s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 25s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 43s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 43s | | the patch passed |
| +1 :green_heart: | compile | 0m 41s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 41s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 35s |
[/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4236/1/artifact/out/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt)
|
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core:
The patch generated 13 new + 1 unchanged - 0 fixed = 14 total (was 1) |
| +1 :green_heart: | mvnsite | 0m 44s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 33s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 40s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 6m 27s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 115m 41s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4236/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4236 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 3e3cb74493a3 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4c061144429cd70a79c02257a304f67681387e0c |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4236/1/testReport/ |
| Max. process+thread count | 1599 (vs. ulimit of 5500) |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4236/1/console |
|
