[jira] [Work logged] (HADOOP-17633) Please upgrade json-smart dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-17633?focusedWorklogId=583146=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-583146 ] ASF GitHub Bot logged work on HADOOP-17633: --- Author: ASF GitHub Bot Created on: 15/Apr/21 05:33 Start Date: 15/Apr/21 05:33 Worklog Time Spent: 10m Work Description: virajjasani commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-820126450 Build#8 has timed-out after 20 hr and has these test failures recorded. Pasting screenshots here as the build data will be archived very soon. https://user-images.githubusercontent.com/34790606/114818757-f2afbe80-9dd9-11eb-8410-45f52814ca9b.png;> https://user-images.githubusercontent.com/34790606/114818827-0eb36000-9dda-11eb-8f84-54f8081c5ad1.png;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 583146) Time Spent: 3h 10m (was: 3h) > Please upgrade json-smart dependency to the latest version > -- > > Key: HADOOP-17633 > URL: https://issues.apache.org/jira/browse/HADOOP-17633 > Project: Hadoop Common > Issue Type: Improvement > Components: auth, build >Affects Versions: 3.3.0, 3.2.1, 3.2.2, 3.4.0 >Reporter: helen huang >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > Please upgrade the json-smart dependency to the latest version available. > Currently hadoop-auth is using version 2.3. Fortify scan picked up a security > issue with this version. Please upgrade to the latest version. > Thanks! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs
virajjasani commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-820126450 Build#8 has timed-out after 20 hr and has these test failures recorded. Pasting screenshots here as the build data will be archived very soon. https://user-images.githubusercontent.com/34790606/114818757-f2afbe80-9dd9-11eb-8410-45f52814ca9b.png;> https://user-images.githubusercontent.com/34790606/114818827-0eb36000-9dda-11eb-8f84-54f8081c5ad1.png;> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17524) Remove EventCounter
[ https://issues.apache.org/jira/browse/HADOOP-17524?focusedWorklogId=583145=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-583145 ] ASF GitHub Bot logged work on HADOOP-17524: --- Author: ASF GitHub Bot Created on: 15/Apr/21 05:28 Start Date: 15/Apr/21 05:28 Worklog Time Spent: 10m Work Description: virajjasani commented on pull request #2909: URL: https://github.com/apache/hadoop/pull/2909#issuecomment-820124354 Test failure has been taken care of in recent commit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 583145) Time Spent: 40m (was: 0.5h) > Remove EventCounter > --- > > Key: HADOOP-17524 > URL: https://issues.apache.org/jira/browse/HADOOP-17524 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > EventCount is using Log4J 1.x API. We need to remove it to drop Log4J 1.x. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #2909: HADOOP-17524. Remove EventCounter and Log counters from JVM Metrics
virajjasani commented on pull request #2909: URL: https://github.com/apache/hadoop/pull/2909#issuecomment-820124354 Test failure has been taken care of in recent commit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17524) Remove EventCounter
[ https://issues.apache.org/jira/browse/HADOOP-17524?focusedWorklogId=583144=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-583144 ] ASF GitHub Bot logged work on HADOOP-17524: --- Author: ASF GitHub Bot Created on: 15/Apr/21 05:27 Start Date: 15/Apr/21 05:27 Worklog Time Spent: 10m Work Description: virajjasani commented on pull request #2909: URL: https://github.com/apache/hadoop/pull/2909#issuecomment-820123401 @Apache9 @aajisaka Could you please take a look? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 583144) Time Spent: 0.5h (was: 20m) > Remove EventCounter > --- > > Key: HADOOP-17524 > URL: https://issues.apache.org/jira/browse/HADOOP-17524 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > EventCount is using Log4J 1.x API. We need to remove it to drop Log4J 1.x. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #2909: HADOOP-17524. Remove EventCounter and Log counters from JVM Metrics
virajjasani commented on pull request #2909: URL: https://github.com/apache/hadoop/pull/2909#issuecomment-820123401 @Apache9 @aajisaka Could you please take a look? Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] functioner commented on pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang
functioner commented on pull request #2737: URL: https://github.com/apache/hadoop/pull/2737#issuecomment-820106508 @Hexiaoqiao thanks for the reminder. I've added: 1. comments for removing `logSyncNotifyExecutor.shutdown()` 2. configuration for the size of `logSyncNotifyExecutor` 3. the docs for that configuration -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao commented on pull request #2889: HDFS-15963. Unreleased volume references cause an infinite loop.
Hexiaoqiao commented on pull request #2889: URL: https://github.com/apache/hadoop/pull/2889#issuecomment-820096203 Thanks @zhangshuyan0 for your great catch here. LGTM. +1 from my side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao commented on pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang
Hexiaoqiao commented on pull request #2737: URL: https://github.com/apache/hadoop/pull/2737#issuecomment-820080877 @functioner after review code, comment from Yiqun seems not resolved yet, do you mind to make thread pool size be configurable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] qizhu-lucas opened a new pull request #2911: YARN-10737: Fix typos in CapacityScheduler#schedule.
qizhu-lucas opened a new pull request #2911: URL: https://github.com/apache/hadoop/pull/2911 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferhui merged pull request #2902: HDFS-15940. Fixing and refactoring tests specific to Block recovery.
ferhui merged pull request #2902: URL: https://github.com/apache/hadoop/pull/2902 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] cxorm commented on a change in pull request #2910: HDFS-15810. RBF: Use BigInteger for Router capacity metrics
cxorm commented on a change in pull request #2910:
URL: https://github.com/apache/hadoop/pull/2910#discussion_r613727691
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/metrics/RBFMetrics.java
##
@@ -371,13 +372,13 @@ private static void setStateStoreVersions(
}
@Override
- public long getTotalCapacity() {
-return getNameserviceAggregatedLong(MembershipStats::getTotalSpace);
+ public BigInteger getTotalCapacity() {
Review comment:
Not sure the change of public API is proper or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on pull request #2902: HDFS-15940. Fixing and refactoring tests specific to Block recovery.
ferhui commented on pull request #2902: URL: https://github.com/apache/hadoop/pull/2902#issuecomment-820025360 @jojochuang Thanks for this PR. @virajjasani Thanks for review! merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2910: HDFS-15810. RBF: Use BigInteger for Router capacity metrics
hadoop-yetus commented on pull request #2910:
URL: https://github.com/apache/hadoop/pull/2910#issuecomment-820010002
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 49s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 43s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 56s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 13s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 20s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 32s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| -1 :x: | spotbugs | 1m 19s |
[/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2910/1/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-rbf.html)
| hadoop-hdfs-project/hadoop-hdfs-rbf generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) |
| +1 :green_heart: | shadedclient | 14m 2s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 17m 48s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2910/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 93m 9s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-hdfs-project/hadoop-hdfs-rbf |
| | Return value of java.math.BigInteger.add(BigInteger) ignored in
org.apache.hadoop.hdfs.server.federation.metrics.RBFMetrics.getNameserviceAggregatedBigInt(ToLongFunction)
At RBFMetrics.java:in
org.apache.hadoop.hdfs.server.federation.metrics.RBFMetrics.getNameserviceAggregatedBigInt(ToLongFunction)
At RBFMetrics.java:[line 795] |
| Failed junit tests |
hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2910/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2910 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux ee336fa6323a 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 19043f9fa20f545ecd1836fdc1598826727cb4fa |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
[GitHub] [hadoop] tomscut commented on pull request #2896: HDFS-15970. Print network topology on the web
tomscut commented on pull request #2896: URL: https://github.com/apache/hadoop/pull/2896#issuecomment-819960710 Hi @tasanuma , could you please do another review? Thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong
tomscut commented on pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#issuecomment-819959935 Those failed unit tests were unrelated to the change. And they work fine locally. Hi @goiri @tasanuma , could you please review the code? Thanks very much. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] fengnanli opened a new pull request #2910: HDFS-15810 RBF: Use BigInteger for Router capacity metrics
fengnanli opened a new pull request #2910: URL: https://github.com/apache/hadoop/pull/2910 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong
hadoop-yetus commented on pull request #2907:
URL: https://github.com/apache/hadoop/pull/2907#issuecomment-819897893
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 59s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 32s | | trunk passed |
| +1 :green_heart: | compile | 28m 48s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 24m 6s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 5m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 8s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 4m 13s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 10m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 21s | | the patch passed |
| +1 :green_heart: | compile | 20m 20s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 20m 20s | | the patch passed |
| +1 :green_heart: | compile | 18m 4s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 18m 4s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 3m 45s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/1/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 5 new + 242 unchanged - 5 fixed = 247 total (was
247) |
| +1 :green_heart: | mvnsite | 4m 10s | | the patch passed |
| +1 :green_heart: | javadoc | 3m 0s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 4m 3s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 8m 42s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 41s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 25s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 36s | | hadoop-hdfs-client in the patch
passed. |
| -1 :x: | unit | 246m 51s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 1m 12s | | The patch does not
generate ASF License warnings. |
| | | 495m 56s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.hdfs.TestReconstructStripedFile |
| | hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys |
| | hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2907/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2907 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux d880014b6357 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk /
[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: Test Pre-Commits.
hadoop-yetus commented on pull request #2860:
URL: https://github.com/apache/hadoop/pull/2860#issuecomment-819872752
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 57s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 15s | | trunk passed |
| +1 :green_heart: | compile | 20m 49s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 18m 12s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 51s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 3s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 4m 51s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 0m 49s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 15m 15s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 3m 0s | | the patch passed |
| +1 :green_heart: | compile | 19m 58s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 19m 58s | | the patch passed |
| +1 :green_heart: | compile | 18m 10s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 18m 10s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 3m 50s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2860/12/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 3 new + 177 unchanged - 1 fixed = 180 total (was
178) |
| +1 :green_heart: | hadolint | 0m 2s | | No new issues. |
| +1 :green_heart: | mvnsite | 4m 50s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 4m 0s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 4m 51s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +0 :ok: | spotbugs | 0m 45s | | hadoop-project has no data from
spotbugs |
| +1 :green_heart: | shadedclient | 15m 26s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 45s | | hadoop-project in the patch
passed. |
| +1 :green_heart: | unit | 2m 45s | | hadoop-hdfs-client in the patch
passed. |
| -1 :x: | unit | 231m 38s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2860/12/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | unit | 18m 23s | | hadoop-hdfs-rbf in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 21s | | The patch does not
generate ASF License warnings. |
| | | 461m 1s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2860/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2860 |
| Optional Tests | dupname asflicense codespell hadolint shellcheck
shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs
checkstyle xml |
| uname | Linux 93205950b95d 4.15.0-136-generic
[GitHub] [hadoop] hadoop-yetus commented on pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang
hadoop-yetus commented on pull request #2737:
URL: https://github.com/apache/hadoop/pull/2737#issuecomment-819811557
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 1s | | trunk passed |
| +1 :green_heart: | compile | 1m 21s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 16s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 19s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 54s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 24s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 4s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 4s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 16m 23s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 9s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 1m 9s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 52s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2737/3/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 2 unchanged -
0 fixed = 3 total (was 2) |
| +1 :green_heart: | mvnsite | 1m 11s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 16s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 52s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 231m 29s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2737/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 317m 56s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.balancer.TestBalancer |
| | hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys |
| | hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots |
| | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2737/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2737 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux f6092b2b51d2 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / bbc76cc242d2ba7c79924d77adab96a30d7eb013 |
| Default Java | Private
[jira] [Work logged] (HADOOP-17524) Remove EventCounter
[
https://issues.apache.org/jira/browse/HADOOP-17524?focusedWorklogId=582862=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582862
]
ASF GitHub Bot logged work on HADOOP-17524:
---
Author: ASF GitHub Bot
Created on: 14/Apr/21 20:29
Start Date: 14/Apr/21 20:29
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2909:
URL: https://github.com/apache/hadoop/pull/2909#issuecomment-819810914
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 21s | | trunk passed |
| +1 :green_heart: | compile | 21m 10s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 18m 6s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 33s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 42s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 53s | | the patch passed |
| +1 :green_heart: | compile | 20m 16s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 20m 16s | | the patch passed |
| +1 :green_heart: | compile | 18m 13s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 18m 13s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 6s | |
hadoop-common-project/hadoop-common: The patch generated 0 new + 25 unchanged -
5 fixed = 25 total (was 30) |
| +1 :green_heart: | mvnsite | 1m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 36s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 17m 35s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2909/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 57s | | The patch does not
generate ASF License warnings. |
| | | 179m 49s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.metrics2.source.TestJvmMetrics |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2909/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2909 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 7efd521557c3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 207f35fc728284afe04e0e1b481877b6de7009c4 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
[GitHub] [hadoop] hadoop-yetus commented on pull request #2909: HADOOP-17524. Remove EventCounter and Log counters from JVM Metrics
hadoop-yetus commented on pull request #2909:
URL: https://github.com/apache/hadoop/pull/2909#issuecomment-819810914
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 21s | | trunk passed |
| +1 :green_heart: | compile | 21m 10s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 18m 6s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 1m 5s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 33s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 42s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 15m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 53s | | the patch passed |
| +1 :green_heart: | compile | 20m 16s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 20m 16s | | the patch passed |
| +1 :green_heart: | compile | 18m 13s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 18m 13s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 6s | |
hadoop-common-project/hadoop-common: The patch generated 0 new + 25 unchanged -
5 fixed = 25 total (was 30) |
| +1 :green_heart: | mvnsite | 1m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 36s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 2m 31s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 17m 35s |
[/patch-unit-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2909/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt)
| hadoop-common in the patch passed. |
| +1 :green_heart: | asflicense | 0m 57s | | The patch does not
generate ASF License warnings. |
| | | 179m 49s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.metrics2.source.TestJvmMetrics |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2909/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2909 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 7efd521557c3 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22
16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 207f35fc728284afe04e0e1b481877b6de7009c4 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2909/1/testReport/ |
| Max. process+thread count | 1378 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2909/1/console |
| versions | git=2.25.1 maven=3.6.3
[jira] [Assigned] (HADOOP-16206) Migrate from Log4j1 to Log4j2
[ https://issues.apache.org/jira/browse/HADOOP-16206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Hussein reassigned HADOOP-16206: -- Assignee: Duo Zhang > Migrate from Log4j1 to Log4j2 > - > > Key: HADOOP-16206 > URL: https://issues.apache.org/jira/browse/HADOOP-16206 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.3.0 >Reporter: Akira Ajisaka >Assignee: Duo Zhang >Priority: Major > Attachments: HADOOP-16206-wip.001.patch > > > This sub-task is to remove log4j1 dependency and add log4j2 dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jbrennan333 merged pull request #2899: YARN-10733. TimelineService Hbase tests failing with timeouts
jbrennan333 merged pull request #2899: URL: https://github.com/apache/hadoop/pull/2899 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2908: HDFS-15976. Make mkdtemp cross platform
hadoop-yetus commented on pull request #2908:
URL: https://github.com/apache/hadoop/pull/2908#issuecomment-819729658
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 54s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 34m 51s | | trunk passed |
| +1 :green_heart: | compile | 2m 38s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 2m 39s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | mvnsite | 0m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 54m 12s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 16s | | the patch passed |
| +1 :green_heart: | compile | 2m 34s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | cc | 2m 34s | | the patch passed |
| +1 :green_heart: | golang | 2m 34s | | the patch passed |
| +1 :green_heart: | javac | 2m 34s | | the patch passed |
| +1 :green_heart: | compile | 2m 32s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | cc | 2m 32s | | the patch passed |
| +1 :green_heart: | golang | 2m 32s | | the patch passed |
| +1 :green_heart: | javac | 2m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 19s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 21s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 116m 2s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2908/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt)
| hadoop-hdfs-native-client in the patch failed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 193m 3s | | |
| Reason | Tests |
|---:|:--|
| Failed CTEST tests | configuration |
| | memcheck_configuration |
| | hdfs_configuration |
| | memcheck_hdfs_configuration |
| | hdfs_builder_test |
| | memcheck_hdfs_builder_test |
| | hdfs_config_connect_bugs |
| | memcheck_hdfs_config_connect_bugs |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2908/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2908 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux 8bfbfed8d78b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 5df023b90978abf2bf5ec131b5e401c1f880d268 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| CTEST |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2908/2/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt
|
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2908/2/testReport/ |
| Max. process+thread count | 713 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2908/2/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
[GitHub] [hadoop] hadoop-yetus commented on pull request #2908: HDFS-15976. Make mkdtemp cross platform
hadoop-yetus commented on pull request #2908:
URL: https://github.com/apache/hadoop/pull/2908#issuecomment-819718466
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 54s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 4s | | trunk passed |
| +1 :green_heart: | compile | 2m 57s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 2m 52s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | mvnsite | 0m 22s | | trunk passed |
| +1 :green_heart: | shadedclient | 60m 3s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 13s | | the patch passed |
| +1 :green_heart: | compile | 2m 42s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | cc | 2m 42s | | the patch passed |
| +1 :green_heart: | golang | 2m 42s | | the patch passed |
| +1 :green_heart: | javac | 2m 42s | | the patch passed |
| +1 :green_heart: | compile | 2m 44s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | cc | 2m 44s | | the patch passed |
| +1 :green_heart: | golang | 2m 44s | | the patch passed |
| +1 :green_heart: | javac | 2m 44s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 0m 15s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 52s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 91m 44s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2908/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt)
| hadoop-hdfs-native-client in the patch failed. |
| +1 :green_heart: | asflicense | 0m 30s | | The patch does not
generate ASF License warnings. |
| | | 177m 20s | | |
| Reason | Tests |
|---:|:--|
| Failed CTEST tests | configuration |
| | memcheck_configuration |
| | hdfs_configuration |
| | memcheck_hdfs_configuration |
| | hdfs_builder_test |
| | memcheck_hdfs_builder_test |
| | hdfs_config_connect_bugs |
| | memcheck_hdfs_config_connect_bugs |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2908/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2908 |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit
codespell golang |
| uname | Linux cd68af073374 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 5df023b90978abf2bf5ec131b5e401c1f880d268 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| CTEST |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2908/1/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt
|
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2908/1/testReport/ |
| Max. process+thread count | 569 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-native-client U:
hadoop-hdfs-project/hadoop-hdfs-native-client |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2908/1/console |
| versions | git=2.25.1 maven=3.6.3 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please
[jira] [Work started] (HADOOP-17524) Remove EventCounter
[ https://issues.apache.org/jira/browse/HADOOP-17524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HADOOP-17524 started by Viraj Jasani. - > Remove EventCounter > --- > > Key: HADOOP-17524 > URL: https://issues.apache.org/jira/browse/HADOOP-17524 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > EventCount is using Log4J 1.x API. We need to remove it to drop Log4J 1.x. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17524) Remove EventCounter
[ https://issues.apache.org/jira/browse/HADOOP-17524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HADOOP-17524: -- Status: Patch Available (was: In Progress) > Remove EventCounter > --- > > Key: HADOOP-17524 > URL: https://issues.apache.org/jira/browse/HADOOP-17524 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > EventCount is using Log4J 1.x API. We need to remove it to drop Log4J 1.x. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17524) Remove EventCounter
[ https://issues.apache.org/jira/browse/HADOOP-17524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-17524: Labels: pull-request-available (was: ) > Remove EventCounter > --- > > Key: HADOOP-17524 > URL: https://issues.apache.org/jira/browse/HADOOP-17524 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > EventCount is using Log4J 1.x API. We need to remove it to drop Log4J 1.x. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17524) Remove EventCounter
[ https://issues.apache.org/jira/browse/HADOOP-17524?focusedWorklogId=582719=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582719 ] ASF GitHub Bot logged work on HADOOP-17524: --- Author: ASF GitHub Bot Created on: 14/Apr/21 17:28 Start Date: 14/Apr/21 17:28 Worklog Time Spent: 10m Work Description: virajjasani opened a new pull request #2909: URL: https://github.com/apache/hadoop/pull/2909 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582719) Remaining Estimate: 0h Time Spent: 10m > Remove EventCounter > --- > > Key: HADOOP-17524 > URL: https://issues.apache.org/jira/browse/HADOOP-17524 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Akira Ajisaka >Assignee: Viraj Jasani >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > EventCount is using Log4J 1.x API. We need to remove it to drop Log4J 1.x. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani opened a new pull request #2909: HADOOP-17524. Remove EventCounter and Log counters from JVM Metrics
virajjasani opened a new pull request #2909: URL: https://github.com/apache/hadoop/pull/2909 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2902: HDFS-15940. Fixing and refactoring tests specific to Block recovery.
hadoop-yetus commented on pull request #2902:
URL: https://github.com/apache/hadoop/pull/2902#issuecomment-819685591
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 4s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ branch-3.2 Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 27s | | branch-3.2 passed |
| +1 :green_heart: | compile | 1m 1s | | branch-3.2 passed |
| +1 :green_heart: | checkstyle | 0m 46s | | branch-3.2 passed |
| +1 :green_heart: | mvnsite | 1m 9s | | branch-3.2 passed |
| +1 :green_heart: | javadoc | 0m 56s | | branch-3.2 passed |
| +1 :green_heart: | spotbugs | 2m 53s | | branch-3.2 passed |
| +1 :green_heart: | shadedclient | 16m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 6s | | the patch passed |
| +1 :green_heart: | compile | 0m 55s | | the patch passed |
| +1 :green_heart: | javac | 0m 55s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 40s | |
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 56 unchanged - 7
fixed = 56 total (was 63) |
| +1 :green_heart: | mvnsite | 1m 1s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed |
| +1 :green_heart: | spotbugs | 3m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 32s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 192m 23s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2902/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch failed. |
| +1 :green_heart: | asflicense | 0m 36s | | The patch does not
generate ASF License warnings. |
| | | 269m 55s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.TestRedudantBlocks |
| | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | hadoop.hdfs.server.namenode.TestEditLogRace |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2902/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2902 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 82eff6c5936f 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | branch-3.2 / 3c8ee877e3bf8fbc99e0b64692c6e71a25ec4b61 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~18.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2902/2/testReport/ |
| Max. process+thread count | 2364 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2902/2/console |
| versions | git=2.17.1 maven=3.6.0 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] amahussein commented on a change in pull request #2878: HDFS-15957. The ignored IOException in the RPC response sent by FSEditLogAsync can cause the HDFS client to hang
amahussein commented on a change in pull request #2878:
URL: https://github.com/apache/hadoop/pull/2878#discussion_r613382377
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java
##
@@ -378,13 +381,18 @@ public void logSyncWait() {
@Override
public void logSyncNotify(RuntimeException syncEx) {
- try {
-if (syncEx == null) {
- call.sendResponse();
-} else {
- call.abortResponse(syncEx);
+ for (int retries = 0; retries <= RESPONSE_SEND_RETRIES; retries++) {
Review comment:
@functioner Thanks for the explanation.
Adding an explicit IOException alters the purpose of the implementation. In
order to observe the symptom of client, the faultInjection needs to be closer
to what the code was implemented for. For example, instead of throwing an
exception, a faultijection should be added to the rpc-call so that
`call.sendResponse()/call.abortResponse(syncEx)` would throw the appropriate
exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17471) ABFS to collect IOStatistics
[
https://issues.apache.org/jira/browse/HADOOP-17471?focusedWorklogId=582646=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582646
]
ASF GitHub Bot logged work on HADOOP-17471:
---
Author: ASF GitHub Bot
Created on: 14/Apr/21 16:04
Start Date: 14/Apr/21 16:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2731:
URL: https://github.com/apache/hadoop/pull/2731#issuecomment-819633848
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 36s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 23m 9s | | trunk passed |
| +1 :green_heart: | compile | 22m 38s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 19m 18s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 4m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 35s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 4s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 24s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 47s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 21s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 21m 55s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 21m 55s | | the patch passed |
| +1 :green_heart: | compile | 19m 16s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 19m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 2s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 10s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 33s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 36s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 2s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 208m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2731/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2731 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 8b5358ce85a7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 16d11277eeb86d0dbe6a423f9464f7e09227bf93 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2731/5/testReport/ |
| Max. process+thread count | 1274 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure
U:
[GitHub] [hadoop] functioner commented on a change in pull request #2878: HDFS-15957. The ignored IOException in the RPC response sent by FSEditLogAsync can cause the HDFS client to hang
functioner commented on a change in pull request #2878:
URL: https://github.com/apache/hadoop/pull/2878#discussion_r613361072
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java
##
@@ -378,13 +381,18 @@ public void logSyncWait() {
@Override
public void logSyncNotify(RuntimeException syncEx) {
- try {
-if (syncEx == null) {
- call.sendResponse();
-} else {
- call.abortResponse(syncEx);
+ for (int retries = 0; retries <= RESPONSE_SEND_RETRIES; retries++) {
Review comment:
@daryn-sharp, in our report
([HDFS-15957](https://issues.apache.org/jira/browse/HDFS-15957)), we are doing
fault injection testing, and we inject an IOException in `call.sendResponse()`,
and then we observe the symptom that the client gets stuck. In this scenario,
retry can help.
> If the response cannot be sent it's either because the connection is
already closed or there's a bug preventing the encoding of the response.
Within `call.sendResponse()`, there are lots of code; some of them is
related to the OS kernel I/O, and some of them may get changed in the future,
so I think (now & future) there should be multiple reasons for an IOException,
besides the two you list here. For example, for transient connection issues,
retry would help. Furthermore, in the scenarios you describe, retry wouldn't
make it worst.
Actually, it's not like retry won't help with connection issue, it's just a
matter of whether our fix implements the retry correctly or not, i.e., whether
we should re-create a connection object or not. Therefore, I think it's still
worth adding the retry logic here, although it might not be able to handle the
two scenarios you describe here. Do you agree?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2731: HADOOP-17471. ABFS to collect IOStatistics
hadoop-yetus commented on pull request #2731:
URL: https://github.com/apache/hadoop/pull/2731#issuecomment-819633848
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 36s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 23m 9s | | trunk passed |
| +1 :green_heart: | compile | 22m 38s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 19m 18s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 4m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 35s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 4s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 24s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 47s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 21s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 21m 55s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 21m 55s | | the patch passed |
| +1 :green_heart: | compile | 19m 16s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 19m 16s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 2s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 10s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 33s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 3s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 5s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 36s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 2s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 45s | | The patch does not
generate ASF License warnings. |
| | | 208m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2731/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2731 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 8b5358ce85a7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 16d11277eeb86d0dbe6a423f9464f7e09227bf93 |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2731/5/testReport/ |
| Max. process+thread count | 1274 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-azure
U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2731/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
[GitHub] [hadoop] functioner commented on a change in pull request #2878: HDFS-15957. The ignored IOException in the RPC response sent by FSEditLogAsync can cause the HDFS client to hang
functioner commented on a change in pull request #2878:
URL: https://github.com/apache/hadoop/pull/2878#discussion_r613361072
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java
##
@@ -378,13 +381,18 @@ public void logSyncWait() {
@Override
public void logSyncNotify(RuntimeException syncEx) {
- try {
-if (syncEx == null) {
- call.sendResponse();
-} else {
- call.abortResponse(syncEx);
+ for (int retries = 0; retries <= RESPONSE_SEND_RETRIES; retries++) {
Review comment:
@daryn-sharp, in our report
([HDFS-15957](https://issues.apache.org/jira/browse/HDFS-15957)), we are doing
fault injection testing, and we inject an IOException in `call.sendResponse()`,
and then we observe the symptom that the client gets stuck. In this scenario,
retry can help.
> If the response cannot be sent it's either because the connection is
already closed or there's a bug preventing the encoding of the response.
Within `call.sendResponse()`, there are lots of code; some of them is
related to the OS kernel I/O, and some of them may get changed in the future,
so I think (now & future) there should be multiple reasons for an IOException,
besides the two you list here. Furthermore, in the scenarios you describe,
retry wouldn't make it worst.
Therefore, I think it's still worth adding the retry logic here, although it
might not be able to handle the two scenarios you describe here. Do you agree?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao commented on pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang
Hexiaoqiao commented on pull request #2737: URL: https://github.com/apache/hadoop/pull/2737#issuecomment-819615655 > I think we also need to add some comments in the code to show why we don't shutdown this executor, in case some developers may get confused, because at the first glance it's also normal for people to think that this executor should shutdown on `close`. +1. > Furthermore, I think your argument implies that the `FSEditLogAsync` is singleton in terms of the namenode process, otherwise we may create multiple executors without shutting down any of them. Is `FSEditLogAsync` really always singleton (now & future)? IMO, it is true that `FSEditLogAsync` is singleton instance for NameNode process. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] functioner commented on pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang
functioner commented on pull request #2737: URL: https://github.com/apache/hadoop/pull/2737#issuecomment-819605128 @Hexiaoqiao Thanks for the comment, I removed `logSyncNotifyExecutor.shutdown()`. I think we also need to add some comments in the code to show why we don't shutdown this executor, in case some developers may get confused, because at the first glance it's also normal for people to think that this executor should shutdown on `close`. Furthermore, I think your argument implies that the `FSEditLogAsync` is singleton in terms of the namenode process, otherwise we may create multiple executors without shutting down any of them. Is `FSEditLogAsync` really always singleton (now & future)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] szilard-nemeth merged pull request #2906: YARN-10736. Fix GetApplicationsRequest JavaDoc. Contributed by Miklos Gergely.
szilard-nemeth merged pull request #2906: URL: https://github.com/apache/hadoop/pull/2906 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] szilard-nemeth commented on pull request #2906: YARN-10736. Fix GetApplicationsRequest JavaDoc. Contributed by Miklos Gergely.
szilard-nemeth commented on pull request #2906: URL: https://github.com/apache/hadoop/pull/2906#issuecomment-819599123 @miklosgergely Thanks for working on this patch. The build only fails because of test4tests, which is okay in this case. LGTM, committed to trunk. Thanks @virajjasani for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong
tomscut commented on pull request #2907: URL: https://github.com/apache/hadoop/pull/2907#issuecomment-819592967 > Nice one. For metrics and statistics, `LongAdder` provides much better throughput. Thanks @virajjasani for the review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] daryn-sharp commented on a change in pull request #2878: HDFS-15957. The ignored IOException in the RPC response sent by FSEditLogAsync can cause the HDFS client to hang
daryn-sharp commented on a change in pull request #2878:
URL: https://github.com/apache/hadoop/pull/2878#discussion_r61355
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java
##
@@ -378,13 +381,18 @@ public void logSyncWait() {
@Override
public void logSyncNotify(RuntimeException syncEx) {
- try {
-if (syncEx == null) {
- call.sendResponse();
-} else {
- call.abortResponse(syncEx);
+ for (int retries = 0; retries <= RESPONSE_SEND_RETRIES; retries++) {
Review comment:
This appears to be a "solution" to the contrived fault injection. If
the response cannot be sent it's either because the connection is already
closed or there's a bug preventing the encoding of the response. In either
case, retrying is not going to help.
Have you observed a real problem or just noticed this by casual inspection
of the code?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] GauthamBanasandra opened a new pull request #2908: HDFS-15976. Make mkdtemp cross platform
GauthamBanasandra opened a new pull request #2908: URL: https://github.com/apache/hadoop/pull/2908 * mkdtemp is used for creating temporary directory, adhering to the given pattern. It's not available on Visual C++. Need to make this cross platform. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #2737: HDFS-15869. Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang
Hexiaoqiao commented on a change in pull request #2737:
URL: https://github.com/apache/hadoop/pull/2737#discussion_r613305815
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogAsync.java
##
@@ -117,6 +125,7 @@ void openForWrite(int layoutVersion) throws IOException {
public void close() {
super.close();
stopSyncThread();
+logSyncNotifyExecutor.shutdown();
Review comment:
It could meet issue when transition active to standby then transition
back here. Because executor has been shutdown and no response will be sent.
Moreover I suspect that namenode process could fatal in this case. In my
opinion, we do not need to shutdown this executor. FYI. Thanks.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tomscut opened a new pull request #2907: HDFS-15975. Use LongAdder instead of AtomicLong
tomscut opened a new pull request #2907: URL: https://github.com/apache/hadoop/pull/2907 JIRA: [HDFS-15975](https://issues.apache.org/jira/browse/HDFS-15975) When counting some indicators, we can use LongAdder instead of AtomicLong to improve performance. The long value is not an atomic snapshot in LongAdder, but I think we can tolerate that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17633) Please upgrade json-smart dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-17633?focusedWorklogId=582539=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582539 ] ASF GitHub Bot logged work on HADOOP-17633: --- Author: ASF GitHub Bot Created on: 14/Apr/21 14:25 Start Date: 14/Apr/21 14:25 Worklog Time Spent: 10m Work Description: virajjasani commented on a change in pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#discussion_r613292135 ## File path: hadoop-project/pom.xml ## @@ -210,6 +210,8 @@ 1.0.7.Final 1.0.2 5.3.0 +2.4.2 +9.8 Review comment: That's correct, 9.8.1 seems latest. Maven repository showed highest usage for 9.8 and 9.8.1 is quite recent, hence I thought of using 9.8. However, I think we can use 9.8.1 as things look good with 9.8 so there should not be much difference with 9.8.1 either. Edit: Just realized 9.8 is also recent only, so I agree we should go with 9.8.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582539) Time Spent: 3h (was: 2h 50m) > Please upgrade json-smart dependency to the latest version > -- > > Key: HADOOP-17633 > URL: https://issues.apache.org/jira/browse/HADOOP-17633 > Project: Hadoop Common > Issue Type: Improvement > Components: auth, build >Affects Versions: 3.3.0, 3.2.1, 3.2.2, 3.4.0 >Reporter: helen huang >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > Please upgrade the json-smart dependency to the latest version available. > Currently hadoop-auth is using version 2.3. Fortify scan picked up a security > issue with this version. Please upgrade to the latest version. > Thanks! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2906: YARN-10736. Fix GetApplicationsRequest JavaDoc. Contributed by Miklos Gergely.
hadoop-yetus commented on pull request #2906:
URL: https://github.com/apache/hadoop/pull/2906#issuecomment-819559929
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 38s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 47s | | trunk passed |
| +1 :green_heart: | compile | 0m 51s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 28s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 51s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 55s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 2s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 37s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 43s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 29s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 58s | | the patch passed |
| +1 :green_heart: | shadedclient | 15m 48s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 53s | | hadoop-yarn-api in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 31s | | The patch does not
generate ASF License warnings. |
| | | 80m 35s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2906/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2906 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 0effd44a6961 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0665119764b7811c06312cbb89a3bede792c059f |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2906/1/testReport/ |
| Max. process+thread count | 540 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2906/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact
[GitHub] [hadoop] virajjasani commented on a change in pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs
virajjasani commented on a change in pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#discussion_r613292135 ## File path: hadoop-project/pom.xml ## @@ -210,6 +210,8 @@ 1.0.7.Final 1.0.2 5.3.0 +2.4.2 +9.8 Review comment: That's correct, 9.8.1 seems latest. Maven repository showed highest usage for 9.8 and 9.8.1 is quite recent, hence I thought of using 9.8. However, I think we can use 9.8.1 as things look good with 9.8 so there should not be much difference with 9.8.1 either. Edit: Just realized 9.8 is also recent only, so I agree we should go with 9.8.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17633) Please upgrade json-smart dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-17633?focusedWorklogId=582537=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582537 ] ASF GitHub Bot logged work on HADOOP-17633: --- Author: ASF GitHub Bot Created on: 14/Apr/21 14:21 Start Date: 14/Apr/21 14:21 Worklog Time Spent: 10m Work Description: virajjasani commented on a change in pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#discussion_r613292135 ## File path: hadoop-project/pom.xml ## @@ -210,6 +210,8 @@ 1.0.7.Final 1.0.2 5.3.0 +2.4.2 +9.8 Review comment: That's correct, 9.8.1 seems latest. Maven repository showed highest usage for 9.8 and 9.8.1 is quite recent, hence I thought of using 9.8. However, I think we can use 9.8.1 as things look good with 9.8 so there should not be much difference with 9.8.1 either. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582537) Time Spent: 2h 50m (was: 2h 40m) > Please upgrade json-smart dependency to the latest version > -- > > Key: HADOOP-17633 > URL: https://issues.apache.org/jira/browse/HADOOP-17633 > Project: Hadoop Common > Issue Type: Improvement > Components: auth, build >Affects Versions: 3.3.0, 3.2.1, 3.2.2, 3.4.0 >Reporter: helen huang >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > Please upgrade the json-smart dependency to the latest version available. > Currently hadoop-auth is using version 2.3. Fortify scan picked up a security > issue with this version. Please upgrade to the latest version. > Thanks! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on a change in pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs
virajjasani commented on a change in pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#discussion_r613292135 ## File path: hadoop-project/pom.xml ## @@ -210,6 +210,8 @@ 1.0.7.Final 1.0.2 5.3.0 +2.4.2 +9.8 Review comment: That's correct, 9.8.1 seems latest. Maven repository showed highest usage for 9.8 and 9.8.1 is quite recent, hence I thought of using 9.8. However, I think we can use 9.8.1 as things look good with 9.8 so there should not be much difference with 9.8.1 either. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: Test Pre-Commits.
hadoop-yetus commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-819551391 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2860/12/console in case of problems. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17633) Please upgrade json-smart dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-17633?focusedWorklogId=582532=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582532 ] ASF GitHub Bot logged work on HADOOP-17633: --- Author: ASF GitHub Bot Created on: 14/Apr/21 14:08 Start Date: 14/Apr/21 14:08 Worklog Time Spent: 10m Work Description: ayushtkn commented on a change in pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#discussion_r613280902 ## File path: hadoop-project/pom.xml ## @@ -210,6 +210,8 @@ 1.0.7.Final 1.0.2 5.3.0 +2.4.2 +9.8 Review comment: Guess 9.8.1 is also available? Or did I check it wrong, If so we can choose that? Otherwise the build seems happy, -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582532) Time Spent: 2h 40m (was: 2.5h) > Please upgrade json-smart dependency to the latest version > -- > > Key: HADOOP-17633 > URL: https://issues.apache.org/jira/browse/HADOOP-17633 > Project: Hadoop Common > Issue Type: Improvement > Components: auth, build >Affects Versions: 3.3.0, 3.2.1, 3.2.2, 3.4.0 >Reporter: helen huang >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > Please upgrade the json-smart dependency to the latest version available. > Currently hadoop-auth is using version 2.3. Fortify scan picked up a security > issue with this version. Please upgrade to the latest version. > Thanks! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ayushtkn commented on a change in pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs
ayushtkn commented on a change in pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#discussion_r613280902 ## File path: hadoop-project/pom.xml ## @@ -210,6 +210,8 @@ 1.0.7.Final 1.0.2 5.3.0 +2.4.2 +9.8 Review comment: Guess 9.8.1 is also available? Or did I check it wrong, If so we can choose that? Otherwise the build seems happy, -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-11245) Update NFS gateway to use Netty4
[
https://issues.apache.org/jira/browse/HADOOP-11245?focusedWorklogId=582476=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582476
]
ASF GitHub Bot logged work on HADOOP-11245:
---
Author: ASF GitHub Bot
Created on: 14/Apr/21 13:16
Start Date: 14/Apr/21 13:16
Worklog Time Spent: 10m
Work Description: szetszwo commented on a change in pull request #2832:
URL: https://github.com/apache/hadoop/pull/2832#discussion_r613193567
##
File path:
hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleTcpServer.java
##
@@ -59,35 +64,41 @@ public SimpleTcpServer(int port, RpcProgram program, int
workercount) {
public void run() {
// Configure the Server.
-ChannelFactory factory;
+bossGroup = new NioEventLoopGroup();
+
if (workerCount == 0) {
// Use default workers: 2 * the number of available processors
- factory = new NioServerSocketChannelFactory(
- Executors.newCachedThreadPool(), Executors.newCachedThreadPool());
+ workerGroup = new NioEventLoopGroup(0, Executors.newCachedThreadPool());
} else {
- factory = new NioServerSocketChannelFactory(
- Executors.newCachedThreadPool(), Executors.newCachedThreadPool(),
- workerCount);
+ workerGroup = new NioEventLoopGroup(workerCount,
Executors.newCachedThreadPool());
}
Review comment:
The if-else is no longer need. Simply pass workerCount for both cases
as below.
```
workerGroup = new NioEventLoopGroup(workerCount,
Executors.newCachedThreadPool());
```
##
File path:
hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleTcpServer.java
##
@@ -39,9 +42,11 @@
LoggerFactory.getLogger(SimpleTcpServer.class);
protected final int port;
protected int boundPort = -1; // Will be set after server starts
- protected final SimpleChannelUpstreamHandler rpcProgram;
+ protected final ChannelInboundHandlerAdapter rpcProgram;
private ServerBootstrap server;
private Channel ch;
+ private EventLoopGroup bossGroup;
+ private EventLoopGroup workerGroup;
Review comment:
Should the non-final fields (boundPort, server, ch, bossGroup,
workerGroup) be volatile? Not sure if they are synchronized in somewhere.
##
File path:
hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/XDR.java
##
@@ -242,26 +242,26 @@ public static boolean verifyLength(XDR xdr, int len) {
* @param last specifies last request or not
* @return TCP buffer
*/
- public static ChannelBuffer writeMessageTcp(XDR request, boolean last) {
+ public static ByteBuf writeMessageTcp(XDR request, boolean last) {
Preconditions.checkState(request.state == XDR.State.WRITING);
ByteBuffer b = request.buf.duplicate();
b.flip();
byte[] fragmentHeader = XDR.recordMark(b.limit(), last);
ByteBuffer headerBuf = ByteBuffer.wrap(fragmentHeader);
// TODO: Investigate whether making a copy of the buffer is necessary.
-return ChannelBuffers.copiedBuffer(headerBuf, b);
+return Unpooled.wrappedBuffer(headerBuf, b);
Review comment:
The new code uses wrappedBuffer but old code uses copiedBuffer. Is it
intended?
##
File path: hadoop-common-project/hadoop-nfs/src/test/resources/log4j.properties
##
@@ -15,4 +15,4 @@ log4j.rootLogger=info,stdout
log4j.threshold=ALL
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
-log4j.appender.stdout.layout.ConversionPattern=%d{ISO8601} %-5p %c{2}
(%F:%M(%L)) - %m%n
+log4j.appender.stdout.layout.ConversionPattern=%d{ISO8601} %-5p %c{2}
(%F:%M(%L)) - %m%n
Review comment:
Let's revert this whitespace change in order to keep log4j.properties
untouched.
##
File path:
hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/portmap/TestPortmap.java
##
@@ -92,6 +92,7 @@ public void testRegistration() throws IOException,
InterruptedException {
DatagramPacket p = new DatagramPacket(reqBuf, reqBuf.length,
pm.getUdpServerLoAddress());
try {
+
Review comment:
Let's revert this whitespace change.
##
File path:
hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleUdpServer.java
##
@@ -84,8 +97,6 @@ public void shutdown() {
if (ch != null) {
ch.close().awaitUninterruptibly();
}
-if (server != null) {
- server.releaseExternalResources();
-}
+workerGroup.shutdownGracefully();
Review comment:
Check null.
##
File path:
hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/portmap/Portmap.java
##
@@ -95,36 +108,59 @@ RpcProgramPortmap getHandler() {
void start(final int idleTimeMilliSeconds, final SocketAddress tcpAddress,
[GitHub] [hadoop] szetszwo commented on a change in pull request #2832: HADOOP-11245. Update NFS gateway to use Netty4
szetszwo commented on a change in pull request #2832:
URL: https://github.com/apache/hadoop/pull/2832#discussion_r613193567
##
File path:
hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleTcpServer.java
##
@@ -59,35 +64,41 @@ public SimpleTcpServer(int port, RpcProgram program, int
workercount) {
public void run() {
// Configure the Server.
-ChannelFactory factory;
+bossGroup = new NioEventLoopGroup();
+
if (workerCount == 0) {
// Use default workers: 2 * the number of available processors
- factory = new NioServerSocketChannelFactory(
- Executors.newCachedThreadPool(), Executors.newCachedThreadPool());
+ workerGroup = new NioEventLoopGroup(0, Executors.newCachedThreadPool());
} else {
- factory = new NioServerSocketChannelFactory(
- Executors.newCachedThreadPool(), Executors.newCachedThreadPool(),
- workerCount);
+ workerGroup = new NioEventLoopGroup(workerCount,
Executors.newCachedThreadPool());
}
Review comment:
The if-else is no longer need. Simply pass workerCount for both cases
as below.
```
workerGroup = new NioEventLoopGroup(workerCount,
Executors.newCachedThreadPool());
```
##
File path:
hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleTcpServer.java
##
@@ -39,9 +42,11 @@
LoggerFactory.getLogger(SimpleTcpServer.class);
protected final int port;
protected int boundPort = -1; // Will be set after server starts
- protected final SimpleChannelUpstreamHandler rpcProgram;
+ protected final ChannelInboundHandlerAdapter rpcProgram;
private ServerBootstrap server;
private Channel ch;
+ private EventLoopGroup bossGroup;
+ private EventLoopGroup workerGroup;
Review comment:
Should the non-final fields (boundPort, server, ch, bossGroup,
workerGroup) be volatile? Not sure if they are synchronized in somewhere.
##
File path:
hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/XDR.java
##
@@ -242,26 +242,26 @@ public static boolean verifyLength(XDR xdr, int len) {
* @param last specifies last request or not
* @return TCP buffer
*/
- public static ChannelBuffer writeMessageTcp(XDR request, boolean last) {
+ public static ByteBuf writeMessageTcp(XDR request, boolean last) {
Preconditions.checkState(request.state == XDR.State.WRITING);
ByteBuffer b = request.buf.duplicate();
b.flip();
byte[] fragmentHeader = XDR.recordMark(b.limit(), last);
ByteBuffer headerBuf = ByteBuffer.wrap(fragmentHeader);
// TODO: Investigate whether making a copy of the buffer is necessary.
-return ChannelBuffers.copiedBuffer(headerBuf, b);
+return Unpooled.wrappedBuffer(headerBuf, b);
Review comment:
The new code uses wrappedBuffer but old code uses copiedBuffer. Is it
intended?
##
File path: hadoop-common-project/hadoop-nfs/src/test/resources/log4j.properties
##
@@ -15,4 +15,4 @@ log4j.rootLogger=info,stdout
log4j.threshold=ALL
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
-log4j.appender.stdout.layout.ConversionPattern=%d{ISO8601} %-5p %c{2}
(%F:%M(%L)) - %m%n
+log4j.appender.stdout.layout.ConversionPattern=%d{ISO8601} %-5p %c{2}
(%F:%M(%L)) - %m%n
Review comment:
Let's revert this whitespace change in order to keep log4j.properties
untouched.
##
File path:
hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/portmap/TestPortmap.java
##
@@ -92,6 +92,7 @@ public void testRegistration() throws IOException,
InterruptedException {
DatagramPacket p = new DatagramPacket(reqBuf, reqBuf.length,
pm.getUdpServerLoAddress());
try {
+
Review comment:
Let's revert this whitespace change.
##
File path:
hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleUdpServer.java
##
@@ -84,8 +97,6 @@ public void shutdown() {
if (ch != null) {
ch.close().awaitUninterruptibly();
}
-if (server != null) {
- server.releaseExternalResources();
-}
+workerGroup.shutdownGracefully();
Review comment:
Check null.
##
File path:
hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/portmap/Portmap.java
##
@@ -95,36 +108,59 @@ RpcProgramPortmap getHandler() {
void start(final int idleTimeMilliSeconds, final SocketAddress tcpAddress,
final SocketAddress udpAddress) {
-tcpServer = new ServerBootstrap(new NioServerSocketChannelFactory(
-Executors.newCachedThreadPool(), Executors.newCachedThreadPool()));
-tcpServer.setPipelineFactory(new ChannelPipelineFactory() {
- private final HashedWheelTimer timer = new HashedWheelTimer();
- private final IdleStateHandler idleStateHandler = new IdleStateHandler(
- timer, 0, 0,
[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context
[
https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=582464=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582464
]
ASF GitHub Bot logged work on HADOOP-17511:
---
Author: ASF GitHub Bot
Created on: 14/Apr/21 13:04
Start Date: 14/Apr/21 13:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2807:
URL: https://github.com/apache/hadoop/pull/2807#issuecomment-818230736
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 3s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 43 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 36s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 7s | | trunk passed |
| +1 :green_heart: | compile | 20m 43s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 18m 0s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 40s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 41s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 19m 49s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 19m 48s |
[/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 generated 1 new + 1936 unchanged - 1
fixed = 1937 total (was 1937) |
| +1 :green_heart: | compile | 18m 1s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| -1 :x: | javac | 18m 1s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt)
| root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 1 new + 1831
unchanged - 1 fixed = 1832 total (was 1832) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 3m 49s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 7 new + 185 unchanged - 4 fixed = 192 total (was
189) |
| +1 :green_heart: | mvnsite | 2m 24s | | the patch passed |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 1m 40s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javadoc | 0m 45s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 6 new +
80
[jira] [Work logged] (HADOOP-17511) Add an Audit plugin point for S3A auditing/context
[
https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=582463=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582463
]
ASF GitHub Bot logged work on HADOOP-17511:
---
Author: ASF GitHub Bot
Created on: 14/Apr/21 13:04
Start Date: 14/Apr/21 13:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2807:
URL: https://github.com/apache/hadoop/pull/2807#issuecomment-818255439
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 43 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 9s | | trunk passed |
| +1 :green_heart: | compile | 20m 32s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 17m 51s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 50s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 41s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 22s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 33s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 19m 49s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 19m 49s |
[/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 generated 1 new + 1937 unchanged - 1
fixed = 1938 total (was 1938) |
| +1 :green_heart: | compile | 17m 55s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| -1 :x: | javac | 17m 55s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt)
| root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 1 new + 1832
unchanged - 1 fixed = 1833 total (was 1833) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 3m 44s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 7 new + 185 unchanged - 4 fixed = 192 total (was
189) |
| +1 :green_heart: | mvnsite | 2m 23s | | the patch passed |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 1m 40s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javadoc | 0m 45s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 6 new +
80
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2807: HADOOP-17511. Add audit/telemetry logging to S3A connector
hadoop-yetus removed a comment on pull request #2807:
URL: https://github.com/apache/hadoop/pull/2807#issuecomment-818230736
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 3s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 43 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 36s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 7s | | trunk passed |
| +1 :green_heart: | compile | 20m 43s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 18m 0s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 40s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 21s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 41s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 19m 49s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 19m 48s |
[/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 generated 1 new + 1936 unchanged - 1
fixed = 1937 total (was 1937) |
| +1 :green_heart: | compile | 18m 1s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| -1 :x: | javac | 18m 1s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt)
| root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 1 new + 1831
unchanged - 1 fixed = 1832 total (was 1832) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 3m 49s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 7 new + 185 unchanged - 4 fixed = 192 total (was
189) |
| +1 :green_heart: | mvnsite | 2m 24s | | the patch passed |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 1m 40s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javadoc | 0m 45s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 6 new +
80 unchanged - 8 fixed = 86 total (was 88) |
| -1 :x: | spotbugs | 1m 32s |
[/new-spotbugs-hadoop-tools_hadoop-aws.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/10/artifact/out/new-spotbugs-hadoop-tools_hadoop-aws.html)
| hadoop-tools/hadoop-aws generated 3 new + 0 unchanged - 0 fixed = 3 total
(was 0) |
| +1 :green_heart: | shadedclient | 14m 51s | | patch has no errors
when
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2807: HADOOP-17511. Add audit/telemetry logging to S3A connector
hadoop-yetus removed a comment on pull request #2807:
URL: https://github.com/apache/hadoop/pull/2807#issuecomment-818255439
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 43 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 24s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 9s | | trunk passed |
| +1 :green_heart: | compile | 20m 32s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 17m 51s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 3m 50s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 41s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 22s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 3m 33s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 26s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 19m 49s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javac | 19m 49s |
[/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt)
| root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 generated 1 new + 1937 unchanged - 1
fixed = 1938 total (was 1938) |
| +1 :green_heart: | compile | 17m 55s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| -1 :x: | javac | 17m 55s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt)
| root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 1 new + 1832
unchanged - 1 fixed = 1833 total (was 1833) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/blanks-eol.txt)
| The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 3m 44s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 7 new + 185 unchanged - 4 fixed = 192 total (was
189) |
| +1 :green_heart: | mvnsite | 2m 23s | | the patch passed |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 1m 40s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| -1 :x: | javadoc | 0m 45s |
[/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 generated 6 new +
80 unchanged - 8 fixed = 86 total (was 88) |
| -1 :x: | spotbugs | 1m 32s |
[/new-spotbugs-hadoop-tools_hadoop-aws.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2807/11/artifact/out/new-spotbugs-hadoop-tools_hadoop-aws.html)
| hadoop-tools/hadoop-aws generated 3 new + 0 unchanged - 0 fixed = 3 total
(was 0) |
| +1 :green_heart: | shadedclient | 14m 44s | | patch has no errors
when
[GitHub] [hadoop] miklosgergely opened a new pull request #2906: YARN-10736. Fix GetApplicationsRequest JavaDoc. Contributed by Miklos Gergely.
miklosgergely opened a new pull request #2906: URL: https://github.com/apache/hadoop/pull/2906 getName and setName javadoc comments are mixed up -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on pull request #2889: HDFS-15963. Unreleased volume references cause an infinite loop.
jojochuang commented on pull request #2889: URL: https://github.com/apache/hadoop/pull/2889#issuecomment-819475596 Please fix the spotbugs warning. Other than that I am +1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bshashikant commented on pull request #2881: HDFS-15961. standby namenode failed to start ordered snapshot deletion is enabled while having snapshottable directories
bshashikant commented on pull request #2881: URL: https://github.com/apache/hadoop/pull/2881#issuecomment-819454476 > > I think We should hold this off, I think the code has issues, as I said. Earlier I thought, there is some catch but I don’t think. It is misbehaving only. Ideally such features should go in a branch first > > @ayushtkn , IMO its not a misbehaviour here. Once the snapshotTrash feature is enabled, the .Trash directory has to be present to make the feature work. As you can have pre existing snapshottable directories , one way to create the .Trash inside these was to create right after the startup is done. The other solution was to explicitly provision the .Trash using a cmd line option. The choice was made to do it automatically on restart, and fail the Namenode in case any any issues occur. > > The other solution is not fail the namenode , but log an warning and later, if any Trash operations gets performed inside the snapshottable root, will fail. > > The discussion is here: [#2682 (comment)](https://github.com/apache/hadoop/pull/2682#discussion_r570461526) Coming to think of it, if providing an external command to create the Trash directory by admins is feasible and makes sense, i think its ok to remove the NN startup logic to create Trash directories inside snapshottable root. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17633) Please upgrade json-smart dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-17633?focusedWorklogId=582400=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582400 ] ASF GitHub Bot logged work on HADOOP-17633: --- Author: ASF GitHub Bot Created on: 14/Apr/21 11:32 Start Date: 14/Apr/21 11:32 Worklog Time Spent: 10m Work Description: virajjasani commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-819448373 > And I now know how to force a full retest after a JAR update. +1, I also feel so good to know about this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582400) Time Spent: 2.5h (was: 2h 20m) > Please upgrade json-smart dependency to the latest version > -- > > Key: HADOOP-17633 > URL: https://issues.apache.org/jira/browse/HADOOP-17633 > Project: Hadoop Common > Issue Type: Improvement > Components: auth, build >Affects Versions: 3.3.0, 3.2.1, 3.2.2, 3.4.0 >Reporter: helen huang >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > Please upgrade the json-smart dependency to the latest version available. > Currently hadoop-auth is using version 2.3. Fortify scan picked up a security > issue with this version. Please upgrade to the latest version. > Thanks! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs
virajjasani commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-819448373 > And I now know how to force a full retest after a JAR update. +1, I also feel so good to know about this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17633) Please upgrade json-smart dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-17633?focusedWorklogId=582388=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582388 ] ASF GitHub Bot logged work on HADOOP-17633: --- Author: ASF GitHub Bot Created on: 14/Apr/21 11:11 Start Date: 14/Apr/21 11:11 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-819437080 well, we did successfully run the full 17+ build test run. And I now know how to force a full retest after a JAR update. Looking at the test failures, I'm going to say "unrelated"...certainly the ZK one is known to be brittle -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582388) Time Spent: 2h 20m (was: 2h 10m) > Please upgrade json-smart dependency to the latest version > -- > > Key: HADOOP-17633 > URL: https://issues.apache.org/jira/browse/HADOOP-17633 > Project: Hadoop Common > Issue Type: Improvement > Components: auth, build >Affects Versions: 3.3.0, 3.2.1, 3.2.2, 3.4.0 >Reporter: helen huang >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Please upgrade the json-smart dependency to the latest version available. > Currently hadoop-auth is using version 2.3. Fortify scan picked up a security > issue with this version. Please upgrade to the latest version. > Thanks! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs
steveloughran commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-819437080 well, we did successfully run the full 17+ build test run. And I now know how to force a full retest after a JAR update. Looking at the test failures, I'm going to say "unrelated"...certainly the ZK one is known to be brittle -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2819: HDFS-15923. Authentication failed when rename accross sub clusters.
hadoop-yetus commented on pull request #2819:
URL: https://github.com/apache/hadoop/pull/2819#issuecomment-819407274
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 0s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 36m 27s | | trunk passed |
| +1 :green_heart: | compile | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 0m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 36s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 51s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 23s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 8s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 0m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 16s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2819/3/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-project/hadoop-hdfs-rbf: The patch generated 5 new + 0
unchanged - 0 fixed = 5 total (was 0) |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 0m 50s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 1m 23s | | the patch passed |
| +1 :green_heart: | shadedclient | 17m 9s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 25m 2s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2819/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 0m 28s | | The patch does not
generate ASF License warnings. |
| | | 108m 52s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.federation.router.TestRouterRpc |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2819/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2819 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 3f4a540e031b 4.15.0-101-generic #102-Ubuntu SMP Mon May 11
10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8a2cd53a4d27271f9498f4e02e3901d17c5c1a9c |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2819/3/testReport/ |
| Max. process+thread count | 2062 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-rbf U:
hadoop-hdfs-project/hadoop-hdfs-rbf |
| Console output |
[jira] [Work logged] (HADOOP-17611) Distcp parallel file copy breaks the modification time
[ https://issues.apache.org/jira/browse/HADOOP-17611?focusedWorklogId=582333=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582333 ] ASF GitHub Bot logged work on HADOOP-17611: --- Author: ASF GitHub Bot Created on: 14/Apr/21 10:03 Start Date: 14/Apr/21 10:03 Worklog Time Spent: 10m Work Description: amaroti edited a comment on pull request #2897: URL: https://github.com/apache/hadoop/pull/2897#issuecomment-819387790 @bgaborg I have tested this manully by hand using two clusters. I have not yet looked into how the unit tests look like for hadoop. I will take a look at it. Also Ayush Saxena (@ayushtkn) had made some usefull points on the jira ticket that I will take a look at shortly: https://issues.apache.org/jira/browse/HADOOP-17611?focusedCommentId=17320445=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17320445 > Seems there are two PRs, More or less doing the same thing I guess, I just had a glance on the second one, > > So, who ever plans to chase this, Couple of points to keep in mind: > > We need a test in AbstractContractDistCpTest which all the FileSystems can also use > Should cover two scenarios. First When preserve Time is specified it should preserve time and when not it shouldn't in case of parallel copy. The latter case is working ok as of now, To make sure we don't change the behaviour > The parent modification time is to be preserved when the parent is in the scope of copy, not always. say your are copying /dir/fil1 to /dir1/file2 using parallel copy, then we don't touch /dir1 AFAIK > > The above are the basic requirements, Now the below stuff, If possible we should do: > > For parent directories preserve only once, say if you have 10K files under that parent, then do that setTimes 10K times. > And if the parallel copy is enabled, there is no point of preserving before concat operation, we can save that call. > > > > This isn't a one liner, and throw some challenges, So, please decide who wants to chase this and together work on one PR only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582333) Time Spent: 2h 50m (was: 2h 40m) > Distcp parallel file copy breaks the modification time > -- > > Key: HADOOP-17611 > URL: https://issues.apache.org/jira/browse/HADOOP-17611 > Project: Hadoop Common > Issue Type: Bug >Reporter: Adam Maroti >Assignee: Adam Maroti >Priority: Major > Labels: pull-request-available > Time Spent: 2h 50m > Remaining Estimate: 0h > > The commit HADOOP-11794. Enable distcp to copy blocks in parallel. > (bf3fb585aaf2b179836e139c041fc87920a3c886) broke the modification time of > large files. > > In CopyCommitter.java inside concatFileChunks Filesystem.concat is called > which changes the modification time therefore the modification times of files > copeid by distcp will not match the source files. However this only occurs > for large enough files, which are copied by splitting them up by distcp. > In concatFileChunks before calling concat extract the modification time and > apply that to the concatenated result-file after the concat. (probably best > -after- before the rename()). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] amaroti edited a comment on pull request #2897: HADOOP-17611 Restore modification and access times after concat
amaroti edited a comment on pull request #2897: URL: https://github.com/apache/hadoop/pull/2897#issuecomment-819387790 @bgaborg I have tested this manully by hand using two clusters. I have not yet looked into how the unit tests look like for hadoop. I will take a look at it. Also Ayush Saxena (@ayushtkn) had made some usefull points on the jira ticket that I will take a look at shortly: https://issues.apache.org/jira/browse/HADOOP-17611?focusedCommentId=17320445=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17320445 > Seems there are two PRs, More or less doing the same thing I guess, I just had a glance on the second one, > > So, who ever plans to chase this, Couple of points to keep in mind: > > We need a test in AbstractContractDistCpTest which all the FileSystems can also use > Should cover two scenarios. First When preserve Time is specified it should preserve time and when not it shouldn't in case of parallel copy. The latter case is working ok as of now, To make sure we don't change the behaviour > The parent modification time is to be preserved when the parent is in the scope of copy, not always. say your are copying /dir/fil1 to /dir1/file2 using parallel copy, then we don't touch /dir1 AFAIK > > The above are the basic requirements, Now the below stuff, If possible we should do: > > For parent directories preserve only once, say if you have 10K files under that parent, then do that setTimes 10K times. > And if the parallel copy is enabled, there is no point of preserving before concat operation, we can save that call. > > > > This isn't a one liner, and throw some challenges, So, please decide who wants to chase this and together work on one PR only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17611) Distcp parallel file copy breaks the modification time
[ https://issues.apache.org/jira/browse/HADOOP-17611?focusedWorklogId=582330=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582330 ] ASF GitHub Bot logged work on HADOOP-17611: --- Author: ASF GitHub Bot Created on: 14/Apr/21 09:46 Start Date: 14/Apr/21 09:46 Worklog Time Spent: 10m Work Description: amaroti commented on pull request #2897: URL: https://github.com/apache/hadoop/pull/2897#issuecomment-819387790 @bgaborg I have tested this manully with hand using two clusters. I have not yet looked into how the unit tests look like for hadoop. I will take a look at it. Also Ayush Saxena (@ayushtkn) had made some usefull points on the jira ticket that I will take a look at shortly: https://issues.apache.org/jira/browse/HADOOP-17611?focusedCommentId=17320445=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17320445 > Seems there are two PRs, More or less doing the same thing I guess, I just had a glance on the second one, > > So, who ever plans to chase this, Couple of points to keep in mind: > > We need a test in AbstractContractDistCpTest which all the FileSystems can also use > Should cover two scenarios. First When preserve Time is specified it should preserve time and when not it shouldn't in case of parallel copy. The latter case is working ok as of now, To make sure we don't change the behaviour > The parent modification time is to be preserved when the parent is in the scope of copy, not always. say your are copying /dir/fil1 to /dir1/file2 using parallel copy, then we don't touch /dir1 AFAIK > > The above are the basic requirements, Now the below stuff, If possible we should do: > > For parent directories preserve only once, say if you have 10K files under that parent, then do that setTimes 10K times. > And if the parallel copy is enabled, there is no point of preserving before concat operation, we can save that call. > > > > This isn't a one liner, and throw some challenges, So, please decide who wants to chase this and together work on one PR only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582330) Time Spent: 2h 40m (was: 2.5h) > Distcp parallel file copy breaks the modification time > -- > > Key: HADOOP-17611 > URL: https://issues.apache.org/jira/browse/HADOOP-17611 > Project: Hadoop Common > Issue Type: Bug >Reporter: Adam Maroti >Assignee: Adam Maroti >Priority: Major > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > The commit HADOOP-11794. Enable distcp to copy blocks in parallel. > (bf3fb585aaf2b179836e139c041fc87920a3c886) broke the modification time of > large files. > > In CopyCommitter.java inside concatFileChunks Filesystem.concat is called > which changes the modification time therefore the modification times of files > copeid by distcp will not match the source files. However this only occurs > for large enough files, which are copied by splitting them up by distcp. > In concatFileChunks before calling concat extract the modification time and > apply that to the concatenated result-file after the concat. (probably best > -after- before the rename()). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] amaroti commented on pull request #2897: HADOOP-17611 Restore modification and access times after concat
amaroti commented on pull request #2897: URL: https://github.com/apache/hadoop/pull/2897#issuecomment-819387790 @bgaborg I have tested this manully with hand using two clusters. I have not yet looked into how the unit tests look like for hadoop. I will take a look at it. Also Ayush Saxena (@ayushtkn) had made some usefull points on the jira ticket that I will take a look at shortly: https://issues.apache.org/jira/browse/HADOOP-17611?focusedCommentId=17320445=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17320445 > Seems there are two PRs, More or less doing the same thing I guess, I just had a glance on the second one, > > So, who ever plans to chase this, Couple of points to keep in mind: > > We need a test in AbstractContractDistCpTest which all the FileSystems can also use > Should cover two scenarios. First When preserve Time is specified it should preserve time and when not it shouldn't in case of parallel copy. The latter case is working ok as of now, To make sure we don't change the behaviour > The parent modification time is to be preserved when the parent is in the scope of copy, not always. say your are copying /dir/fil1 to /dir1/file2 using parallel copy, then we don't touch /dir1 AFAIK > > The above are the basic requirements, Now the below stuff, If possible we should do: > > For parent directories preserve only once, say if you have 10K files under that parent, then do that setTimes 10K times. > And if the parallel copy is enabled, there is no point of preserving before concat operation, we can save that call. > > > > This isn't a one liner, and throw some challenges, So, please decide who wants to chase this and together work on one PR only. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17604) Separate string metric from tag in hadoop metrics2
[
https://issues.apache.org/jira/browse/HADOOP-17604?focusedWorklogId=582307=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582307
]
ASF GitHub Bot logged work on HADOOP-17604:
---
Author: ASF GitHub Bot
Created on: 14/Apr/21 09:00
Start Date: 14/Apr/21 09:00
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2904:
URL: https://github.com/apache/hadoop/pull/2904#issuecomment-819357312
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 27s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 54s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 22m 59s | | trunk passed |
| +1 :green_heart: | compile | 22m 40s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 20m 21s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 4m 9s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 2s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 39s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 4m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 46s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 21s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 2s | | the patch passed |
| +1 :green_heart: | compile | 23m 25s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 23m 25s | | the patch passed |
| +1 :green_heart: | compile | 19m 32s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 19m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 0s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2904/2/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 4 new + 107 unchanged - 2 fixed = 111 total (was
109) |
| +1 :green_heart: | mvnsite | 3m 0s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 4s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 41s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 5m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 19s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 13s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 1s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 219m 53s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2904/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2904 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 9a4bbdaf3b5f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f45bb925996318cd44d5bc0f507045cd080fa96d |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
[GitHub] [hadoop] hadoop-yetus commented on pull request #2904: HADOOP-17604 Fix method metric prefix for string type
hadoop-yetus commented on pull request #2904:
URL: https://github.com/apache/hadoop/pull/2904#issuecomment-819357312
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 27s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 13m 54s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 22m 59s | | trunk passed |
| +1 :green_heart: | compile | 22m 40s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 20m 21s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | checkstyle | 4m 9s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 2s | | trunk passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 39s | | trunk passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 4m 40s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 46s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 21s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 2s | | the patch passed |
| +1 :green_heart: | compile | 23m 25s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 23m 25s | | the patch passed |
| +1 :green_heart: | compile | 19m 32s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | javac | 19m 32s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 0s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2904/2/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 4 new + 107 unchanged - 2 fixed = 111 total (was
109) |
| +1 :green_heart: | mvnsite | 3m 0s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 4s | | the patch passed with JDK
Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 2m 41s | | the patch passed with JDK
Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| +1 :green_heart: | spotbugs | 5m 11s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 58s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 19s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 13s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | unit | 2m 1s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 219m 53s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2904/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2904 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 9a4bbdaf3b5f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / f45bb925996318cd44d5bc0f507045cd080fa96d |
| Default Java | Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2904/2/testReport/ |
| Max. process+thread count | 3137 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws
hadoop-tools/hadoop-azure U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2904/2/console |
| versions |
[jira] [Commented] (HADOOP-17613) Log not flushed fully when daemon shutdown
[ https://issues.apache.org/jira/browse/HADOOP-17613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320748#comment-17320748 ] Xiaoqiao He commented on HADOOP-17613: -- [~prasad-acit] cherry-pick to branch-3.3. Thanks. > Log not flushed fully when daemon shutdown > -- > > Key: HADOOP-17613 > URL: https://issues.apache.org/jira/browse/HADOOP-17613 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 2.7.2, 3.1.1 >Reporter: Renukaprasad C >Assignee: Renukaprasad C >Priority: Major > Fix For: 3.3.1, 3.4.0 > > Attachments: HADOOP-17613.001.patch > > > When server generating large amount of logs and gets stopped, doesnt print > all the logs. Need to call LogManager.shutdown(); to flush all the pending > log to be written before shutdown. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17613) Log not flushed fully when daemon shutdown
[ https://issues.apache.org/jira/browse/HADOOP-17613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoqiao He updated HADOOP-17613: - Fix Version/s: 3.3.1 > Log not flushed fully when daemon shutdown > -- > > Key: HADOOP-17613 > URL: https://issues.apache.org/jira/browse/HADOOP-17613 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 2.7.2, 3.1.1 >Reporter: Renukaprasad C >Assignee: Renukaprasad C >Priority: Major > Fix For: 3.3.1, 3.4.0 > > Attachments: HADOOP-17613.001.patch > > > When server generating large amount of logs and gets stopped, doesnt print > all the logs. Need to call LogManager.shutdown(); to flush all the pending > log to be written before shutdown. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bshashikant edited a comment on pull request #2881: HDFS-15961. standby namenode failed to start ordered snapshot deletion is enabled while having snapshottable directories
bshashikant edited a comment on pull request #2881: URL: https://github.com/apache/hadoop/pull/2881#issuecomment-819258964 > I think We should hold this off, I think the code has issues, as I said. Earlier I thought, there is some catch but I don’t think. It is misbehaving only. Ideally such features should go in a branch first @ayushtkn , IMO its not a misbehaviour here. Once the snapshotTrash feature is enabled, the .Trash directory has to be present to make the feature work. As you can have pre existing snapshottable directories , one way to create the .Trash inside these was to create right after the startup is done. The other solution was to explicitly provision the .Trash using a cmd line option. The choice was made to do it automatically on restart, and fail the Namenode in case any any issues occur. The other solution is not fail the namenode , but log an warning and later, if any Trash operations gets performed inside the snapshottable root, will fail. The discussion is here: https://github.com/apache/hadoop/pull/2682#discussion_r570461526 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17633) Please upgrade json-smart dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-17633?focusedWorklogId=582237=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582237 ] ASF GitHub Bot logged work on HADOOP-17633: --- Author: ASF GitHub Bot Created on: 14/Apr/21 06:27 Start Date: 14/Apr/21 06:27 Worklog Time Spent: 10m Work Description: virajjasani edited a comment on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-819251359 It looks like, since we touched `LICENSE-binary` with this PR, the build is running tests for all modules and hence taking 17+ hr ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582237) Time Spent: 2h 10m (was: 2h) > Please upgrade json-smart dependency to the latest version > -- > > Key: HADOOP-17633 > URL: https://issues.apache.org/jira/browse/HADOOP-17633 > Project: Hadoop Common > Issue Type: Improvement > Components: auth, build >Affects Versions: 3.3.0, 3.2.1, 3.2.2, 3.4.0 >Reporter: helen huang >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 2h 10m > Remaining Estimate: 0h > > Please upgrade the json-smart dependency to the latest version available. > Currently hadoop-auth is using version 2.3. Fortify scan picked up a security > issue with this version. Please upgrade to the latest version. > Thanks! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani edited a comment on pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs
virajjasani edited a comment on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-819251359 It looks like, since we touched `LICENSE-binary` with this PR, the build is running tests for all modules and hence taking 17+ hr ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17633) Please upgrade json-smart dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-17633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani updated HADOOP-17633: -- Target Version/s: 3.3.1, 3.4.0, 3.1.5, 3.2.3 (was: 3.3.0, 3.4.0) > Please upgrade json-smart dependency to the latest version > -- > > Key: HADOOP-17633 > URL: https://issues.apache.org/jira/browse/HADOOP-17633 > Project: Hadoop Common > Issue Type: Improvement > Components: auth, build >Affects Versions: 3.3.0, 3.2.1, 3.2.2, 3.4.0 >Reporter: helen huang >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > Please upgrade the json-smart dependency to the latest version available. > Currently hadoop-auth is using version 2.3. Fortify scan picked up a security > issue with this version. Please upgrade to the latest version. > Thanks! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] bshashikant commented on pull request #2881: HDFS-15961. standby namenode failed to start ordered snapshot deletion is enabled while having snapshottable directories
bshashikant commented on pull request #2881: URL: https://github.com/apache/hadoop/pull/2881#issuecomment-819258964 > I think We should hold this off, I think the code has issues, as I said. Earlier I thought, there is some catch but I don’t think. It is misbehaving only. Ideally such features should go in a branch first @ayushtkn , IMO its not a misbehaviour here. Once the snapshotTrash feature is enabled, the .Trash directory has to be present to make the feature work. As you can have pre existing snapshottable directories , one way to create the .Trash inside these was to create right after the startup is done. The other solution was to explicitly provision the .Trash using a cmd line option. The choice was made to do it automatically on restart, and fail the Namenode in case any any issues occur. The other solution is not fail the namenode , but log an warning and later, if any Trash operations get performes inside the snapshottable root, will fail. The discussion is here: https://github.com/apache/hadoop/pull/2682#discussion_r570461526 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17611) Distcp parallel file copy breaks the modification time
[ https://issues.apache.org/jira/browse/HADOOP-17611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HADOOP-17611: Assignee: Adam Maroti > Distcp parallel file copy breaks the modification time > -- > > Key: HADOOP-17611 > URL: https://issues.apache.org/jira/browse/HADOOP-17611 > Project: Hadoop Common > Issue Type: Bug >Reporter: Adam Maroti >Assignee: Adam Maroti >Priority: Major > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > The commit HADOOP-11794. Enable distcp to copy blocks in parallel. > (bf3fb585aaf2b179836e139c041fc87920a3c886) broke the modification time of > large files. > > In CopyCommitter.java inside concatFileChunks Filesystem.concat is called > which changes the modification time therefore the modification times of files > copeid by distcp will not match the source files. However this only occurs > for large enough files, which are copied by splitting them up by distcp. > In concatFileChunks before calling concat extract the modification time and > apply that to the concatenated result-file after the concat. (probably best > -after- before the rename()). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17633) Please upgrade json-smart dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-17633?focusedWorklogId=582228=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582228 ] ASF GitHub Bot logged work on HADOOP-17633: --- Author: ASF GitHub Bot Created on: 14/Apr/21 06:12 Start Date: 14/Apr/21 06:12 Worklog Time Spent: 10m Work Description: jojochuang commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-819256687 I don't worry too much about these two dependency bump. Netty, Jetty and jackson-databind are usually more brittle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582228) Time Spent: 2h (was: 1h 50m) > Please upgrade json-smart dependency to the latest version > -- > > Key: HADOOP-17633 > URL: https://issues.apache.org/jira/browse/HADOOP-17633 > Project: Hadoop Common > Issue Type: Improvement > Components: auth, build >Affects Versions: 3.3.0, 3.2.1, 3.2.2, 3.4.0 >Reporter: helen huang >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 2h > Remaining Estimate: 0h > > Please upgrade the json-smart dependency to the latest version available. > Currently hadoop-auth is using version 2.3. Fortify scan picked up a security > issue with this version. Please upgrade to the latest version. > Thanks! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jojochuang commented on pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs
jojochuang commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-819256687 I don't worry too much about these two dependency bump. Netty, Jetty and jackson-databind are usually more brittle. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2860: Test Pre-Commits.
hadoop-yetus commented on pull request #2860: URL: https://github.com/apache/hadoop/pull/2860#issuecomment-819251753 (!) A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2860/11/console in case of problems. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17633) Please upgrade json-smart dependency to the latest version
[ https://issues.apache.org/jira/browse/HADOOP-17633?focusedWorklogId=582219=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-582219 ] ASF GitHub Bot logged work on HADOOP-17633: --- Author: ASF GitHub Bot Created on: 14/Apr/21 06:00 Start Date: 14/Apr/21 06:00 Worklog Time Spent: 10m Work Description: virajjasani commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-819251359 It looks like, since we touched `LICENSE-binary` with this PR, the build is running tests for all modules and hence taking 17+ hr. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 582219) Time Spent: 1h 50m (was: 1h 40m) > Please upgrade json-smart dependency to the latest version > -- > > Key: HADOOP-17633 > URL: https://issues.apache.org/jira/browse/HADOOP-17633 > Project: Hadoop Common > Issue Type: Improvement > Components: auth, build >Affects Versions: 3.3.0, 3.2.1, 3.2.2, 3.4.0 >Reporter: helen huang >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Please upgrade the json-smart dependency to the latest version available. > Currently hadoop-auth is using version 2.3. Fortify scan picked up a security > issue with this version. Please upgrade to the latest version. > Thanks! > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #2895: HADOOP-17633. Bump json-smart to 2.4.2 and nimbus-jose-jwt to 9.8 due to CVEs
virajjasani commented on pull request #2895: URL: https://github.com/apache/hadoop/pull/2895#issuecomment-819251359 It looks like, since we touched `LICENSE-binary` with this PR, the build is running tests for all modules and hence taking 17+ hr. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
