[GitHub] [hadoop] tomscut commented on pull request #3140: HDFS-16088. Standby NameNode process getLiveDatanodeStorageReport req…
tomscut commented on pull request #3140: URL: https://github.com/apache/hadoop/pull/3140#issuecomment-874499453 Thanks @Hexiaoqiao for your comments and suggestions. I fixed the problems. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ArkenKiran opened a new pull request #3177: HADOOP-166887 WIP
ArkenKiran opened a new pull request #3177: URL: https://github.com/apache/hadoop/pull/3177 ## NOTICE Please create an issue in ASF JIRA before opening a pull request, and you need to set the title of the pull request which starts with the corresponding JIRA issue number. (e.g. HADOOP-X. Fix a typo in YYY.) For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Hexiaoqiao commented on a change in pull request #3140: HDFS-16088. Standby NameNode process getLiveDatanodeStorageReport req…
Hexiaoqiao commented on a change in pull request #3140:
URL: https://github.com/apache/hadoop/pull/3140#discussion_r664253973
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithHANameNodes.java
##
@@ -236,4 +241,93 @@ private void testBalancerWithObserver(boolean
withObserverFailure)
}
}
}
+
+ /**
+ * Comparing the results of getLiveDatanodeStorageReport()
+ * from the active and standby NameNodes,
+ * the results should be the same.
+ */
+ @Test(timeout = 6)
+ public void testGetLiveDatanodeStorageReport() throws Exception {
+Configuration conf = new HdfsConfiguration();
+TestBalancer.initConf(conf);
+assertEquals(TEST_CAPACITIES.length, TEST_RACKS.length);
+NNConf nn1Conf = new MiniDFSNNTopology.NNConf("nn1");
+nn1Conf.setIpcPort(HdfsClientConfigKeys.DFS_NAMENODE_RPC_PORT_DEFAULT);
+Configuration copiedConf = new Configuration(conf);
+// Try capture NameNodeConnector log.
+LogCapturer log =LogCapturer.captureLogs(
+LoggerFactory.getLogger(NameNodeConnector.class));
+// We needs to assert datanode info from ANN and SNN, so the
+// heartbeat should disabled for the duration of method execution
+copiedConf.setInt(DFSConfigKeys.DFS_HEARTBEAT_INTERVAL_KEY, 6);
+cluster = new MiniDFSCluster.Builder(copiedConf)
+.nnTopology(MiniDFSNNTopology.simpleHATopology())
+.numDataNodes(TEST_CAPACITIES.length)
+.racks(TEST_RACKS)
+.simulatedCapacities(TEST_CAPACITIES)
+.build();
+HATestUtil.setFailoverConfigurations(cluster, conf);
+try {
+ cluster.waitActive();
+ cluster.transitionToActive(0);
+ URI namenode = (URI) DFSUtil.getInternalNsRpcUris(conf)
+ .toArray()[0];
+ String nsId = DFSUtilClient.getNameServiceIds(conf)
+ .toArray()[0].toString();
+
+ // request to active namenode
+ NameNodeConnector nncActive = new NameNodeConnector(
+ "nncActive", namenode,
+ nsId, new Path("/test"),
+ null, conf, NameNodeConnector.DEFAULT_MAX_IDLE_ITERATIONS);
+ DatanodeStorageReport[] ldspFromAnn =
Review comment:
`ldspFromAnn` here is not very explicit IMO, is `datanodeStorageReports`
more clear here?
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithHANameNodes.java
##
@@ -236,4 +241,93 @@ private void testBalancerWithObserver(boolean
withObserverFailure)
}
}
}
+
+ /**
+ * Comparing the results of getLiveDatanodeStorageReport()
+ * from the active and standby NameNodes,
+ * the results should be the same.
+ */
+ @Test(timeout = 6)
+ public void testGetLiveDatanodeStorageReport() throws Exception {
+Configuration conf = new HdfsConfiguration();
+TestBalancer.initConf(conf);
+assertEquals(TEST_CAPACITIES.length, TEST_RACKS.length);
+NNConf nn1Conf = new MiniDFSNNTopology.NNConf("nn1");
+nn1Conf.setIpcPort(HdfsClientConfigKeys.DFS_NAMENODE_RPC_PORT_DEFAULT);
+Configuration copiedConf = new Configuration(conf);
+// Try capture NameNodeConnector log.
+LogCapturer log =LogCapturer.captureLogs(
+LoggerFactory.getLogger(NameNodeConnector.class));
+// We needs to assert datanode info from ANN and SNN, so the
+// heartbeat should disabled for the duration of method execution
+copiedConf.setInt(DFSConfigKeys.DFS_HEARTBEAT_INTERVAL_KEY, 6);
+cluster = new MiniDFSCluster.Builder(copiedConf)
+.nnTopology(MiniDFSNNTopology.simpleHATopology())
+.numDataNodes(TEST_CAPACITIES.length)
+.racks(TEST_RACKS)
+.simulatedCapacities(TEST_CAPACITIES)
+.build();
+HATestUtil.setFailoverConfigurations(cluster, conf);
+try {
+ cluster.waitActive();
+ cluster.transitionToActive(0);
+ URI namenode = (URI) DFSUtil.getInternalNsRpcUris(conf)
+ .toArray()[0];
+ String nsId = DFSUtilClient.getNameServiceIds(conf)
+ .toArray()[0].toString();
+
+ // request to active namenode
Review comment:
It is better to begin with uppercase character and end with period for
annotation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HADOOP-17770) WASB : Support disabling buffered reads in positional reads
[ https://issues.apache.org/jira/browse/HADOOP-17770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375234#comment-17375234 ] Anoop Sam John edited comment on HADOOP-17770 at 7/6/21, 5:38 AM: -- Ran below perf test with an HBase usage Have a 3 RS node cluster. Region Server node SKU is DS13 v2. Storage is premium Block blob storage account. Region Server Xmx = 32 GB Created one table with 15 presplits ( So that 5 regions/RS) Pumped 100 GB of data using HBase PE tool. Single column/row with 1 KB row size. Table is NOT major compacted. Every region is having 2 or 3 HFiles under it. Doing random read PE tests (Every read req is a multi get, getting 300 rows). Ran with different number of tests. 1st case is where we make sure 0% cache hit always (skipping the data caching) 2nd case is when RS is having file mode cache with cache size ~50% of data size. The cache hit ration ~50% Case #1 !Throughput_0 cache hit.png! !AvgLatency_0 cache hit.png! Case #2 !Throughput_50% cache hit.png! !AvgLatency_50% cache hit.png! was (Author: anoop.hbase): Ran below perf test with an HBase usage Have a 3 RS node cluster. Region Server node SKU is DS13 v2. Storage is premium Block blob storage account. Region Server Xmx = 32 GB Created one table with 15 presplits ( So that 5 regions/RS) Pumped 100 GB of data using HBase PE tool. Single column/row with 1 KB row size. Doing random read PE tests (Every read req is a multi get, getting 300 rows). Ran with different number of tests. 1st case is where we make sure 0% cache hit always (skipping the data caching) 2nd case is when RS is having file mode cache with cache size ~50% of data size. The cache hit ration ~50% Case #1 !Throughput_0 cache hit.png! !AvgLatency_0 cache hit.png! Case #2 !Throughput_50% cache hit.png! !AvgLatency_50% cache hit.png! > WASB : Support disabling buffered reads in positional reads > --- > > Key: HADOOP-17770 > URL: https://issues.apache.org/jira/browse/HADOOP-17770 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: AvgLatency_0 cache hit.png, AvgLatency_50% cache > hit.png, Throughput_0 cache hit.png, Throughput_50% cache hit.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > This is just like HADOOP-17038 > Right now it will do a seek to the position , read and then seek back to the > old position. (As per the impl in the super class) > In HBase kind of workloads we rely mostly on short preads. (like 64 KB size > by default). So would be ideal to support a pure pos read API which will not > even keep the data in a buffer but will only read the required data as what > is asked for by the caller. (Not reading ahead more data as per the read size > config) > Allow an optional boolean config to be specified while opening file for read > using which buffered pread can be disabled. > FutureDataInputStreamBuilder openFile(Path path) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17770) WASB : Support disabling buffered reads in positional reads
[ https://issues.apache.org/jira/browse/HADOOP-17770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375234#comment-17375234 ] Anoop Sam John commented on HADOOP-17770: - Ran below perf test with an HBase usage Have a 3 RS node cluster. Region Server node SKU is DS13 v2. Storage is premium Block blob storage account. Region Server Xmx = 32 GB Created one table with 15 presplits ( So that 5 regions/RS) Pumped 100 GB of data using HBase PE tool. Single column/row with 1 KB row size. Doing random read PE tests (Every read req is a multi get, getting 300 rows). Ran with different number of tests. 1st case is where we make sure 0% cache hit always (skipping the data caching) 2nd case is when RS is having file mode cache with cache size ~50% of data size. The cache hit ration ~50% Case #1 !Throughput_0 cache hit.png! !AvgLatency_0 cache hit.png! Case #2 !Throughput_50% cache hit.png! !AvgLatency_50% cache hit.png! > WASB : Support disabling buffered reads in positional reads > --- > > Key: HADOOP-17770 > URL: https://issues.apache.org/jira/browse/HADOOP-17770 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: AvgLatency_0 cache hit.png, AvgLatency_50% cache > hit.png, Throughput_0 cache hit.png, Throughput_50% cache hit.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > This is just like HADOOP-17038 > Right now it will do a seek to the position , read and then seek back to the > old position. (As per the impl in the super class) > In HBase kind of workloads we rely mostly on short preads. (like 64 KB size > by default). So would be ideal to support a pure pos read API which will not > even keep the data in a buffer but will only read the required data as what > is asked for by the caller. (Not reading ahead more data as per the read size > config) > Allow an optional boolean config to be specified while opening file for read > using which buffered pread can be disabled. > FutureDataInputStreamBuilder openFile(Path path) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17770) WASB : Support disabling buffered reads in positional reads
[ https://issues.apache.org/jira/browse/HADOOP-17770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HADOOP-17770: Attachment: AvgLatency_50% cache hit.png > WASB : Support disabling buffered reads in positional reads > --- > > Key: HADOOP-17770 > URL: https://issues.apache.org/jira/browse/HADOOP-17770 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: AvgLatency_0 cache hit.png, AvgLatency_50% cache > hit.png, Throughput_0 cache hit.png, Throughput_50% cache hit.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > This is just like HADOOP-17038 > Right now it will do a seek to the position , read and then seek back to the > old position. (As per the impl in the super class) > In HBase kind of workloads we rely mostly on short preads. (like 64 KB size > by default). So would be ideal to support a pure pos read API which will not > even keep the data in a buffer but will only read the required data as what > is asked for by the caller. (Not reading ahead more data as per the read size > config) > Allow an optional boolean config to be specified while opening file for read > using which buffered pread can be disabled. > FutureDataInputStreamBuilder openFile(Path path) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17770) WASB : Support disabling buffered reads in positional reads
[ https://issues.apache.org/jira/browse/HADOOP-17770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HADOOP-17770: Attachment: Throughput_50% cache hit.png > WASB : Support disabling buffered reads in positional reads > --- > > Key: HADOOP-17770 > URL: https://issues.apache.org/jira/browse/HADOOP-17770 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: AvgLatency_0 cache hit.png, AvgLatency_50% cache > hit.png, Throughput_0 cache hit.png, Throughput_50% cache hit.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > This is just like HADOOP-17038 > Right now it will do a seek to the position , read and then seek back to the > old position. (As per the impl in the super class) > In HBase kind of workloads we rely mostly on short preads. (like 64 KB size > by default). So would be ideal to support a pure pos read API which will not > even keep the data in a buffer but will only read the required data as what > is asked for by the caller. (Not reading ahead more data as per the read size > config) > Allow an optional boolean config to be specified while opening file for read > using which buffered pread can be disabled. > FutureDataInputStreamBuilder openFile(Path path) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17770) WASB : Support disabling buffered reads in positional reads
[ https://issues.apache.org/jira/browse/HADOOP-17770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HADOOP-17770: Attachment: AvgLatency_0 cache hit.png > WASB : Support disabling buffered reads in positional reads > --- > > Key: HADOOP-17770 > URL: https://issues.apache.org/jira/browse/HADOOP-17770 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: AvgLatency_0 cache hit.png, Throughput_0 cache hit.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > This is just like HADOOP-17038 > Right now it will do a seek to the position , read and then seek back to the > old position. (As per the impl in the super class) > In HBase kind of workloads we rely mostly on short preads. (like 64 KB size > by default). So would be ideal to support a pure pos read API which will not > even keep the data in a buffer but will only read the required data as what > is asked for by the caller. (Not reading ahead more data as per the read size > config) > Allow an optional boolean config to be specified while opening file for read > using which buffered pread can be disabled. > FutureDataInputStreamBuilder openFile(Path path) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-15566) Support OpenTelemetry
[ https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kiran Kumar Maturi updated HADOOP-15566: Attachment: HADOOP-15566-WIP.1.patch > Support OpenTelemetry > - > > Key: HADOOP-15566 > URL: https://issues.apache.org/jira/browse/HADOOP-15566 > Project: Hadoop Common > Issue Type: New Feature > Components: metrics, tracing >Affects Versions: 3.1.0 >Reporter: Todd Lipcon >Assignee: Siyao Meng >Priority: Major > Labels: pull-request-available, security > Attachments: HADOOP-15566-WIP.1.patch, HADOOP-15566.000.WIP.patch, > OpenTelemetry Support Scope Doc v2.pdf, OpenTracing Support Scope Doc.pdf, > Screen Shot 2018-06-29 at 11.59.16 AM.png, ss-trace-s3a.png > > Time Spent: 2h 50m > Remaining Estimate: 0h > > The HTrace incubator project has voted to retire itself and won't be making > further releases. The Hadoop project currently has various hooks with HTrace. > It seems in some cases (eg HDFS-13702) these hooks have had measurable > performance overhead. Given these two factors, I think we should consider > removing the HTrace integration. If there is someone willing to do the work, > replacing it with OpenTracing might be a better choice since there is an > active community. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17770) WASB : Support disabling buffered reads in positional reads
[ https://issues.apache.org/jira/browse/HADOOP-17770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HADOOP-17770: Attachment: Throughput_0 cache hit.png > WASB : Support disabling buffered reads in positional reads > --- > > Key: HADOOP-17770 > URL: https://issues.apache.org/jira/browse/HADOOP-17770 > Project: Hadoop Common > Issue Type: Improvement >Reporter: Anoop Sam John >Assignee: Anoop Sam John >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Attachments: Throughput_0 cache hit.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > This is just like HADOOP-17038 > Right now it will do a seek to the position , read and then seek back to the > old position. (As per the impl in the super class) > In HBase kind of workloads we rely mostly on short preads. (like 64 KB size > by default). So would be ideal to support a pure pos read API which will not > even keep the data in a buffer but will only read the required data as what > is asked for by the caller. (Not reading ahead more data as per the read size > config) > Allow an optional boolean config to be specified while opening file for read > using which buffered pread can be disabled. > FutureDataInputStreamBuilder openFile(Path path) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on pull request #2971: MAPREDUCE-7341. Intermediate Manifest Committer
mukund-thakur commented on pull request #2971: URL: https://github.com/apache/hadoop/pull/2971#issuecomment-874472280 > I propose we add a dir for job stats into which success/failure JSON goes (we will add a success/failure bit and a stack trace serialization on failures)...saving to this dir will be best-effort. will need to track whether this has been written by the current job so that an abort() after success doesn't overwrite it, etc. > > also: dump stats in cleanup & task commit/aborts Yes this is a good idea. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17788) Replace IOUtils#closeQuietly usages
[
https://issues.apache.org/jira/browse/HADOOP-17788?focusedWorklogId=618878&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618878
]
ASF GitHub Bot logged work on HADOOP-17788:
---
Author: ASF GitHub Bot
Created on: 06/Jul/21 05:21
Start Date: 06/Jul/21 05:21
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3171:
URL: https://github.com/apache/hadoop/pull/3171#issuecomment-874468390
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 14 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 42s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 1s | | trunk passed |
| +1 :green_heart: | compile | 21m 47s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 18m 20s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 11m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 9m 44s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 10m 35s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 19m 5s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 29s | | the patch passed |
| +1 :green_heart: | compile | 20m 8s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 20m 8s | |
root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 0 new + 1920 unchanged - 68
fixed = 1920 total (was 1988) |
| +1 :green_heart: | compile | 18m 11s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 18m 11s | |
root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 0 new + 1794 unchanged -
68 fixed = 1794 total (was 1862) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 52s | | root: The patch generated
0 new + 927 unchanged - 7 fixed = 927 total (was 934) |
| +1 :green_heart: | mvnsite | 11m 40s | | the patch passed |
| +1 :green_heart: | javadoc | 9m 44s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 10m 27s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 21m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 45s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 8s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 44s | | hadoop-kms in the patch passed.
|
| +1 :green_heart: | unit | 2m 40s | | hadoop-hdfs-client in the patch
passed. |
| -1 :x: | unit | 413m 24s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3171/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | unit | 5m 26s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 7s | |
hadoop-yarn-server-timelineservice in the patch passed. |
| -1 :x: | unit | 112m 45s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3171/6/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
| hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_h
[GitHub] [hadoop] hadoop-yetus commented on pull request #3171: HADOOP-17788. Replace IOUtils#closeQuietly usages by Hadoop's own utility
hadoop-yetus commented on pull request #3171:
URL: https://github.com/apache/hadoop/pull/3171#issuecomment-874468390
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 14 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 42s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 21m 1s | | trunk passed |
| +1 :green_heart: | compile | 21m 47s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 18m 20s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 11m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 9m 44s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 10m 35s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 19m 5s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 7m 29s | | the patch passed |
| +1 :green_heart: | compile | 20m 8s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 20m 8s | |
root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 0 new + 1920 unchanged - 68
fixed = 1920 total (was 1988) |
| +1 :green_heart: | compile | 18m 11s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 18m 11s | |
root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 with JDK Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 0 new + 1794 unchanged -
68 fixed = 1794 total (was 1862) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 52s | | root: The patch generated
0 new + 927 unchanged - 7 fixed = 927 total (was 934) |
| +1 :green_heart: | mvnsite | 11m 40s | | the patch passed |
| +1 :green_heart: | javadoc | 9m 44s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 10m 27s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 21m 21s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 45s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 8s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 44s | | hadoop-kms in the patch passed.
|
| +1 :green_heart: | unit | 2m 40s | | hadoop-hdfs-client in the patch
passed. |
| -1 :x: | unit | 413m 24s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3171/6/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | unit | 5m 26s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 7s | |
hadoop-yarn-server-timelineservice in the patch passed. |
| -1 :x: | unit | 112m 45s |
[/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3171/6/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt)
| hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 23m 49s | | hadoop-yarn-server-nodemanager
in the patch passed. |
| +1 :green_heart: | unit | 27m 59s | | hadoop-yarn-client in the patch
passed. |
| +1 :green_heart: | unit | 9m 21s | | hadoop-mapreduce-client-app in
the patch passed. |
| +1 :green_heart: | unit | 22m 58s | |
hadoop-yarn-applications-distributedshell in the patch passed. |
| +1 :green_heart: | asflicense
[GitHub] [hadoop] tomscut commented on pull request #3140: HDFS-16088. Standby NameNode process getLiveDatanodeStorageReport req…
tomscut commented on pull request #3140: URL: https://github.com/apache/hadoop/pull/3140#issuecomment-874419344 Hi @tasanuma @jojochuang @aajisaka @ayushtkn , could you also help to review the code if you have time? Thanks a lot. We can refer to the discussion in JIRA [HDFS-16088](https://issues.apache.org/jira/browse/HDFS-16088). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #3174: HDFS-16110. Remove unused method reportChecksumFailure in DFSClient
tomscut commented on pull request #3174: URL: https://github.com/apache/hadoop/pull/3174#issuecomment-874418161 Thanks @jojochuang for the merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jojochuang merged pull request #3174: HDFS-16110. Remove unused method reportChecksumFailure in DFSClient
jojochuang merged pull request #3174: URL: https://github.com/apache/hadoop/pull/3174 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #3140: HDFS-16088. Standby NameNode process getLiveDatanodeStorageReport req…
tomscut commented on pull request #3140: URL: https://github.com/apache/hadoop/pull/3140#issuecomment-874412101 Thanks @ferhui for your review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout
[
https://issues.apache.org/jira/browse/HADOOP-17749?focusedWorklogId=618834&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618834
]
ASF GitHub Bot logged work on HADOOP-17749:
---
Author: ASF GitHub Bot
Created on: 06/Jul/21 01:43
Start Date: 06/Jul/21 01:43
Worklog Time Spent: 10m
Work Description: liangxs commented on pull request #3080:
URL: https://github.com/apache/hadoop/pull/3080#issuecomment-874400802
@ferhui @aajisaka Thanks very much for your reviews!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 618834)
Time Spent: 4h 20m (was: 4h 10m)
> Remove lock contention in SelectorPool of SocketIOWithTimeout
> -
>
> Key: HADOOP-17749
> URL: https://issues.apache.org/jira/browse/HADOOP-17749
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: Xuesen Liang
>Assignee: Xuesen Liang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> *SelectorPool* in
> hadoop-common/src/main/java/org/apache/hadoop/net/*SocketIOWithTimeout.java*
> is a point of lock contention.
> For example:
> {code:java}
> $ grep 'waiting to lock <0x7f7d94006d90>' 63692.jstack | uniq -c
> 1005 - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> {code}
> and the thread stack is as follows:
> {code:java}
> "IPC Client (324579982) connection to /100.10.6.10:60020 from user_00" #14139
> daemon prio=5 os_prio=0 tid=0x7f7374039000 nid=0x85cc waiting for monitor
> entry [0x7f6f45939000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.get(SocketIOWithTimeout.java:390)
> - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:325)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> - locked <0x7fa818caf258> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readInt(DataInputStream.java:387)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.readResponse(RpcClientImpl.java:967)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:568)
> {code}
> We should remove the lock contention.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] liangxs commented on pull request #3080: HADOOP-17749. Remove lock contention in SelectorPool of SocketIOWithTimeout
liangxs commented on pull request #3080: URL: https://github.com/apache/hadoop/pull/3080#issuecomment-874400802 @ferhui @aajisaka Thanks very much for your reviews! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout
[
https://issues.apache.org/jira/browse/HADOOP-17749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375092#comment-17375092
]
Hui Fei commented on HADOOP-17749:
--
Could not access gitbox, cherrypick to branch-3.2 branch-3.3 later
> Remove lock contention in SelectorPool of SocketIOWithTimeout
> -
>
> Key: HADOOP-17749
> URL: https://issues.apache.org/jira/browse/HADOOP-17749
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: Xuesen Liang
>Assignee: Xuesen Liang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> *SelectorPool* in
> hadoop-common/src/main/java/org/apache/hadoop/net/*SocketIOWithTimeout.java*
> is a point of lock contention.
> For example:
> {code:java}
> $ grep 'waiting to lock <0x7f7d94006d90>' 63692.jstack | uniq -c
> 1005 - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> {code}
> and the thread stack is as follows:
> {code:java}
> "IPC Client (324579982) connection to /100.10.6.10:60020 from user_00" #14139
> daemon prio=5 os_prio=0 tid=0x7f7374039000 nid=0x85cc waiting for monitor
> entry [0x7f6f45939000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.get(SocketIOWithTimeout.java:390)
> - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:325)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> - locked <0x7fa818caf258> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readInt(DataInputStream.java:387)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.readResponse(RpcClientImpl.java:967)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:568)
> {code}
> We should remove the lock contention.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17776) The java agent args should be passed as the VM arguments
[
https://issues.apache.org/jira/browse/HADOOP-17776?focusedWorklogId=618831&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618831
]
ASF GitHub Bot logged work on HADOOP-17776:
---
Author: ASF GitHub Bot
Created on: 06/Jul/21 01:19
Start Date: 06/Jul/21 01:19
Worklog Time Spent: 10m
Work Description: jiwq edited a comment on pull request #3147:
URL: https://github.com/apache/hadoop/pull/3147#issuecomment-874394094
@aajisaka Thanks for your review. I agree with your opinion and I can try to
refactor the `maven-surefire-plugin.argLine` in a separate jira.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 618831)
Time Spent: 1h (was: 50m)
> The java agent args should be passed as the VM arguments
>
>
> Key: HADOOP-17776
> URL: https://issues.apache.org/jira/browse/HADOOP-17776
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Wanqiang Ji
>Assignee: Wanqiang Ji
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Some java agents or maven plugins want works well, must pass the agent
> arguments as the VM arguments, such as JaCoCo etc.
> PS. Use the JaCoCo to generate the code coverage data and put into SonarQube.
> {code:java}
> mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent package
> -Dmaven.test.failure.ignore=true org.jacoco:jacoco-maven-plugin:report
> surefire-report:report
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jiwq edited a comment on pull request #3147: HADOOP-17776. The java agent args should be passed as the VM arguments
jiwq edited a comment on pull request #3147: URL: https://github.com/apache/hadoop/pull/3147#issuecomment-874394094 @aajisaka Thanks for your review. I agree with your opinion and I can try to refactor the `maven-surefire-plugin.argLine` in a separate jira. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17776) The java agent args should be passed as the VM arguments
[
https://issues.apache.org/jira/browse/HADOOP-17776?focusedWorklogId=618830&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618830
]
ASF GitHub Bot logged work on HADOOP-17776:
---
Author: ASF GitHub Bot
Created on: 06/Jul/21 01:18
Start Date: 06/Jul/21 01:18
Worklog Time Spent: 10m
Work Description: jiwq commented on pull request #3147:
URL: https://github.com/apache/hadoop/pull/3147#issuecomment-874394094
@aajisaka Thanks for your review. I agree with your opinion and I can try
refactor the `maven-surefire-plugin.argLine` in a separate jira.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 618830)
Time Spent: 50m (was: 40m)
> The java agent args should be passed as the VM arguments
>
>
> Key: HADOOP-17776
> URL: https://issues.apache.org/jira/browse/HADOOP-17776
> Project: Hadoop Common
> Issue Type: Improvement
>Reporter: Wanqiang Ji
>Assignee: Wanqiang Ji
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Some java agents or maven plugins want works well, must pass the agent
> arguments as the VM arguments, such as JaCoCo etc.
> PS. Use the JaCoCo to generate the code coverage data and put into SonarQube.
> {code:java}
> mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent package
> -Dmaven.test.failure.ignore=true org.jacoco:jacoco-maven-plugin:report
> surefire-report:report
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Reopened] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout
[
https://issues.apache.org/jira/browse/HADOOP-17749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hui Fei reopened HADOOP-17749:
--
> Remove lock contention in SelectorPool of SocketIOWithTimeout
> -
>
> Key: HADOOP-17749
> URL: https://issues.apache.org/jira/browse/HADOOP-17749
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: Xuesen Liang
>Assignee: Xuesen Liang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> *SelectorPool* in
> hadoop-common/src/main/java/org/apache/hadoop/net/*SocketIOWithTimeout.java*
> is a point of lock contention.
> For example:
> {code:java}
> $ grep 'waiting to lock <0x7f7d94006d90>' 63692.jstack | uniq -c
> 1005 - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> {code}
> and the thread stack is as follows:
> {code:java}
> "IPC Client (324579982) connection to /100.10.6.10:60020 from user_00" #14139
> daemon prio=5 os_prio=0 tid=0x7f7374039000 nid=0x85cc waiting for monitor
> entry [0x7f6f45939000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.get(SocketIOWithTimeout.java:390)
> - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:325)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> - locked <0x7fa818caf258> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readInt(DataInputStream.java:387)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.readResponse(RpcClientImpl.java:967)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:568)
> {code}
> We should remove the lock contention.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] jiwq commented on pull request #3147: HADOOP-17776. The java agent args should be passed as the VM arguments
jiwq commented on pull request #3147: URL: https://github.com/apache/hadoop/pull/3147#issuecomment-874394094 @aajisaka Thanks for your review. I agree with your opinion and I can try refactor the `maven-surefire-plugin.argLine` in a separate jira. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] tomscut commented on pull request #3174: HDFS-16110. Remove unused method reportChecksumFailure in DFSClient
tomscut commented on pull request #3174: URL: https://github.com/apache/hadoop/pull/3174#issuecomment-874394011 Thanks @tasanuma for your review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout
[
https://issues.apache.org/jira/browse/HADOOP-17749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hui Fei reassigned HADOOP-17749:
Assignee: Xuesen Liang
> Remove lock contention in SelectorPool of SocketIOWithTimeout
> -
>
> Key: HADOOP-17749
> URL: https://issues.apache.org/jira/browse/HADOOP-17749
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: Xuesen Liang
>Assignee: Xuesen Liang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> *SelectorPool* in
> hadoop-common/src/main/java/org/apache/hadoop/net/*SocketIOWithTimeout.java*
> is a point of lock contention.
> For example:
> {code:java}
> $ grep 'waiting to lock <0x7f7d94006d90>' 63692.jstack | uniq -c
> 1005 - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> {code}
> and the thread stack is as follows:
> {code:java}
> "IPC Client (324579982) connection to /100.10.6.10:60020 from user_00" #14139
> daemon prio=5 os_prio=0 tid=0x7f7374039000 nid=0x85cc waiting for monitor
> entry [0x7f6f45939000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.get(SocketIOWithTimeout.java:390)
> - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:325)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> - locked <0x7fa818caf258> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readInt(DataInputStream.java:387)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.readResponse(RpcClientImpl.java:967)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:568)
> {code}
> We should remove the lock contention.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout
[
https://issues.apache.org/jira/browse/HADOOP-17749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hui Fei resolved HADOOP-17749.
--
Fix Version/s: 3.4.0
Resolution: Fixed
> Remove lock contention in SelectorPool of SocketIOWithTimeout
> -
>
> Key: HADOOP-17749
> URL: https://issues.apache.org/jira/browse/HADOOP-17749
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: Xuesen Liang
>Assignee: Xuesen Liang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> *SelectorPool* in
> hadoop-common/src/main/java/org/apache/hadoop/net/*SocketIOWithTimeout.java*
> is a point of lock contention.
> For example:
> {code:java}
> $ grep 'waiting to lock <0x7f7d94006d90>' 63692.jstack | uniq -c
> 1005 - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> {code}
> and the thread stack is as follows:
> {code:java}
> "IPC Client (324579982) connection to /100.10.6.10:60020 from user_00" #14139
> daemon prio=5 os_prio=0 tid=0x7f7374039000 nid=0x85cc waiting for monitor
> entry [0x7f6f45939000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.get(SocketIOWithTimeout.java:390)
> - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:325)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> - locked <0x7fa818caf258> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readInt(DataInputStream.java:387)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.readResponse(RpcClientImpl.java:967)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:568)
> {code}
> We should remove the lock contention.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout
[
https://issues.apache.org/jira/browse/HADOOP-17749?focusedWorklogId=618828&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618828
]
ASF GitHub Bot logged work on HADOOP-17749:
---
Author: ASF GitHub Bot
Created on: 06/Jul/21 01:11
Start Date: 06/Jul/21 01:11
Worklog Time Spent: 10m
Work Description: ferhui merged pull request #3080:
URL: https://github.com/apache/hadoop/pull/3080
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 618828)
Time Spent: 4h 10m (was: 4h)
> Remove lock contention in SelectorPool of SocketIOWithTimeout
> -
>
> Key: HADOOP-17749
> URL: https://issues.apache.org/jira/browse/HADOOP-17749
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: Xuesen Liang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> *SelectorPool* in
> hadoop-common/src/main/java/org/apache/hadoop/net/*SocketIOWithTimeout.java*
> is a point of lock contention.
> For example:
> {code:java}
> $ grep 'waiting to lock <0x7f7d94006d90>' 63692.jstack | uniq -c
> 1005 - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> {code}
> and the thread stack is as follows:
> {code:java}
> "IPC Client (324579982) connection to /100.10.6.10:60020 from user_00" #14139
> daemon prio=5 os_prio=0 tid=0x7f7374039000 nid=0x85cc waiting for monitor
> entry [0x7f6f45939000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.get(SocketIOWithTimeout.java:390)
> - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:325)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> - locked <0x7fa818caf258> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readInt(DataInputStream.java:387)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.readResponse(RpcClientImpl.java:967)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:568)
> {code}
> We should remove the lock contention.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferhui merged pull request #3080: HADOOP-17749. Remove lock contention in SelectorPool of SocketIOWithTimeout
ferhui merged pull request #3080: URL: https://github.com/apache/hadoop/pull/3080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17749) Remove lock contention in SelectorPool of SocketIOWithTimeout
[
https://issues.apache.org/jira/browse/HADOOP-17749?focusedWorklogId=618827&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618827
]
ASF GitHub Bot logged work on HADOOP-17749:
---
Author: ASF GitHub Bot
Created on: 06/Jul/21 01:10
Start Date: 06/Jul/21 01:10
Worklog Time Spent: 10m
Work Description: ferhui commented on pull request #3080:
URL: https://github.com/apache/hadoop/pull/3080#issuecomment-874391947
@liangxs Thanks for contribution. @aajisaka Thanks for review!
Merged to trunk.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 618827)
Time Spent: 4h (was: 3h 50m)
> Remove lock contention in SelectorPool of SocketIOWithTimeout
> -
>
> Key: HADOOP-17749
> URL: https://issues.apache.org/jira/browse/HADOOP-17749
> Project: Hadoop Common
> Issue Type: Improvement
> Components: common
>Reporter: Xuesen Liang
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> *SelectorPool* in
> hadoop-common/src/main/java/org/apache/hadoop/net/*SocketIOWithTimeout.java*
> is a point of lock contention.
> For example:
> {code:java}
> $ grep 'waiting to lock <0x7f7d94006d90>' 63692.jstack | uniq -c
> 1005 - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> {code}
> and the thread stack is as follows:
> {code:java}
> "IPC Client (324579982) connection to /100.10.6.10:60020 from user_00" #14139
> daemon prio=5 os_prio=0 tid=0x7f7374039000 nid=0x85cc waiting for monitor
> entry [0x7f6f45939000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.get(SocketIOWithTimeout.java:390)
> - waiting to lock <0x7f7d94006d90> (a
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool)
> at
> org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:325)
> at
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)
> at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)
> at java.io.FilterInputStream.read(FilterInputStream.java:133)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
> - locked <0x7fa818caf258> (a java.io.BufferedInputStream)
> at java.io.DataInputStream.readInt(DataInputStream.java:387)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.readResponse(RpcClientImpl.java:967)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.run(RpcClientImpl.java:568)
> {code}
> We should remove the lock contention.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] ferhui commented on pull request #3080: HADOOP-17749. Remove lock contention in SelectorPool of SocketIOWithTimeout
ferhui commented on pull request #3080: URL: https://github.com/apache/hadoop/pull/3080#issuecomment-874391947 @liangxs Thanks for contribution. @aajisaka Thanks for review! Merged to trunk. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2971: MAPREDUCE-7341. Intermediate Manifest Committer
hadoop-yetus commented on pull request #2971:
URL: https://github.com/apache/hadoop/pull/2971#issuecomment-874362787
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 35s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 21 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 48s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 18s | | trunk passed |
| +1 :green_heart: | compile | 20m 46s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 18m 4s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 50s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 13s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 41s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +0 :ok: | spotbugs | 0m 41s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 14m 36s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 15s | | the patch passed |
| +1 :green_heart: | compile | 20m 16s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| -1 :x: | javac | 20m 16s |
[/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/17/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt)
| root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 2 new + 1985 unchanged - 1
fixed = 1987 total (was 1986) |
| +1 :green_heart: | compile | 18m 10s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| -1 :x: | javac | 18m 10s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/17/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt)
| root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 2 new + 1858
unchanged - 1 fixed = 1860 total (was 1859) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/17/artifact/out/blanks-eol.txt)
| The patch has 7 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 3m 49s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/17/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 59 new + 0 unchanged - 0 fixed = 59 total (was 0)
|
| +1 :green_heart: | mvnsite | 4m 4s | | the patch passed |
| +1 :green_heart: | xml | 0m 8s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 3m 11s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 40s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +0 :ok: | spotbugs | 0m 38s | | hadoop-project has no data from
spotbugs |
| -1 :x: | spotbugs | 1m 43s |
[/new-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/17/artifact/out/new-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html)
|
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | shadedclient | 14m 41s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 37s | | hadoop-project in
[jira] [Created] (HADOOP-17791) TestActivitiesManager is flaky
Szilard Nemeth created HADOOP-17791: --- Summary: TestActivitiesManager is flaky Key: HADOOP-17791 URL: https://issues.apache.org/jira/browse/HADOOP-17791 Project: Hadoop Common Issue Type: Bug Reporter: Szilard Nemeth I noticed in our internal testing environment that org.apache.hadoop.yarn.server.resourcemanager.scheduler.activities.TestActivitiesManager.testAppActivitiesTTL failed a couple of times, quite randomly. By checking the Jira and searching for the name of the class, there are some results from this year as well: [https://issues.apache.org/jira/issues/?jql=text%20~%20TestActivitiesManager%20ORDER%20BY%20updated%20DESC] I don't know exactly how to reproduce this though. Tries to run the whole test class 60 times and it hasn't failed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran closed pull request #2349: MAPREDUCE-7282. Move away from V2 commit algorithm
steveloughran closed pull request #2349: URL: https://github.com/apache/hadoop/pull/2349 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2971: MAPREDUCE-7341. Intermediate Manifest Committer
steveloughran commented on pull request #2971: URL: https://github.com/apache/hadoop/pull/2971#issuecomment-874312227 I propose we add a dir for job stats into which success/failure JSON goes (we will add a success/failure bit and a stack trace serialization on failures)...saving to this dir will be best-effort. will need to track whether this has been written by the current job so that an abort() after success doesn't overwrite it, etc. also: dump stats in cleanup & task commit/aborts -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2971: MAPREDUCE-7341. Intermediate Manifest Committer
hadoop-yetus removed a comment on pull request #2971:
URL: https://github.com/apache/hadoop/pull/2971#issuecomment-861004924
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 1s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 21 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 47s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 20m 25s | | trunk passed |
| +1 :green_heart: | compile | 21m 30s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 18m 21s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 3m 51s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 4s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 14s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 45s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +0 :ok: | spotbugs | 0m 42s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| +1 :green_heart: | shadedclient | 14m 49s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 28s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 15s | | the patch passed |
| +1 :green_heart: | compile | 20m 47s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| -1 :x: | javac | 20m 47s |
[/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/16/artifact/out/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt)
| root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 2 new + 1985 unchanged - 1
fixed = 1987 total (was 1986) |
| +1 :green_heart: | compile | 21m 30s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| -1 :x: | javac | 21m 30s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/16/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt)
| root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 generated 2 new + 1858
unchanged - 1 fixed = 1860 total (was 1859) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/16/artifact/out/blanks-eol.txt)
| The patch has 7 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| -0 :warning: | checkstyle | 4m 26s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/16/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 59 new + 0 unchanged - 0 fixed = 59 total (was 0)
|
| +1 :green_heart: | mvnsite | 4m 23s | | the patch passed |
| +1 :green_heart: | xml | 0m 9s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 3m 23s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 52s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +0 :ok: | spotbugs | 0m 34s | | hadoop-project has no data from
spotbugs |
| -1 :x: | spotbugs | 2m 11s |
[/new-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2971/16/artifact/out/new-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.html)
|
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) |
| +1 :green_heart: | shadedclient | 19m 30s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 0m 31s | | hadoop-pro
[jira] [Work logged] (HADOOP-16680) Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
[ https://issues.apache.org/jira/browse/HADOOP-16680?focusedWorklogId=618790&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618790 ] ASF GitHub Bot logged work on HADOOP-16680: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:12 Start Date: 05/Jul/21 20:12 Worklog Time Spent: 10m Work Description: steveloughran closed pull request #1690: URL: https://github.com/apache/hadoop/pull/1690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618790) Time Spent: 1h 40m (was: 1.5h) > Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider > --- > > Key: HADOOP-16680 > URL: https://issues.apache.org/jira/browse/HADOOP-16680 > Project: Hadoop Common > Issue Type: New Feature > Components: auth, common, security >Affects Versions: 3.3.0 >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Add MicrosoftGraphGroupsMapping that uses > https://developer.microsoft.com/en-us/graph to retrieve user groups. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16680) Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
[ https://issues.apache.org/jira/browse/HADOOP-16680?focusedWorklogId=618789&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618789 ] ASF GitHub Bot logged work on HADOOP-16680: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:12 Start Date: 05/Jul/21 20:12 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #1690: URL: https://github.com/apache/hadoop/pull/1690#issuecomment-874310490 closing as it is superceded by #1727. Let's see if we can get it wrapped up in there -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618789) Time Spent: 1.5h (was: 1h 20m) > Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider > --- > > Key: HADOOP-16680 > URL: https://issues.apache.org/jira/browse/HADOOP-16680 > Project: Hadoop Common > Issue Type: New Feature > Components: auth, common, security >Affects Versions: 3.3.0 >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > Add MicrosoftGraphGroupsMapping that uses > https://developer.microsoft.com/en-us/graph to retrieve user groups. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16680) Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
[ https://issues.apache.org/jira/browse/HADOOP-16680?focusedWorklogId=618788&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618788 ] ASF GitHub Bot logged work on HADOOP-16680: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:12 Start Date: 05/Jul/21 20:12 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #1690: URL: https://github.com/apache/hadoop/pull/1690#issuecomment-667156807 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618788) Time Spent: 1h 20m (was: 1h 10m) > Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider > --- > > Key: HADOOP-16680 > URL: https://issues.apache.org/jira/browse/HADOOP-16680 > Project: Hadoop Common > Issue Type: New Feature > Components: auth, common, security >Affects Versions: 3.3.0 >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Labels: pull-request-available > Time Spent: 1h 20m > Remaining Estimate: 0h > > Add MicrosoftGraphGroupsMapping that uses > https://developer.microsoft.com/en-us/graph to retrieve user groups. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran closed pull request #1690: HADOOP-16680. Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
steveloughran closed pull request #1690: URL: https://github.com/apache/hadoop/pull/1690 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #1690: HADOOP-16680. Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
steveloughran commented on pull request #1690: URL: https://github.com/apache/hadoop/pull/1690#issuecomment-874310490 closing as it is superceded by #1727. Let's see if we can get it wrapped up in there -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #1690: HADOOP-16680. Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
hadoop-yetus removed a comment on pull request #1690: URL: https://github.com/apache/hadoop/pull/1690#issuecomment-667156807 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16680) Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
[ https://issues.apache.org/jira/browse/HADOOP-16680?focusedWorklogId=618787&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618787 ] ASF GitHub Bot logged work on HADOOP-16680: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:11 Start Date: 05/Jul/21 20:11 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #1727: URL: https://github.com/apache/hadoop/pull/1727#issuecomment-874310092 @goiri looks like some other pom change has broken the merge. Rebase and we can look at it again -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618787) Time Spent: 1h 10m (was: 1h) > Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider > --- > > Key: HADOOP-16680 > URL: https://issues.apache.org/jira/browse/HADOOP-16680 > Project: Hadoop Common > Issue Type: New Feature > Components: auth, common, security >Affects Versions: 3.3.0 >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > > Add MicrosoftGraphGroupsMapping that uses > https://developer.microsoft.com/en-us/graph to retrieve user groups. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16680) Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
[ https://issues.apache.org/jira/browse/HADOOP-16680?focusedWorklogId=618786&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618786 ] ASF GitHub Bot logged work on HADOOP-16680: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:11 Start Date: 05/Jul/21 20:11 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #1727: URL: https://github.com/apache/hadoop/pull/1727#issuecomment-557717462 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618786) Time Spent: 1h (was: 50m) > Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider > --- > > Key: HADOOP-16680 > URL: https://issues.apache.org/jira/browse/HADOOP-16680 > Project: Hadoop Common > Issue Type: New Feature > Components: auth, common, security >Affects Versions: 3.3.0 >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Add MicrosoftGraphGroupsMapping that uses > https://developer.microsoft.com/en-us/graph to retrieve user groups. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #1727: HADOOP-16680. Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
steveloughran commented on pull request #1727: URL: https://github.com/apache/hadoop/pull/1727#issuecomment-874310092 @goiri looks like some other pom change has broken the merge. Rebase and we can look at it again -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #1727: HADOOP-16680. Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
hadoop-yetus removed a comment on pull request #1727: URL: https://github.com/apache/hadoop/pull/1727#issuecomment-557717462 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16680) Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
[ https://issues.apache.org/jira/browse/HADOOP-16680?focusedWorklogId=618785&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618785 ] ASF GitHub Bot logged work on HADOOP-16680: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:10 Start Date: 05/Jul/21 20:10 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #1727: URL: https://github.com/apache/hadoop/pull/1727#issuecomment-667180788 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 20m 4s | trunk passed | | +1 :green_heart: | compile | 20m 14s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 17m 41s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | checkstyle | 2m 34s | trunk passed | | +1 :green_heart: | mvnsite | 3m 38s | trunk passed | | +1 :green_heart: | shadedclient | 20m 28s | branch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 36s | hadoop-tools in trunk failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | +1 :green_heart: | javadoc | 2m 34s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +0 :ok: | spotbugs | 5m 37s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +0 :ok: | findbugs | 0m 28s | branch/hadoop-project no findbugs output file (findbugsXml.xml) | | +0 :ok: | findbugs | 0m 30s | branch/hadoop-tools/hadoop-tools-dist no findbugs output file (findbugsXml.xml) | | -0 :warning: | patch | 5m 55s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 0m 13s | hadoop-tools in the patch failed. | | -1 :x: | mvninstall | 0m 11s | hadoop-msgraph in the patch failed. | | -1 :x: | mvninstall | 0m 16s | hadoop-tools-dist in the patch failed. | | -1 :x: | compile | 0m 14s | root in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | -1 :x: | javac | 0m 14s | root in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | -1 :x: | compile | 0m 14s | root in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -1 :x: | javac | 0m 14s | root in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -0 :warning: | checkstyle | 0m 12s | The patch fails to run checkstyle in root | | -1 :x: | mvnsite | 0m 13s | hadoop-tools in the patch failed. | | -1 :x: | mvnsite | 0m 11s | hadoop-msgraph in the patch failed. | | -1 :x: | mvnsite | 0m 12s | hadoop-tools-dist in the patch failed. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 5s | The patch has no ill-formed XML file. | | -1 :x: | shadedclient | 0m 17s | patch has errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 13s | hadoop-tools in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | -1 :x: | javadoc | 0m 11s | hadoop-msgraph in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | -1 :x: | javadoc | 0m 13s | hadoop-tools-dist in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | -1 :x: | javadoc | 0m 13s | hadoop-tools in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -1 :x: | javadoc | 0m 11s | hadoop-msgraph in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -1 :x: | javadoc | 0m 13s | hadoop-tools-dist in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | +0 :ok: | findbugs | 0m 13s | hadoop-project has no data from findbugs | | -1 :x: | findbugs | 0m 13s | hadoop-too
[jira] [Work logged] (HADOOP-16680) Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
[ https://issues.apache.org/jira/browse/HADOOP-16680?focusedWorklogId=618784&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618784 ] ASF GitHub Bot logged work on HADOOP-16680: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:10 Start Date: 05/Jul/21 20:10 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #1727: URL: https://github.com/apache/hadoop/pull/1727#issuecomment-557384227 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 6s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 19m 30s | trunk passed | | +1 :green_heart: | compile | 18m 0s | trunk passed | | +1 :green_heart: | checkstyle | 2m 41s | trunk passed | | +1 :green_heart: | mvnsite | 3m 27s | trunk passed | | +1 :green_heart: | shadedclient | 21m 37s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 2m 25s | trunk passed | | +0 :ok: | spotbugs | 5m 56s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +0 :ok: | findbugs | 0m 23s | branch/hadoop-project no findbugs output file (findbugsXml.xml) | | +0 :ok: | findbugs | 0m 21s | branch/hadoop-tools/hadoop-tools-dist no findbugs output file (findbugsXml.xml) | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 2m 29s | hadoop-tools in the patch failed. | | -1 :x: | mvninstall | 0m 19s | hadoop-tools-dist in the patch failed. | | +1 :green_heart: | compile | 19m 35s | the patch passed | | -1 :x: | javac | 19m 35s | root generated 2 new + 1869 unchanged - 0 fixed = 1871 total (was 1869) | | -0 :warning: | checkstyle | 3m 7s | root: The patch generated 10 new + 0 unchanged - 0 fixed = 10 total (was 0) | | +1 :green_heart: | mvnsite | 4m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 6s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 15m 28s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 3m 19s | the patch passed | | +0 :ok: | findbugs | 0m 29s | hadoop-project has no data from findbugs | | -1 :x: | findbugs | 0m 33s | hadoop-msgraph in the patch failed. | | -1 :x: | findbugs | 6m 12s | hadoop-tools in the patch failed. | | +0 :ok: | findbugs | 0m 32s | hadoop-tools/hadoop-tools-dist has no data from findbugs | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 30s | hadoop-project in the patch passed. | | +1 :green_heart: | unit | 0m 38s | hadoop-msgraph in the patch passed. | | -1 :x: | unit | 0m 33s | hadoop-tools in the patch failed. | | +1 :green_heart: | unit | 0m 31s | hadoop-tools-dist in the patch passed. | | -1 :x: | asflicense | 0m 55s | The patch generated 3 ASF License warnings. | | | | 135m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1727 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle | | uname | Linux 064379991d46 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / b89fd4d | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/patch-mvninstall-hadoop-tools.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/patch-mvninstall-hadoop-tools_hadoop-tools-dist.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/diff-compile-javac-root.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/diff-ch
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #1727: HADOOP-16680. Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
hadoop-yetus removed a comment on pull request #1727: URL: https://github.com/apache/hadoop/pull/1727#issuecomment-667180788 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +0 :ok: | markdownlint | 0m 0s | markdownlint was not available. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 0m 28s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 20m 4s | trunk passed | | +1 :green_heart: | compile | 20m 14s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 17m 41s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +1 :green_heart: | checkstyle | 2m 34s | trunk passed | | +1 :green_heart: | mvnsite | 3m 38s | trunk passed | | +1 :green_heart: | shadedclient | 20m 28s | branch has no errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 36s | hadoop-tools in trunk failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | +1 :green_heart: | javadoc | 2m 34s | trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09 | | +0 :ok: | spotbugs | 5m 37s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +0 :ok: | findbugs | 0m 28s | branch/hadoop-project no findbugs output file (findbugsXml.xml) | | +0 :ok: | findbugs | 0m 30s | branch/hadoop-tools/hadoop-tools-dist no findbugs output file (findbugsXml.xml) | | -0 :warning: | patch | 5m 55s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 0m 13s | hadoop-tools in the patch failed. | | -1 :x: | mvninstall | 0m 11s | hadoop-msgraph in the patch failed. | | -1 :x: | mvninstall | 0m 16s | hadoop-tools-dist in the patch failed. | | -1 :x: | compile | 0m 14s | root in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | -1 :x: | javac | 0m 14s | root in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | -1 :x: | compile | 0m 14s | root in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -1 :x: | javac | 0m 14s | root in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -0 :warning: | checkstyle | 0m 12s | The patch fails to run checkstyle in root | | -1 :x: | mvnsite | 0m 13s | hadoop-tools in the patch failed. | | -1 :x: | mvnsite | 0m 11s | hadoop-msgraph in the patch failed. | | -1 :x: | mvnsite | 0m 12s | hadoop-tools-dist in the patch failed. | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 5s | The patch has no ill-formed XML file. | | -1 :x: | shadedclient | 0m 17s | patch has errors when building and testing our client artifacts. | | -1 :x: | javadoc | 0m 13s | hadoop-tools in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | -1 :x: | javadoc | 0m 11s | hadoop-msgraph in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | -1 :x: | javadoc | 0m 13s | hadoop-tools-dist in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1. | | -1 :x: | javadoc | 0m 13s | hadoop-tools in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -1 :x: | javadoc | 0m 11s | hadoop-msgraph in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | -1 :x: | javadoc | 0m 13s | hadoop-tools-dist in the patch failed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09. | | +0 :ok: | findbugs | 0m 13s | hadoop-project has no data from findbugs | | -1 :x: | findbugs | 0m 13s | hadoop-tools in the patch failed. | | -1 :x: | findbugs | 0m 12s | hadoop-msgraph in the patch failed. | | -1 :x: | findbugs | 0m 13s | hadoop-tools-dist in the patch failed. | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 13s | hadoop-project in the patch passed. | | -1 :x: | unit | 0m 13s | hadoop-tools in the patch failed. | | -1 :x: | unit | 0m 11s | hadoop-msgraph in the patch fa
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #1727: HADOOP-16680. Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
hadoop-yetus removed a comment on pull request #1727: URL: https://github.com/apache/hadoop/pull/1727#issuecomment-557384227 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 11s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +0 :ok: | mvndep | 1m 6s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 19m 30s | trunk passed | | +1 :green_heart: | compile | 18m 0s | trunk passed | | +1 :green_heart: | checkstyle | 2m 41s | trunk passed | | +1 :green_heart: | mvnsite | 3m 27s | trunk passed | | +1 :green_heart: | shadedclient | 21m 37s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 2m 25s | trunk passed | | +0 :ok: | spotbugs | 5m 56s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +0 :ok: | findbugs | 0m 23s | branch/hadoop-project no findbugs output file (findbugsXml.xml) | | +0 :ok: | findbugs | 0m 21s | branch/hadoop-tools/hadoop-tools-dist no findbugs output file (findbugsXml.xml) | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 35s | Maven dependency ordering for patch | | -1 :x: | mvninstall | 2m 29s | hadoop-tools in the patch failed. | | -1 :x: | mvninstall | 0m 19s | hadoop-tools-dist in the patch failed. | | +1 :green_heart: | compile | 19m 35s | the patch passed | | -1 :x: | javac | 19m 35s | root generated 2 new + 1869 unchanged - 0 fixed = 1871 total (was 1869) | | -0 :warning: | checkstyle | 3m 7s | root: The patch generated 10 new + 0 unchanged - 0 fixed = 10 total (was 0) | | +1 :green_heart: | mvnsite | 4m 28s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | xml | 0m 6s | The patch has no ill-formed XML file. | | +1 :green_heart: | shadedclient | 15m 28s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 3m 19s | the patch passed | | +0 :ok: | findbugs | 0m 29s | hadoop-project has no data from findbugs | | -1 :x: | findbugs | 0m 33s | hadoop-msgraph in the patch failed. | | -1 :x: | findbugs | 6m 12s | hadoop-tools in the patch failed. | | +0 :ok: | findbugs | 0m 32s | hadoop-tools/hadoop-tools-dist has no data from findbugs | ||| _ Other Tests _ | | +1 :green_heart: | unit | 0m 30s | hadoop-project in the patch passed. | | +1 :green_heart: | unit | 0m 38s | hadoop-msgraph in the patch passed. | | -1 :x: | unit | 0m 33s | hadoop-tools in the patch failed. | | +1 :green_heart: | unit | 0m 31s | hadoop-tools-dist in the patch passed. | | -1 :x: | asflicense | 0m 55s | The patch generated 3 ASF License warnings. | | | | 135m 59s | | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1727 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle | | uname | Linux 064379991d46 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / b89fd4d | | Default Java | 1.8.0_222 | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/patch-mvninstall-hadoop-tools.txt | | mvninstall | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/patch-mvninstall-hadoop-tools_hadoop-tools-dist.txt | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/diff-compile-javac-root.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/diff-checkstyle-root.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/patch-findbugs-hadoop-tools_hadoop-msgraph.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/patch-findbugs-hadoop-tools.txt | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1727/1/artifact/out/patch-unit-hadoop-tools.txt | | Test Results |
[jira] [Work logged] (HADOOP-16680) Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
[ https://issues.apache.org/jira/browse/HADOOP-16680?focusedWorklogId=618783&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618783 ] ASF GitHub Bot logged work on HADOOP-16680: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:09 Start Date: 05/Jul/21 20:09 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #1690: URL: https://github.com/apache/hadoop/pull/1690#issuecomment-548993831 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 33 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1232 | trunk passed | | +1 | compile | 1058 | trunk passed | | +1 | checkstyle | 44 | trunk passed | | +1 | mvnsite | 81 | trunk passed | | +1 | shadedclient | 943 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 86 | trunk passed | | 0 | spotbugs | 126 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 123 | trunk passed | | -0 | patch | 157 | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 57 | the patch passed | | +1 | compile | 1040 | the patch passed | | -1 | javac | 1040 | root generated 2 new + 1888 unchanged - 0 fixed = 1890 total (was 1888) | | -0 | checkstyle | 46 | hadoop-common-project/hadoop-common: The patch generated 12 new + 0 unchanged - 0 fixed = 12 total (was 0) | | +1 | mvnsite | 77 | the patch passed | | +1 | whitespace | 1 | The patch has no whitespace issues. | | +1 | xml | 1 | The patch has no ill-formed XML file. | | -1 | shadedclient | 176 | patch has errors when building and testing our client artifacts. | | +1 | javadoc | 84 | the patch passed | | -1 | findbugs | 135 | hadoop-common-project/hadoop-common generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | ||| _ Other Tests _ | | -1 | unit | 534 | hadoop-common in the patch failed. | | -1 | asflicense | 45 | The patch generated 11 ASF License warnings. | | | | 5875 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-common-project/hadoop-common | | | Found reliance on default encoding in org.apache.hadoop.security.oauth2.AzureADAuthenticator.consumeInputStream(InputStream):in org.apache.hadoop.security.oauth2.AzureADAuthenticator.consumeInputStream(InputStream): new String(byte[], int, int) At AzureADAuthenticator.java:[line 153] | | | org.apache.hadoop.security.oauth2.QueryParams.serialize() makes inefficient use of keySet iterator instead of entrySet iterator At QueryParams.java:keySet iterator instead of entrySet iterator At QueryParams.java:[line 44] | | Failed junit tests | hadoop.conf.TestCommonConfigurationFields | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1690 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle | | uname | Linux 6de7a5250d7e 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 7d7acb0 | | Default Java | 1.8.0_222 | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/diff-compile-javac-root.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/testReport/ | | asflicense | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/patch-asflicense-problems.txt | | Max. process+thread count | 1442 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://b
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #1690: HADOOP-16680. Add MicrosoftGraphGroupsMapping GroupMappingServiceProvider
hadoop-yetus removed a comment on pull request #1690: URL: https://github.com/apache/hadoop/pull/1690#issuecomment-548993831 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | 0 | reexec | 33 | Docker mode activated. | ||| _ Prechecks _ | | +1 | dupname | 0 | No case conflicting files found. | | +1 | @author | 0 | The patch does not contain any @author tags. | | +1 | test4tests | 0 | The patch appears to include 3 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 | mvninstall | 1232 | trunk passed | | +1 | compile | 1058 | trunk passed | | +1 | checkstyle | 44 | trunk passed | | +1 | mvnsite | 81 | trunk passed | | +1 | shadedclient | 943 | branch has no errors when building and testing our client artifacts. | | +1 | javadoc | 86 | trunk passed | | 0 | spotbugs | 126 | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 | findbugs | 123 | trunk passed | | -0 | patch | 157 | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 | mvninstall | 57 | the patch passed | | +1 | compile | 1040 | the patch passed | | -1 | javac | 1040 | root generated 2 new + 1888 unchanged - 0 fixed = 1890 total (was 1888) | | -0 | checkstyle | 46 | hadoop-common-project/hadoop-common: The patch generated 12 new + 0 unchanged - 0 fixed = 12 total (was 0) | | +1 | mvnsite | 77 | the patch passed | | +1 | whitespace | 1 | The patch has no whitespace issues. | | +1 | xml | 1 | The patch has no ill-formed XML file. | | -1 | shadedclient | 176 | patch has errors when building and testing our client artifacts. | | +1 | javadoc | 84 | the patch passed | | -1 | findbugs | 135 | hadoop-common-project/hadoop-common generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) | ||| _ Other Tests _ | | -1 | unit | 534 | hadoop-common in the patch failed. | | -1 | asflicense | 45 | The patch generated 11 ASF License warnings. | | | | 5875 | | | Reason | Tests | |---:|:--| | FindBugs | module:hadoop-common-project/hadoop-common | | | Found reliance on default encoding in org.apache.hadoop.security.oauth2.AzureADAuthenticator.consumeInputStream(InputStream):in org.apache.hadoop.security.oauth2.AzureADAuthenticator.consumeInputStream(InputStream): new String(byte[], int, int) At AzureADAuthenticator.java:[line 153] | | | org.apache.hadoop.security.oauth2.QueryParams.serialize() makes inefficient use of keySet iterator instead of entrySet iterator At QueryParams.java:keySet iterator instead of entrySet iterator At QueryParams.java:[line 44] | | Failed junit tests | hadoop.conf.TestCommonConfigurationFields | | Subsystem | Report/Notes | |--:|:-| | Docker | Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/1690 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml findbugs checkstyle | | uname | Linux 6de7a5250d7e 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | personality/hadoop.sh | | git revision | trunk / 7d7acb0 | | Default Java | 1.8.0_222 | | javac | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/diff-compile-javac-root.txt | | checkstyle | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt | | findbugs | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/new-findbugs-hadoop-common-project_hadoop-common.html | | unit | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt | | Test Results | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/testReport/ | | asflicense | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/artifact/out/patch-asflicense-problems.txt | | Max. process+thread count | 1442 (vs. ulimit of 5500) | | modules | C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common | | Console output | https://builds.apache.org/job/hadoop-multibranch/job/PR-1690/1/console | | versions | git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1 | | Powered by | Apache Yetus 0.10.0 http://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,
[jira] [Work logged] (HADOOP-17402) Add GCS FS impl reference to core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-17402?focusedWorklogId=618781&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618781 ] ASF GitHub Bot logged work on HADOOP-17402: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:08 Start Date: 05/Jul/21 20:08 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #2638: URL: https://github.com/apache/hadoop/pull/2638#issuecomment-874308502 Merged to trunk. @ravwojdyla : if you cherrypick this to branch-3.3 and submit a new PR there, I'll merge as soon as yetus is happy. Leaving the JIRA open for now thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618781) Time Spent: 3h 40m (was: 3.5h) > Add GCS FS impl reference to core-default.xml > - > > Key: HADOOP-17402 > URL: https://issues.apache.org/jira/browse/HADOOP-17402 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Reporter: Rafal Wojdyla >Priority: Major > Labels: pull-request-available > Time Spent: 3h 40m > Remaining Estimate: 0h > > Akin to current S3 default configuration add GCS configuration, specifically > to declare the GCS implementation. [GCS > connector|https://cloud.google.com/dataproc/docs/concepts/connectors/cloud-storage]. > Has this not been done since the GCS connector is not part of the hadoop/ASF > codebase, or is there any other blocker? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #2638: HADOOP-17402. Add GCS config to the core-site
steveloughran commented on pull request #2638: URL: https://github.com/apache/hadoop/pull/2638#issuecomment-874308502 Merged to trunk. @ravwojdyla : if you cherrypick this to branch-3.3 and submit a new PR there, I'll merge as soon as yetus is happy. Leaving the JIRA open for now thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17402) Add GCS FS impl reference to core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-17402?focusedWorklogId=618780&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618780 ] ASF GitHub Bot logged work on HADOOP-17402: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:07 Start Date: 05/Jul/21 20:07 Worklog Time Spent: 10m Work Description: steveloughran merged pull request #2638: URL: https://github.com/apache/hadoop/pull/2638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618780) Time Spent: 3.5h (was: 3h 20m) > Add GCS FS impl reference to core-default.xml > - > > Key: HADOOP-17402 > URL: https://issues.apache.org/jira/browse/HADOOP-17402 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Reporter: Rafal Wojdyla >Priority: Major > Labels: pull-request-available > Time Spent: 3.5h > Remaining Estimate: 0h > > Akin to current S3 default configuration add GCS configuration, specifically > to declare the GCS implementation. [GCS > connector|https://cloud.google.com/dataproc/docs/concepts/connectors/cloud-storage]. > Has this not been done since the GCS connector is not part of the hadoop/ASF > codebase, or is there any other blocker? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran merged pull request #2638: HADOOP-17402. Add GCS config to the core-site
steveloughran merged pull request #2638: URL: https://github.com/apache/hadoop/pull/2638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17139) Re-enable optimized copyFromLocal implementation in S3AFileSystem
[
https://issues.apache.org/jira/browse/HADOOP-17139?focusedWorklogId=618777&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618777
]
ASF GitHub Bot logged work on HADOOP-17139:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 20:04
Start Date: 05/Jul/21 20:04
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#3101:
URL: https://github.com/apache/hadoop/pull/3101#discussion_r664110702
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractCopyFromLocalTest.java
##
@@ -0,0 +1,285 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.contract;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.io.IOUtils;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.PathExistsException;
+import org.junit.Test;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+
+import static org.apache.hadoop.test.LambdaTestUtils.intercept;
+
+public abstract class AbstractContractCopyFromLocalTest extends
+AbstractFSContractTestBase {
+
+private static final Charset ASCII = StandardCharsets.US_ASCII;
+private File file;
+
+@Override
+public void teardown() throws Exception {
+super.teardown();
+if (file != null) {
+file.delete();
+}
+}
+
+@Test
+public void testCopyEmptyFile() throws Throwable {
+file = File.createTempFile("test", ".txt");
+Path dest = copyFromLocal(file, true);
+assertPathExists("uploaded file", dest);
+}
+
+@Test
+public void testCopyFile() throws Throwable {
+String message = "hello";
+file = createTempFile(message);
+Path dest = copyFromLocal(file, true);
+
+assertPathExists("uploaded file not found", dest);
+// TODO: Should this be assertFileExists?
+assertTrue("source file deleted", Files.exists(file.toPath()));
+
+FileSystem fs = getFileSystem();
+FileStatus status = fs.getFileStatus(dest);
+assertEquals("File length of " + status,
+message.getBytes(ASCII).length, status.getLen());
+assertFileTextEquals(dest, message);
+}
+
+@Test
+public void testCopyFileNoOverwrite() throws Throwable {
+file = createTempFile("hello");
+copyFromLocal(file, true);
+intercept(PathExistsException.class,
+() -> copyFromLocal(file, false));
+}
+
+@Test
+public void testCopyFileOverwrite() throws Throwable {
+file = createTempFile("hello");
+Path dest = copyFromLocal(file, true);
+String updated = "updated";
+FileUtils.write(file, updated, ASCII);
+copyFromLocal(file, true);
+assertFileTextEquals(dest, updated);
+}
+
+@Test
+public void testCopyMissingFile() throws Throwable {
+file = createTempFile("test");
+file.delete();
+// first upload to create
+intercept(FileNotFoundException.class, "",
+() -> copyFromLocal(file, true));
+}
+
+@Test
+public void testSourceIsFileAndDelSrcTrue() throws Throwable {
+describe("Source is a file delSrc flag is set to true");
+
+file = createTempFile("test");
+copyFromLocal(file, false, true);
+
+assertFalse("uploaded file", Files.exists(file.toPath()));
+}
+
+@Test
+public void testSourceIsFileAndDestinationIsDirectory() throws Throwable {
+describe("Source is a file and destination is a directory. File" +
+"should be copied inside the directory.");
+
+file = createTempFile("test");
+Path source = new Path(file.toURI());
+FileSystem fs = getFileSystem();
+
+File dir = createTempDirectory("test");
+Path destination = fi
[jira] [Work logged] (HADOOP-17402) Add GCS FS impl reference to core-default.xml
[
https://issues.apache.org/jira/browse/HADOOP-17402?focusedWorklogId=618779&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618779
]
ASF GitHub Bot logged work on HADOOP-17402:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 20:04
Start Date: 05/Jul/21 20:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2638:
URL: https://github.com/apache/hadoop/pull/2638#issuecomment-770508993
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 28s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 25s | | trunk passed |
| +1 :green_heart: | compile | 20m 26s | | trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 17m 47s | | trunk passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +1 :green_heart: | checkstyle | 1m 9s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 21s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +0 :ok: | spotbugs | 2m 18s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 16s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 53s | | the patch passed |
| +1 :green_heart: | compile | 20m 0s | | the patch passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 20m 0s | | the patch passed |
| +1 :green_heart: | compile | 17m 45s | | the patch passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +1 :green_heart: | javac | 17m 45s | | the patch passed |
| +1 :green_heart: | checkstyle | 1m 4s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 27s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 13m 17s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 4s | | the patch passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +1 :green_heart: | findbugs | 2m 23s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 19s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 174m 52s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2638/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 9e384faea1bd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6fafdd76829 |
| Default Java | Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2638/3/testReport/ |
| Max. process+thread count | 3149 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2638/3/console |
|
[jira] [Work logged] (HADOOP-17402) Add GCS FS impl reference to core-default.xml
[ https://issues.apache.org/jira/browse/HADOOP-17402?focusedWorklogId=618778&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618778 ] ASF GitHub Bot logged work on HADOOP-17402: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:04 Start Date: 05/Jul/21 20:04 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2638: URL: https://github.com/apache/hadoop/pull/2638#issuecomment-764861125 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618778) Time Spent: 3h 10m (was: 3h) > Add GCS FS impl reference to core-default.xml > - > > Key: HADOOP-17402 > URL: https://issues.apache.org/jira/browse/HADOOP-17402 > Project: Hadoop Common > Issue Type: Improvement > Components: fs >Reporter: Rafal Wojdyla >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > Akin to current S3 default configuration add GCS configuration, specifically > to declare the GCS implementation. [GCS > connector|https://cloud.google.com/dataproc/docs/concepts/connectors/cloud-storage]. > Has this not been done since the GCS connector is not part of the hadoop/ASF > codebase, or is there any other blocker? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2638: HADOOP-17402. Add GCS config to the core-site
hadoop-yetus removed a comment on pull request #2638:
URL: https://github.com/apache/hadoop/pull/2638#issuecomment-770508993
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 28s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | | 0m 0s | [test4tests](test4tests) | The patch
appears to include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 25s | | trunk passed |
| +1 :green_heart: | compile | 20m 26s | | trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 17m 47s | | trunk passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +1 :green_heart: | checkstyle | 1m 9s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 16m 21s | | branch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 4s | | trunk passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 36s | | trunk passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +0 :ok: | spotbugs | 2m 18s | | Used deprecated FindBugs config;
considering switching to SpotBugs. |
| +1 :green_heart: | findbugs | 2m 16s | | trunk passed |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 53s | | the patch passed |
| +1 :green_heart: | compile | 20m 0s | | the patch passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 20m 0s | | the patch passed |
| +1 :green_heart: | compile | 17m 45s | | the patch passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +1 :green_heart: | javac | 17m 45s | | the patch passed |
| +1 :green_heart: | checkstyle | 1m 4s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 27s | | the patch passed |
| +1 :green_heart: | whitespace | 0m 0s | | The patch has no
whitespace issues. |
| +1 :green_heart: | xml | 0m 1s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | shadedclient | 13m 17s | | patch has no errors
when building and testing our client artifacts. |
| +1 :green_heart: | javadoc | 1m 4s | | the patch passed with JDK
Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed with JDK
Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| +1 :green_heart: | findbugs | 2m 23s | | the patch passed |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 19s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 174m 52s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2638/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2638 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient xml findbugs checkstyle |
| uname | Linux 9e384faea1bd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 6fafdd76829 |
| Default Java | Private Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_275-8u275-b01-0ubuntu1~20.04-b01 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2638/3/testReport/ |
| Max. process+thread count | 3149 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2638/3/console |
| versions | git=2.25.1 maven=3.6.3 findbugs=4.0.6 |
| Powered by | Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For quer
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2638: HADOOP-17402. Add GCS config to the core-site
hadoop-yetus removed a comment on pull request #2638: URL: https://github.com/apache/hadoop/pull/2638#issuecomment-764861125 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #3101: HADOOP-17139 Re-enable optimized copyFromLocal implementation in S3AFileSystem
steveloughran commented on a change in pull request #3101:
URL: https://github.com/apache/hadoop/pull/3101#discussion_r664110702
##
File path:
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractCopyFromLocalTest.java
##
@@ -0,0 +1,285 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.fs.contract;
+
+import org.apache.commons.io.FileUtils;
+import org.apache.commons.io.IOUtils;
+import org.apache.hadoop.fs.FileStatus;
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.fs.PathExistsException;
+import org.junit.Test;
+
+import java.io.File;
+import java.io.FileNotFoundException;
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+
+import static org.apache.hadoop.test.LambdaTestUtils.intercept;
+
+public abstract class AbstractContractCopyFromLocalTest extends
+AbstractFSContractTestBase {
+
+private static final Charset ASCII = StandardCharsets.US_ASCII;
+private File file;
+
+@Override
+public void teardown() throws Exception {
+super.teardown();
+if (file != null) {
+file.delete();
+}
+}
+
+@Test
+public void testCopyEmptyFile() throws Throwable {
+file = File.createTempFile("test", ".txt");
+Path dest = copyFromLocal(file, true);
+assertPathExists("uploaded file", dest);
+}
+
+@Test
+public void testCopyFile() throws Throwable {
+String message = "hello";
+file = createTempFile(message);
+Path dest = copyFromLocal(file, true);
+
+assertPathExists("uploaded file not found", dest);
+// TODO: Should this be assertFileExists?
+assertTrue("source file deleted", Files.exists(file.toPath()));
+
+FileSystem fs = getFileSystem();
+FileStatus status = fs.getFileStatus(dest);
+assertEquals("File length of " + status,
+message.getBytes(ASCII).length, status.getLen());
+assertFileTextEquals(dest, message);
+}
+
+@Test
+public void testCopyFileNoOverwrite() throws Throwable {
+file = createTempFile("hello");
+copyFromLocal(file, true);
+intercept(PathExistsException.class,
+() -> copyFromLocal(file, false));
+}
+
+@Test
+public void testCopyFileOverwrite() throws Throwable {
+file = createTempFile("hello");
+Path dest = copyFromLocal(file, true);
+String updated = "updated";
+FileUtils.write(file, updated, ASCII);
+copyFromLocal(file, true);
+assertFileTextEquals(dest, updated);
+}
+
+@Test
+public void testCopyMissingFile() throws Throwable {
+file = createTempFile("test");
+file.delete();
+// first upload to create
+intercept(FileNotFoundException.class, "",
+() -> copyFromLocal(file, true));
+}
+
+@Test
+public void testSourceIsFileAndDelSrcTrue() throws Throwable {
+describe("Source is a file delSrc flag is set to true");
+
+file = createTempFile("test");
+copyFromLocal(file, false, true);
+
+assertFalse("uploaded file", Files.exists(file.toPath()));
+}
+
+@Test
+public void testSourceIsFileAndDestinationIsDirectory() throws Throwable {
+describe("Source is a file and destination is a directory. File" +
+"should be copied inside the directory.");
+
+file = createTempFile("test");
+Path source = new Path(file.toURI());
+FileSystem fs = getFileSystem();
+
+File dir = createTempDirectory("test");
+Path destination = fileToPath(dir);
+fs.delete(destination, false);
+mkdirs(destination);
+
+fs.copyFromLocalFile(source, destination);
+System.out.println("Did this work?");
+}
+
+@Test
+public void testSrcIsDirWithFilesAndCopySuccessful() throws Throwable {
+describe("Source is a directory with files, copy should copy all" +
+" dir contents to source");
+String firstChild =
[jira] [Work logged] (HADOOP-17139) Re-enable optimized copyFromLocal implementation in S3AFileSystem
[ https://issues.apache.org/jira/browse/HADOOP-17139?focusedWorklogId=618776&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618776 ] ASF GitHub Bot logged work on HADOOP-17139: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:01 Start Date: 05/Jul/21 20:01 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #3101: URL: https://github.com/apache/hadoop/pull/3101#issuecomment-868703568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618776) Time Spent: 3h (was: 2h 50m) > Re-enable optimized copyFromLocal implementation in S3AFileSystem > - > > Key: HADOOP-17139 > URL: https://issues.apache.org/jira/browse/HADOOP-17139 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Affects Versions: 3.3.0, 3.2.1 >Reporter: Sahil Takiar >Assignee: Bogdan Stolojan >Priority: Minor > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > It looks like HADOOP-15932 disabled the optimized copyFromLocal > implementation in S3A for correctness reasons. innerCopyFromLocalFile should > be fixed and re-enabled. The current implementation uses > FileSystem.copyFromLocal which will open an input stream from the local fs > and an output stream to the destination fs, and then call IOUtils.copyBytes. > With default configs, this will cause S3A to read the file into memory, write > it back to a file on the local fs, and then when the file is closed, upload > it to S3. > The optimized version of copyFromLocal in innerCopyFromLocalFile, directly > creates a PutObjectRequest request with the local file as the input. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3101: HADOOP-17139 Re-enable optimized copyFromLocal implementation in S3AFileSystem
hadoop-yetus removed a comment on pull request #3101: URL: https://github.com/apache/hadoop/pull/3101#issuecomment-868703568 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-16752) ABFS: test failure testLastModifiedTime()
[
https://issues.apache.org/jira/browse/HADOOP-16752?focusedWorklogId=618775&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618775
]
ASF GitHub Bot logged work on HADOOP-16752:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 20:00
Start Date: 05/Jul/21 20:00
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #3045:
URL: https://github.com/apache/hadoop/pull/3045#issuecomment-846952997
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 10m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 11s | | trunk passed |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 2m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 39s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 47s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 28s | | The patch does not
generate ASF License warnings. |
| | | 74m 1s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3045/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3045 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux bb000f84c894 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fdb5344ed2ffa3a83b4bb4d7b97a1e81b6191a8c |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3045/1/testReport/ |
| Max. process+thread count | 592 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3045/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This
[jira] [Work logged] (HADOOP-16752) ABFS: test failure testLastModifiedTime()
[ https://issues.apache.org/jira/browse/HADOOP-16752?focusedWorklogId=618774&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618774 ] ASF GitHub Bot logged work on HADOOP-16752: --- Author: ASF GitHub Bot Created on: 05/Jul/21 20:00 Start Date: 05/Jul/21 20:00 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #3045: URL: https://github.com/apache/hadoop/pull/3045#issuecomment-874304825 Looking at it some more. It's getting modtime from the store, create time from the clock. 1. Tests under load can have an endtime > modtime 2. clock skew could cause errors if client clock ahead of store Maybe we should just allow for +1 five minutes. All we want to know its is in local time, not UTC. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618774) Time Spent: 50m (was: 40m) > ABFS: test failure testLastModifiedTime() > - > > Key: HADOOP-16752 > URL: https://issues.apache.org/jira/browse/HADOOP-16752 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Reporter: Da Zhou >Assignee: Sneha Vijayarajan >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > java.lang.AssertionError: lastModifiedTime should be after minCreateStartTime > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.assertTrue(Assert.java:41) > at > org.apache.hadoop.fs.azurebfs.ITestAzureBlobFileSystemFileStatus.testLastModifiedTime(ITestAzureBlobFileSystemFileStatus.java:138) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) > at > org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) > at > org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.lang.Thread.run(Thread.java:748) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3045: HADOOP-16752. assertion failure in ITestAzureBlobFileSystemFileStatus
hadoop-yetus removed a comment on pull request #3045:
URL: https://github.com/apache/hadoop/pull/3045#issuecomment-846952997
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 10m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 11s | | trunk passed |
| +1 :green_heart: | compile | 0m 35s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 23s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 36s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 27s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 2m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 28s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 0s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 39s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 47s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 28s | | The patch does not
generate ASF License warnings. |
| | | 74m 1s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3045/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3045 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux bb000f84c894 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / fdb5344ed2ffa3a83b4bb4d7b97a1e81b6191a8c |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3045/1/testReport/ |
| Max. process+thread count | 592 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3045/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
[GitHub] [hadoop] steveloughran commented on pull request #3045: HADOOP-16752. assertion failure in ITestAzureBlobFileSystemFileStatus
steveloughran commented on pull request #3045: URL: https://github.com/apache/hadoop/pull/3045#issuecomment-874304825 Looking at it some more. It's getting modtime from the store, create time from the clock. 1. Tests under load can have an endtime > modtime 2. clock skew could cause errors if client clock ahead of store Maybe we should just allow for +1 five minutes. All we want to know its is in local time, not UTC. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17764) S3AInputStream read does not re-open the input stream on the second read retry attempt
[
https://issues.apache.org/jira/browse/HADOOP-17764?focusedWorklogId=618771&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618771
]
ASF GitHub Bot logged work on HADOOP-17764:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 19:56
Start Date: 05/Jul/21 19:56
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #3109:
URL: https://github.com/apache/hadoop/pull/3109#issuecomment-874303444
Note: we could also backport to 3.2.x if you want to cherrypick the 3.3
changes and retest...then provide a new PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 618771)
Time Spent: 9h (was: 8h 50m)
> S3AInputStream read does not re-open the input stream on the second read
> retry attempt
> --
>
> Key: HADOOP-17764
> URL: https://issues.apache.org/jira/browse/HADOOP-17764
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
>Affects Versions: 3.3.1
>Reporter: Zamil Majdy
>Assignee: Zamil Majdy
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.2
>
> Time Spent: 9h
> Remaining Estimate: 0h
>
> *Bug description:*
> The read method in S3AInputStream has this following behaviour when an
> IOException happening during the read:
> * {{reopen and read quickly}}: The client after failing in the first attempt
> of {{read}}, will reopen the stream and try reading again without {{sleep}}.
> * {{reopen and wait for fixed duration}}: The client after failing in the
> attempt of {{read}}, will reopen the stream, sleep for
> {{fs.s3a.retry.interval}} milliseconds (defaults to 500 ms), and then try
> reading from the stream.
> While doing the {{reopen and read quickly}} process, the subsequent read will
> be retried without reopening the input stream in case of the second failure
> happened. This leads to some of the bytes read being skipped which results to
> corrupt/less data than required.
>
> *Scenario to reproduce:*
> * Execute S3AInputStream `read()` or `read(b, off, len)`.
> * The read failed and throws `Connection Reset` exception after reading some
> data.
> * The InputStream is re-opened and another `read()` or `read(b, off, len)`
> is executed
> * The read failed for the second time and throws `Connection Reset`
> exception after reading some data.
> * The InputStream is not re-opened and another `read()` or `read(b, off,
> len)` is executed after sleep
> * The read succeed, but it skips the first few bytes that has already been
> read on the second failure.
>
> *Proposed fix:*
> [https://github.com/apache/hadoop/pull/3109]
> Added the test that reproduces the issue along with the fix
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #3109: HADOOP-17764. S3AInputStream read does not re-open the input stream on the second read retry attempt
steveloughran commented on pull request #3109: URL: https://github.com/apache/hadoop/pull/3109#issuecomment-874303444 Note: we could also backport to 3.2.x if you want to cherrypick the 3.3 changes and retest...then provide a new PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17765) ABFS: Use Unique File Paths in Tests
[
https://issues.apache.org/jira/browse/HADOOP-17765?focusedWorklogId=618769&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618769
]
ASF GitHub Bot logged work on HADOOP-17765:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 19:53
Start Date: 05/Jul/21 19:53
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#3122:
URL: https://github.com/apache/hadoop/pull/3122#discussion_r664106486
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemBackCompat.java
##
@@ -50,13 +52,17 @@ public void testBlobBackCompat() throws Exception {
CloudBlobContainer container =
blobClient.getContainerReference(this.getFileSystemName());
container.createIfNotExists();
-CloudBlockBlob blockBlob =
container.getBlockBlobReference("test/10/10/10");
+String testPath = "test" + UUID.randomUUID();
+CloudBlockBlob blockBlob = container
+.getBlockBlobReference(testPath + "/10/10/10");
blockBlob.uploadText("");
-blockBlob = container.getBlockBlobReference("test/10/123/3/2/1/3");
+blockBlob = container.getBlockBlobReference(testPath + "/10/123/3/2/1/3");
blockBlob.uploadText("");
-FileStatus[] fileStatuses = fs.listStatus(new Path("/test/10/"));
+System.out.println(new Path(String.format("%s/10/", testPath)));
Review comment:
use SLF4J logger
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestCustomerProvidedKey.java
##
@@ -106,7 +106,7 @@ public ITestCustomerProvidedKey() throws Exception {
@Test
public void testReadWithCPK() throws Exception {
final AzureBlobFileSystem fs = getAbfs(true);
-String fileName = path("/" + methodName.getMethodName()).toString();
+String fileName = "/" + methodName.getMethodName();
Review comment:
so this moves away from path(). why?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 618769)
Time Spent: 2h 20m (was: 2h 10m)
> ABFS: Use Unique File Paths in Tests
>
>
> Key: HADOOP-17765
> URL: https://issues.apache.org/jira/browse/HADOOP-17765
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
>Affects Versions: 3.3.1
>Reporter: Sumangala Patki
>Assignee: Sumangala Patki
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> Many of ABFS driver tests use common names for file paths (e.g.,
> "/testfile"). This poses a risk of errors during parallel test runs when
> static variables (such as those for monitoring stats) affected by file paths
> are introduced.
> Using unique test file names will avoid possible errors arising from shared
> resources during parallel runs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on a change in pull request #3122: HADOOP-17765. ABFS: Use Unique File Paths in Tests
steveloughran commented on a change in pull request #3122:
URL: https://github.com/apache/hadoop/pull/3122#discussion_r664106486
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemBackCompat.java
##
@@ -50,13 +52,17 @@ public void testBlobBackCompat() throws Exception {
CloudBlobContainer container =
blobClient.getContainerReference(this.getFileSystemName());
container.createIfNotExists();
-CloudBlockBlob blockBlob =
container.getBlockBlobReference("test/10/10/10");
+String testPath = "test" + UUID.randomUUID();
+CloudBlockBlob blockBlob = container
+.getBlockBlobReference(testPath + "/10/10/10");
blockBlob.uploadText("");
-blockBlob = container.getBlockBlobReference("test/10/123/3/2/1/3");
+blockBlob = container.getBlockBlobReference(testPath + "/10/123/3/2/1/3");
blockBlob.uploadText("");
-FileStatus[] fileStatuses = fs.listStatus(new Path("/test/10/"));
+System.out.println(new Path(String.format("%s/10/", testPath)));
Review comment:
use SLF4J logger
##
File path:
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestCustomerProvidedKey.java
##
@@ -106,7 +106,7 @@ public ITestCustomerProvidedKey() throws Exception {
@Test
public void testReadWithCPK() throws Exception {
final AzureBlobFileSystem fs = getAbfs(true);
-String fileName = path("/" + methodName.getMethodName()).toString();
+String fileName = "/" + methodName.getMethodName();
Review comment:
so this moves away from path(). why?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17779) Lock File System Creator Semaphore Uninterruptibly
[
https://issues.apache.org/jira/browse/HADOOP-17779?focusedWorklogId=618768&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618768
]
ASF GitHub Bot logged work on HADOOP-17779:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 19:48
Start Date: 05/Jul/21 19:48
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #3158:
URL: https://github.com/apache/hadoop/pull/3158#issuecomment-874300831
why do we need to make it uninterruptible?
Is there a risk of problems if one thread is waiting for the semaphore and
the process is exited?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 618768)
Time Spent: 50m (was: 40m)
> Lock File System Creator Semaphore Uninterruptibly
> --
>
> Key: HADOOP-17779
> URL: https://issues.apache.org/jira/browse/HADOOP-17779
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
>Affects Versions: 3.3.1
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {{FileSystem}} "creator permits" are acquired in an interruptable way. This
> changed the behavior of the call because previous callers were handling the
> IOException as a critical error. An interrupt was handled in the typical
> {{InterruptedException}} way. Lastly, there was no documentation of this new
> event so again, callers are not prepared.
> Restore the previous behavior and lock the semaphore Uninterruptibly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #3158: HADOOP-17779: Lock File System Creator Semaphore Uninterruptibly
steveloughran commented on pull request #3158: URL: https://github.com/apache/hadoop/pull/3158#issuecomment-874300831 why do we need to make it uninterruptible? Is there a risk of problems if one thread is waiting for the semaphore and the process is exited? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17779) Lock File System Creator Semaphore Uninterruptibly
[
https://issues.apache.org/jira/browse/HADOOP-17779?focusedWorklogId=618767&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618767
]
ASF GitHub Bot logged work on HADOOP-17779:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 19:46
Start Date: 05/Jul/21 19:46
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #3158:
URL: https://github.com/apache/hadoop/pull/3158#issuecomment-871086628
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 14s | | trunk passed |
| +1 :green_heart: | compile | 27m 25s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 22m 48s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 33s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 2m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 0s | | the patch passed |
| +1 :green_heart: | compile | 26m 24s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 26m 24s | | the patch passed |
| +1 :green_heart: | compile | 22m 45s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 22m 45s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 59s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3158/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 1 new + 75
unchanged - 0 fixed = 76 total (was 75) |
| +1 :green_heart: | mvnsite | 1m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 3s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 2m 56s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 28s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 47s | | The patch does not
generate ASF License warnings. |
| | | 206m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3158/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3158 |
| JIRA Issue | HADOOP-17779 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 3a6de050e081 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4016b27eea0bfcb421f36e0881a6839747d3 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://c
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #3158: HADOOP-17779: Lock File System Creator Semaphore Uninterruptibly
hadoop-yetus removed a comment on pull request #3158:
URL: https://github.com/apache/hadoop/pull/3158#issuecomment-871086628
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 35m 14s | | trunk passed |
| +1 :green_heart: | compile | 27m 25s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 22m 48s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 1s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 38s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 3s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 33s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 2m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 18m 30s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 0s | | the patch passed |
| +1 :green_heart: | compile | 26m 24s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 26m 24s | | the patch passed |
| +1 :green_heart: | compile | 22m 45s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 22m 45s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 59s |
[/results-checkstyle-hadoop-common-project_hadoop-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3158/1/artifact/out/results-checkstyle-hadoop-common-project_hadoop-common.txt)
| hadoop-common-project/hadoop-common: The patch generated 1 new + 75
unchanged - 0 fixed = 76 total (was 75) |
| +1 :green_heart: | mvnsite | 1m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 3s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 38s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 2m 56s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 46s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 28s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 47s | | The patch does not
generate ASF License warnings. |
| | | 206m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3158/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3158 |
| JIRA Issue | HADOOP-17779 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 3a6de050e081 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4016b27eea0bfcb421f36e0881a6839747d3 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3158/1/testReport/ |
| Max. process+thread count | 2137 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3158/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT htt
[jira] [Work logged] (HADOOP-17788) Replace IOUtils#closeQuietly usages
[ https://issues.apache.org/jira/browse/HADOOP-17788?focusedWorklogId=618766&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618766 ] ASF GitHub Bot logged work on HADOOP-17788: --- Author: ASF GitHub Bot Created on: 05/Jul/21 19:44 Start Date: 05/Jul/21 19:44 Worklog Time Spent: 10m Work Description: steveloughran commented on pull request #3171: URL: https://github.com/apache/hadoop/pull/3171#issuecomment-874299709 @aajisaka > To fix deprecation warning, I perfer using try-with-resources rather than replacing with our internal replacement. Mixed feelings there. I went through a lot of grief in HADOOP-16998/#2073 related to exceptions being raised in try with resources while a prior exception has been raised. The JVM will attach any ex raised in the resource closed to the initial ex via addSuppressed(), but if somehow it's the same exception you get an IllegalArgumentException with all stack traces lost. so, good * exception gets retained and passed up bad * none of our code (e.g. marshalling of exceptions, junit logging, log4j... does anything with these ugly * any attempt to be clever with caching and rethrowing exceptions can trigger failures -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618766) Time Spent: 1.5h (was: 1h 20m) > Replace IOUtils#closeQuietly usages > --- > > Key: HADOOP-17788 > URL: https://issues.apache.org/jira/browse/HADOOP-17788 > Project: Hadoop Common > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 1.5h > Remaining Estimate: 0h > > IOUtils#closeQuietly is deprecated since 2.6 release of commons-io without > any replacement. Since we already have good replacement available in Hadoop's > own IOUtils, we should use it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] steveloughran commented on pull request #3171: HADOOP-17788. Replace IOUtils#closeQuietly usages by Hadoop's own utility
steveloughran commented on pull request #3171: URL: https://github.com/apache/hadoop/pull/3171#issuecomment-874299709 @aajisaka > To fix deprecation warning, I perfer using try-with-resources rather than replacing with our internal replacement. Mixed feelings there. I went through a lot of grief in HADOOP-16998/#2073 related to exceptions being raised in try with resources while a prior exception has been raised. The JVM will attach any ex raised in the resource closed to the initial ex via addSuppressed(), but if somehow it's the same exception you get an IllegalArgumentException with all stack traces lost. so, good * exception gets retained and passed up bad * none of our code (e.g. marshalling of exceptions, junit logging, log4j... does anything with these ugly * any attempt to be clever with caching and rethrowing exceptions can trigger failures -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-17790) ABFS: Modifications to TracingContext
[ https://issues.apache.org/jira/browse/HADOOP-17790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17374974#comment-17374974 ] Steve Loughran commented on HADOOP-17790: - I'd really like this to pick up and integrate with the work of the S3A auditing, HADOOP-17511. This adds the code to hadoop-common to create a referrer header containing call context operation (FS API call, paths, principal), including job UUID which will span entire jobs (e.g. all spark workers) # Can the context use this and attach to requests # is there a way for myself and others to get at the logs. I don't think I can currently see the X-trace headers, can I? > ABFS: Modifications to TracingContext > - > > Key: HADOOP-17790 > URL: https://issues.apache.org/jira/browse/HADOOP-17790 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.1 >Reporter: Sumangala Patki >Priority: Major > > TracingContext introduced in the PR > [HADOOP-17290|https://github.com/apache/hadoop/pull/2520] tracks a request > with identifiers as it passes through ABFS layers. Creating this Jira to > track suggested modifications and improvements to the tracing structure. > * rename Listener; remove it from TracingContext constructor, confine to > setter > * make fields (identifiers) immutable; eliminate retry count field => reduce > instance cloning > * introduce child classes of TracingContext to handle stream/continuation ops -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17290) ABFS: Add Identifiers to Client Request Header
[
https://issues.apache.org/jira/browse/HADOOP-17290?focusedWorklogId=618763&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618763
]
ASF GitHub Bot logged work on HADOOP-17290:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 19:29
Start Date: 05/Jul/21 19:29
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2520:
URL: https://github.com/apache/hadoop/pull/2520#issuecomment-872117658
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 37 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 45s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 11s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 14m 29s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 2s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 11s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 72m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2520/23/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2520 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint |
| uname | Linux 093ca10a13ea 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8cf0fe32c89351a55c475cdd91261335591129f8 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2520/23/testReport/ |
| Max. process+thread count | 543 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2520: HADOOP-17290. ABFS: Add Identifiers to Client Request Header
hadoop-yetus removed a comment on pull request #2520:
URL: https://github.com/apache/hadoop/pull/2520#issuecomment-872117658
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 39s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 37 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 45s | | trunk passed |
| +1 :green_heart: | compile | 0m 40s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 34s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 11s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 14m 29s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 32s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 15s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 2s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 11s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 72m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2520/23/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2520 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint |
| uname | Linux 093ca10a13ea 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 8cf0fe32c89351a55c475cdd91261335591129f8 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2520/23/testReport/ |
| Max. process+thread count | 543 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2520/23/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on
[jira] [Work logged] (HADOOP-17290) ABFS: Add Identifiers to Client Request Header
[
https://issues.apache.org/jira/browse/HADOOP-17290?focusedWorklogId=618762&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618762
]
ASF GitHub Bot logged work on HADOOP-17290:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 19:28
Start Date: 05/Jul/21 19:28
Worklog Time Spent: 10m
Work Description: hadoop-yetus removed a comment on pull request #2520:
URL: https://github.com/apache/hadoop/pull/2520#issuecomment-872998551
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 37 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 40s | | trunk passed |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 13m 58s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 14m 17s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 27s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 3s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 72m 18s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2520/24/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2520 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint |
| uname | Linux f7e3def8158c 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0dc83f4367c6d674a7d8f0bfba0498c26c4551e1 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2520/24/testReport/ |
| Max. process+thread count | 615 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoo
[jira] [Work logged] (HADOOP-17290) ABFS: Add Identifiers to Client Request Header
[ https://issues.apache.org/jira/browse/HADOOP-17290?focusedWorklogId=618761&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618761 ] ASF GitHub Bot logged work on HADOOP-17290: --- Author: ASF GitHub Bot Created on: 05/Jul/21 19:28 Start Date: 05/Jul/21 19:28 Worklog Time Spent: 10m Work Description: hadoop-yetus removed a comment on pull request #2520: URL: https://github.com/apache/hadoop/pull/2520#issuecomment-870058533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618761) Time Spent: 8h 50m (was: 8h 40m) > ABFS: Add Identifiers to Client Request Header > -- > > Key: HADOOP-17290 > URL: https://issues.apache.org/jira/browse/HADOOP-17290 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sumangala Patki >Assignee: Sumangala Patki >Priority: Major > Labels: abfsactive, pull-request-available > Fix For: 3.4.0 > > Time Spent: 8h 50m > Remaining Estimate: 0h > > Adding unique values to the client request header to assist in correlating > requests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2520: HADOOP-17290. ABFS: Add Identifiers to Client Request Header
hadoop-yetus removed a comment on pull request #2520:
URL: https://github.com/apache/hadoop/pull/2520#issuecomment-872998551
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 58s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 2s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 37 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 30m 40s | | trunk passed |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 42s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 13m 58s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 14m 17s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 27s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 3s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 72m 18s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2520/24/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2520 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell markdownlint |
| uname | Linux f7e3def8158c 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 0dc83f4367c6d674a7d8f0bfba0498c26c4551e1 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2520/24/testReport/ |
| Max. process+thread count | 615 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2520/24/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log o
[GitHub] [hadoop] hadoop-yetus removed a comment on pull request #2520: HADOOP-17290. ABFS: Add Identifiers to Client Request Header
hadoop-yetus removed a comment on pull request #2520: URL: https://github.com/apache/hadoop/pull/2520#issuecomment-870058533 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17631) Configuration ${env.VAR:-FALLBACK} should eval FALLBACK when restrictSystemProps=true
[
https://issues.apache.org/jira/browse/HADOOP-17631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-17631:
Description:
When configuration reads in resources with a restricted parser, it skips
evaluaging system ${env. } vars. But it also skips evaluating fallbacks
As a result, a property like {{fs.s3a.buffer.dir}}
{code}
${env.LOCAL_DIRS:-${hadoop.tmp.dir}} ends up evaluating as
${env.LOCAL_DIRS:-${hadoop.tmp.dir}}
{code}
It should instead fall back to the "env var unset" option of ${hadoop.tmp.dir}.
This allows for configs (like for s3a buffer dirs) which are usable in
restricted mode as well as unrestricted deployments.
was:
When configuration reads in resources with a restricted parser, it skips
evaluaging system ${env. } vars. But it also skips evaluating fallbacks
As a result, a property like
${env.LOCAL_DIRS:-${hadoop.tmp.dir}} ends up evaluating as
${env.LOCAL_DIRS:-${hadoop.tmp.dir}}
It should instead fall back to the "env var unset" option of ${hadoop.tmp.dir}.
This allows for configs (like for s3a buffer dirs) which are usable in
restricted mode as well as unrestricted deployments.
> Configuration ${env.VAR:-FALLBACK} should eval FALLBACK when
> restrictSystemProps=true
> --
>
> Key: HADOOP-17631
> URL: https://issues.apache.org/jira/browse/HADOOP-17631
> Project: Hadoop Common
> Issue Type: Bug
> Components: common
>Affects Versions: 3.3.0
>Reporter: Steve Loughran
>Assignee: Steve Loughran
>Priority: Minor
> Labels: pull-request-available
> Fix For: 3.3.2
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> When configuration reads in resources with a restricted parser, it skips
> evaluaging system ${env. } vars. But it also skips evaluating fallbacks
> As a result, a property like {{fs.s3a.buffer.dir}}
> {code}
> ${env.LOCAL_DIRS:-${hadoop.tmp.dir}} ends up evaluating as
> ${env.LOCAL_DIRS:-${hadoop.tmp.dir}}
> {code}
> It should instead fall back to the "env var unset" option of
> ${hadoop.tmp.dir}. This allows for configs (like for s3a buffer dirs) which
> are usable in restricted mode as well as unrestricted deployments.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Work logged] (HADOOP-17682) ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters
[
https://issues.apache.org/jira/browse/HADOOP-17682?focusedWorklogId=618742&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618742
]
ASF GitHub Bot logged work on HADOOP-17682:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 17:48
Start Date: 05/Jul/21 17:48
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2975:
URL: https://github.com/apache/hadoop/pull/2975#issuecomment-874257716
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 1s | | trunk passed |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 12s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 14m 31s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 18s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2975/10/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 1 new + 2 unchanged - 0
fixed = 3 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 55s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 3s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 72m 26s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2975/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2975 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux f1b2078475f4 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 94f6884fb64f1d768c6ab075db22fa467e803b0a |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache
[GitHub] [hadoop] hadoop-yetus commented on pull request #2975: HADOOP-17682. ABFS: Support FileStatus input to OpenFileWithOptions() via OpenFileParameters
hadoop-yetus commented on pull request #2975:
URL: https://github.com/apache/hadoop/pull/2975#issuecomment-874257716
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 2s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 1s | | trunk passed |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 29s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 12s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 14m 31s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 31s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 18s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2975/10/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 1 new + 2 unchanged - 0
fixed = 3 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 55s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 3s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 72m 26s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2975/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2975 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux f1b2078475f4 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17
17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 94f6884fb64f1d768c6ab075db22fa467e803b0a |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2975/10/testReport/ |
| Max. process+thread count | 545 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2975/10/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
[jira] [Created] (HADOOP-17790) ABFS: Modifications to TracingContext
Sumangala Patki created HADOOP-17790: Summary: ABFS: Modifications to TracingContext Key: HADOOP-17790 URL: https://issues.apache.org/jira/browse/HADOOP-17790 Project: Hadoop Common Issue Type: Sub-task Components: fs/azure Affects Versions: 3.3.1 Reporter: Sumangala Patki TracingContext introduced in the PR [HADOOP-17290|https://github.com/apache/hadoop/pull/2520] tracks a request with identifiers as it passes through ABFS layers. Creating this Jira to track suggested modifications and improvements to the tracing structure. * rename Listener; remove it from TracingContext constructor, confine to setter * make fields (identifiers) immutable; eliminate retry count field => reduce instance cloning * introduce child classes of TracingContext to handle stream/continuation ops -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-17250) ABFS: Random read perf improvement
[ https://issues.apache.org/jira/browse/HADOOP-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur resolved HADOOP-17250. Resolution: Fixed > ABFS: Random read perf improvement > -- > > Key: HADOOP-17250 > URL: https://issues.apache.org/jira/browse/HADOOP-17250 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sneha Vijayarajan >Assignee: Mukund Thakur >Priority: Major > Labels: abfsactive, pull-request-available > Fix For: 3.3.2 > > Time Spent: 5h 10m > Remaining Estimate: 0h > > Random read if marginally read ahead was seen to improve perf for a TPCH > query. > > Introducing fs.azure.readahead.range parameter which can be set by user. > Data will be populated in buffer for random reads as well which leads to > lesser > remote calls. > This patch also changes the seek implementation to perform a lazy seek. Actual > seek is done when a read is initiated and data is not present in buffer else > date is returned from buffer thus reducing the number of remote calls. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17250) ABFS: Random read perf improvement
[ https://issues.apache.org/jira/browse/HADOOP-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur updated HADOOP-17250: --- Description: Random read if marginally read ahead was seen to improve perf for a TPCH query. Introducing fs.azure.readahead.range parameter which can be set by user. Data will be populated in buffer for random reads as well which leads to lesser remote calls. This patch also changes the seek implementation to perform a lazy seek. Actual seek is done when a read is initiated and data is not present in buffer else date is returned from buffer thus reducing the number of remote calls. was: Random read if marginally read ahead was seen to improve perf for a TPCH query. > ABFS: Random read perf improvement > -- > > Key: HADOOP-17250 > URL: https://issues.apache.org/jira/browse/HADOOP-17250 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sneha Vijayarajan >Assignee: Mukund Thakur >Priority: Major > Labels: abfsactive, pull-request-available > Fix For: 3.3.2 > > Time Spent: 5h 10m > Remaining Estimate: 0h > > Random read if marginally read ahead was seen to improve perf for a TPCH > query. > > Introducing fs.azure.readahead.range parameter which can be set by user. > Data will be populated in buffer for random reads as well which leads to > lesser > remote calls. > This patch also changes the seek implementation to perform a lazy seek. Actual > seek is done when a read is initiated and data is not present in buffer else > date is returned from buffer thus reducing the number of remote calls. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-17250) ABFS: Random read perf improvement
[ https://issues.apache.org/jira/browse/HADOOP-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur updated HADOOP-17250: --- Fix Version/s: 3.3.2 > ABFS: Random read perf improvement > -- > > Key: HADOOP-17250 > URL: https://issues.apache.org/jira/browse/HADOOP-17250 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sneha Vijayarajan >Assignee: Mukund Thakur >Priority: Major > Labels: abfsactive, pull-request-available > Fix For: 3.3.2 > > Time Spent: 5h 10m > Remaining Estimate: 0h > > Random read if marginally read ahead was seen to improve perf for a TPCH > query. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-17250) ABFS: Random read perf improvement
[ https://issues.apache.org/jira/browse/HADOOP-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur reassigned HADOOP-17250: -- Assignee: Mukund Thakur (was: Mukund Thakur) > ABFS: Random read perf improvement > -- > > Key: HADOOP-17250 > URL: https://issues.apache.org/jira/browse/HADOOP-17250 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sneha Vijayarajan >Assignee: Mukund Thakur >Priority: Major > Labels: abfsactive, pull-request-available > Time Spent: 5h 10m > Remaining Estimate: 0h > > Random read if marginally read ahead was seen to improve perf for a TPCH > query. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #2549: Hadoop 17428. ABFS: Implementation for getContentSummary
hadoop-yetus commented on pull request #2549:
URL: https://github.com/apache/hadoop/pull/2549#issuecomment-874062488
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 52s | | trunk passed |
| +1 :green_heart: | compile | 0m 39s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 36s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 39s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 14m 15s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 14m 35s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 31s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | compile | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 28s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 18s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 31s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 14m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 6s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 34s | | The patch does not
generate ASF License warnings. |
| | | 73m 15s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2549/31/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/2549 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 5e510ce929d1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / be0e94cb8d74789efc490dadbbdcaeac18e07bab |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2549/31/testReport/ |
| Max. process+thread count | 687 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2549/31/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-un
[jira] [Work logged] (HADOOP-17765) ABFS: Use Unique File Paths in Tests
[
https://issues.apache.org/jira/browse/HADOOP-17765?focusedWorklogId=618606&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618606
]
ASF GitHub Bot logged work on HADOOP-17765:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 11:04
Start Date: 05/Jul/21 11:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3153:
URL: https://github.com/apache/hadoop/pull/3153#issuecomment-874022678
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 25 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 7s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 3s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 17m 20s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 27s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 51s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 78m 57s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3153/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3153 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 2816e4e33139 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13
01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b9977b30700db14e420c44d14deafda4530ce614 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3153/3/testReport/ |
| Max. process+thread count | 625 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-mu
[GitHub] [hadoop] hadoop-yetus commented on pull request #3153: HADOOP-17765. ABFS: Use Unique File Paths in Tests
hadoop-yetus commented on pull request #3153:
URL: https://github.com/apache/hadoop/pull/3153#issuecomment-874022678
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 52s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 25 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 7s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 32s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 25s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 26s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 59s | | trunk passed |
| +1 :green_heart: | shadedclient | 17m 3s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 17m 20s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 29s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 17s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 22s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 16m 27s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 51s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 29s | | The patch does not
generate ASF License warnings. |
| | | 78m 57s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3153/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3153 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 2816e4e33139 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13
01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / b9977b30700db14e420c44d14deafda4530ce614 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3153/3/testReport/ |
| Max. process+thread count | 625 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3153/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-u
[jira] [Work logged] (HADOOP-17250) ABFS: Random read perf improvement
[
https://issues.apache.org/jira/browse/HADOOP-17250?focusedWorklogId=618601&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618601
]
ASF GitHub Bot logged work on HADOOP-17250:
---
Author: ASF GitHub Bot
Created on: 05/Jul/21 10:45
Start Date: 05/Jul/21 10:45
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3110:
URL: https://github.com/apache/hadoop/pull/3110#issuecomment-874011511
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 31s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 13m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3110/6/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 1 new + 2 unchanged - 0
fixed = 3 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 53s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 71m 57s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3110/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 3e167b4cf1d2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / cc79131214eb6eebbd3580bb17398ff9bd1d0049 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3110/6/testReport/ |
| Max. process+thread count | 545 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
[GitHub] [hadoop] hadoop-yetus commented on pull request #3110: HADOOP-17250 Lot of short reads can be merged with readahead.
hadoop-yetus commented on pull request #3110:
URL: https://github.com/apache/hadoop/pull/3110#issuecomment-874011511
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 4 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 31m 31s | | trunk passed |
| +1 :green_heart: | compile | 0m 38s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 2s | | trunk passed |
| +1 :green_heart: | shadedclient | 13m 57s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 30s | | the patch passed |
| +1 :green_heart: | compile | 0m 26s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 26s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 19s |
[/results-checkstyle-hadoop-tools_hadoop-azure.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3110/6/artifact/out/results-checkstyle-hadoop-tools_hadoop-azure.txt)
| hadoop-tools/hadoop-azure: The patch generated 1 new + 2 unchanged - 0
fixed = 3 total (was 2) |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 1m 1s | | the patch passed |
| +1 :green_heart: | shadedclient | 13m 53s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 1m 56s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 33s | | The patch does not
generate ASF License warnings. |
| | | 71m 57s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3110/6/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3110 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 3e167b4cf1d2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / cc79131214eb6eebbd3580bb17398ff9bd1d0049 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3110/6/testReport/ |
| Max. process+thread count | 545 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3110/6/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to g
[jira] [Work logged] (HADOOP-17250) ABFS: Random read perf improvement
[ https://issues.apache.org/jira/browse/HADOOP-17250?focusedWorklogId=618600&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-618600 ] ASF GitHub Bot logged work on HADOOP-17250: --- Author: ASF GitHub Bot Created on: 05/Jul/21 10:19 Start Date: 05/Jul/21 10:19 Worklog Time Spent: 10m Work Description: mukund-thakur merged pull request #3110: URL: https://github.com/apache/hadoop/pull/3110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 618600) Time Spent: 5h (was: 4h 50m) > ABFS: Random read perf improvement > -- > > Key: HADOOP-17250 > URL: https://issues.apache.org/jira/browse/HADOOP-17250 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure >Affects Versions: 3.3.0 >Reporter: Sneha Vijayarajan >Assignee: Mukund Thakur >Priority: Major > Labels: abfsactive, pull-request-available > Time Spent: 5h > Remaining Estimate: 0h > > Random read if marginally read ahead was seen to improve perf for a TPCH > query. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
