[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
slfan1989 commented on code in PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#discussion_r949817992
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestRouterWebServiceUtil.java:
##
@@ -592,4 +595,90 @@ public static AppAttemptInfo generateAppAttemptInfo(int
attemptId) {
when(appAttemptInfo.getLogsLink()).thenReturn("LogLink_" + attemptId);
return appAttemptInfo;
}
+
+ @Test
+ public void testMergeApplicationStatisticsInfo() {
+ApplicationStatisticsInfo infoA = new ApplicationStatisticsInfo();
+ApplicationStatisticsInfo infoB = new ApplicationStatisticsInfo();
+
+StatisticsItemInfo item1 =
+new StatisticsItemInfo(YarnApplicationState.ACCEPTED, "*", 10);
+StatisticsItemInfo item2 =
+new StatisticsItemInfo(YarnApplicationState.ACCEPTED, "*", 20);
+
+infoA.add(item1);
+infoB.add(item2);
+
+List lists = new ArrayList<>();
+lists.add(infoA);
+lists.add(infoB);
+
+ApplicationStatisticsInfo mergeInfo =
+RouterWebServiceUtil.mergeApplicationStatisticsInfo(lists);
+
+Assert.assertEquals(1, mergeInfo.getStatItems().size());
+Assert.assertEquals(item1.getCount() + item2.getCount(),
+mergeInfo.getStatItems().get(0).getCount());
Review Comment:
I will modify the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
slfan1989 commented on code in PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#discussion_r949814907
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/MockDefaultRequestInterceptorREST.java:
##
@@ -661,4 +683,108 @@ public Response updateAppQueue(AppQueue targetQueue,
HttpServletRequest hsr, Str
AppQueue targetAppQueue = new AppQueue(targetQueue.getQueue());
return Response.status(Status.OK).entity(targetAppQueue).build();
}
+
+ public void updateApplicationState(YarnApplicationState appState, String
appId)
+ throws AuthorizationException, YarnException, InterruptedException,
IOException {
+validateRunning();
+ApplicationId applicationId = ApplicationId.fromString(appId);
+if (!applicationMap.containsKey(applicationId)) {
+ throw new NotFoundException("app with id: " + appId + " not found");
+}
+ApplicationReport appReport = applicationMap.get(applicationId);
+appReport.setYarnApplicationState(appState);
+ }
+
+ @Override
+ public ApplicationStatisticsInfo getAppStatistics(
+ HttpServletRequest hsr, Set stateQueries, Set
typeQueries) {
+if (!isRunning) {
+ throw new RuntimeException("RM is stopped");
+}
+
+Map itemInfoMap = new HashMap<>();
+
+for (HashMap.Entry item :
applicationMap.entrySet()) {
+
+ ApplicationReport applicationReport = item.getValue();
+ YarnApplicationState appState =
applicationReport.getYarnApplicationState();
+ String appType = applicationReport.getApplicationType();
+
+ if (stateQueries.contains(appState.name()) &&
typeQueries.contains(appType)) {
+String itemInfoMapKey = appState.toString() + "_" + appType;
+StatisticsItemInfo itemInfo = itemInfoMap.getOrDefault(itemInfoMapKey,
null);
+if (itemInfo == null) {
+ itemInfo = new StatisticsItemInfo(appState, appType, 1);
+} else {
+ long newCount = itemInfo.getCount() + 1;
+ itemInfo.setCount(newCount);
+}
+itemInfoMap.put(itemInfoMapKey, itemInfo);
+ }
+}
+
+ArrayList itemInfos = new
ArrayList<>(itemInfoMap.values());
+
+return new ApplicationStatisticsInfo(itemInfos);
Review Comment:
Thank you for your suggestion, I will modify the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
slfan1989 commented on code in PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#discussion_r949814907
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/MockDefaultRequestInterceptorREST.java:
##
@@ -661,4 +683,108 @@ public Response updateAppQueue(AppQueue targetQueue,
HttpServletRequest hsr, Str
AppQueue targetAppQueue = new AppQueue(targetQueue.getQueue());
return Response.status(Status.OK).entity(targetAppQueue).build();
}
+
+ public void updateApplicationState(YarnApplicationState appState, String
appId)
+ throws AuthorizationException, YarnException, InterruptedException,
IOException {
+validateRunning();
+ApplicationId applicationId = ApplicationId.fromString(appId);
+if (!applicationMap.containsKey(applicationId)) {
+ throw new NotFoundException("app with id: " + appId + " not found");
+}
+ApplicationReport appReport = applicationMap.get(applicationId);
+appReport.setYarnApplicationState(appState);
+ }
+
+ @Override
+ public ApplicationStatisticsInfo getAppStatistics(
+ HttpServletRequest hsr, Set stateQueries, Set
typeQueries) {
+if (!isRunning) {
+ throw new RuntimeException("RM is stopped");
+}
+
+Map itemInfoMap = new HashMap<>();
+
+for (HashMap.Entry item :
applicationMap.entrySet()) {
+
+ ApplicationReport applicationReport = item.getValue();
+ YarnApplicationState appState =
applicationReport.getYarnApplicationState();
+ String appType = applicationReport.getApplicationType();
+
+ if (stateQueries.contains(appState.name()) &&
typeQueries.contains(appType)) {
+String itemInfoMapKey = appState.toString() + "_" + appType;
+StatisticsItemInfo itemInfo = itemInfoMap.getOrDefault(itemInfoMapKey,
null);
+if (itemInfo == null) {
+ itemInfo = new StatisticsItemInfo(appState, appType, 1);
+} else {
+ long newCount = itemInfo.getCount() + 1;
+ itemInfo.setCount(newCount);
+}
+itemInfoMap.put(itemInfoMapKey, itemInfo);
+ }
+}
+
+ArrayList itemInfos = new
ArrayList<>(itemInfoMap.values());
+
+return new ApplicationStatisticsInfo(itemInfos);
Review Comment:
Thank you for your suggestion!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
slfan1989 commented on code in PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#discussion_r949815067
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/MockDefaultRequestInterceptorREST.java:
##
@@ -661,4 +683,108 @@ public Response updateAppQueue(AppQueue targetQueue,
HttpServletRequest hsr, Str
AppQueue targetAppQueue = new AppQueue(targetQueue.getQueue());
return Response.status(Status.OK).entity(targetAppQueue).build();
}
+
+ public void updateApplicationState(YarnApplicationState appState, String
appId)
+ throws AuthorizationException, YarnException, InterruptedException,
IOException {
+validateRunning();
+ApplicationId applicationId = ApplicationId.fromString(appId);
+if (!applicationMap.containsKey(applicationId)) {
+ throw new NotFoundException("app with id: " + appId + " not found");
+}
+ApplicationReport appReport = applicationMap.get(applicationId);
+appReport.setYarnApplicationState(appState);
+ }
+
+ @Override
+ public ApplicationStatisticsInfo getAppStatistics(
+ HttpServletRequest hsr, Set stateQueries, Set
typeQueries) {
+if (!isRunning) {
+ throw new RuntimeException("RM is stopped");
+}
+
+Map itemInfoMap = new HashMap<>();
+
+for (HashMap.Entry item :
applicationMap.entrySet()) {
+
+ ApplicationReport applicationReport = item.getValue();
+ YarnApplicationState appState =
applicationReport.getYarnApplicationState();
+ String appType = applicationReport.getApplicationType();
+
+ if (stateQueries.contains(appState.name()) &&
typeQueries.contains(appType)) {
+String itemInfoMapKey = appState.toString() + "_" + appType;
+StatisticsItemInfo itemInfo = itemInfoMap.getOrDefault(itemInfoMapKey,
null);
+if (itemInfo == null) {
+ itemInfo = new StatisticsItemInfo(appState, appType, 1);
+} else {
+ long newCount = itemInfo.getCount() + 1;
+ itemInfo.setCount(newCount);
+}
+itemInfoMap.put(itemInfoMapKey, itemInfo);
+ }
+}
+
+ArrayList itemInfos = new
ArrayList<>(itemInfoMap.values());
+
+return new ApplicationStatisticsInfo(itemInfos);
Review Comment:
I will fix it.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/MockDefaultRequestInterceptorREST.java:
##
@@ -661,4 +683,108 @@ public Response updateAppQueue(AppQueue targetQueue,
HttpServletRequest hsr, Str
AppQueue targetAppQueue = new AppQueue(targetQueue.getQueue());
return Response.status(Status.OK).entity(targetAppQueue).build();
}
+
+ public void updateApplicationState(YarnApplicationState appState, String
appId)
+ throws AuthorizationException, YarnException, InterruptedException,
IOException {
+validateRunning();
+ApplicationId applicationId = ApplicationId.fromString(appId);
+if (!applicationMap.containsKey(applicationId)) {
+ throw new NotFoundException("app with id: " + appId + " not found");
+}
+ApplicationReport appReport = applicationMap.get(applicationId);
+appReport.setYarnApplicationState(appState);
+ }
+
+ @Override
+ public ApplicationStatisticsInfo getAppStatistics(
+ HttpServletRequest hsr, Set stateQueries, Set
typeQueries) {
+if (!isRunning) {
+ throw new RuntimeException("RM is stopped");
+}
+
+Map itemInfoMap = new HashMap<>();
+
+for (HashMap.Entry item :
applicationMap.entrySet()) {
+
+ ApplicationReport applicationReport = item.getValue();
+ YarnApplicationState appState =
applicationReport.getYarnApplicationState();
+ String appType = applicationReport.getApplicationType();
+
+ if (stateQueries.contains(appState.name()) &&
typeQueries.contains(appType)) {
+String itemInfoMapKey = appState.toString() + "_" + appType;
+StatisticsItemInfo itemInfo = itemInfoMap.getOrDefault(itemInfoMapKey,
null);
+if (itemInfo == null) {
+ itemInfo = new StatisticsItemInfo(appState, appType, 1);
+} else {
+ long newCount = itemInfo.getCount() + 1;
+ itemInfo.setCount(newCount);
+}
+itemInfoMap.put(itemInfoMapKey, itemInfo);
+ }
+}
+
+ArrayList itemInfos = new
ArrayList<>(itemInfoMap.values());
+
+return new ApplicationStatisticsInfo(itemInfos);
+ }
+
+ @Override
+ public AppActivitiesInfo getAppActivities(
+ HttpServletRequest hsr, String appId, String time, Set
requestPriorities,
+ Set allocationRequestIds, String groupBy, String limit,
Set actions,
+ boolean summarize) {
+if (!isRunning) {
+ throw new RuntimeException("RM is stopped");
+}
+
+ApplicationId applicationId = ApplicationId.fromString(appId);
+if (!applicationMap.containsKey(applicationId)) {
+ throw new
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
slfan1989 commented on code in PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#discussion_r949814739
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/MockDefaultRequestInterceptorREST.java:
##
@@ -661,4 +683,108 @@ public Response updateAppQueue(AppQueue targetQueue,
HttpServletRequest hsr, Str
AppQueue targetAppQueue = new AppQueue(targetQueue.getQueue());
return Response.status(Status.OK).entity(targetAppQueue).build();
}
+
+ public void updateApplicationState(YarnApplicationState appState, String
appId)
+ throws AuthorizationException, YarnException, InterruptedException,
IOException {
+validateRunning();
+ApplicationId applicationId = ApplicationId.fromString(appId);
+if (!applicationMap.containsKey(applicationId)) {
+ throw new NotFoundException("app with id: " + appId + " not found");
+}
+ApplicationReport appReport = applicationMap.get(applicationId);
+appReport.setYarnApplicationState(appState);
+ }
+
+ @Override
+ public ApplicationStatisticsInfo getAppStatistics(
+ HttpServletRequest hsr, Set stateQueries, Set
typeQueries) {
+if (!isRunning) {
+ throw new RuntimeException("RM is stopped");
+}
+
+Map itemInfoMap = new HashMap<>();
+
+for (HashMap.Entry item :
applicationMap.entrySet()) {
+
+ ApplicationReport applicationReport = item.getValue();
Review Comment:
I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
slfan1989 commented on code in PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#discussion_r949814084
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/RouterWebServiceUtil.java:
##
@@ -540,4 +542,34 @@ public static NodeToLabelsInfo mergeNodeToLabels(
return new NodeToLabelsInfo(nodeToLabels);
}
+
+ public static ApplicationStatisticsInfo mergeApplicationStatisticsInfo(
+ Collection appStatistics) {
+ApplicationStatisticsInfo result = new ApplicationStatisticsInfo();
+HashMap statisticsItemMap = new HashMap();
+
+appStatistics.stream().forEach(appStatistic -> {
+List statisticsItemInfos =
appStatistic.getStatItems();
+ for (StatisticsItemInfo statisticsItemInfo : statisticsItemInfos) {
+String statisticsItemKey = statisticsItemInfo.getType() + "_" +
statisticsItemInfo.getState().toString();
+StatisticsItemInfo statisticsItemValue =
+statisticsItemMap.getOrDefault(statisticsItemKey, null);
+if (statisticsItemValue != null) {
Review Comment:
I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
slfan1989 commented on code in PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#discussion_r949813097
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationInterceptorREST.java:
##
@@ -1129,13 +1129,49 @@ public AppActivitiesInfo
getAppActivities(HttpServletRequest hsr,
String appId, String time, Set requestPriorities,
Set allocationRequestIds, String groupBy, String limit,
Set actions, boolean summarize) {
-throw new NotImplementedException("Code is not implemented");
+
+// Only verify the app_id, because the specific subCluster needs to be
found according to the app_id,
+// and other verifications are directly handed over to the corresponding
subCluster RM
+if (appId == null || appId.isEmpty()) {
+ throw new IllegalArgumentException("Parameter error, the appId is empty
or null.");
+}
+
+try {
+ SubClusterInfo subClusterInfo = getHomeSubClusterInfoByAppId(appId);
+ DefaultRequestInterceptorREST interceptor =
getOrCreateInterceptorForSubCluster(
+ subClusterInfo.getSubClusterId(),
subClusterInfo.getRMWebServiceAddress());
+
+ final HttpServletRequest hsrCopy = clone(hsr);
+ return interceptor.getAppActivities(hsrCopy, appId, time,
requestPriorities,
+ allocationRequestIds, groupBy, limit, actions, summarize);
+} catch (IllegalArgumentException e) {
+ RouterServerUtil.logAndThrowRunTimeException(e,
Review Comment:
I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
slfan1989 commented on code in PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#discussion_r949812767
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ApplicationStatisticsInfo.java:
##
@@ -33,12 +34,15 @@ public class ApplicationStatisticsInfo {
public ApplicationStatisticsInfo() {
} // JAXB needs this
+ public ApplicationStatisticsInfo(Collection items) {
+statItem.addAll(items);
+ }
+
public void add(StatisticsItemInfo statItem) {
this.statItem.add(statItem);
}
public ArrayList getStatItems() {
return statItem;
}
-
Review Comment:
Thanks for your help reviewing the code, I will modify the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
hadoop-yetus commented on PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#issuecomment-1220246203
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 57s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 23s | | trunk passed |
| +1 :green_heart: | compile | 4m 4s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 28s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 31s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 59s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 47s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 43s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 31s | | the patch passed |
| +1 :green_heart: | compile | 3m 46s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 3m 46s | | the patch passed |
| +1 :green_heart: | compile | 3m 15s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 3m 15s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 24s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 19s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 6s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 35s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 98m 32s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 3m 41s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 225m 14s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4757/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4757 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 3ce0f9113e88 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / ac69043d91045fe2c701e668a453263c98a3c3c3 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4757/2/testReport/ |
| Max. process+thread count | 1744 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router
U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4757/2/console |
|
[GitHub] [hadoop] jianghuazhu opened a new pull request, #4761: HDFS-16733.Improve INode#isRoot().
jianghuazhu opened a new pull request, #4761: URL: https://github.com/apache/hadoop/pull/4761 ### Description of PR When constructing an INodeFile or INodeDirectory, if the given name is null, an exception will be thrown when checking whether it is root. Details: HDFS-16733 ### How was this patch tested? When constructing an INodeFile or INodeDirectory, if the given name is null, an exception will not occur when checking whether it is root. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] goiri merged pull request #4741: YARN-11252. Yarn Federation Router Supports Update / Delete Reservation in MemoryStore.
goiri merged PR #4741: URL: https://github.com/apache/hadoop/pull/4741 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18302) Remove WhiteBox in hadoop-common module.
[
https://issues.apache.org/jira/browse/HADOOP-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581634#comment-17581634
]
ASF GitHub Bot commented on HADOOP-18302:
-
hadoop-yetus commented on PR #4457:
URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1220224493
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 11 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 39s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 40s | | trunk passed |
| +1 :green_heart: | compile | 23m 16s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 58s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 56s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 26s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 37s | | the patch passed |
| +1 :green_heart: | compile | 22m 29s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 22m 29s | |
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 2833 unchanged - 17
fixed = 2833 total (was 2850) |
| +1 :green_heart: | compile | 20m 44s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 44s | |
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2629 unchanged -
17 fixed = 2629 total (was 2646) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 49s |
[/results-checkstyle-hadoop-common-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/23/artifact/out/results-checkstyle-hadoop-common-project.txt)
| hadoop-common-project: The patch generated 2 new + 378 unchanged - 233
fixed = 380 total (was 611) |
| +1 :green_heart: | mvnsite | 3m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 52s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 46s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 1m 33s | | hadoop-nfs in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 31s | | The patch does not
generate ASF License warnings. |
| | | 233m 32s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/23/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4457 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 7f2a96fcfa54 4.15.0-191-generic #202-Ubuntu
[GitHub] [hadoop] hadoop-yetus commented on pull request #4457: HADOOP-18302. Remove WhiteBox in hadoop-common module.
hadoop-yetus commented on PR #4457:
URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1220224493
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 11 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 39s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 40s | | trunk passed |
| +1 :green_heart: | compile | 23m 16s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 56s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 58s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 37s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 56s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 26s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 33s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 37s | | the patch passed |
| +1 :green_heart: | compile | 22m 29s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 22m 29s | |
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 2833 unchanged - 17
fixed = 2833 total (was 2850) |
| +1 :green_heart: | compile | 20m 44s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 44s | |
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2629 unchanged -
17 fixed = 2629 total (was 2646) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 49s |
[/results-checkstyle-hadoop-common-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/23/artifact/out/results-checkstyle-hadoop-common-project.txt)
| hadoop-common-project: The patch generated 2 new + 378 unchanged - 233
fixed = 380 total (was 611) |
| +1 :green_heart: | mvnsite | 3m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 52s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 46s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 1m 33s | | hadoop-nfs in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 31s | | The patch does not
generate ASF License warnings. |
| | | 233m 32s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/23/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4457 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 7f2a96fcfa54 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / d835bfe92237ecf5b375d99e040b512dca7f9dd6 |
| Default Java | Private
[GitHub] [hadoop] hadoop-yetus commented on pull request #4127: HDFS-13522. RBF: Support observer node from Router-Based Federation
hadoop-yetus commented on PR #4127:
URL: https://github.com/apache/hadoop/pull/4127#issuecomment-1220183368
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 57s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | buf | 0m 1s | | buf was not available. |
| +0 :ok: | xmllint | 0m 1s | | xmllint was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 16 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 30s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 18s | | trunk passed |
| +1 :green_heart: | compile | 23m 19s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 7m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 27s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 6m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 12m 21s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 33s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 12s | | the patch passed |
| +1 :green_heart: | compile | 22m 25s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 22m 25s | | the patch passed |
| +1 :green_heart: | javac | 22m 25s | | the patch passed |
| +1 :green_heart: | compile | 20m 46s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 20m 46s | | the patch passed |
| +1 :green_heart: | javac | 20m 46s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 18s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/30/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 4 new + 539 unchanged - 2 fixed = 543 total (was
541) |
| +1 :green_heart: | mvnsite | 7m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 6m 21s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 6m 49s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 13m 3s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 54s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 46s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 17s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 380m 43s | | hadoop-hdfs in the patch
passed. |
| -1 :x: | unit | 2m 5s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/30/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch failed. |
| +1 :green_heart: | asflicense | 1m 57s | | The patch does not
generate ASF License warnings. |
| | | 667m 23s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/30/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4127 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets cc buflint
bufcompat xmllint |
| uname | Linux 4f8d04985f75 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
|
[jira] [Commented] (HADOOP-18407) Improve vectored IO api spec.
[
https://issues.apache.org/jira/browse/HADOOP-18407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581587#comment-17581587
]
ASF GitHub Bot commented on HADOOP-18407:
-
hadoop-yetus commented on PR #4760:
URL: https://github.com/apache/hadoop/pull/4760#issuecomment-1220128378
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 0s | | trunk passed |
| +1 :green_heart: | compile | 24m 51s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 21m 0s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 44s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 39s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 14s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 23m 48s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 23m 48s | | the patch passed |
| +1 :green_heart: | compile | 21m 40s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 40s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 12s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 15s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 3s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 34s | | The patch does not
generate ASF License warnings. |
| | | 224m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4760/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4760 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 16edb2be83be 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 82d080042f2ce0379a05a27e7293eab1bb1a9c5b |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4760/1/testReport/ |
| Max. process+thread count | 1251 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
[GitHub] [hadoop] hadoop-yetus commented on pull request #4760: HADOOP-18407. Improve readVectored() api spec
hadoop-yetus commented on PR #4760:
URL: https://github.com/apache/hadoop/pull/4760#issuecomment-1220128378
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 0s | | trunk passed |
| +1 :green_heart: | compile | 24m 51s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 21m 0s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 44s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 39s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 14s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 11s | | the patch passed |
| +1 :green_heart: | compile | 23m 48s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 23m 48s | | the patch passed |
| +1 :green_heart: | compile | 21m 40s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 40s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 12s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 39s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 15s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 14s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 3s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 34s | | The patch does not
generate ASF License warnings. |
| | | 224m 42s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4760/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4760 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 16edb2be83be 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 82d080042f2ce0379a05a27e7293eab1bb1a9c5b |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4760/1/testReport/ |
| Max. process+thread count | 1251 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common U:
hadoop-common-project/hadoop-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4760/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an
[GitHub] [hadoop] goiri commented on a diff in pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
goiri commented on code in PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#discussion_r949711369
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/dao/ApplicationStatisticsInfo.java:
##
@@ -33,12 +34,15 @@ public class ApplicationStatisticsInfo {
public ApplicationStatisticsInfo() {
} // JAXB needs this
+ public ApplicationStatisticsInfo(Collection items) {
+statItem.addAll(items);
+ }
+
public void add(StatisticsItemInfo statItem) {
this.statItem.add(statItem);
}
public ArrayList getStatItems() {
return statItem;
}
-
Review Comment:
Avoid
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/RouterWebServiceUtil.java:
##
@@ -540,4 +542,34 @@ public static NodeToLabelsInfo mergeNodeToLabels(
return new NodeToLabelsInfo(nodeToLabels);
}
+
+ public static ApplicationStatisticsInfo mergeApplicationStatisticsInfo(
+ Collection appStatistics) {
+ApplicationStatisticsInfo result = new ApplicationStatisticsInfo();
+HashMap statisticsItemMap = new HashMap();
+
+appStatistics.stream().forEach(appStatistic -> {
+List statisticsItemInfos =
appStatistic.getStatItems();
+ for (StatisticsItemInfo statisticsItemInfo : statisticsItemInfos) {
+String statisticsItemKey = statisticsItemInfo.getType() + "_" +
statisticsItemInfo.getState().toString();
+StatisticsItemInfo statisticsItemValue =
+statisticsItemMap.getOrDefault(statisticsItemKey, null);
+if (statisticsItemValue != null) {
Review Comment:
Cleaner to do contains or similar.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/TestRouterWebServiceUtil.java:
##
@@ -592,4 +595,90 @@ public static AppAttemptInfo generateAppAttemptInfo(int
attemptId) {
when(appAttemptInfo.getLogsLink()).thenReturn("LogLink_" + attemptId);
return appAttemptInfo;
}
+
+ @Test
+ public void testMergeApplicationStatisticsInfo() {
+ApplicationStatisticsInfo infoA = new ApplicationStatisticsInfo();
+ApplicationStatisticsInfo infoB = new ApplicationStatisticsInfo();
+
+StatisticsItemInfo item1 =
+new StatisticsItemInfo(YarnApplicationState.ACCEPTED, "*", 10);
+StatisticsItemInfo item2 =
+new StatisticsItemInfo(YarnApplicationState.ACCEPTED, "*", 20);
+
+infoA.add(item1);
+infoB.add(item2);
+
+List lists = new ArrayList<>();
+lists.add(infoA);
+lists.add(infoB);
+
+ApplicationStatisticsInfo mergeInfo =
+RouterWebServiceUtil.mergeApplicationStatisticsInfo(lists);
+
+Assert.assertEquals(1, mergeInfo.getStatItems().size());
+Assert.assertEquals(item1.getCount() + item2.getCount(),
+mergeInfo.getStatItems().get(0).getCount());
Review Comment:
extract get(0) and check it first.
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/test/java/org/apache/hadoop/yarn/server/router/webapp/MockDefaultRequestInterceptorREST.java:
##
@@ -661,4 +683,108 @@ public Response updateAppQueue(AppQueue targetQueue,
HttpServletRequest hsr, Str
AppQueue targetAppQueue = new AppQueue(targetQueue.getQueue());
return Response.status(Status.OK).entity(targetAppQueue).build();
}
+
+ public void updateApplicationState(YarnApplicationState appState, String
appId)
+ throws AuthorizationException, YarnException, InterruptedException,
IOException {
+validateRunning();
+ApplicationId applicationId = ApplicationId.fromString(appId);
+if (!applicationMap.containsKey(applicationId)) {
+ throw new NotFoundException("app with id: " + appId + " not found");
+}
+ApplicationReport appReport = applicationMap.get(applicationId);
+appReport.setYarnApplicationState(appState);
+ }
+
+ @Override
+ public ApplicationStatisticsInfo getAppStatistics(
+ HttpServletRequest hsr, Set stateQueries, Set
typeQueries) {
+if (!isRunning) {
+ throw new RuntimeException("RM is stopped");
+}
+
+Map itemInfoMap = new HashMap<>();
+
+for (HashMap.Entry item :
applicationMap.entrySet()) {
+
+ ApplicationReport applicationReport = item.getValue();
Review Comment:
If we don't do getKey, we can iterate .values()
##
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-router/src/main/java/org/apache/hadoop/yarn/server/router/webapp/FederationInterceptorREST.java:
##
@@ -1129,13 +1129,49 @@ public AppActivitiesInfo
getAppActivities(HttpServletRequest hsr,
String appId, String time, Set requestPriorities,
Set allocationRequestIds, String groupBy, String limit,
Set actions, boolean summarize) {
-throw new
[GitHub] [hadoop] hadoop-yetus commented on pull request #4311: HDFS-13522: IPC changes to support observer reads through routers.
hadoop-yetus commented on PR #4311:
URL: https://github.com/apache/hadoop/pull/4311#issuecomment-1220105115
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 37s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 27s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 22s | | trunk passed |
| +1 :green_heart: | compile | 23m 9s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 42s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 26s | | trunk passed |
| +1 :green_heart: | mvnsite | 7m 40s | | trunk passed |
| +1 :green_heart: | javadoc | 6m 22s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 6m 42s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 12m 19s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 34s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 9s | | the patch passed |
| +1 :green_heart: | compile | 22m 23s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 22m 23s | | the patch passed |
| +1 :green_heart: | javac | 22m 23s | | the patch passed |
| +1 :green_heart: | compile | 20m 40s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 20m 40s | | the patch passed |
| -1 :x: | javac | 20m 40s |
[/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4311/23/artifact/out/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
| root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new + 2643
unchanged - 1 fixed = 2644 total (was 2644) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 4m 16s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4311/23/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 4 new + 396 unchanged - 2 fixed = 400 total (was
398) |
| +1 :green_heart: | mvnsite | 7m 37s | | the patch passed |
| +1 :green_heart: | javadoc | 6m 17s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 6m 47s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 12m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 44s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 16s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 240m 37s | | hadoop-hdfs in the patch
passed. |
| -1 :x: | unit | 2m 6s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4311/23/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch failed. |
| +1 :green_heart: | asflicense | 1m 55s | | The patch does not
generate ASF License warnings. |
| | | 525m 38s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4311/23/artifact/out/Dockerfile
|
| GITHUB PR |
[jira] [Commented] (HADOOP-18302) Remove WhiteBox in hadoop-common module.
[ https://issues.apache.org/jira/browse/HADOOP-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581575#comment-17581575 ] ASF GitHub Bot commented on HADOOP-18302: - slfan1989 commented on code in PR #4457: URL: https://github.com/apache/hadoop/pull/4457#discussion_r949683088 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/impl/TestStatsDMetrics.java: ## @@ -31,13 +31,13 @@ import java.util.List; import java.util.Set; +import org.apache.commons.lang3.reflect.FieldUtils; Review Comment: `org.apache.commons.lang3.reflect.FieldUtils` has been removed, and TestStatsDMetrics.java has also been moved to the package org.apache.hadoop.metrics2.sink. > Remove WhiteBox in hadoop-common module. > > > Key: HADOOP-18302 > URL: https://issues.apache.org/jira/browse/HADOOP-18302 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.4.0, 3.3.9 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > > WhiteBox is deprecated, try to remove this method in hadoop-common. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18302) Remove WhiteBox in hadoop-common module.
[ https://issues.apache.org/jira/browse/HADOOP-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581576#comment-17581576 ] ASF GitHub Bot commented on HADOOP-18302: - hadoop-yetus commented on PR #4457: URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1220080729 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 0s | | Docker mode activated. | | -1 :x: | patch | 0m 20s | | https://github.com/apache/hadoop/pull/4457 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/4457 | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/22/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org | This message was automatically generated. > Remove WhiteBox in hadoop-common module. > > > Key: HADOOP-18302 > URL: https://issues.apache.org/jira/browse/HADOOP-18302 > Project: Hadoop Common > Issue Type: Sub-task >Affects Versions: 3.4.0, 3.3.9 >Reporter: fanshilun >Assignee: fanshilun >Priority: Minor > Labels: pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > > WhiteBox is deprecated, try to remove this method in hadoop-common. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4457: HADOOP-18302. Remove WhiteBox in hadoop-common module.
hadoop-yetus commented on PR #4457: URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1220080729 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Logfile | Comment | |::|--:|:|::|:---:| | +0 :ok: | reexec | 0m 0s | | Docker mode activated. | | -1 :x: | patch | 0m 20s | | https://github.com/apache/hadoop/pull/4457 does not apply to trunk. Rebase required? Wrong Branch? See https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute for help. | | Subsystem | Report/Notes | |--:|:-| | GITHUB PR | https://github.com/apache/hadoop/pull/4457 | | Console output | https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/22/console | | versions | git=2.17.1 | | Powered by | Apache Yetus 0.14.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4457: HADOOP-18302. Remove WhiteBox in hadoop-common module.
slfan1989 commented on code in PR #4457: URL: https://github.com/apache/hadoop/pull/4457#discussion_r949683088 ## hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/metrics2/impl/TestStatsDMetrics.java: ## @@ -31,13 +31,13 @@ import java.util.List; import java.util.Set; +import org.apache.commons.lang3.reflect.FieldUtils; Review Comment: `org.apache.commons.lang3.reflect.FieldUtils` has been removed, and TestStatsDMetrics.java has also been moved to the package org.apache.hadoop.metrics2.sink. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18302) Remove WhiteBox in hadoop-common module.
[
https://issues.apache.org/jira/browse/HADOOP-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581572#comment-17581572
]
ASF GitHub Bot commented on HADOOP-18302:
-
slfan1989 commented on code in PR #4457:
URL: https://github.com/apache/hadoop/pull/4457#discussion_r949678639
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/GraphiteSink.java:
##
@@ -37,171 +37,176 @@
import java.nio.charset.StandardCharsets;
/**
- * A metrics sink that writes to a Graphite server
+ * A metrics sink that writes to a Graphite server.
*/
@InterfaceAudience.Public
@InterfaceStability.Evolving
public class GraphiteSink implements MetricsSink, Closeable {
-private static final Logger LOG =
-LoggerFactory.getLogger(GraphiteSink.class);
-private static final String SERVER_HOST_KEY = "server_host";
-private static final String SERVER_PORT_KEY = "server_port";
-private static final String METRICS_PREFIX = "metrics_prefix";
-private String metricsPrefix = null;
-private Graphite graphite = null;
-
-@Override
-public void init(SubsetConfiguration conf) {
-// Get Graphite host configurations.
-final String serverHost = conf.getString(SERVER_HOST_KEY);
-final int serverPort =
Integer.parseInt(conf.getString(SERVER_PORT_KEY));
-
-// Get Graphite metrics graph prefix.
-metricsPrefix = conf.getString(METRICS_PREFIX);
-if (metricsPrefix == null)
-metricsPrefix = "";
-
-graphite = new Graphite(serverHost, serverPort);
-graphite.connect();
+ private static final Logger LOG =
+ LoggerFactory.getLogger(GraphiteSink.class);
+ private static final String SERVER_HOST_KEY = "server_host";
+ private static final String SERVER_PORT_KEY = "server_port";
+ private static final String METRICS_PREFIX = "metrics_prefix";
+ private String metricsPrefix = null;
+ private Graphite graphite = null;
+
+ @Override
+ public void init(SubsetConfiguration conf) {
+// Get Graphite host configurations.
+final String serverHost = conf.getString(SERVER_HOST_KEY);
+final int serverPort = Integer.parseInt(conf.getString(SERVER_PORT_KEY));
+
+// Get Graphite metrics graph prefix.
+metricsPrefix = conf.getString(METRICS_PREFIX);
+if (metricsPrefix == null) {
+ metricsPrefix = "";
}
-@Override
-public void putMetrics(MetricsRecord record) {
-StringBuilder lines = new StringBuilder();
-StringBuilder metricsPathPrefix = new StringBuilder();
-
-// Configure the hierarchical place to display the graph.
-metricsPathPrefix.append(metricsPrefix).append(".")
-.append(record.context()).append(".").append(record.name());
-
-for (MetricsTag tag : record.tags()) {
-if (tag.value() != null) {
-metricsPathPrefix.append(".")
-.append(tag.name())
-.append("=")
-.append(tag.value());
-}
-}
-
-// The record timestamp is in milliseconds while Graphite expects an
epoc time in seconds.
-long timestamp = record.timestamp() / 1000L;
+graphite = new Graphite(serverHost, serverPort);
+graphite.connect();
+ }
+
+ @Override
+ public void putMetrics(MetricsRecord record) {
+StringBuilder lines = new StringBuilder();
+StringBuilder metricsPathPrefix = new StringBuilder();
+
+// Configure the hierarchical place to display the graph.
+metricsPathPrefix.append(metricsPrefix).append(".")
+.append(record.context()).append(".").append(record.name());
+
+for (MetricsTag tag : record.tags()) {
+ if (tag.value() != null) {
+metricsPathPrefix.append(".")
+.append(tag.name())
+.append("=")
+.append(tag.value());
+ }
+}
-// Collect datapoints.
-for (AbstractMetric metric : record.metrics()) {
-lines.append(
-metricsPathPrefix.toString() + "."
-+ metric.name().replace(' ', '.')).append(" ")
-.append(metric.value()).append(" ").append(timestamp)
-.append("\n");
-}
+// The record timestamp is in milliseconds while Graphite expects an epoc
time in seconds.
+long timestamp = record.timestamp() / 1000L;
-try {
- graphite.write(lines.toString());
-} catch (Exception e) {
- LOG.warn("Error sending metrics to Graphite", e);
- try {
-graphite.close();
- } catch (Exception e1) {
-throw new MetricsException("Error closing connection to Graphite",
e1);
- }
-}
+// Collect datapoints.
+for (AbstractMetric metric : record.metrics()) {
+ lines.append(metricsPathPrefix + "." + metric.name().replace(' ',
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4457: HADOOP-18302. Remove WhiteBox in hadoop-common module.
slfan1989 commented on code in PR #4457:
URL: https://github.com/apache/hadoop/pull/4457#discussion_r949678639
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/sink/GraphiteSink.java:
##
@@ -37,171 +37,176 @@
import java.nio.charset.StandardCharsets;
/**
- * A metrics sink that writes to a Graphite server
+ * A metrics sink that writes to a Graphite server.
*/
@InterfaceAudience.Public
@InterfaceStability.Evolving
public class GraphiteSink implements MetricsSink, Closeable {
-private static final Logger LOG =
-LoggerFactory.getLogger(GraphiteSink.class);
-private static final String SERVER_HOST_KEY = "server_host";
-private static final String SERVER_PORT_KEY = "server_port";
-private static final String METRICS_PREFIX = "metrics_prefix";
-private String metricsPrefix = null;
-private Graphite graphite = null;
-
-@Override
-public void init(SubsetConfiguration conf) {
-// Get Graphite host configurations.
-final String serverHost = conf.getString(SERVER_HOST_KEY);
-final int serverPort =
Integer.parseInt(conf.getString(SERVER_PORT_KEY));
-
-// Get Graphite metrics graph prefix.
-metricsPrefix = conf.getString(METRICS_PREFIX);
-if (metricsPrefix == null)
-metricsPrefix = "";
-
-graphite = new Graphite(serverHost, serverPort);
-graphite.connect();
+ private static final Logger LOG =
+ LoggerFactory.getLogger(GraphiteSink.class);
+ private static final String SERVER_HOST_KEY = "server_host";
+ private static final String SERVER_PORT_KEY = "server_port";
+ private static final String METRICS_PREFIX = "metrics_prefix";
+ private String metricsPrefix = null;
+ private Graphite graphite = null;
+
+ @Override
+ public void init(SubsetConfiguration conf) {
+// Get Graphite host configurations.
+final String serverHost = conf.getString(SERVER_HOST_KEY);
+final int serverPort = Integer.parseInt(conf.getString(SERVER_PORT_KEY));
+
+// Get Graphite metrics graph prefix.
+metricsPrefix = conf.getString(METRICS_PREFIX);
+if (metricsPrefix == null) {
+ metricsPrefix = "";
}
-@Override
-public void putMetrics(MetricsRecord record) {
-StringBuilder lines = new StringBuilder();
-StringBuilder metricsPathPrefix = new StringBuilder();
-
-// Configure the hierarchical place to display the graph.
-metricsPathPrefix.append(metricsPrefix).append(".")
-.append(record.context()).append(".").append(record.name());
-
-for (MetricsTag tag : record.tags()) {
-if (tag.value() != null) {
-metricsPathPrefix.append(".")
-.append(tag.name())
-.append("=")
-.append(tag.value());
-}
-}
-
-// The record timestamp is in milliseconds while Graphite expects an
epoc time in seconds.
-long timestamp = record.timestamp() / 1000L;
+graphite = new Graphite(serverHost, serverPort);
+graphite.connect();
+ }
+
+ @Override
+ public void putMetrics(MetricsRecord record) {
+StringBuilder lines = new StringBuilder();
+StringBuilder metricsPathPrefix = new StringBuilder();
+
+// Configure the hierarchical place to display the graph.
+metricsPathPrefix.append(metricsPrefix).append(".")
+.append(record.context()).append(".").append(record.name());
+
+for (MetricsTag tag : record.tags()) {
+ if (tag.value() != null) {
+metricsPathPrefix.append(".")
+.append(tag.name())
+.append("=")
+.append(tag.value());
+ }
+}
-// Collect datapoints.
-for (AbstractMetric metric : record.metrics()) {
-lines.append(
-metricsPathPrefix.toString() + "."
-+ metric.name().replace(' ', '.')).append(" ")
-.append(metric.value()).append(" ").append(timestamp)
-.append("\n");
-}
+// The record timestamp is in milliseconds while Graphite expects an epoc
time in seconds.
+long timestamp = record.timestamp() / 1000L;
-try {
- graphite.write(lines.toString());
-} catch (Exception e) {
- LOG.warn("Error sending metrics to Graphite", e);
- try {
-graphite.close();
- } catch (Exception e1) {
-throw new MetricsException("Error closing connection to Graphite",
e1);
- }
-}
+// Collect datapoints.
+for (AbstractMetric metric : record.metrics()) {
+ lines.append(metricsPathPrefix + "." + metric.name().replace(' ',
'.')).append(" ")
+ .append(metric.value()).append(" ").append(timestamp)
+ .append("\n");
}
-@Override
-public void flush() {
+try {
+ graphite.write(lines.toString());
+} catch (Exception e) {
+
[jira] [Commented] (HADOOP-18302) Remove WhiteBox in hadoop-common module.
[
https://issues.apache.org/jira/browse/HADOOP-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581571#comment-17581571
]
ASF GitHub Bot commented on HADOOP-18302:
-
slfan1989 commented on code in PR #4457:
URL: https://github.com/apache/hadoop/pull/4457#discussion_r949677121
##
hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/portmap/TestPortmap.java:
##
@@ -76,7 +75,7 @@ public void testIdle() throws InterruptedException,
IOException {
}
@Test(timeout = 1)
- public void testRegistration() throws IOException, InterruptedException {
+ public void testRegistration() throws IOException, InterruptedException,
IllegalAccessException {
Review Comment:
I will fix it.
> Remove WhiteBox in hadoop-common module.
>
>
> Key: HADOOP-18302
> URL: https://issues.apache.org/jira/browse/HADOOP-18302
> Project: Hadoop Common
> Issue Type: Sub-task
>Affects Versions: 3.4.0, 3.3.9
>Reporter: fanshilun
>Assignee: fanshilun
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> WhiteBox is deprecated, try to remove this method in hadoop-common.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on a diff in pull request #4457: HADOOP-18302. Remove WhiteBox in hadoop-common module.
slfan1989 commented on code in PR #4457:
URL: https://github.com/apache/hadoop/pull/4457#discussion_r949677121
##
hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/portmap/TestPortmap.java:
##
@@ -76,7 +75,7 @@ public void testIdle() throws InterruptedException,
IOException {
}
@Test(timeout = 1)
- public void testRegistration() throws IOException, InterruptedException {
+ public void testRegistration() throws IOException, InterruptedException,
IllegalAccessException {
Review Comment:
I will fix it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] slfan1989 commented on pull request #4741: YARN-11252. Yarn Federation Router Supports Update / Delete Reservation in MemoryStore.
slfan1989 commented on PR #4741: URL: https://github.com/apache/hadoop/pull/4741#issuecomment-1220070014 @goiri Can you help merge this pr into trunk branch? I will follow up with YARN-11177. Thank you very much! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18403) Fix FileSystem leak in ITestS3AAWSCredentialsProvider
[ https://issues.apache.org/jira/browse/HADOOP-18403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur resolved HADOOP-18403. Fix Version/s: 3.3.9 Resolution: Fixed > Fix FileSystem leak in ITestS3AAWSCredentialsProvider > - > > Key: HADOOP-18403 > URL: https://issues.apache.org/jira/browse/HADOOP-18403 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Labels: pull-request-available > Fix For: 3.3.9 > > > ITestS3AAWSCredentialsProvider#testAnonymousProvider has FileSystem leak that > should be fixed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18403) Fix FileSystem leak in ITestS3AAWSCredentialsProvider
[ https://issues.apache.org/jira/browse/HADOOP-18403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581567#comment-17581567 ] ASF GitHub Bot commented on HADOOP-18403: - mukund-thakur commented on PR #4737: URL: https://github.com/apache/hadoop/pull/4737#issuecomment-1220057944 merged to branch-3.3 as well. > Fix FileSystem leak in ITestS3AAWSCredentialsProvider > - > > Key: HADOOP-18403 > URL: https://issues.apache.org/jira/browse/HADOOP-18403 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Labels: pull-request-available > > ITestS3AAWSCredentialsProvider#testAnonymousProvider has FileSystem leak that > should be fixed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on pull request #4737: HADOOP-18403. Fix FileSystem leak in ITestS3AAWSCredentialsProvider
mukund-thakur commented on PR #4737: URL: https://github.com/apache/hadoop/pull/4737#issuecomment-1220057944 merged to branch-3.3 as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xkrogen commented on pull request #4560: HDFS-16659. JournalNode should throw CacheMissException when SinceTxId is bigger than HighestWrittenTxId
xkrogen commented on PR #4560: URL: https://github.com/apache/hadoop/pull/4560#issuecomment-1220049310 Thanks for trying to tackle this issue! Actually @shvachko and I discussed this potential issue long ago but had not observed problems in practice; I guess it is made much worse by using cross-DC JNs. I don't feel that a `CacheMissException` is correct. The situation where the NN requests edits newer than what the JNs have is expected to be common, especially if the transaction rate is low, since in this situation the NN will constantly poll the JNs for new edits by sending `sinceTxID = highestWrittenTxId + 1`. I see you're trying to handle this by special-casing when the `sinceTxId` is `getHighestWrittenTxId() + 1`, but it seems pretty hacky/brittle. My initial thought is that we should make a special-case return value when `sinceTxId > highestWrittenTxId` (maybe `-1`) and on the NN side, if you find some responses with `txnCount > 0` and some responses with `txnCount < 0`, then you only use the responses with `txnCount > 0`. The main issue I see with this is that `AsyncLoggerSet#waitForWriteQuorum()` isn't set up to handle this kind of situation; it will just return as soon as there are a quorum of non-error responses. As an alternative, we could create a new exception different from `CacheMissException`, like `NewerTxnIdException`, which the JN throws in the situation of `startTxId > highestWrittenTxId`. Since it's an exception, `waitForWriteQuorum()` will try to throw away JNs that threw it. If only some JNs throw the exception, then we still get a valid result from `waitForWriteQuorum()`. If too many JNs throw the exception, then we can catch it on the NN side and swallow the exception to treat it as a normal/expected situation. I think this would avoid us having to special-case `startTxId + 1` on the JN side. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18403) Fix FileSystem leak in ITestS3AAWSCredentialsProvider
[ https://issues.apache.org/jira/browse/HADOOP-18403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581563#comment-17581563 ] ASF GitHub Bot commented on HADOOP-18403: - mukund-thakur merged PR #4737: URL: https://github.com/apache/hadoop/pull/4737 > Fix FileSystem leak in ITestS3AAWSCredentialsProvider > - > > Key: HADOOP-18403 > URL: https://issues.apache.org/jira/browse/HADOOP-18403 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Labels: pull-request-available > > ITestS3AAWSCredentialsProvider#testAnonymousProvider has FileSystem leak that > should be fixed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur merged pull request #4737: HADOOP-18403. Fix FileSystem leak in ITestS3AAWSCredentialsProvider
mukund-thakur merged PR #4737: URL: https://github.com/apache/hadoop/pull/4737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18341) upgrade to commons-configuration2 2.8.0
[
https://issues.apache.org/jira/browse/HADOOP-18341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581553#comment-17581553
]
ASF GitHub Bot commented on HADOOP-18341:
-
hadoop-yetus commented on PR #4578:
URL: https://github.com/apache/hadoop/pull/4578#issuecomment-1219991154
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 26s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 11s | | trunk passed |
| +1 :green_heart: | compile | 25m 8s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 21m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 20m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 9m 42s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 8m 38s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 27s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| -1 :x: | shadedclient | 45m 31s | | branch has errors when building
and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 25m 40s | | the patch passed |
| +1 :green_heart: | compile | 24m 33s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 24m 33s | |
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 2845 unchanged - 6
fixed = 2845 total (was 2851) |
| +1 :green_heart: | compile | 21m 53s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 53s | |
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2642 unchanged -
6 fixed = 2642 total (was 2648) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 22s | | the patch passed |
| +1 :green_heart: | mvnsite | 19m 53s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 8m 25s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 7m 36s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 24s | | hadoop-project has no data from
spotbugs |
| +1 :green_heart: | shadedclient | 58m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 1051m 32s |
[/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4578/7/artifact/out/patch-unit-root.txt)
| root in the patch passed. |
| +1 :green_heart: | asflicense | 2m 14s | | The patch does not
generate ASF License warnings. |
| | | 1414m 1s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4578/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4578 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux bd06c83538a3 4.15.0-191-generic
[GitHub] [hadoop] hadoop-yetus commented on pull request #4578: HADOOP-18341: upgrade commons-configuration2 to 2.8.0
hadoop-yetus commented on PR #4578:
URL: https://github.com/apache/hadoop/pull/4578#issuecomment-1219991154
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 50s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | shelldocs | 0m 0s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 26s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 11s | | trunk passed |
| +1 :green_heart: | compile | 25m 8s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 21m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 20m 6s | | trunk passed |
| +1 :green_heart: | javadoc | 9m 42s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 8m 38s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 27s | | branch/hadoop-project no spotbugs
output file (spotbugsXml.xml) |
| -1 :x: | shadedclient | 45m 31s | | branch has errors when building
and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 25m 40s | | the patch passed |
| +1 :green_heart: | compile | 24m 33s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 24m 33s | |
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 2845 unchanged - 6
fixed = 2845 total (was 2851) |
| +1 :green_heart: | compile | 21m 53s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 53s | |
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2642 unchanged -
6 fixed = 2642 total (was 2648) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 22s | | the patch passed |
| +1 :green_heart: | mvnsite | 19m 53s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | javadoc | 8m 25s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 7m 36s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +0 :ok: | spotbugs | 0m 24s | | hadoop-project has no data from
spotbugs |
| +1 :green_heart: | shadedclient | 58m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 1051m 32s |
[/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4578/7/artifact/out/patch-unit-root.txt)
| root in the patch passed. |
| +1 :green_heart: | asflicense | 2m 14s | | The patch does not
generate ASF License warnings. |
| | | 1414m 1s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.yarn.server.timeline.TestLevelDBCacheTimelineStore |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4578/7/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4578 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
shellcheck shelldocs |
| uname | Linux bd06c83538a3 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1273b25ed800f4f812d94d4c156a5b1d08530175 |
| Default Java |
[jira] [Commented] (HADOOP-18407) Improve vectored IO api spec.
[ https://issues.apache.org/jira/browse/HADOOP-18407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581546#comment-17581546 ] ASF GitHub Bot commented on HADOOP-18407: - mukund-thakur commented on PR #4760: URL: https://github.com/apache/hadoop/pull/4760#issuecomment-1219977111 CC @steveloughran > Improve vectored IO api spec. > -- > > Key: HADOOP-18407 > URL: https://issues.apache.org/jira/browse/HADOOP-18407 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs, fs/s3 >Reporter: Mukund Thakur >Priority: Minor > Labels: pull-request-available > > Let's add more details to the vectored IO api spec for better clarity. > * the position returned by getPos(); is undefined afterwards. > * note that if a file is changed during a read, the output is again > undefined. some ranges may be old data, some may be new, *and some may be both > * note that while reads are active, normal fs api calls may block. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18407) Improve vectored IO api spec.
[ https://issues.apache.org/jira/browse/HADOOP-18407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur updated HADOOP-18407: --- Summary: Improve vectored IO api spec. (was: Update vectored IO api spec. ) > Improve vectored IO api spec. > -- > > Key: HADOOP-18407 > URL: https://issues.apache.org/jira/browse/HADOOP-18407 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs, fs/s3 >Reporter: Mukund Thakur >Priority: Minor > > Let's add more details to the vectored IO api spec for better clarity. > * the position returned by getPos(); is undefined afterwards. > * note that if a file is changed during a read, the output is again > undefined. some ranges may be old data, some may be new, *and some may be both > * note that while reads are active, normal fs api calls may block. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18407) Improve vectored IO api spec.
[ https://issues.apache.org/jira/browse/HADOOP-18407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581545#comment-17581545 ] ASF GitHub Bot commented on HADOOP-18407: - mukund-thakur opened a new pull request, #4760: URL: https://github.com/apache/hadoop/pull/4760 part of HADOOP-18103. ### Description of PR ### How was this patch tested? Not required as it is a doc only change. ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > Improve vectored IO api spec. > -- > > Key: HADOOP-18407 > URL: https://issues.apache.org/jira/browse/HADOOP-18407 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs, fs/s3 >Reporter: Mukund Thakur >Priority: Minor > > Let's add more details to the vectored IO api spec for better clarity. > * the position returned by getPos(); is undefined afterwards. > * note that if a file is changed during a read, the output is again > undefined. some ranges may be old data, some may be new, *and some may be both > * note that while reads are active, normal fs api calls may block. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18407) Improve vectored IO api spec.
[ https://issues.apache.org/jira/browse/HADOOP-18407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HADOOP-18407: Labels: pull-request-available (was: ) > Improve vectored IO api spec. > -- > > Key: HADOOP-18407 > URL: https://issues.apache.org/jira/browse/HADOOP-18407 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs, fs/s3 >Reporter: Mukund Thakur >Priority: Minor > Labels: pull-request-available > > Let's add more details to the vectored IO api spec for better clarity. > * the position returned by getPos(); is undefined afterwards. > * note that if a file is changed during a read, the output is again > undefined. some ranges may be old data, some may be new, *and some may be both > * note that while reads are active, normal fs api calls may block. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on pull request #4760: HADOOP-18407. Improve readVectored() api spec
mukund-thakur commented on PR #4760: URL: https://github.com/apache/hadoop/pull/4760#issuecomment-1219977111 CC @steveloughran -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur opened a new pull request, #4760: HADOOP-18407. Improve readVectored() api spec
mukund-thakur opened a new pull request, #4760: URL: https://github.com/apache/hadoop/pull/4760 part of HADOOP-18103. ### Description of PR ### How was this patch tested? Not required as it is a doc only change. ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18391) harden VectoredReadUtils
[ https://issues.apache.org/jira/browse/HADOOP-18391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur updated HADOOP-18391: --- Parent: HADOOP-18103 Issue Type: Sub-task (was: Improvement) > harden VectoredReadUtils > > > Key: HADOOP-18391 > URL: https://issues.apache.org/jira/browse/HADOOP-18391 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 3.3.9 >Reporter: Steve Loughran >Assignee: Mukund Thakur >Priority: Major > > harden the VectoredReadUtils methods for consistent and more robust use, > especially in those filesystems which don't have the api. > VectoredReadUtils.readInDirectBuffer should allocate a max buffer size, .e.g > 4mb, then do repeated reads and copies; this ensures that you don't OOM with > many threads doing ranged requests. other libs do this. > readVectored to call validateNonOverlappingAndReturnSortedRanges before > iterating > this ensures the abfs/s3a requirements are always met, and that because > ranges will be read in order, prefetching by other clients will keep their > performance good. > readVectored to add special handling for 0 byte ranges -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18403) Fix FileSystem leak in ITestS3AAWSCredentialsProvider
[
https://issues.apache.org/jira/browse/HADOOP-18403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581541#comment-17581541
]
ASF GitHub Bot commented on HADOOP-18403:
-
hadoop-yetus commented on PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#issuecomment-1219965582
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 4s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 32s | | trunk passed |
| +1 :green_heart: | compile | 0m 54s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 42s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 42s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 23s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 49s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 43s | | The patch does not
generate ASF License warnings. |
| | | 105m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4737 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e7079b49869c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / be4360a997c9aebbf9629f8b7cf2ffa8425a73f1 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/3/testReport/ |
| Max. process+thread count | 607 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Fix FileSystem leak in
[GitHub] [hadoop] hadoop-yetus commented on pull request #4737: HADOOP-18403. Fix FileSystem leak in ITestS3AAWSCredentialsProvider
hadoop-yetus commented on PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#issuecomment-1219965582
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 4s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 1s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 32s | | trunk passed |
| +1 :green_heart: | compile | 0m 54s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 18s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 36s | | the patch passed |
| +1 :green_heart: | compile | 0m 42s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 42s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 23s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 27s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 49s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 43s | | The patch does not
generate ASF License warnings. |
| | | 105m 56s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4737 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux e7079b49869c 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / be4360a997c9aebbf9629f8b7cf2ffa8425a73f1 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/3/testReport/ |
| Max. process+thread count | 607 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/3/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please
[GitHub] [hadoop] ashutoshcipher commented on pull request #4248: MAPREDUCE-7370. Parallelize MultipleOutputs#close call
ashutoshcipher commented on PR #4248: URL: https://github.com/apache/hadoop/pull/4248#issuecomment-1219961390 Hi @cnauroth - I have tried addressing all your comments - Can you please review it again? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18272) replace with HashSet/TreeSet constructor directly in hadoop-yarn-project
[
https://issues.apache.org/jira/browse/HADOOP-18272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581533#comment-17581533
]
ASF GitHub Bot commented on HADOOP-18272:
-
hadoop-yetus commented on PR #4759:
URL: https://github.com/apache/hadoop/pull/4759#issuecomment-1219941433
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 47s | | trunk passed |
| +1 :green_heart: | compile | 1m 5s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 55s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 46s | | the patch passed |
| +1 :green_heart: | compile | 0m 49s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 0m 49s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4759/1/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 7 new + 63
unchanged - 0 fixed = 70 total (was 63) |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 0m 43s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4759/1/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 7 new
+ 56 unchanged - 0 fixed = 63 total (was 56) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 35s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4759/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common: The patch generated 14
new + 56 unchanged - 2 fixed = 70 total (was 58) |
| +1 :green_heart: | mvnsite | 0m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 4m 59s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 104m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker |
[GitHub] [hadoop] hadoop-yetus commented on pull request #4759: HADOOP-18272. [WIP] converting sets.hashset to java HashSet in hadoop-yarn-project
hadoop-yetus commented on PR #4759:
URL: https://github.com/apache/hadoop/pull/4759#issuecomment-1219941433
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 6 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 47s | | trunk passed |
| +1 :green_heart: | compile | 1m 5s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 1s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 58s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 5s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 12s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 1s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 7s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 55s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 46s | | the patch passed |
| +1 :green_heart: | compile | 0m 49s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 0m 49s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4759/1/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 7 new + 63
unchanged - 0 fixed = 70 total (was 63) |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 0m 43s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4759/1/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 7 new
+ 56 unchanged - 0 fixed = 63 total (was 56) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 35s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4759/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common: The patch generated 14
new + 56 unchanged - 2 fixed = 70 total (was 58) |
| +1 :green_heart: | mvnsite | 0m 46s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 47s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 46s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 47s | | the patch passed |
| +1 :green_heart: | shadedclient | 21m 4s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 4m 59s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 104m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4759/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4759 |
| Optional Tests | dupname asflicense compile
[GitHub] [hadoop] attilapiros commented on a diff in pull request #4728: MAPREDUCE-7403. manifest-committer dynamic partitioning support.
attilapiros commented on code in PR #4728:
URL: https://github.com/apache/hadoop/pull/4728#discussion_r949572501
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/TestManifestCommitProtocol.java:
##
@@ -1549,6 +1550,21 @@ public void testOutputFormatIntegration() throws
Throwable {
ManifestCommitter committer = (ManifestCommitter)
outputFormat.getOutputCommitter(tContext);
+// check path capabilities directly
+Assertions.assertThat(committer.hasCapability(
+ManifestCommitterConstants.CAPABILITY_DYNAMIC_PARTITIONING))
+.describedAs("dynamic partitioning capability in committer %s",
+committer);
+// and through a binding committer -passthrough is critical
+// for the spark binding.
+BindingPathOutputCommitter bindingCommitter =
+new BindingPathOutputCommitter(outputDir, tContext);
+Assertions.assertThat(bindingCommitter.hasCapability(
+ManifestCommitterConstants.CAPABILITY_DYNAMIC_PARTITIONING))
+.describedAs("dynamic partitioning capability in committer %s",
+bindingCommitter);
Review Comment:
```suggestion
bindingCommitter).isTrue();
```
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/TestManifestCommitProtocol.java:
##
@@ -1549,6 +1550,21 @@ public void testOutputFormatIntegration() throws
Throwable {
ManifestCommitter committer = (ManifestCommitter)
outputFormat.getOutputCommitter(tContext);
+// check path capabilities directly
+Assertions.assertThat(committer.hasCapability(
+ManifestCommitterConstants.CAPABILITY_DYNAMIC_PARTITIONING))
+.describedAs("dynamic partitioning capability in committer %s",
+committer);
Review Comment:
```suggestion
committer).isTrue();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18403) Fix FileSystem leak in ITestS3AAWSCredentialsProvider
[
https://issues.apache.org/jira/browse/HADOOP-18403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581508#comment-17581508
]
ASF GitHub Bot commented on HADOOP-18403:
-
hadoop-yetus commented on PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#issuecomment-1219894591
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 22s | | trunk passed |
| +1 :green_heart: | compile | 0m 53s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 39s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 24s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 46s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 105m 6s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4737 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 7e6ca432ae51 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 762d33d68f0046754529576e6b0b12af76c6b2e2 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/2/testReport/ |
| Max. process+thread count | 604 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
> Fix FileSystem leak in
[GitHub] [hadoop] hadoop-yetus commented on pull request #4737: HADOOP-18403. Fix FileSystem leak in ITestS3AAWSCredentialsProvider
hadoop-yetus commented on PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#issuecomment-1219894591
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 22s | | trunk passed |
| +1 :green_heart: | compile | 0m 53s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 39s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 25s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 56s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 40s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 33s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 24s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 39s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 36s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 46s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 105m 6s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4737 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 7e6ca432ae51 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 762d33d68f0046754529576e6b0b12af76c6b2e2 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/2/testReport/ |
| Max. process+thread count | 604 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4737/2/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please
[jira] [Commented] (HADOOP-18403) Fix FileSystem leak in ITestS3AAWSCredentialsProvider
[
https://issues.apache.org/jira/browse/HADOOP-18403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581499#comment-17581499
]
ASF GitHub Bot commented on HADOOP-18403:
-
virajjasani commented on code in PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#discussion_r949515329
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java:
##
@@ -162,12 +162,13 @@ public void testAnonymousProvider() throws Exception {
conf.set(AWS_CREDENTIALS_PROVIDER,
AnonymousAWSCredentialsProvider.class.getName());
Path testFile = getCSVTestPath(conf);
-FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf);
-assertNotNull(fs);
-assertTrue(fs instanceof S3AFileSystem);
-FileStatus stat = fs.getFileStatus(testFile);
-assertNotNull(stat);
-assertEquals(testFile, stat.getPath());
+try (FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf)) {
+ assertNotNull("S3AFileSystem instance must not be null", fs);
+ assertTrue(fs instanceof S3AFileSystem);
Review Comment:
Ah, missed it. Done, thanks
> Fix FileSystem leak in ITestS3AAWSCredentialsProvider
> -
>
> Key: HADOOP-18403
> URL: https://issues.apache.org/jira/browse/HADOOP-18403
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
> Labels: pull-request-available
>
> ITestS3AAWSCredentialsProvider#testAnonymousProvider has FileSystem leak that
> should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on a diff in pull request #4737: HADOOP-18403. Fix FileSystem leak in ITestS3AAWSCredentialsProvider
virajjasani commented on code in PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#discussion_r949515329
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java:
##
@@ -162,12 +162,13 @@ public void testAnonymousProvider() throws Exception {
conf.set(AWS_CREDENTIALS_PROVIDER,
AnonymousAWSCredentialsProvider.class.getName());
Path testFile = getCSVTestPath(conf);
-FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf);
-assertNotNull(fs);
-assertTrue(fs instanceof S3AFileSystem);
-FileStatus stat = fs.getFileStatus(testFile);
-assertNotNull(stat);
-assertEquals(testFile, stat.getPath());
+try (FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf)) {
+ assertNotNull("S3AFileSystem instance must not be null", fs);
+ assertTrue(fs instanceof S3AFileSystem);
Review Comment:
Ah, missed it. Done, thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18272) replace with HashSet/TreeSet constructor directly in hadoop-yarn-project
[ https://issues.apache.org/jira/browse/HADOOP-18272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581491#comment-17581491 ] ASF GitHub Bot commented on HADOOP-18272: - Samrat002 opened a new pull request, #4759: URL: https://github.com/apache/hadoop/pull/4759 ### Description of PR [HADOOP-18272](https://issues.apache.org/jira/browse/HADOOP-18272) ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? > replace with HashSet/TreeSet constructor directly in hadoop-yarn-project > - > > Key: HADOOP-18272 > URL: https://issues.apache.org/jira/browse/HADOOP-18272 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Samrat Deb >Assignee: Samrat Deb >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Samrat002 opened a new pull request, #4759: HADOOP-18272. converting sets.hashset to java HashSet in hadoop-yarn-project
Samrat002 opened a new pull request, #4759: URL: https://github.com/apache/hadoop/pull/4759 ### Description of PR [HADOOP-18272](https://issues.apache.org/jira/browse/HADOOP-18272) ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18403) Fix FileSystem leak in ITestS3AAWSCredentialsProvider
[
https://issues.apache.org/jira/browse/HADOOP-18403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581484#comment-17581484
]
ASF GitHub Bot commented on HADOOP-18403:
-
mukund-thakur commented on code in PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#discussion_r949474664
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java:
##
@@ -162,12 +162,13 @@ public void testAnonymousProvider() throws Exception {
conf.set(AWS_CREDENTIALS_PROVIDER,
AnonymousAWSCredentialsProvider.class.getName());
Path testFile = getCSVTestPath(conf);
-FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf);
-assertNotNull(fs);
-assertTrue(fs instanceof S3AFileSystem);
-FileStatus stat = fs.getFileStatus(testFile);
-assertNotNull(stat);
-assertEquals(testFile, stat.getPath());
+try (FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf)) {
+ assertNotNull("S3AFileSystem instance must not be null", fs);
+ assertTrue(fs instanceof S3AFileSystem);
Review Comment:
here as well please.
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java:
##
@@ -162,12 +162,13 @@ public void testAnonymousProvider() throws Exception {
conf.set(AWS_CREDENTIALS_PROVIDER,
AnonymousAWSCredentialsProvider.class.getName());
Path testFile = getCSVTestPath(conf);
-FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf);
-assertNotNull(fs);
-assertTrue(fs instanceof S3AFileSystem);
-FileStatus stat = fs.getFileStatus(testFile);
-assertNotNull(stat);
-assertEquals(testFile, stat.getPath());
+try (FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf)) {
+ assertNotNull("S3AFileSystem instance must not be null", fs);
+ assertTrue(fs instanceof S3AFileSystem);
+ FileStatus stat = fs.getFileStatus(testFile);
+ assertNotNull("FileStatus with qualified path must not be null", stat);
+ assertEquals(testFile, stat.getPath());
Review Comment:
and here.
> Fix FileSystem leak in ITestS3AAWSCredentialsProvider
> -
>
> Key: HADOOP-18403
> URL: https://issues.apache.org/jira/browse/HADOOP-18403
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
> Labels: pull-request-available
>
> ITestS3AAWSCredentialsProvider#testAnonymousProvider has FileSystem leak that
> should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on a diff in pull request #4737: HADOOP-18403. Fix FileSystem leak in ITestS3AAWSCredentialsProvider
mukund-thakur commented on code in PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#discussion_r949474664
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java:
##
@@ -162,12 +162,13 @@ public void testAnonymousProvider() throws Exception {
conf.set(AWS_CREDENTIALS_PROVIDER,
AnonymousAWSCredentialsProvider.class.getName());
Path testFile = getCSVTestPath(conf);
-FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf);
-assertNotNull(fs);
-assertTrue(fs instanceof S3AFileSystem);
-FileStatus stat = fs.getFileStatus(testFile);
-assertNotNull(stat);
-assertEquals(testFile, stat.getPath());
+try (FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf)) {
+ assertNotNull("S3AFileSystem instance must not be null", fs);
+ assertTrue(fs instanceof S3AFileSystem);
Review Comment:
here as well please.
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java:
##
@@ -162,12 +162,13 @@ public void testAnonymousProvider() throws Exception {
conf.set(AWS_CREDENTIALS_PROVIDER,
AnonymousAWSCredentialsProvider.class.getName());
Path testFile = getCSVTestPath(conf);
-FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf);
-assertNotNull(fs);
-assertTrue(fs instanceof S3AFileSystem);
-FileStatus stat = fs.getFileStatus(testFile);
-assertNotNull(stat);
-assertEquals(testFile, stat.getPath());
+try (FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf)) {
+ assertNotNull("S3AFileSystem instance must not be null", fs);
+ assertTrue(fs instanceof S3AFileSystem);
+ FileStatus stat = fs.getFileStatus(testFile);
+ assertNotNull("FileStatus with qualified path must not be null", stat);
+ assertEquals(testFile, stat.getPath());
Review Comment:
and here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] xkrogen commented on pull request #4744: HDFS-16689. Standby NameNode crashes when transitioning to Active with in-progress tailer
xkrogen commented on PR #4744: URL: https://github.com/apache/hadoop/pull/4744#issuecomment-1219804467 > Before `catchupDuringFailover`, whatever active crashed or successfully changed to standby, the last segment in majority journalnode should be finalized. If the active crashed, then the segment won't be finalized, right? > The same processing idea has also appeared in [HDFS-14806](https://issues.apache.org/jira/browse/HDFS-14806). [Here](https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java#L113) This is different. Bootstrap standby doesn't need to get _all_ transactions, it's more like a best-effort to load most of the transactions. Later, when the standby transitions to active, _then_ it will call `catchupDuringFailover` to load the remainder. Though generally I agree that the idea is similar. Perhaps we should add a way for callers of `QuorumJournalManager` to indicate that they want to use the streaming mechanism, as opposed to RPC. Disabling in-progress edits achieves this, but is too strong (note that the streaming mechanism can also load in-progress edits). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18384) ITestS3AFileSystemStatistic failure in prefetch feature branch
[
https://issues.apache.org/jira/browse/HADOOP-18384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581480#comment-17581480
]
Steve Loughran commented on HADOOP-18384:
-
yes, target trunk for changes
> ITestS3AFileSystemStatistic failure in prefetch feature branch
> --
>
> Key: HADOOP-18384
> URL: https://issues.apache.org/jira/browse/HADOOP-18384
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Reporter: Steve Loughran
>Assignee: Samrat Deb
>Priority: Major
>
> testing the rebased prefetch feature branch; got a failure in
> ITestS3AFileSystemStatistic
>
> {code}
> tics.ITestS3AFileSystemStatistic
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.489
> s <<< FAILURE! - in
> org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic
> [ERROR]
> testBytesReadWithStream(org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic)
> Time elapsed: 1.489 s <<< FAILURE!
> java.lang.AssertionError: Mismatch in number of FS bytes read by InputStreams
> expected:<2048> but was:<69537130>
> at
> org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic.testBytesReadWithStream(ITestS3AFileSystemStatistic.java:72)
> {code}
> that;s 64MB + ~237 kb, the kind of values you would get from prefetching
> but, prefetch was disabled in this test run.
> maybe its just the fs stats aren't being reset between test cases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18384) ITestS3AFileSystemStatistic failure in prefetch feature branch
[
https://issues.apache.org/jira/browse/HADOOP-18384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18384:
Affects Version/s: 3.4.0
> ITestS3AFileSystemStatistic failure in prefetch feature branch
> --
>
> Key: HADOOP-18384
> URL: https://issues.apache.org/jira/browse/HADOOP-18384
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 3.4.0
>Reporter: Steve Loughran
>Assignee: Samrat Deb
>Priority: Major
>
> testing the rebased prefetch feature branch; got a failure in
> ITestS3AFileSystemStatistic
>
> {code}
> tics.ITestS3AFileSystemStatistic
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.489
> s <<< FAILURE! - in
> org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic
> [ERROR]
> testBytesReadWithStream(org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic)
> Time elapsed: 1.489 s <<< FAILURE!
> java.lang.AssertionError: Mismatch in number of FS bytes read by InputStreams
> expected:<2048> but was:<69537130>
> at
> org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic.testBytesReadWithStream(ITestS3AFileSystemStatistic.java:72)
> {code}
> that;s 64MB + ~237 kb, the kind of values you would get from prefetching
> but, prefetch was disabled in this test run.
> maybe its just the fs stats aren't being reset between test cases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-18384) ITestS3AFileSystemStatistic failure in prefetch feature branch
[
https://issues.apache.org/jira/browse/HADOOP-18384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18384:
Priority: Minor (was: Major)
> ITestS3AFileSystemStatistic failure in prefetch feature branch
> --
>
> Key: HADOOP-18384
> URL: https://issues.apache.org/jira/browse/HADOOP-18384
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Affects Versions: 3.4.0
>Reporter: Steve Loughran
>Assignee: Samrat Deb
>Priority: Minor
>
> testing the rebased prefetch feature branch; got a failure in
> ITestS3AFileSystemStatistic
>
> {code}
> tics.ITestS3AFileSystemStatistic
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.489
> s <<< FAILURE! - in
> org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic
> [ERROR]
> testBytesReadWithStream(org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic)
> Time elapsed: 1.489 s <<< FAILURE!
> java.lang.AssertionError: Mismatch in number of FS bytes read by InputStreams
> expected:<2048> but was:<69537130>
> at
> org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic.testBytesReadWithStream(ITestS3AFileSystemStatistic.java:72)
> {code}
> that;s 64MB + ~237 kb, the kind of values you would get from prefetching
> but, prefetch was disabled in this test run.
> maybe its just the fs stats aren't being reset between test cases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18403) Fix FileSystem leak in ITestS3AAWSCredentialsProvider
[
https://issues.apache.org/jira/browse/HADOOP-18403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581479#comment-17581479
]
ASF GitHub Bot commented on HADOOP-18403:
-
virajjasani commented on code in PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#discussion_r949454666
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java:
##
@@ -162,12 +162,13 @@ public void testAnonymousProvider() throws Exception {
conf.set(AWS_CREDENTIALS_PROVIDER,
AnonymousAWSCredentialsProvider.class.getName());
Path testFile = getCSVTestPath(conf);
-FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf);
-assertNotNull(fs);
-assertTrue(fs instanceof S3AFileSystem);
-FileStatus stat = fs.getFileStatus(testFile);
-assertNotNull(stat);
-assertEquals(testFile, stat.getPath());
+try (FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf)) {
+ assertNotNull(fs);
Review Comment:
Done, thanks @mukund-thakur
> Fix FileSystem leak in ITestS3AAWSCredentialsProvider
> -
>
> Key: HADOOP-18403
> URL: https://issues.apache.org/jira/browse/HADOOP-18403
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
> Labels: pull-request-available
>
> ITestS3AAWSCredentialsProvider#testAnonymousProvider has FileSystem leak that
> should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on a diff in pull request #4737: HADOOP-18403. Fix FileSystem leak in ITestS3AAWSCredentialsProvider
virajjasani commented on code in PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#discussion_r949454666
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java:
##
@@ -162,12 +162,13 @@ public void testAnonymousProvider() throws Exception {
conf.set(AWS_CREDENTIALS_PROVIDER,
AnonymousAWSCredentialsProvider.class.getName());
Path testFile = getCSVTestPath(conf);
-FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf);
-assertNotNull(fs);
-assertTrue(fs instanceof S3AFileSystem);
-FileStatus stat = fs.getFileStatus(testFile);
-assertNotNull(stat);
-assertEquals(testFile, stat.getPath());
+try (FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf)) {
+ assertNotNull(fs);
Review Comment:
Done, thanks @mukund-thakur
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18384) ITestS3AFileSystemStatistic failure in prefetch feature branch
[
https://issues.apache.org/jira/browse/HADOOP-18384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581477#comment-17581477
]
Samrat Deb commented on HADOOP-18384:
-
hi [~ste...@apache.org]
greetings !
checked your branch got merged to trunk ! Is it good now for me to work on this
test case ?
i will try to fix this test error !
> ITestS3AFileSystemStatistic failure in prefetch feature branch
> --
>
> Key: HADOOP-18384
> URL: https://issues.apache.org/jira/browse/HADOOP-18384
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Reporter: Steve Loughran
>Assignee: Samrat Deb
>Priority: Major
>
> testing the rebased prefetch feature branch; got a failure in
> ITestS3AFileSystemStatistic
>
> {code}
> tics.ITestS3AFileSystemStatistic
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.489
> s <<< FAILURE! - in
> org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic
> [ERROR]
> testBytesReadWithStream(org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic)
> Time elapsed: 1.489 s <<< FAILURE!
> java.lang.AssertionError: Mismatch in number of FS bytes read by InputStreams
> expected:<2048> but was:<69537130>
> at
> org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic.testBytesReadWithStream(ITestS3AFileSystemStatistic.java:72)
> {code}
> that;s 64MB + ~237 kb, the kind of values you would get from prefetching
> but, prefetch was disabled in this test run.
> maybe its just the fs stats aren't being reset between test cases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Assigned] (HADOOP-18384) ITestS3AFileSystemStatistic failure in prefetch feature branch
[
https://issues.apache.org/jira/browse/HADOOP-18384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Samrat Deb reassigned HADOOP-18384:
---
Assignee: Samrat Deb
> ITestS3AFileSystemStatistic failure in prefetch feature branch
> --
>
> Key: HADOOP-18384
> URL: https://issues.apache.org/jira/browse/HADOOP-18384
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Reporter: Steve Loughran
>Assignee: Samrat Deb
>Priority: Major
>
> testing the rebased prefetch feature branch; got a failure in
> ITestS3AFileSystemStatistic
>
> {code}
> tics.ITestS3AFileSystemStatistic
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.489
> s <<< FAILURE! - in
> org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic
> [ERROR]
> testBytesReadWithStream(org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic)
> Time elapsed: 1.489 s <<< FAILURE!
> java.lang.AssertionError: Mismatch in number of FS bytes read by InputStreams
> expected:<2048> but was:<69537130>
> at
> org.apache.hadoop.fs.s3a.statistics.ITestS3AFileSystemStatistic.testBytesReadWithStream(ITestS3AFileSystemStatistic.java:72)
> {code}
> that;s 64MB + ~237 kb, the kind of values you would get from prefetching
> but, prefetch was disabled in this test run.
> maybe its just the fs stats aren't being reset between test cases
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18382) Upgrade AWS SDK to V2 - Prerequisites
[
https://issues.apache.org/jira/browse/HADOOP-18382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581465#comment-17581465
]
ASF GitHub Bot commented on HADOOP-18382:
-
hadoop-yetus commented on PR #4698:
URL: https://github.com/apache/hadoop/pull/4698#issuecomment-1219766110
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 13 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 14s | | trunk passed |
| +1 :green_heart: | compile | 0m 51s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 52s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 14s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 42s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 0m 42s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/13/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42
unchanged - 2 fixed = 57 total (was 44) |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 0m 33s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/13/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new
+ 41 unchanged - 2 fixed = 56 total (was 43) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 23s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/13/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 28 unchanged - 0 fixed
= 29 total (was 28) |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 43s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 106m 4s | | |
| Subsystem | Report/Notes |
[GitHub] [hadoop] hadoop-yetus commented on pull request #4698: HADOOP-18382. SDK upgrade prerequisites
hadoop-yetus commented on PR #4698:
URL: https://github.com/apache/hadoop/pull/4698#issuecomment-1219766110
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 13 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 42m 14s | | trunk passed |
| +1 :green_heart: | compile | 0m 51s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 46s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 28s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 52s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 14s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 42s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 0m 42s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/13/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42
unchanged - 2 fixed = 57 total (was 44) |
| +1 :green_heart: | compile | 0m 33s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 0m 33s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/13/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new
+ 41 unchanged - 2 fixed = 56 total (was 43) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 23s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/13/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 28 unchanged - 0 fixed
= 29 total (was 28) |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 43s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 106m 4s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4698 |
| Optional Tests |
[jira] [Commented] (HADOOP-18382) Upgrade AWS SDK to V2 - Prerequisites
[
https://issues.apache.org/jira/browse/HADOOP-18382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581462#comment-17581462
]
ASF GitHub Bot commented on HADOOP-18382:
-
hadoop-yetus commented on PR #4698:
URL: https://github.com/apache/hadoop/pull/4698#issuecomment-1219758179
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 13 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 22s | | trunk passed |
| +1 :green_heart: | compile | 0m 56s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 6s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 33s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 0m 43s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/12/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42
unchanged - 2 fixed = 57 total (was 44) |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 0m 37s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/12/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new
+ 41 unchanged - 2 fixed = 56 total (was 43) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 28s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/12/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 28 unchanged - 0 fixed
= 29 total (was 28) |
| +1 :green_heart: | mvnsite | 0m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 44s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 6s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 100m 23s | | |
| Subsystem | Report/Notes |
[GitHub] [hadoop] hadoop-yetus commented on pull request #4698: HADOOP-18382. SDK upgrade prerequisites
hadoop-yetus commented on PR #4698:
URL: https://github.com/apache/hadoop/pull/4698#issuecomment-1219758179
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 6s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 13 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 40m 22s | | trunk passed |
| +1 :green_heart: | compile | 0m 56s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 50s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 53s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 6s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 33s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 39s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 0m 43s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/12/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42
unchanged - 2 fixed = 57 total (was 44) |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 0m 37s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/12/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new
+ 41 unchanged - 2 fixed = 56 total (was 43) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 28s |
[/results-checkstyle-hadoop-tools_hadoop-aws.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/12/artifact/out/results-checkstyle-hadoop-tools_hadoop-aws.txt)
| hadoop-tools/hadoop-aws: The patch generated 1 new + 28 unchanged - 0 fixed
= 29 total (was 28) |
| +1 :green_heart: | mvnsite | 0m 42s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 24s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 44s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 6s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 49s | | The patch does not
generate ASF License warnings. |
| | | 100m 23s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4698 |
| Optional Tests |
[jira] [Commented] (HADOOP-18403) Fix FileSystem leak in ITestS3AAWSCredentialsProvider
[
https://issues.apache.org/jira/browse/HADOOP-18403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581458#comment-17581458
]
ASF GitHub Bot commented on HADOOP-18403:
-
mukund-thakur commented on code in PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#discussion_r949414334
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java:
##
@@ -162,12 +162,13 @@ public void testAnonymousProvider() throws Exception {
conf.set(AWS_CREDENTIALS_PROVIDER,
AnonymousAWSCredentialsProvider.class.getName());
Path testFile = getCSVTestPath(conf);
-FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf);
-assertNotNull(fs);
-assertTrue(fs instanceof S3AFileSystem);
-FileStatus stat = fs.getFileStatus(testFile);
-assertNotNull(stat);
-assertEquals(testFile, stat.getPath());
+try (FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf)) {
+ assertNotNull(fs);
Review Comment:
Now as you are fixing this already, why don't we add error messages in the
assert statements for better error reporting. thx.
> Fix FileSystem leak in ITestS3AAWSCredentialsProvider
> -
>
> Key: HADOOP-18403
> URL: https://issues.apache.org/jira/browse/HADOOP-18403
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Minor
> Labels: pull-request-available
>
> ITestS3AAWSCredentialsProvider#testAnonymousProvider has FileSystem leak that
> should be fixed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] mukund-thakur commented on a diff in pull request #4737: HADOOP-18403. Fix FileSystem leak in ITestS3AAWSCredentialsProvider
mukund-thakur commented on code in PR #4737:
URL: https://github.com/apache/hadoop/pull/4737#discussion_r949414334
##
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/ITestS3AAWSCredentialsProvider.java:
##
@@ -162,12 +162,13 @@ public void testAnonymousProvider() throws Exception {
conf.set(AWS_CREDENTIALS_PROVIDER,
AnonymousAWSCredentialsProvider.class.getName());
Path testFile = getCSVTestPath(conf);
-FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf);
-assertNotNull(fs);
-assertTrue(fs instanceof S3AFileSystem);
-FileStatus stat = fs.getFileStatus(testFile);
-assertNotNull(stat);
-assertEquals(testFile, stat.getPath());
+try (FileSystem fs = FileSystem.newInstance(testFile.toUri(), conf)) {
+ assertNotNull(fs);
Review Comment:
Now as you are fixing this already, why don't we add error messages in the
assert statements for better error reporting. thx.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18272) replace with HashSet/TreeSet constructor directly in hadoop-yarn-project
[ https://issues.apache.org/jira/browse/HADOOP-18272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581456#comment-17581456 ] ASF GitHub Bot commented on HADOOP-18272: - Samrat002 commented on PR #4411: URL: https://github.com/apache/hadoop/pull/4411#issuecomment-1219745872 closing it to create new one from the trunk > replace with HashSet/TreeSet constructor directly in hadoop-yarn-project > - > > Key: HADOOP-18272 > URL: https://issues.apache.org/jira/browse/HADOOP-18272 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Samrat Deb >Assignee: Samrat Deb >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18272) replace with HashSet/TreeSet constructor directly in hadoop-yarn-project
[ https://issues.apache.org/jira/browse/HADOOP-18272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581457#comment-17581457 ] ASF GitHub Bot commented on HADOOP-18272: - Samrat002 closed pull request #4411: HADOOP-18272. Removing util.set from yarn project URL: https://github.com/apache/hadoop/pull/4411 > replace with HashSet/TreeSet constructor directly in hadoop-yarn-project > - > > Key: HADOOP-18272 > URL: https://issues.apache.org/jira/browse/HADOOP-18272 > Project: Hadoop Common > Issue Type: Sub-task >Reporter: Samrat Deb >Assignee: Samrat Deb >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Samrat002 closed pull request #4411: HADOOP-18272. Removing util.set from yarn project
Samrat002 closed pull request #4411: HADOOP-18272. Removing util.set from yarn project URL: https://github.com/apache/hadoop/pull/4411 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] Samrat002 commented on pull request #4411: HADOOP-18272. Removing util.set from yarn project
Samrat002 commented on PR #4411: URL: https://github.com/apache/hadoop/pull/4411#issuecomment-1219745872 closing it to create new one from the trunk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4746: YARN-9708. Yarn Federation Router Support DelegationToken.
hadoop-yetus commented on PR #4746:
URL: https://github.com/apache/hadoop/pull/4746#issuecomment-1219741441
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 53s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | xmllint | 0m 0s | | xmllint was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +0 :ok: | buf | 0m 0s | | buf was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 5 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 4s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 14s | | trunk passed |
| +1 :green_heart: | compile | 10m 28s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 9m 8s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 2m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 4m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 4m 25s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 4m 21s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 48s | | trunk passed |
| +1 :green_heart: | shadedclient | 24m 34s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 25s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 44s | | the patch passed |
| +1 :green_heart: | compile | 9m 48s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | cc | 9m 48s | | the patch passed |
| -1 :x: | javac | 9m 48s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4746/5/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 1 new +
740 unchanged - 0 fixed = 741 total (was 740) |
| +1 :green_heart: | compile | 9m 6s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | cc | 9m 6s | | the patch passed |
| -1 :x: | javac | 9m 6s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4746/5/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 3 new
+ 649 unchanged - 2 fixed = 652 total (was 651) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 56s | |
hadoop-yarn-project/hadoop-yarn: The patch generated 0 new + 26 unchanged - 2
fixed = 26 total (was 28) |
| +1 :green_heart: | mvnsite | 4m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 4m 2s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 3m 53s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 7m 58s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 11s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 5m 5s | | hadoop-yarn-common in the patch
passed. |
| +1 :green_heart: | unit | 3m 10s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 102m 27s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 4m 4s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart:
[jira] [Commented] (HADOOP-18408) [ABFS]: Ignore run of ITestAbfsRenameStageFailure for NonHNS-SharedKey configuration
[
https://issues.apache.org/jira/browse/HADOOP-18408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581446#comment-17581446
]
ASF GitHub Bot commented on HADOOP-18408:
-
hadoop-yetus commented on PR #4758:
URL: https://github.com/apache/hadoop/pull/4758#issuecomment-1219733637
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 27s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 19s | | trunk passed |
| +1 :green_heart: | compile | 24m 4s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 19s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 41s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 30s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 52s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 24m 5s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 24m 5s | | the patch passed |
| +1 :green_heart: | compile | 20m 40s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 59s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 32s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 20s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 7m 31s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 3m 5s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 33s | | The patch does not
generate ASF License warnings. |
| | | 232m 59s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4758/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4758 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux ae7be0d2f606 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1060102066533c90f7dab85d20ffb31497416587 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4758/1/testReport/ |
| Max. process+thread count | 1567 (vs. ulimit of 5500) |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
[GitHub] [hadoop] hadoop-yetus commented on pull request #4758: HADOOP-18408. ABFS: Ignoring testResilienceAsExpected for nonHNS configuration
hadoop-yetus commented on PR #4758:
URL: https://github.com/apache/hadoop/pull/4758#issuecomment-1219733637
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 42s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 27s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 19s | | trunk passed |
| +1 :green_heart: | compile | 24m 4s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 19s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 1s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 41s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 30s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 52s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 34s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 24s | | the patch passed |
| +1 :green_heart: | compile | 24m 5s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 24m 5s | | the patch passed |
| +1 :green_heart: | compile | 20m 40s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 40s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 4m 13s | | the patch passed |
| +1 :green_heart: | mvnsite | 2m 59s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 32s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 16s | | the patch passed |
| +1 :green_heart: | shadedclient | 25m 20s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 7m 31s | | hadoop-mapreduce-client-core in
the patch passed. |
| +1 :green_heart: | unit | 3m 5s | | hadoop-azure in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 33s | | The patch does not
generate ASF License warnings. |
| | | 232m 59s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4758/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4758 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux ae7be0d2f606 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 1060102066533c90f7dab85d20ffb31497416587 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4758/1/testReport/ |
| Max. process+thread count | 1567 (vs. ulimit of 5500) |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-tools/hadoop-azure U: . |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4758/1/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
[jira] [Updated] (HADOOP-18403) Fix FileSystem leak in ITestS3AAWSCredentialsProvider
[ https://issues.apache.org/jira/browse/HADOOP-18403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukund Thakur updated HADOOP-18403: --- Parent: HADOOP-18067 Issue Type: Sub-task (was: Test) > Fix FileSystem leak in ITestS3AAWSCredentialsProvider > - > > Key: HADOOP-18403 > URL: https://issues.apache.org/jira/browse/HADOOP-18403 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Minor > Labels: pull-request-available > > ITestS3AAWSCredentialsProvider#testAnonymousProvider has FileSystem leak that > should be fixed. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18302) Remove WhiteBox in hadoop-common module.
[
https://issues.apache.org/jira/browse/HADOOP-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581437#comment-17581437
]
ASF GitHub Bot commented on HADOOP-18302:
-
hadoop-yetus commented on PR #4457:
URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1219714273
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 10 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 26s | | trunk passed |
| +1 :green_heart: | compile | 23m 16s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 59s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 56s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 27s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 34s | | the patch passed |
| +1 :green_heart: | compile | 22m 28s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 22m 28s | |
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 2833 unchanged - 17
fixed = 2833 total (was 2850) |
| +1 :green_heart: | compile | 20m 48s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 48s | |
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2629 unchanged -
17 fixed = 2629 total (was 2646) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 48s |
[/results-checkstyle-hadoop-common-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/21/artifact/out/results-checkstyle-hadoop-common-project.txt)
| hadoop-common-project: The patch generated 2 new + 378 unchanged - 233
fixed = 380 total (was 611) |
| +1 :green_heart: | mvnsite | 3m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 48s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 24s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 50s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 6s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 1m 24s | | hadoop-nfs in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 20s | | The patch does not
generate ASF License warnings. |
| | | 233m 54s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/21/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4457 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 298785f96891 4.15.0-191-generic #202-Ubuntu
[GitHub] [hadoop] hadoop-yetus commented on pull request #4457: HADOOP-18302. Remove WhiteBox in hadoop-common module.
hadoop-yetus commented on PR #4457:
URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1219714273
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 56s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 10 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 23s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 26s | | trunk passed |
| +1 :green_heart: | compile | 23m 16s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 20m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 57s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 28s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 59s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 33s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 38s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 56s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 27s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 34s | | the patch passed |
| +1 :green_heart: | compile | 22m 28s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 22m 28s | |
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 2833 unchanged - 17
fixed = 2833 total (was 2850) |
| +1 :green_heart: | compile | 20m 48s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 20m 48s | |
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2629 unchanged -
17 fixed = 2629 total (was 2646) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 48s |
[/results-checkstyle-hadoop-common-project.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/21/artifact/out/results-checkstyle-hadoop-common-project.txt)
| hadoop-common-project: The patch generated 2 new + 378 unchanged - 233
fixed = 380 total (was 611) |
| +1 :green_heart: | mvnsite | 3m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 48s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 24s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 50s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 57s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 19m 6s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 1m 24s | | hadoop-nfs in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 20s | | The patch does not
generate ASF License warnings. |
| | | 233m 54s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/21/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4457 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 298785f96891 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / bcaa569abc0a78375f15b9852b5d5da8f73d860b |
| Default Java | Private
[jira] [Commented] (HADOOP-18302) Remove WhiteBox in hadoop-common module.
[
https://issues.apache.org/jira/browse/HADOOP-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581436#comment-17581436
]
ASF GitHub Bot commented on HADOOP-18302:
-
hadoop-yetus commented on PR #4457:
URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1219711690
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 2s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 22s | | trunk passed |
| +1 :green_heart: | compile | 24m 35s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 22m 6s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 42s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 30s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 9s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 41s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 11s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 34s | | the patch passed |
| +1 :green_heart: | compile | 23m 29s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 23m 29s | |
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 2833 unchanged - 17
fixed = 2833 total (was 2850) |
| +1 :green_heart: | compile | 21m 43s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 43s | |
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2629 unchanged -
17 fixed = 2629 total (was 2646) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 45s | | hadoop-common-project:
The patch generated 0 new + 377 unchanged - 233 fixed = 377 total (was 610) |
| +1 :green_heart: | mvnsite | 2m 59s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 20s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 50s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 43s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 1m 33s | | hadoop-nfs in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 34s | | The patch does not
generate ASF License warnings. |
| | | 235m 41s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/20/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4457 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4e0742742bd8 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk /
[GitHub] [hadoop] hadoop-yetus commented on pull request #4457: HADOOP-18302. Remove WhiteBox in hadoop-common module.
hadoop-yetus commented on PR #4457:
URL: https://github.com/apache/hadoop/pull/4457#issuecomment-1219711690
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 7 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 2s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 22s | | trunk passed |
| +1 :green_heart: | compile | 24m 35s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 22m 6s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 42s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 30s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 9s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 16s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 41s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 11s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 32s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 34s | | the patch passed |
| +1 :green_heart: | compile | 23m 29s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 23m 29s | |
root-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 2833 unchanged - 17
fixed = 2833 total (was 2850) |
| +1 :green_heart: | compile | 21m 43s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 43s | |
root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 with JDK Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new + 2629 unchanged -
17 fixed = 2629 total (was 2646) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 45s | | hadoop-common-project:
The patch generated 0 new + 377 unchanged - 233 fixed = 377 total (was 610) |
| +1 :green_heart: | mvnsite | 2m 59s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 20s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 2m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 44s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 50s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 18m 43s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 1m 33s | | hadoop-nfs in the patch passed.
|
| +1 :green_heart: | asflicense | 1m 34s | | The patch does not
generate ASF License warnings. |
| | | 235m 41s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4457/20/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4457 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 4e0742742bd8 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / bcbafea1f6582356c7932a67061ccff7e9962c01 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
[jira] [Commented] (HADOOP-18382) Upgrade AWS SDK to V2 - Prerequisites
[
https://issues.apache.org/jira/browse/HADOOP-18382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581435#comment-17581435
]
ASF GitHub Bot commented on HADOOP-18382:
-
hadoop-yetus commented on PR #4698:
URL: https://github.com/apache/hadoop/pull/4698#issuecomment-1219710219
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 13 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 4s | | trunk passed |
| +1 :green_heart: | compile | 1m 0s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 53s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 53s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 48s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 57s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 24s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 0m 43s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/11/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42
unchanged - 2 fixed = 57 total (was 44) |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 0m 37s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/11/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new
+ 41 unchanged - 2 fixed = 56 total (was 43) |
| -1 :x: | blanks | 0m 1s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/11/artifact/out/blanks-eol.txt)
| The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 0m 28s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 26s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 49s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 98m 41s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
[GitHub] [hadoop] hadoop-yetus commented on pull request #4698: HADOOP-18382. SDK upgrade prerequisites
hadoop-yetus commented on PR #4698:
URL: https://github.com/apache/hadoop/pull/4698#issuecomment-1219710219
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 45s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 13 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 4s | | trunk passed |
| +1 :green_heart: | compile | 1m 0s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 53s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 53s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 2s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 48s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 50s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 34s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 57s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 21m 24s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 40s | | the patch passed |
| +1 :green_heart: | compile | 0m 43s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 0m 43s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/11/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
| hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 15 new + 42
unchanged - 2 fixed = 57 total (was 44) |
| +1 :green_heart: | compile | 0m 37s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 0m 37s |
[/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/11/artifact/out/results-compile-javac-hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 15 new
+ 41 unchanged - 2 fixed = 56 total (was 43) |
| -1 :x: | blanks | 0m 1s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/11/artifact/out/blanks-eol.txt)
| The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 0m 28s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 23s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 31s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 14s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 26s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 49s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 98m 41s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/11/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4698 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit
[jira] [Commented] (HADOOP-18365) Updated addresses are still accessed using the old IP address
[ https://issues.apache.org/jira/browse/HADOOP-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581428#comment-17581428 ] ASF GitHub Bot commented on HADOOP-18365: - saintstack merged PR #4692: URL: https://github.com/apache/hadoop/pull/4692 > Updated addresses are still accessed using the old IP address > - > > Key: HADOOP-18365 > URL: https://issues.apache.org/jira/browse/HADOOP-18365 > Project: Hadoop Common > Issue Type: Improvement > Components: common > Environment: Demonstrated in a Kubernetes environment running Java 11. >Reporter: Steve Vaughan >Assignee: Steve Vaughan >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > When the IPC Client recognizes that an IP address has changed, it updates the > server field and logs a message: > Address change detected. Old: > journalnode-1.journalnode.hdfs.svc.cluster.local/10.1.0.178:8485 New: > journalnode-1.journalnode.hdfs.svc.cluster.local/10.1.0.182:8485 > Although the change is detected, the client will continue to connect to the > old IP address, resulting in repeated log messages. This is seen in managed > environments when JournalNode syncing is enabled and a JournalNode is > restarted, with the remaining nodes in the set repeatedly logging this > message when syncing to the restarted JournalNode. > The source of the problem is that the remoteId.address is not updated. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] saintstack merged pull request #4692: HADOOP-18365. Update the remote address when a change is detected
saintstack merged PR #4692: URL: https://github.com/apache/hadoop/pull/4692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] hadoop-yetus commented on pull request #4741: YARN-11252. Yarn Federation Router Supports Update / Delete Reservation in MemoryStore.
hadoop-yetus commented on PR #4741:
URL: https://github.com/apache/hadoop/pull/4741#issuecomment-1219685883
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 18s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 26m 7s | | trunk passed |
| +1 :green_heart: | compile | 4m 23s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 7s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 50s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 34s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 47s | | trunk passed |
| +1 :green_heart: | shadedclient | 22m 11s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 22m 37s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 30s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 35s | | the patch passed |
| +1 :green_heart: | compile | 4m 15s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 4m 15s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4741/5/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 6 new +
443 unchanged - 6 fixed = 449 total (was 449) |
| +1 :green_heart: | compile | 3m 35s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 3m 35s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4741/5/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 6 new
+ 367 unchanged - 6 fixed = 373 total (was 373) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 12s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 56s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 28s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 1s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 3m 5s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | unit | 98m 44s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | asflicense | 0m 46s | | The patch does not
generate ASF License warnings. |
| | | 229m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
[jira] [Commented] (HADOOP-18365) Updated addresses are still accessed using the old IP address
[
https://issues.apache.org/jira/browse/HADOOP-18365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581419#comment-17581419
]
ASF GitHub Bot commented on HADOOP-18365:
-
saintstack commented on code in PR #4692:
URL: https://github.com/apache/hadoop/pull/4692#discussion_r949336930
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java:
##
@@ -1753,7 +1756,28 @@ public ConnectionId(InetSocketAddress address, Class
protocol,
InetSocketAddress getAddress() {
return address;
}
-
+
+/**
+ * This is used to update the remote address when an address change is
detected. This method
+ * ensures that the {@link #hashCode()} won't change.
+ *
+ * @param address the updated address
+ * @throws IllegalArgumentException if the hostname or port doesn't match
+ * @see Connection#updateAddress()
Review Comment:
The IllegalAddressException throw is unsettling but makes sense as long as
the caller #updateAddress keeps making the currentAddr in the same way... I was
going to say this method comment could be more explicit about what is happening
in here and presumptions but I think the pointer to #updateAddress takes care
of it
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java:
##
@@ -815,6 +817,81 @@ public Void call() throws IOException {
}
}
+ /**
+ * The {@link ConnectionId#hashCode} has to be stable despite updates that
occur as the the
+ * address evolves over time. The {@link ConnectionId} is used as a primary
key in maps, so
+ * its hashCode can't change.
+ *
+ * @throws IOException if there is a client or server failure
+ */
+ @Test
+ public void testStableHashCode() throws IOException {
+Server server = new TestServer(5, false);
+try {
+ server.start();
+
+ // Leave host unresolved to start. Use "localhost" as opposed
+ // to local IP from NetUtils.getConnectAddress(server) to force
+ // resolution later
+ InetSocketAddress unresolvedAddr = InetSocketAddress.createUnresolved(
+ "localhost", NetUtils.getConnectAddress(server).getPort());
+
+ // Setup: Create a ConnectionID using an unresolved address, and get
it's hashCode to serve
+ // as a point of comparison.
+ int rpcTimeout = MIN_SLEEP_TIME * 2;
+ final ConnectionId remoteId = getConnectionId(unresolvedAddr,
rpcTimeout, conf);
+ int expected = remoteId.hashCode();
+
+ // Start client
+ Client.setConnectTimeout(conf, 100);
+ Client client = new Client(LongWritable.class, conf);
+ try {
+// Test: Call should re-resolve host and succeed
+LongWritable param = new LongWritable(RANDOM.nextLong());
+client.call(RPC.RpcKind.RPC_BUILTIN, param, remoteId,
+RPC.RPC_SERVICE_CLASS_DEFAULT, null);
+int actual = remoteId.hashCode();
+
+// Verify: The hashCode should match, although the InetAddress is
different since it has
+// now been resolved
+assertThat(remoteId.getAddress()).isNotEqualTo(unresolvedAddr);
+
assertThat(remoteId.getAddress().getHostName()).isEqualTo(unresolvedAddr.getHostName());
+assertThat(remoteId.hashCode()).isEqualTo(expected);
+
+// Test: Call should succeed without having to re-resolve
+InetSocketAddress expectedSocketAddress = remoteId.getAddress();
+param = new LongWritable(RANDOM.nextLong());
+client.call(RPC.RpcKind.RPC_BUILTIN, param, remoteId,
+RPC.RPC_SERVICE_CLASS_DEFAULT, null);
+
+// Verify: The same instance of the InetSocketAddress has been used to
make the second
+// call
+assertThat(remoteId.getAddress()).isSameAs(expectedSocketAddress);
+
+// Verify: The hashCode is protected against updates to the host name
+String hostName = InetAddress.getLocalHost().getHostName();
+InetSocketAddress mismatchedHostName = NetUtils.createSocketAddr(
+InetAddress.getLocalHost().getHostName(),
+remoteId.getAddress().getPort());
+assertThatExceptionOfType(IllegalArgumentException.class)
+.isThrownBy(() -> remoteId.setAddress(mismatchedHostName))
+.withMessageStartingWith("Hostname must match");
+
+// Verify: The hashCode is protected against updates to the port
+InetSocketAddress mismatchedPort = NetUtils.createSocketAddr(
+remoteId.getAddress().getHostName(),
+remoteId.getAddress().getPort() + 1);
+assertThatExceptionOfType(IllegalArgumentException.class)
+.isThrownBy(() -> remoteId.setAddress(mismatchedPort))
+.withMessageStartingWith("Port must match");
+ } finally {
+client.stop();
+ }
+} finally {
+ server.stop();
+}
+ }
+
Review Comment:
Nice test.
[GitHub] [hadoop] saintstack commented on a diff in pull request #4692: HADOOP-18365. Update the remote address when a change is detected
saintstack commented on code in PR #4692:
URL: https://github.com/apache/hadoop/pull/4692#discussion_r949336930
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Client.java:
##
@@ -1753,7 +1756,28 @@ public ConnectionId(InetSocketAddress address, Class
protocol,
InetSocketAddress getAddress() {
return address;
}
-
+
+/**
+ * This is used to update the remote address when an address change is
detected. This method
+ * ensures that the {@link #hashCode()} won't change.
+ *
+ * @param address the updated address
+ * @throws IllegalArgumentException if the hostname or port doesn't match
+ * @see Connection#updateAddress()
Review Comment:
The IllegalAddressException throw is unsettling but makes sense as long as
the caller #updateAddress keeps making the currentAddr in the same way... I was
going to say this method comment could be more explicit about what is happening
in here and presumptions but I think the pointer to #updateAddress takes care
of it
##
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestIPC.java:
##
@@ -815,6 +817,81 @@ public Void call() throws IOException {
}
}
+ /**
+ * The {@link ConnectionId#hashCode} has to be stable despite updates that
occur as the the
+ * address evolves over time. The {@link ConnectionId} is used as a primary
key in maps, so
+ * its hashCode can't change.
+ *
+ * @throws IOException if there is a client or server failure
+ */
+ @Test
+ public void testStableHashCode() throws IOException {
+Server server = new TestServer(5, false);
+try {
+ server.start();
+
+ // Leave host unresolved to start. Use "localhost" as opposed
+ // to local IP from NetUtils.getConnectAddress(server) to force
+ // resolution later
+ InetSocketAddress unresolvedAddr = InetSocketAddress.createUnresolved(
+ "localhost", NetUtils.getConnectAddress(server).getPort());
+
+ // Setup: Create a ConnectionID using an unresolved address, and get
it's hashCode to serve
+ // as a point of comparison.
+ int rpcTimeout = MIN_SLEEP_TIME * 2;
+ final ConnectionId remoteId = getConnectionId(unresolvedAddr,
rpcTimeout, conf);
+ int expected = remoteId.hashCode();
+
+ // Start client
+ Client.setConnectTimeout(conf, 100);
+ Client client = new Client(LongWritable.class, conf);
+ try {
+// Test: Call should re-resolve host and succeed
+LongWritable param = new LongWritable(RANDOM.nextLong());
+client.call(RPC.RpcKind.RPC_BUILTIN, param, remoteId,
+RPC.RPC_SERVICE_CLASS_DEFAULT, null);
+int actual = remoteId.hashCode();
+
+// Verify: The hashCode should match, although the InetAddress is
different since it has
+// now been resolved
+assertThat(remoteId.getAddress()).isNotEqualTo(unresolvedAddr);
+
assertThat(remoteId.getAddress().getHostName()).isEqualTo(unresolvedAddr.getHostName());
+assertThat(remoteId.hashCode()).isEqualTo(expected);
+
+// Test: Call should succeed without having to re-resolve
+InetSocketAddress expectedSocketAddress = remoteId.getAddress();
+param = new LongWritable(RANDOM.nextLong());
+client.call(RPC.RpcKind.RPC_BUILTIN, param, remoteId,
+RPC.RPC_SERVICE_CLASS_DEFAULT, null);
+
+// Verify: The same instance of the InetSocketAddress has been used to
make the second
+// call
+assertThat(remoteId.getAddress()).isSameAs(expectedSocketAddress);
+
+// Verify: The hashCode is protected against updates to the host name
+String hostName = InetAddress.getLocalHost().getHostName();
+InetSocketAddress mismatchedHostName = NetUtils.createSocketAddr(
+InetAddress.getLocalHost().getHostName(),
+remoteId.getAddress().getPort());
+assertThatExceptionOfType(IllegalArgumentException.class)
+.isThrownBy(() -> remoteId.setAddress(mismatchedHostName))
+.withMessageStartingWith("Hostname must match");
+
+// Verify: The hashCode is protected against updates to the port
+InetSocketAddress mismatchedPort = NetUtils.createSocketAddr(
+remoteId.getAddress().getHostName(),
+remoteId.getAddress().getPort() + 1);
+assertThatExceptionOfType(IllegalArgumentException.class)
+.isThrownBy(() -> remoteId.setAddress(mismatchedPort))
+.withMessageStartingWith("Port must match");
+ } finally {
+client.stop();
+ }
+} finally {
+ server.stop();
+}
+ }
+
Review Comment:
Nice test.
##
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/WritableRpcEngine.java:
##
@@ -323,7 +323,7 @@ public ProtocolProxy getProxy(Class protocol,
long clientVersion,
Client.ConnectionId connId,
[jira] [Updated] (HADOOP-18311) Upgrade dependencies to address several CVEs
[ https://issues.apache.org/jira/browse/HADOOP-18311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated HADOOP-18311: Fix Version/s: 3.3.9 (was: 3.3.4) > Upgrade dependencies to address several CVEs > > > Key: HADOOP-18311 > URL: https://issues.apache.org/jira/browse/HADOOP-18311 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Affects Versions: 3.3.3, 3.3.4 >Reporter: Steve Vaughan >Priority: Major > Labels: pull-request-available > Fix For: 3.3.9 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > The following CVEs can be addressed by upgrading dependencies within the > build. This includes a replacement of HTrace with a noop implementation. > * CVE-2018-7489 > * CVE-2020-10663 > * CVE-2020-28491 > * CVE-2020-35490 > * CVE-2020-35491 > * CVE-2020-36518 > * PRISMA-2021-0182 > This addresses all of the CVEs from 3.3.3 except for ones that would require > upgrading Netty to 4.x. I'll be submitting a pull request for 3.3.4. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Resolved] (HADOOP-18385) ITestS3ACannedACLs failure; not in a span
[
https://issues.apache.org/jira/browse/HADOOP-18385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-18385.
-
Fix Version/s: 3.3.9
Resolution: Fixed
> ITestS3ACannedACLs failure; not in a span
> -
>
> Key: HADOOP-18385
> URL: https://issues.apache.org/jira/browse/HADOOP-18385
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, test
>Reporter: Steve Loughran
>Assignee: groot
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.3.9
>
>
> seen in a test of the prefetcn feature branch, but it looks more like this
> has been lurking for a long time, or just that some code change has moved the
> api call out of a span.
> {code}
> [INFO] Running org.apache.hadoop.fs.s3a.ITestS3ACannedACLs
> [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.592
> s <<< FAILURE! - in org.apache.hadoop.fs.s3a.ITestS3ACannedACLs
> [ERROR]
> testCreatedObjectsHaveACLs(org.apache.hadoop.fs.s3a.ITestS3ACannedACLs) Time
> elapsed: 0.591 s <<< ERROR!
> org.apache.hadoop.fs.s3a.audit.AuditFailureException:
> dbb71c86-e022-4b76-99cf-c1f64dd21389-00013058 unaudited operation executing a
> request outside an audit span
> {com.amazonaws.services.s3.model.GetObjectAclRequest size=0, mutating=true}
> at
> org.apache.hadoop.fs.s3a.ITestS3ACannedACLs.assertObjectHasLoggingGrant(ITestS3ACannedACLs.java:94)
> at
> org.apache.hadoop.fs.s3a.ITestS3ACannedACLs.testCreatedObjectsHaveACLs(ITestS3ACannedACLs.java:69)
> {code}
> fix is trivial: do the operation within a span
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-18382) Upgrade AWS SDK to V2 - Prerequisites
[
https://issues.apache.org/jira/browse/HADOOP-18382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581412#comment-17581412
]
ASF GitHub Bot commented on HADOOP-18382:
-
hadoop-yetus commented on PR #4698:
URL: https://github.com/apache/hadoop/pull/4698#issuecomment-1219663088
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 13 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 42s | | trunk passed |
| +1 :green_heart: | compile | 0m 53s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 55s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 17s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 41s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 41s | |
hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with
JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 42
unchanged - 2 fixed = 42 total (was 44) |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 32s | |
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new +
41 unchanged - 2 fixed = 41 total (was 43) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/10/artifact/out/blanks-eol.txt)
| The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 0m 23s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 13s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 41s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 105m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4698 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 9028e0c9900e 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk /
[GitHub] [hadoop] hadoop-yetus commented on pull request #4698: HADOOP-18382. SDK upgrade prerequisites
hadoop-yetus commented on PR #4698:
URL: https://github.com/apache/hadoop/pull/4698#issuecomment-1219663088
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +0 :ok: | markdownlint | 0m 0s | | markdownlint was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 13 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 41m 42s | | trunk passed |
| +1 :green_heart: | compile | 0m 53s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 0m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 43s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 52s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 38s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 40s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 30s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 55s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 17s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 35s | | the patch passed |
| +1 :green_heart: | compile | 0m 41s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 0m 41s | |
hadoop-tools_hadoop-aws-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with
JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 0 new + 42
unchanged - 2 fixed = 42 total (was 44) |
| +1 :green_heart: | compile | 0m 32s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 32s | |
hadoop-tools_hadoop-aws-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 0 new +
41 unchanged - 2 fixed = 41 total (was 43) |
| -1 :x: | blanks | 0m 0s |
[/blanks-eol.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/10/artifact/out/blanks-eol.txt)
| The patch has 3 line(s) that end in blanks. Use git apply --whitespace=fix
<>. Refer https://git-scm.com/docs/git-apply |
| +1 :green_heart: | checkstyle | 0m 23s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 38s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 19s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 28s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 1m 13s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 19s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 41s | | hadoop-aws in the patch passed.
|
| +1 :green_heart: | asflicense | 0m 42s | | The patch does not
generate ASF License warnings. |
| | | 105m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4698/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4698 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets markdownlint
|
| uname | Linux 9028e0c9900e 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4c652b1d88b6ee21e76eaef30f752836377d5354 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
[GitHub] [hadoop] hadoop-yetus commented on pull request #4757: YARN-11219. [Federation] Add getAppActivities, getAppStatistics REST APIs for Router.
hadoop-yetus commented on PR #4757:
URL: https://github.com/apache/hadoop/pull/4757#issuecomment-1219659464
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 40s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 3 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 51s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 25m 29s | | trunk passed |
| +1 :green_heart: | compile | 4m 4s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 3m 31s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 30s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 57s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 45s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 40s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 31s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 1m 31s | | the patch passed |
| +1 :green_heart: | compile | 3m 49s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| -1 :x: | javac | 3m 49s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4757/1/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 1 new +
447 unchanged - 0 fixed = 448 total (was 447) |
| +1 :green_heart: | compile | 3m 16s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| -1 :x: | javac | 3m 16s |
[/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4757/1/artifact/out/results-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt)
|
hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 generated 1 new
+ 371 unchanged - 0 fixed = 372 total (was 371) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 1m 12s |
[/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4757/1/artifact/out/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server.txt)
| hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 8
new + 8 unchanged - 0 fixed = 16 total (was 8) |
| +1 :green_heart: | mvnsite | 1m 41s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 23s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 20s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 13s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 98m 14s | |
hadoop-yarn-server-resourcemanager in the patch passed. |
| +1 :green_heart: | unit | 3m 42s | | hadoop-yarn-server-router in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 224m 36s | | |
| Subsystem | Report/Notes |
[GitHub] [hadoop] hadoop-yetus commented on pull request #4738: YARN-11250. Capture the Performance Metrics of ZookeeperFederationStateStore.
hadoop-yetus commented on PR #4738:
URL: https://github.com/apache/hadoop/pull/4738#issuecomment-1219651885
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 18m 17s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 5s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 5s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 70m 11s | | trunk passed |
| +1 :green_heart: | compile | 1m 20s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | compile | 1m 12s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 0m 54s | | trunk passed |
| +1 :green_heart: | mvnsite | 2m 12s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 58s | | trunk passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 43s | | trunk passed |
| +1 :green_heart: | shadedclient | 37m 7s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 37m 41s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 6s | | the patch passed |
| +1 :green_heart: | compile | 1m 0s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javac | 1m 0s | | the patch passed |
| +1 :green_heart: | compile | 0m 49s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 0m 49s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 30s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 51s | | the patch passed |
| +1 :green_heart: | javadoc | 1m 2s | | the patch passed with JDK
Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 |
| +1 :green_heart: | javadoc | 0m 49s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 2m 12s | | the patch passed |
| +1 :green_heart: | shadedclient | 36m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 5m 26s | | hadoop-yarn-server-common in
the patch passed. |
| +1 :green_heart: | asflicense | 0m 58s | | The patch does not
generate ASF License warnings. |
| | | 190m 17s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4738/13/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4738 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 080ef1274d18 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4
01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 15d0370f523e0508a4e5af9c6257f68dff1a9ab6 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Private
Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4738/13/testReport/ |
| Max. process+thread count | 608 (vs. ulimit of 5500) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4738/13/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message
[jira] [Commented] (HADOOP-18408) [ABFS]: Ignore run of ITestAbfsRenameStageFailure for NonHNS-SharedKey configuration
[
https://issues.apache.org/jira/browse/HADOOP-18408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581403#comment-17581403
]
ASF GitHub Bot commented on HADOOP-18408:
-
pranavsaxena-microsoft commented on code in PR #4758:
URL: https://github.com/apache/hadoop/pull/4758#discussion_r949289883
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/TestRenameStageFailure.java:
##
@@ -121,6 +121,7 @@ protected boolean requireRenameResilience() {
@Test
public void testResilienceAsExpected() throws Throwable {
+Assume.assumeTrue(etagsPreserved);
Review Comment:
What if its HNS account and etags are not preserved due to code-issue, this
test will be ignored. Is it possible to know if its non-HNS account and then
ignore on basis of that?
> [ABFS]: Ignore run of ITestAbfsRenameStageFailure for NonHNS-SharedKey
> configuration
>
>
> Key: HADOOP-18408
> URL: https://issues.apache.org/jira/browse/HADOOP-18408
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, test
>Reporter: Pranav Saxena
>Assignee: Sree Bhattacharyya
>Priority: Minor
> Labels: pull-request-available
>
> ITestAbfsRenameStageFailure fails for NonHNS-SharedKey configuration.
> Failure:
> [ERROR]
> ITestAbfsRenameStageFailure>TestRenameStageFailure.testResilienceAsExpected:126
> [resilient commit support] expected:<[tru]e> but was:<[fals]e>
> RCA:
> ResilientCommit looks for whether etags are preserved in rename, if not then
> it throws an exception and the flag for resilientCommitByRename stays null,
> leading ultimately to the test failure
> Mitigation:
> Since, etags are not preserved in the case of rename in nonHNS account, test
> run for nonHNS account is not valid case. Hence, as part of this task, we
> shall ignore this test for nonHNS configuraiton.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] pranavsaxena-microsoft commented on a diff in pull request #4758: HADOOP-18408. ABFS: Ignoring testResilienceAsExpected for nonHNS configuration
pranavsaxena-microsoft commented on code in PR #4758:
URL: https://github.com/apache/hadoop/pull/4758#discussion_r949289883
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/TestRenameStageFailure.java:
##
@@ -121,6 +121,7 @@ protected boolean requireRenameResilience() {
@Test
public void testResilienceAsExpected() throws Throwable {
+Assume.assumeTrue(etagsPreserved);
Review Comment:
What if its HNS account and etags are not preserved due to code-issue, this
test will be ignored. Is it possible to know if its non-HNS account and then
ignore on basis of that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] saintstack commented on a diff in pull request #4723: HDFS-16684. Exclude the current JournalNode
saintstack commented on code in PR #4723:
URL: https://github.com/apache/hadoop/pull/4723#discussion_r949291014
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalNodeSyncer.java:
##
@@ -153,7 +156,8 @@ private boolean getOtherJournalNodeProxies() {
LOG.warn("Could not add proxy for Journal at addresss " + addr, e);
}
}
-if (otherJNProxies.isEmpty()) {
+// Check if any of there are any resolvable JournalNodes before starting
the sync.
Review Comment:
s/there/them/? Extra 'any'?
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalNodeSyncer.java:
##
@@ -310,12 +314,23 @@ private List
getOtherJournalNodeAddrs() {
return null;
}
- private List getJournalAddrList(String uriStr) throws
+ @VisibleForTesting
+ protected List getJournalAddrList(String uriStr) throws
Review Comment:
This is @private audience class. We are changing method visibility here just
for testing. I think that ok.
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalNodeSyncer.java:
##
@@ -310,12 +314,23 @@ private List
getOtherJournalNodeAddrs() {
return null;
}
- private List getJournalAddrList(String uriStr) throws
+ @VisibleForTesting
+ protected List getJournalAddrList(String uriStr) throws
URISyntaxException,
IOException {
URI uri = new URI(uriStr);
-return Util.getLoggerAddresses(uri,
-new HashSet<>(Arrays.asList(jn.getBoundIpcAddress())), conf);
+
+InetSocketAddress boundIpcAddress = jn.getBoundIpcAddress();
+Set excluded = Sets.newHashSet(boundIpcAddress);
+List addrList = Util.getLoggerAddresses(uri, excluded,
conf);
+
+// Exclude the current JournalNode instance. If we are bound to a local
address on the same
Review Comment:
exclude 'any' port?
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/TestJournalNodeSync.java:
##
@@ -420,7 +456,7 @@ public void testSyncDuringRollingUpgrade() throws Exception
{
// Restart the current standby NN (previously active)
dfsCluster.restartNameNode(standbyNNindex, true,
"-rollingUpgrade", "started");
-Assert.assertEquals(info, dfsActive.rollingUpgrade(
+assertEquals(info, dfsActive.rollingUpgrade(
Review Comment:
FYI, best to not pollute your PR w/ this sort of non-related clean-up.
##
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/qjournal/server/TestJournalNodeSync.java:
##
@@ -96,12 +99,45 @@ public void shutDownMiniCluster() throws IOException {
}
}
+ /**
+ * Test that the "self exclusion" works when there are multiple JournalNode
instances running on
+ * the same server, but on different ports.
+ */
+ @Test
+ public void testJournalNodeExcludesSelfMultilpePorts() throws
URISyntaxException, IOException {
+String uri =
qjmhaCluster.getJournalCluster().getQuorumJournalURI("ns1").toString();
+JournalNodeSyncer syncer =
jCluster.getJournalNode(0).getJournalSyncer("ns1");
+
+// Test: Get the Journal address list for the default configuration
+List addrList = syncer.getJournalAddrList(uri);
+
+// Verify: One of the addresses should be excluded so that the node isn't
syncing with itself
+assertEquals(2, addrList.size());
+ }
+
+ /**
+ * Test that the "self exclusion" works when there a host uses a wildcard
address.
+ */
+ @Test
+ public void testJournalNodeExcludesSelfWildCard() throws URISyntaxException,
IOException {
+String uri =
qjmhaCluster.getJournalCluster().getQuorumJournalURI("ns1").toString();
+JournalNodeSyncer syncer =
jCluster.getJournalNode(0).getJournalSyncer("ns1");
+
+// Test: Request the same Journal address list, but using the IPv4
"0.0.0.0" which is commonly
+// used as a bind host.
+String boundHostUri = uri.replaceAll("127.0.0.1", "0.0.0.0");
+List boundHostAddrList =
syncer.getJournalAddrList(boundHostUri);
+
+// Verify: One of the address should be excluded so that the node isn't
syncing with itself
+assertEquals(2, boundHostAddrList.size());
+ }
+
Review Comment:
Nice tests.
##
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/server/JournalNodeSyncer.java:
##
@@ -153,7 +156,8 @@ private boolean getOtherJournalNodeProxies() {
LOG.warn("Could not add proxy for Journal at addresss " + addr, e);
}
}
-if (otherJNProxies.isEmpty()) {
+// Check if any of there are any resolvable JournalNodes before starting
the sync.
+if (otherJNProxies.stream().filter(jnp ->
!jnp.jnAddr.isUnresolved()).count() == 0) {
Review Comment:
We are waiting on them to resolve? If resolve fails, we make no progress ...
This is better (and aligns w/ other checks done above in this method
[jira] [Commented] (HADOOP-18408) [ABFS]: Ignore run of ITestAbfsRenameStageFailure for NonHNS-SharedKey configuration
[
https://issues.apache.org/jira/browse/HADOOP-18408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17581397#comment-17581397
]
ASF GitHub Bot commented on HADOOP-18408:
-
pranavsaxena-microsoft commented on code in PR #4758:
URL: https://github.com/apache/hadoop/pull/4758#discussion_r949261952
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/commit/ITestAbfsRenameStageFailure.java:
##
@@ -45,7 +45,7 @@ public ITestAbfsRenameStageFailure() throws Exception {
public void setup() throws Exception {
binding.setup();
super.setup();
- }
+}
Review Comment:
Extra space to be removed.
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/TestRenameStageFailure.java:
##
@@ -121,6 +121,7 @@ protected boolean requireRenameResilience() {
@Test
public void testResilienceAsExpected() throws Throwable {
+Assume.assumeTrue(etagsPreserved);
Review Comment:
What if its HNS account and etags are not preserved due to code-issue, this
test will be ignored and code shall be merged in trunk. Is it possible to know
if its non-HNS account and then ignore on basis of that?
> [ABFS]: Ignore run of ITestAbfsRenameStageFailure for NonHNS-SharedKey
> configuration
>
>
> Key: HADOOP-18408
> URL: https://issues.apache.org/jira/browse/HADOOP-18408
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure, test
>Reporter: Pranav Saxena
>Assignee: Sree Bhattacharyya
>Priority: Minor
> Labels: pull-request-available
>
> ITestAbfsRenameStageFailure fails for NonHNS-SharedKey configuration.
> Failure:
> [ERROR]
> ITestAbfsRenameStageFailure>TestRenameStageFailure.testResilienceAsExpected:126
> [resilient commit support] expected:<[tru]e> but was:<[fals]e>
> RCA:
> ResilientCommit looks for whether etags are preserved in rename, if not then
> it throws an exception and the flag for resilientCommitByRename stays null,
> leading ultimately to the test failure
> Mitigation:
> Since, etags are not preserved in the case of rename in nonHNS account, test
> run for nonHNS account is not valid case. Hence, as part of this task, we
> shall ignore this test for nonHNS configuraiton.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] pranavsaxena-microsoft commented on a diff in pull request #4758: HADOOP-18408. ABFS: Ignoring testResilienceAsExpected for nonHNS configuration
pranavsaxena-microsoft commented on code in PR #4758:
URL: https://github.com/apache/hadoop/pull/4758#discussion_r949261952
##
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azurebfs/commit/ITestAbfsRenameStageFailure.java:
##
@@ -45,7 +45,7 @@ public ITestAbfsRenameStageFailure() throws Exception {
public void setup() throws Exception {
binding.setup();
super.setup();
- }
+}
Review Comment:
Extra space to be removed.
##
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/TestRenameStageFailure.java:
##
@@ -121,6 +121,7 @@ protected boolean requireRenameResilience() {
@Test
public void testResilienceAsExpected() throws Throwable {
+Assume.assumeTrue(etagsPreserved);
Review Comment:
What if its HNS account and etags are not preserved due to code-issue, this
test will be ignored and code shall be merged in trunk. Is it possible to know
if its non-HNS account and then ignore on basis of that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
