[jira] [Created] (YARN-9008) Extend YARN distributed shell with file localization feature

2018-11-10 Thread Peter Bacsko (JIRA) 
Peter Bacsko created YARN-9008:
--

 Summary: Extend YARN distributed shell with file localization 
feature
 Key: YARN-9008
 URL: https://issues.apache.org/jira/browse/YARN-9008
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: yarn
Affects Versions: 3.1.1, 2.9.1
Reporter: Peter Bacsko
Assignee: Peter Bacsko


YARN distributed shell is a very handy tool to test various features of YARN.

However, it lacks support for file localization - that is, you define files in 
the command like that you wish to be localized remotely. This can be extremely 
useful in certain scenarios.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-11-10 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/

[Nov 9, 2018 1:52:09 AM] (bharat) HDDS-812. 
TestEndPoint#testCheckVersionResponse is failing. Contributed
[Nov 9, 2018 5:28:04 AM] (bharat) HDDS-823. OzoneRestClient is failing with NPE 
on getKeyDetails call.
[Nov 9, 2018 11:06:23 AM] (stevel) HADOOP-15812. ABFS: Improve 
AbfsRestOperationException format to ensure
[Nov 9, 2018 3:24:56 PM] (aajisaka) HADOOP-15916. Upgrade Maven Surefire plugin 
to 3.0.0-M1.
[Nov 9, 2018 6:38:17 PM] (nanda) HDDS-733. Create container if not exist, as 
part of chunk write.




-1 overall


The following subsystems voted -1:
asflicense findbugs hadolint pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.util.TestDiskCheckerWithDiskIo 
   hadoop.util.TestReadWriteDiskValidator 
   hadoop.hdfs.web.TestWebHdfsTimeouts 
   
hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage
 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/diff-compile-javac-root.txt
  [324K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/diff-checkstyle-root.txt
  [17M]

   hadolint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/diff-patch-hadolint.txt
  [4.0K]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/diff-patch-pylint.txt
  [40K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/diff-patch-shellcheck.txt
  [68K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/whitespace-eol.txt
  [9.3M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/whitespace-tabs.txt
  [1.1M]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-hdds_framework.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-ozone_client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-ozone_common.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-ozone_s3gateway.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/branch-findbugs-hadoop-ozone_tools.txt
  [8.0K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/diff-javadoc-javadoc-root.txt
  [752K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
  [176K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [324K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/953/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [80K]

[jira] [Created] (YARN-9007) CS preemption monitor should only select GUARANTEED containers as candidates

2018-11-10 Thread Tao Yang (JIRA) 
Tao Yang created YARN-9007:
--

 Summary: CS preemption monitor should only select GUARANTEED 
containers as candidates
 Key: YARN-9007
 URL: https://issues.apache.org/jira/browse/YARN-9007
 Project: Hadoop YARN
  Issue Type: Bug
  Components: capacityscheduler
Affects Versions: 3.2.1
Reporter: Tao Yang
Assignee: Tao Yang


Currently CS preemption monitor doesn't consider execution type of containers, 
so OPPORTUNISTIC containers maybe selected and killed without effect.
In some scenario with OPPORTUNISTIC containers, not even preemption can't work 
properly to balance resources, but also some apps with OPPORTUNISTIC containers 
maybe effected and unable to work.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-9006) TestNodeLabelContainerAllocation fails on branch-3.0

2018-11-10 Thread Akira Ajisaka (JIRA) 
Akira Ajisaka created YARN-9006:
---

 Summary: TestNodeLabelContainerAllocation fails on branch-3.0
 Key: YARN-9006
 URL: https://issues.apache.org/jira/browse/YARN-9006
 Project: Hadoop YARN
  Issue Type: Bug
  Components: test
Reporter: Akira Ajisaka


{noformat}
[ERROR] Failures: 
[ERROR]   
TestNodeLabelContainerAllocation.testPreferenceOfNeedyPrioritiesUnderSameAppTowardsNodePartitions:848->checkPendingResource:619
 expected:<1024> but was:<0>
[ERROR]   
TestNodeLabelContainerAllocation.testPreferenceOfQueuesTowardsNodePartitions:1047->checkPendingResource:619
 expected:<5120> but was:<0>
[ERROR]   TestNodeLabelContainerAllocation.testQueueMetricsWithLabels:2024 
expected:<0> but was:<1024>
[ERROR]   
TestNodeLabelContainerAllocation.testQueueMetricsWithLabelsOnDefaultLabelNode:2127
 expected:<1024> but was:<2048>
[INFO] 
[ERROR] Tests run: 21, Failures: 4, Errors: 0, Skipped: 0
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org