[jira] [Created] (YARN-2586) Windows Secure Container Executor: use elevated file system operations to cleanup the containers
Remus Rusanu created YARN-2586: -- Summary: Windows Secure Container Executor: use elevated file system operations to cleanup the containers Key: YARN-2586 URL: https://issues.apache.org/jira/browse/YARN-2586 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Remus Rusanu Assignee: Remus Rusanu WSCE needs to use elevated filesystem operations (hadoopwinutilsvc backed) to cleanup the container dirs. The NM does not have enough privileges to delete these dirs. Similar to how LCE does it, using the root container-executor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: YARN-2312 PreCommit Build #5081
Jira: https://issues.apache.org/jira/browse/YARN-2312 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5081/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 5377 lines...] -rw-r--r-- 1 jenkins jenkins 101 Sep 4 06:01 NOTICE.txt -rw-r--r-- 1 jenkins jenkins 1366 Sep 4 06:01 README.txt drwxr-xr-x 2 jenkins jenkins 4096 Sep 19 01:20 dev-support drwxr-xr-x 5 jenkins jenkins 4096 Sep 23 05:26 hadoop-assemblies drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 05:27 hadoop-client drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 05:19 hadoop-common-project drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 05:27 hadoop-dist drwxr-xr-x 6 jenkins jenkins 4096 Sep 23 05:21 hadoop-hdfs-project drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 05:24 hadoop-mapreduce-project drwxr-xr-x 5 jenkins jenkins 4096 Sep 23 05:26 hadoop-maven-plugins drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 05:27 hadoop-minicluster drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 05:18 hadoop-project drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 05:18 hadoop-project-dist drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 05:25 hadoop-tools drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 05:23 hadoop-yarn-project drwxr-xr-x 2 jenkins jenkins 4096 Sep 23 07:54 patchprocess -rw-r--r-- 1 jenkins jenkins 22465 Sep 5 22:03 pom.xml drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 05:35 target total 108 drwxr-xr-x 18 jenkins jenkins 4096 Sep 5 21:43 HADOOP2_Release_Artifacts_Builder drwxr-xr-x 11 jenkins jenkins 4096 Sep 19 17:33 HBase-0.94-JDK7 drwxr-xr-x 22 jenkins jenkins 4096 Aug 28 18:29 HBase-0.98 drwxr-xr-x 4 jenkins jenkins 4096 Sep 19 09:01 Hadoop-Common-0.23-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 22 04:12 Hadoop-Common-2-Commit drwxr-xr-x 16 jenkins jenkins 4096 Sep 11 10:32 Hadoop-Yarn-trunk drwxr-xr-x 3 jenkins jenkins 4096 Sep 22 06:10 PreCommit-Admin drwxr-xr-x 4 jenkins jenkins 4096 Sep 19 09:00 PreCommit-Falcon-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 22 19:19 PreCommit-HADOOP-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 2 18:13 PreCommit-HADOOP-Build@2 drwxr-xr-x 22 jenkins jenkins 4096 Sep 16 08:00 PreCommit-HBASE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 04:49 PreCommit-HDFS-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 29 00:39 PreCommit-HDFS-Build@2 drwxr-xr-x 4 jenkins jenkins 4096 Aug 26 17:32 PreCommit-MAPREDUCE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 08:16 PreCommit-YARN-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 29 05:08 PreCommit-YARN-Build@2 drwxr-xr-x 3 jenkins jenkins 4096 Sep 11 10:10 PreCommit-ZOOKEEPER-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 26 16:43 Samza-0.7.0 drwxr-xr-x 18 jenkins jenkins 4096 Sep 15 02:01 Tajo-0.8.1-nightly drwxr-xr-x 19 jenkins jenkins 4096 Sep 11 01:59 Tajo-master-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 6 02:01 Tajo-master-nightly drwxr-xr-x 16 jenkins jenkins 4096 Sep 21 22:49 Tez-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 08:31 ZooKeeper_branch34 drwxr-xr-x 16 jenkins jenkins 4096 Sep 11 12:18 bookkeeper-trunk drwxr-xr-x 21 jenkins jenkins 4096 Sep 6 00:11 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 5 22:31 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Aug 26 00:45 oozie-trunk-precommit-build@2 mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-YARN-Build #4749 Archived 60 artifacts Archive block size is 32768 Received 0 blocks and 10249244 bytes Compression is 0.0% Took 2.4 sec [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
[jira] [Created] (YARN-2587) Windows Secure Container Executor: classpath in the job classpath-jar is referencing NM nmPrivate files
Remus Rusanu created YARN-2587:
--
Summary: Windows Secure Container Executor: classpath in the job
classpath-jar is referencing NM nmPrivate files
Key: YARN-2587
URL: https://issues.apache.org/jira/browse/YARN-2587
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager
Reporter: Remus Rusanu
Assignee: Remus Rusanu
The classpath-jar is referencing nmPrivate files:
{code}
Manifest-Version: 1.0
Class-Path: file:/c:/Hadoop/Data/Hadoop/local/usercache/HadoopUser/app
cache/application_1411394968079_0006/container_1411394968079_0006_01_ 03
file:/c:/hadoop/hadoop-2.4.1-SNAPSHOT/etc/hadoop/ file:/c:/had
oop/hadoop-2.4.1-SNAPSHOT/share/hadoop/common/hadoop-common-3.0.0-SNA
PSHOT-tests.jar file:/c:/hadoop/hadoop-2.4.1-SNAPSHOT/share/hadoop/co
...
file:/c:/Hadoop/Data/Hadoop/local
/nmPrivate/application_1411394968079_0006/container_1411394968079_000
6_01_03/job.jar/job.jar file:/c:/Hadoop/Data/Hadoop/local/nmPriva
te/application_1411394968079_0006/container_1411394968079_0006_01_000
003/job.jar/classes/
{code}
This is not allowed in the WSCE enviornment, the container has no read
privileges on the nmPrivate paths. Besides that particular jar is copied over
into the user appcache.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (YARN-2586) Windows Secure Container Executor: use elevated file system operations to cleanup the containers
[ https://issues.apache.org/jira/browse/YARN-2586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu resolved YARN-2586. Resolution: Implemented The next patch of YARN-2198 will contain this fix. Windows Secure Container Executor: use elevated file system operations to cleanup the containers Key: YARN-2586 URL: https://issues.apache.org/jira/browse/YARN-2586 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Remus Rusanu Assignee: Remus Rusanu Labels: security, windows Attachments: YARN-2586.1.patch WSCE needs to use elevated filesystem operations (hadoopwinutilsvc backed) to cleanup the container dirs. The NM does not have enough privileges to delete these dirs. Similar to how LCE does it, using the root container-executor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: YARN-2494 PreCommit Build #5082
Jira: https://issues.apache.org/jira/browse/YARN-2494 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5082/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 153 lines...] drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 08:30 b drwxr-xr-x 2 jenkins jenkins 4096 Sep 18 20:40 dev-support drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 08:41 hadoop-assemblies drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 08:41 hadoop-client drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 08:42 hadoop-common-project drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 08:41 hadoop-dist drwxr-xr-x 6 jenkins jenkins 4096 Sep 23 08:43 hadoop-hdfs-project drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 08:41 hadoop-mapreduce-project drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 08:41 hadoop-maven-plugins drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 08:41 hadoop-minicluster drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 08:41 hadoop-project drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 08:41 hadoop-project-dist drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 08:41 hadoop-tools drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 08:40 hadoop-yarn-project drwxr-xr-x 2 jenkins jenkins 4096 Sep 23 08:41 patchprocess -rw-r--r-- 1 jenkins jenkins 22465 Sep 6 03:11 pom.xml total 116 drwxr-xr-x 29 jenkins jenkins 4096 Sep 3 00:32 Apache-falcon drwxr-xr-x 12 jenkins jenkins 4096 Sep 13 07:42 HBase-0.94 drwxr-xr-x 22 jenkins jenkins 4096 Sep 12 02:07 HBase-TRUNK drwxr-xr-x 4 jenkins jenkins 4096 Jul 8 23:10 Hadoop-Common-0.23-Commit drwxr-xr-x 16 jenkins jenkins 4096 Aug 29 11:32 Hadoop-Hdfs-trunk drwxr-xr-x 4 jenkins jenkins 4096 Sep 19 06:00 Hadoop-trunk-java7 drwxr-xr-x 4 jenkins jenkins 4096 Sep 16 23:17 Pig-trunk-commit drwxr-xr-x 3 jenkins jenkins 4096 Sep 10 16:51 PreCommit-Admin drwxr-xr-x 4 jenkins jenkins 4096 Sep 17 18:52 PreCommit-Falcon-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 19 23:08 PreCommit-HADOOP-Build drwxr-xr-x 23 jenkins jenkins 4096 Sep 18 11:57 PreCommit-HBASE-Build drwxr-xr-x 19 jenkins jenkins 4096 Sep 23 06:45 PreCommit-HDFS-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 02:54 PreCommit-HDFS-Build@2 drwxr-xr-x 19 jenkins jenkins 4096 Sep 22 17:00 PreCommit-MAPREDUCE-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 26 00:34 PreCommit-MAPREDUCE-Build@2 drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 08:43 PreCommit-YARN-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 31 01:57 PreCommit-YARN-Build@2 drwxr-xr-x 3 jenkins jenkins 4096 Sep 19 22:30 PreCommit-ZOOKEEPER-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 02:00 Tajo-0.8.1-nightly drwxr-xr-x 19 jenkins jenkins 4096 Sep 16 14:16 Tajo-block_iteration-branch-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 22 00:22 Tajo-master-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 9 02:01 Tajo-master-nightly drwxr-xr-x 17 jenkins jenkins 4096 Sep 19 07:42 Tez-Build drwxr-xr-x 4 jenkins jenkins 4096 Sep 22 08:30 ZooKeeper_branch34 drwxr-xr-x 10 jenkins jenkins 4096 Sep 20 14:06 cayenne-31 drwxr-xr-x 3 jenkins jenkins 4096 Jul 21 14:17 commons-vfs-trunk drwxr-xr-x 21 jenkins jenkins 4096 Sep 8 20:27 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 9 00:17 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Aug 26 00:36 oozie-trunk-precommit-build@2 mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-YARN-Build #4749 Archived 3 artifacts Archive block size is 32768 Received 0 blocks and 3111821 bytes Compression is 0.0% Took 2.4 sec [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
[jira] [Resolved] (YARN-2129) Add scheduling priority to the WindowsSecureContainerExecutor
[ https://issues.apache.org/jira/browse/YARN-2129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu resolved YARN-2129. Resolution: Won't Fix Given the priority boost inheritance model in Windows, there is little incentive to do this. Add scheduling priority to the WindowsSecureContainerExecutor - Key: YARN-2129 URL: https://issues.apache.org/jira/browse/YARN-2129 Project: Hadoop YARN Issue Type: Improvement Components: nodemanager Affects Versions: 3.0.0 Reporter: Remus Rusanu Assignee: Remus Rusanu Labels: security, windows Attachments: YARN-2129.1.patch, YARN-2129.2.patch The WCE (YARN-1972) could and should honor NM_CONTAINER_EXECUTOR_SCHED_PRIORITY. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-2590) Windows Secure Container Executor: containerLaunch environment doe snot get transferred to the container process
Remus Rusanu created YARN-2590: -- Summary: Windows Secure Container Executor: containerLaunch environment doe snot get transferred to the container process Key: YARN-2590 URL: https://issues.apache.org/jira/browse/YARN-2590 Project: Hadoop YARN Issue Type: Sub-task Components: nodemanager Reporter: Remus Rusanu Assignee: Remus Rusanu the sanitized env prepared by the container launch is ignored by the WSCE launcher. The env has to be passwed in the createTaskAsUser call to hadoopwinutilsvc so that is assigns it to the newly spawned process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (YARN-2587) Windows Secure Container Executor: classpath in the job classpath-jar is referencing NM nmPrivate files
[
https://issues.apache.org/jira/browse/YARN-2587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Remus Rusanu resolved YARN-2587.
Resolution: Implemented
Next YARN-2198 patch will contain the fix.
Windows Secure Container Executor: classpath in the job classpath-jar is
referencing NM nmPrivate files
---
Key: YARN-2587
URL: https://issues.apache.org/jira/browse/YARN-2587
Project: Hadoop YARN
Issue Type: Sub-task
Components: nodemanager
Reporter: Remus Rusanu
Assignee: Remus Rusanu
Labels: security, windows
Attachments: YARN-2587.1.patch
The classpath-jar is referencing nmPrivate files:
{code}
Manifest-Version: 1.0
Class-Path: file:/c:/Hadoop/Data/Hadoop/local/usercache/HadoopUser/app
cache/application_1411394968079_0006/container_1411394968079_0006_01_ 03
file:/c:/hadoop/hadoop-2.4.1-SNAPSHOT/etc/hadoop/ file:/c:/had
oop/hadoop-2.4.1-SNAPSHOT/share/hadoop/common/hadoop-common-3.0.0-SNA
PSHOT-tests.jar file:/c:/hadoop/hadoop-2.4.1-SNAPSHOT/share/hadoop/co
...
file:/c:/Hadoop/Data/Hadoop/local
/nmPrivate/application_1411394968079_0006/container_1411394968079_000
6_01_03/job.jar/job.jar file:/c:/Hadoop/Data/Hadoop/local/nmPriva
te/application_1411394968079_0006/container_1411394968079_0006_01_000
003/job.jar/classes/
{code}
This is not allowed in the WSCE enviornment, the container has no read
privileges on the nmPrivate paths. Besides that particular jar is copied over
into the user appcache.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Failed: YARN-2581 PreCommit Build #5084
Jira: https://issues.apache.org/jira/browse/YARN-2581 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5084/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3779 lines...] drwxr-xr-x 17 jenkins jenkins 4096 Sep 23 19:46 PreCommit-YARN-Build drwxr-xr-x 17 jenkins jenkins 4096 Sep 17 23:53 PreCommit-YARN-Build@2 drwxr-xr-x 4 jenkins jenkins 4096 Sep 6 22:15 Qpid-Java-Cpp-Test drwxr-xr-x 5 jenkins jenkins 4096 Sep 9 23:55 Red5-client drwxr-xr-x 3 jenkins jenkins 4096 Sep 10 01:06 Shindig drwxr-xr-x 11 jenkins jenkins 4096 Sep 15 01:56 Stratos-4.0.0-Nightly-Build drwxr-xr-x 11 jenkins jenkins 4096 Sep 11 02:26 Stratos-Master-Nightly-Build drwxr-xr-x 15 jenkins jenkins 4096 Sep 12 09:39 Syncope-trunk drwxr-xr-x 18 jenkins jenkins 4096 Sep 22 02:00 Tajo-0.8.1-nightly drwxr-xr-x 19 jenkins jenkins 4096 Aug 26 02:39 Tajo-master-nightly drwxr-xr-x 3 jenkins jenkins 4096 Sep 6 20:23 Torque4-trunk drwxr-xr-x 11 jenkins jenkins 4096 Sep 7 00:16 Trinidad Core 1.2.x (sanity) drwxr-xr-x 3 jenkins jenkins 4096 Sep 16 00:26 Tuscany-2x drwxr-xr-x 3 jenkins jenkins 4096 Sep 13 00:52 Wink2-Trunk-JDK1.6 drwxr-xr-x 3 jenkins jenkins 4096 Sep 10 11:09 ZooKeeper-trunk-jdk7 drwxr-xr-x 3 jenkins jenkins 4096 Sep 8 20:05 ZooKeeper-trunk-openjdk7 drwxr-xr-x 9 jenkins jenkins 4096 Sep 10 05:26 archiva-master-build drwxr-xr-x 9 jenkins jenkins 4096 Sep 19 00:26 archiva-master-empty-repo drwxr-xr-x 4 jenkins jenkins 4096 Sep 13 17:01 bookkeeper-trunk-find-patches-available drwxr-xr-x 17 jenkins jenkins 4096 Sep 23 15:35 bookkeeper-trunk-precommit-build drwxr-xr-x 5 jenkins jenkins 4096 Sep 10 15:24 central-indexer-test drwxr-xr-x 35 jenkins jenkins 4096 Sep 3 18:44 cloudstack-master-maven drwxr-xr-x 35 jenkins jenkins 4096 Sep 5 09:11 cloudstack-rat-master drwxr-xr-x 21 jenkins jenkins 4096 Aug 30 00:57 flume-trunk drwxr-xr-x 23 jenkins jenkins 4096 Sep 9 05:02 gora-trunk drwxr-xr-x 16 jenkins jenkins 4096 Sep 9 08:53 incubator-brooklyn-master-build drwxr-xr-x 3 jenkins jenkins 4096 Sep 6 11:26 jspf-trunk drwxr-xr-x 2 jenkins jenkins 4096 Aug 22 00:07 lib drwxr-xr-x 9 jenkins jenkins 4096 Aug 28 03:42 lightning-trunk drwxr-xr-x 20 jenkins jenkins 4096 Sep 11 04:09 mailbox drwxr-xr-x 45 jenkins jenkins 4096 Sep 11 19:36 maven-plugins drwxr-xr-x 44 jenkins jenkins 4096 Aug 31 22:23 maven-plugins-ITs-m3.1.x-with-maven-plugin-jdk-1.8 drwxr-xr-x 17 jenkins jenkins 4096 Sep 13 15:50 maven-surefire drwxr-xr-x 17 jenkins jenkins 4096 Sep 11 08:13 maven-surefire-mvn-2.2.1 drwxr-xr-x 9 jenkins jenkins 4096 Sep 10 06:45 maven-wagon drwxr-xr-x 15 jenkins jenkins 4096 Sep 5 01:25 mesos-tnachen drwxr-xr-x 3 jenkins jenkins 4096 Aug 31 22:19 myfaces-orchestra-core drwxr-xr-x 10 jenkins jenkins 4096 Sep 14 22:46 olingo-odata4-cobertura drwxr-xr-x 21 jenkins jenkins 4096 Sep 8 20:51 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 9 01:13 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Sep 6 00:33 oozie-trunk-precommit-build@2 drwxr-xr-x 14 jenkins jenkins 4096 Sep 13 16:17 openmeetings drwxr-xr-x 9 jenkins jenkins 4096 Sep 9 12:09 sirona-1.7 drwxr-xr-x 10 jenkins jenkins 4096 Sep 10 00:21 sis-dev drwxr-xr-x 3 jenkins jenkins 4096 Aug 26 10:55 sling-contrib-1.6 drwxr-xr-x 3 jenkins jenkins 4096 Sep 5 09:17 sling-crankstart drwxr-xr-x 3 jenkins jenkins 4096 Sep 6 18:11 sling-healthcheck-1.6 drwxr-xr-x 3 jenkins jenkins 4096 Sep 11 19:59 sling-trunk-1.8 drwxr-xr-x 3 jenkins jenkins 4096 Aug 27 20:46 tapestry-4.1-trunk drwxr-xr-x 3 jenkins jenkins 4096 Aug 24 08:01 tika-trunk-jdk1.7 drwxr-xr-x 4 jenkins jenkins 4096 Sep 6 19:39 torque4-test-project-derby drwxr-xr-x 12 jenkins jenkins 4096 Sep 6 19:41 torque4-test-project-hsqldb mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
Failed: YARN-2102 PreCommit Build #5085
Jira: https://issues.apache.org/jira/browse/YARN-2102 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5085/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3308 lines...] drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 21:00 hadoop-common-project drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 21:08 hadoop-dist drwxr-xr-x 6 jenkins jenkins 4096 Sep 23 21:02 hadoop-hdfs-project drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 21:05 hadoop-mapreduce-project drwxr-xr-x 5 jenkins jenkins 4096 Sep 23 21:07 hadoop-maven-plugins drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 21:08 hadoop-minicluster drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 20:59 hadoop-project drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 20:59 hadoop-project-dist drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 21:06 hadoop-tools drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 21:04 hadoop-yarn-project drwxr-xr-x 2 jenkins jenkins 4096 Sep 23 21:12 patchprocess -rw-r--r-- 1 jenkins jenkins 22465 Sep 8 15:02 pom.xml drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 21:11 target total 128 drwxr-xr-x 31 jenkins jenkins 4096 Sep 11 00:30 Apache-falcon drwxr-xr-x 12 jenkins jenkins 4096 Aug 25 19:32 HBase-0.94 drwxr-xr-x 12 jenkins jenkins 4096 Sep 10 02:36 HBase-0.94-JDK7 drwxr-xr-x 22 jenkins jenkins 4096 Sep 19 17:28 HBase-0.98 drwxr-xr-x 22 jenkins jenkins 4096 Sep 19 19:12 HBase-TRUNK drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 09:02 Hadoop-Common-0.23-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 01:33 Hadoop-Common-2-Build drwxr-xr-x 16 jenkins jenkins 4096 Sep 13 11:31 Hadoop-Hdfs-trunk drwxr-xr-x 16 jenkins jenkins 4096 Sep 13 10:31 Hadoop-Yarn-trunk drwxr-xr-x 4 jenkins jenkins 4096 Aug 28 05:02 Hadoop-branch2 drwxr-xr-x 4 jenkins jenkins 4096 Sep 3 06:02 Hadoop-trunk-java7 drwxr-xr-x 4 jenkins jenkins 4096 Aug 27 01:19 Pig-trunk-commit drwxr-xr-x 3 jenkins jenkins 4096 Sep 22 07:00 PreCommit-Admin drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 07:12 PreCommit-Falcon-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 28 14:22 PreCommit-Falcon-Build@2 drwxr-xr-x 19 jenkins jenkins 4096 Sep 18 22:33 PreCommit-HADOOP-Build drwxr-xr-x 23 jenkins jenkins 4096 Sep 17 22:12 PreCommit-HBASE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 20:35 PreCommit-HDFS-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 29 01:29 PreCommit-HDFS-Build@2 drwxr-xr-x 18 jenkins jenkins 4096 Sep 17 23:44 PreCommit-MAPREDUCE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 21:14 PreCommit-YARN-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 29 21:58 PreCommit-YARN-Build@2 drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 00:22 PreCommit-ZOOKEEPER-Build drwxr-xr-x 17 jenkins jenkins 4096 Sep 17 23:45 Samza-0.7.0 drwxr-xr-x 18 jenkins jenkins 4096 Sep 13 02:02 Tajo-0.8.1-nightly drwxr-xr-x 19 jenkins jenkins 4096 Sep 19 09:00 Tajo-master-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 22 02:01 Tajo-master-nightly drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 05:50 Tez-Build drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 16:01 bookkeeper-trunk drwxr-xr-x 21 jenkins jenkins 4096 Sep 8 19:31 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 8 23:13 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Aug 26 01:35 oozie-trunk-precommit-build@2 mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-YARN-Build #4749 Archived 25 artifacts Archive block size is 32768 Received 0 blocks and 8990926 bytes Compression is 0.0% Took 2.8 sec [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
[jira] [Resolved] (YARN-2589) RMDelegationTokenSecretManager should update (not store new) master key when key is rolled over
[ https://issues.apache.org/jira/browse/YARN-2589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jian He resolved YARN-2589. --- Resolution: Not a Problem took a closer look, looks like RMDelegationTokenSecretManager just creates a new DelegationKey znode each time the new key is generated. I think we are fine here.. close this. RMDelegationTokenSecretManager should update (not store new) master key when key is rolled over --- Key: YARN-2589 URL: https://issues.apache.org/jira/browse/YARN-2589 Project: Hadoop YARN Issue Type: Bug Reporter: Jian He Assignee: Jian He -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Re: In hindsight... Re: Thinking ahead to hadoop-2.6
Sorry, coming to discussion late. We all agreed that 2.6 would the *last* release supporting JDK6 and hadoop-2.7 would drop support for JDK6. We could easily do 2.7 right after 2.6 (maybe with few critical bug-fixes) with the defining feature of 2.7 being *JDK7 only*. I've checked with HBase, Pig and other communities and they are good with this. I'm thinking I'll follow up 2-3 wks after 2.6 goes out to release 2.7 which drops JDK6. We can certainly add support for JDK8 as early as 2.7 if there are volunteers - clearly we won't make it depend on JDK8 features right away; since it would still need to support JDK7. To recap: hadoop-2.7 onwards would be minimum JDK7, with potential support for JDK8. We can revisit our discussion from a few months ago to discuss when we *drop* support for JDK7; clearly something I'd like to avoid doing in haste. thanks, Arun On Thu, Sep 18, 2014 at 8:41 AM, Alejandro Abdelnur tuc...@gmail.com wrote: Am I missing something, or we already agreed that after 2.5 release we would move trunk and branch-2 to java 7? On Wed, Sep 17, 2014 at 3:33 PM, Travis Thompson travis.r.thomp...@gmail.com wrote: There's actually an umbrella JIRA to track issues with JDK8 (HADOOP-11090), in case anyone missed it. At LinkedIn we've been running our Hadoop 2.3 deployment on JDK8 for about a month now with some mixed results. It definitely works but there are issues, mostly around virtual memory exploding. The reason we took the jump early is there is a company wide push to move to JDK8 ASAP, I suspect this isn't something unique to LinkedIn. To get this to work with security enabled, we've had to apply patches not even in trunk yet because they break JDK6 compatibility. From my perspective, based on what I've seen and people I've talked to, there is a huge push to move to JDK8 ASAP so it's becoming increasingly urgent to at least get support to run on JDK8. On Wed, Sep 17, 2014 at 9:55 AM, Allen Wittenauer a...@altiscale.com wrote: On Sep 17, 2014, at 2:47 AM, Steve Loughran ste...@hortonworks.com wrote: I don't agree. Certainly the stuff I got into Hadoop 2.5 nailed down the filesystem binding with more tests than ever before. FWIW, based upon my survey of JIRA, there are a lot of unit test fixes that are only in trunk. But I am also aware of large organisations that are still on Java 6. Giving a clear roadmap move to Java 7 now, java 8 in XX months can help them plan. Planning is a big thing. That’s one of the reasons why it’d be prudent to start doing 3.0+JDK8 now as well. Even if April slips, other projects and orgs are already moving to 8. These people wonder what our plans are so that they can run one JVM. Right now our answer is ¯\_(ツ)_/¯ . I’m sure I can dig up a user running Hadoop 0.13 because it ran on JDK5. That doesn’t mean the open source project should stall because certain orgs don’t/can't upgrade. Drop the 2.6.0 release, branch trunk, and start rolling a 3.0.0-alpha with JDK8 as the minimum. 2.5.1 becomes the base for all sustaining work. This gives the rest of the community time to move to JDK8 if they haven’t already. For downstream vendors, it gives a roadmap for their customers who will be asking about JDK8 sooner rather than later. By the time 3.0 stabilizes, we’re probably looking at April, which is perfect timing. That delays getting stuff out too much; if april slips it becomes a long time since an ASF release came out. I’m assuming you specifically mean a ‘stable’ release. If, as everyone seems to be saying, that 3.x doesn’t have that much different than 2.x, doesn’t this mean that 3.x should be stable much quicker than 2.x took? In other words, if 2.5 is stable and the biggest differences between it and trunk is the majority of code (450+ JIRAs as of yesterday afternoon) that just also happens to be in 2.6, doesn’t it mean 2.6 is also extremely unstable? (Thus supporting my conjecture that 2.6 is going to be a problematic release?) Saying you must run on Java 8 for this will only scare people off and hold back adoption of 3.x, leaving 2.5 as the last release that ends up being used for a while; the new 1.0.4 From the outside, trunk looks a lot of 0.21 already. From what I can tell, there is zero motivation to get it out the door and on a roadmap. Primarily because there is little different between trunk and branch-2. This is a very dangerous place to be as those few differences, some measured in years old, rot and wither. :( Here's an alternative -2.6 on java 6, announce EOL for Java 6 support -2.7 on Java 7, state that the lifespan of j7 support will be some bounded time period (12-18 mo) -trunk to build and test on Java 8 in jenkins alongside java 7. For that to be useful someone needs to volunteer to care about build failures. are you volunteering, Allen?
Failed: YARN-2102 PreCommit Build #5086
Jira: https://issues.apache.org/jira/browse/YARN-2102 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5086/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3451 lines...] drwxr-xr-x 3 jenkins jenkins 4096 Aug 22 06:04 Trinidad%20Core%20Trunk%20(snapshot) drwxr-xr-x 3 jenkins jenkins 4096 Aug 26 02:08 Wink2-Trunk-JDK1.6 drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 10:30 ZooKeeper-trunk drwxr-xr-x 4 jenkins jenkins 4096 Aug 30 18:55 ZooKeeper-trunk-openjdk7 drwxr-xr-x 4 jenkins jenkins 4096 Jul 14 09:50 ZooKeeper_branch33 drwxr-xr-x 4 jenkins jenkins 4096 Sep 11 09:36 ZooKeeper_branch34_jdk7 drwxr-xr-x 4 jenkins jenkins 4096 Aug 31 09:56 ZooKeeper_branch35_jdk7 drwxr-xr-x 9 jenkins jenkins 4096 Sep 5 00:51 archiva-master-build drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 12:17 bookkeeper-trunk drwxr-xr-x 4 jenkins jenkins 4096 Sep 18 04:30 bookkeeper-trunk-find-patches-available drwxr-xr-x 35 jenkins jenkins 4096 Sep 9 22:33 cloudstack-master-maven drwxr-xr-x 35 jenkins jenkins 4096 Sep 9 21:53 cloudstack-rat-master drwxr-xr-x 5 jenkins jenkins 4096 Sep 17 00:26 core-integration-testing-maven-3-jdk-1.8 drwxr-xr-x 3 jenkins jenkins 4096 Sep 4 00:34 devicemap-java6 drwxr-xr-x 3 jenkins jenkins 4096 Sep 1 05:01 dir-studio-ubuntu-applications drwxr-xr-x 9 jenkins jenkins 4096 Aug 30 20:25 doxia drwxr-xr-x 23 jenkins jenkins 4096 Sep 13 23:06 gora-trunk drwxr-xr-x 16 jenkins jenkins 4096 Sep 7 04:01 incubator-brooklyn-master-build drwxr-xr-x 2 jenkins jenkins 4096 Aug 24 22:30 lib drwxr-xr-x 9 jenkins jenkins 4096 Aug 26 18:30 lightning-trunk drwxr-xr-x 20 jenkins jenkins 4096 Sep 14 04:02 mailbox drwxr-xr-x 17 jenkins jenkins 4096 Sep 22 00:26 maven-3.2-release-status-build drwxr-xr-x 45 jenkins jenkins 4096 Sep 14 20:30 maven-plugins drwxr-xr-x 45 jenkins jenkins 4096 Sep 11 20:16 maven-plugins-ITs-m2 drwxr-xr-x 45 jenkins jenkins 4096 Sep 6 21:19 maven-plugins-ITs-m3.1.x-with-maven-plugin-jdk-1.8 drwxr-xr-x 9 jenkins jenkins 4096 Sep 13 19:59 maven-release drwxr-xr-x 30 jenkins jenkins 4096 Sep 8 21:52 maven-shared drwxr-xr-x 17 jenkins jenkins 4096 Sep 13 22:00 maven-surefire drwxr-xr-x 9 jenkins jenkins 4096 Sep 23 00:26 maven-wagon-1.7 drwxr-xr-x 9 jenkins jenkins 4096 Sep 19 00:25 maven-wagon-with-ssh-embedded drwxr-xr-x 10 jenkins jenkins 4096 Sep 11 00:43 olingo-odata2 drwxr-xr-x 10 jenkins jenkins 4096 Sep 6 23:14 olingo-odata4 drwxr-xr-x 10 jenkins jenkins 4096 Sep 11 23:11 olingo-odata4-cobertura drwxr-xr-x 21 jenkins jenkins 4096 Sep 8 21:20 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 22 22:21 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Sep 2 22:15 oozie-trunk-precommit-build@2 drwxr-xr-x 14 jenkins jenkins 4096 Sep 13 22:28 openmeetings drwxr-xr-x 3 jenkins jenkins 4096 Sep 6 00:12 portals-jetspeed-2-portal-trunk drwxr-xr-x 9 jenkins jenkins 4096 Sep 9 12:10 sirona-1.8 drwxr-xr-x 10 jenkins jenkins 4096 Sep 11 23:27 sis-dev drwxr-xr-x 3 jenkins jenkins 4096 Sep 5 09:55 sling-contrib-1.6 drwxr-xr-x 3 jenkins jenkins 4096 Aug 26 09:54 sling-samples-1.6 drwxr-xr-x 3 jenkins jenkins 4096 Aug 25 06:48 sling-trunk-1.7 drwxr-xr-x 3 jenkins jenkins 4096 Aug 27 19:54 tika-trunk-jdk1.6 drwxr-xr-x 4 jenkins jenkins 4096 Sep 3 21:36 torque4-test-project-derby drwxr-xr-x 12 jenkins jenkins 4096 Sep 6 20:39 torque4-test-project-hsqldb mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-YARN-Build #4749 Archived 25 artifacts Archive block size is 32768 Received 0 blocks and 8986081 bytes Compression is 0.0% Took 2.3 sec [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
[jira] [Created] (YARN-2591) AHSWebServices should return FORBIDDEN(403) if the request user doesn't have access to the history data
Zhijie Shen created YARN-2591: - Summary: AHSWebServices should return FORBIDDEN(403) if the request user doesn't have access to the history data Key: YARN-2591 URL: https://issues.apache.org/jira/browse/YARN-2591 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Reporter: Zhijie Shen AHSWebServices should return FORBIDDEN(403) if the request user doesn't have access to the history data. Currently, it is going to return INTERNAL_SERVER_ERROR(500). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-2592) Preemption can kill containers to fulfil need of already over-capacity queue.
Eric Payne created YARN-2592:
Summary: Preemption can kill containers to fulfil need of already
over-capacity queue.
Key: YARN-2592
URL: https://issues.apache.org/jira/browse/YARN-2592
Project: Hadoop YARN
Issue Type: Bug
Affects Versions: 2.5.1, 3.0.0
Reporter: Eric Payne
There are scenarios in which one over-capacity queue can cause preemption of
another over-capacity queue. However, since killing containers may lose work,
it doesn't make sense to me to kill containers to feed an already over-capacity
queue.
Consider the following:
{code}
root has A,B,C, total capacity = 90
A.guaranteed = 30, A.pending = 5, A.current = 40
B.guaranteed = 30, B.pending = 0, B.current = 50
C.guaranteed = 30, C.pending = 0, C.current = 0
{code}
In this case, the queue preemption monitor will kill 5 resources from queue B
so that queue A can pick them up, even though queue A is already over its
capacity. This could lose any work that those containers in B had already done.
Is there a use case for this behavior? It seems to me that if a queue is
already over its capacity, it shouldn't destroy the work of other queues. If
the over-capacity queue needs more resources, that seems to be a problem that
should be solved by increasing its guarantee.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: Thinking ahead to hadoop-2.6
Hey Arun, Maybe we could do a quick run through of the Roadmap wiki and add/retarget things accordingly? I think the KMS and transparent encryption are ready to go. We've got a very few further bug fixes pending, but that's it. Two HDFS things that I think probably won't make the end of the week are archival storage (HDFS-6584) and single replica memory writes (HDFS-6581), which I believe are under the HSM banner. HDFS-6484 was just merged to trunk and I think needs a little more work before it goes into branch-2. HDFS-6581 hasn't even been merged to trunk yet, so seems a bit further off yet. Just my 2c as I did not work directly on these features. I just generally shy away from shipping bits quite this fresh. Thanks, Andrew On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy a...@hortonworks.com wrote: Looks like most of the content is in and hadoop-2.6 is shaping up nicely. I'll create branch-2.6 by end of the week and we can go from there to stabilize it - hopefully in the next few weeks. Thoughts? thanks, Arun On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, With hadoop-2.5 nearly done, it's time to start thinking ahead to hadoop-2.6. Currently, here is the Roadmap per the wiki: • HADOOP • Credential provider HADOOP-10607 • HDFS • Heterogeneous storage (Phase 2) - Support APIs for using storage tiers by the applications HDFS-5682 • Memory as storage tier HDFS-5851 • YARN • Dynamic Resource Configuration YARN-291 • NodeManager Restart YARN-1336 • ResourceManager HA Phase 2 YARN-556 • Support for admin-specified labels in YARN YARN-796 • Support for automatic, shared cache for YARN application artifacts YARN-1492 • Support NodeGroup layer topology on YARN YARN-18 • Support for Docker containers in YARN YARN-1964 • YARN service registry YARN-913 My suspicion is, as is normal, some will make the cut and some won't. Please do add/subtract from the list as appropriate. Ideally, it would be good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep up a cadence. More importantly, as we discussed previously, we'd like hadoop-2.6 to be the *last* Apache Hadoop 2.x release which support JDK6. I'll start a discussion with other communities (HBase, Pig, Hive, Oozie etc.) and see how they feel about this. thanks, Arun -- -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Failed: YARN-668 PreCommit Build #5087
Jira: https://issues.apache.org/jira/browse/YARN-668 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5087/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 4171 lines...] drwxr-xr-x 2 jenkins jenkins 4096 Sep 18 20:40 dev-support drwxr-xr-x 5 jenkins jenkins 4096 Sep 23 22:06 hadoop-assemblies drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 22:07 hadoop-client drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 21:59 hadoop-common-project drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 22:07 hadoop-dist drwxr-xr-x 6 jenkins jenkins 4096 Sep 23 22:01 hadoop-hdfs-project drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 22:04 hadoop-mapreduce-project drwxr-xr-x 5 jenkins jenkins 4096 Sep 23 22:06 hadoop-maven-plugins drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 22:07 hadoop-minicluster drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 21:58 hadoop-project drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 21:58 hadoop-project-dist drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 22:05 hadoop-tools drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 22:03 hadoop-yarn-project drwxr-xr-x 2 jenkins jenkins 4096 Sep 23 22:54 patchprocess -rw-r--r-- 1 jenkins jenkins 22465 Sep 6 03:11 pom.xml drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 22:12 target total 116 drwxr-xr-x 29 jenkins jenkins 4096 Sep 3 00:32 Apache-falcon drwxr-xr-x 12 jenkins jenkins 4096 Sep 13 07:42 HBase-0.94 drwxr-xr-x 22 jenkins jenkins 4096 Sep 23 17:05 HBase-TRUNK drwxr-xr-x 4 jenkins jenkins 4096 Jul 8 23:10 Hadoop-Common-0.23-Commit drwxr-xr-x 16 jenkins jenkins 4096 Aug 29 11:32 Hadoop-Hdfs-trunk drwxr-xr-x 4 jenkins jenkins 4096 Sep 19 06:00 Hadoop-trunk-java7 drwxr-xr-x 4 jenkins jenkins 4096 Sep 16 23:17 Pig-trunk-commit drwxr-xr-x 3 jenkins jenkins 4096 Sep 10 16:51 PreCommit-Admin drwxr-xr-x 4 jenkins jenkins 4096 Sep 17 18:52 PreCommit-Falcon-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 19 23:08 PreCommit-HADOOP-Build drwxr-xr-x 23 jenkins jenkins 4096 Sep 18 11:57 PreCommit-HBASE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 21:35 PreCommit-HDFS-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 02:54 PreCommit-HDFS-Build@2 drwxr-xr-x 19 jenkins jenkins 4096 Sep 22 17:00 PreCommit-MAPREDUCE-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 26 00:34 PreCommit-MAPREDUCE-Build@2 drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 22:56 PreCommit-YARN-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 31 01:57 PreCommit-YARN-Build@2 drwxr-xr-x 3 jenkins jenkins 4096 Sep 19 22:30 PreCommit-ZOOKEEPER-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 02:00 Tajo-0.8.1-nightly drwxr-xr-x 19 jenkins jenkins 4096 Sep 16 14:16 Tajo-block_iteration-branch-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 23 15:05 Tajo-master-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 9 02:01 Tajo-master-nightly drwxr-xr-x 17 jenkins jenkins 4096 Sep 19 07:42 Tez-Build drwxr-xr-x 4 jenkins jenkins 4096 Sep 22 08:30 ZooKeeper_branch34 drwxr-xr-x 10 jenkins jenkins 4096 Sep 20 14:06 cayenne-31 drwxr-xr-x 3 jenkins jenkins 4096 Jul 21 14:17 commons-vfs-trunk drwxr-xr-x 21 jenkins jenkins 4096 Sep 8 20:27 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 9 00:17 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Aug 26 00:36 oozie-trunk-precommit-build@2 mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-YARN-Build #4749 Archived 45 artifacts Archive block size is 32768 Received 0 blocks and 9503258 bytes Compression is 0.0% Took 2.9 sec [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
Re: Thinking ahead to hadoop-2.6
I actually would like to see both archival storage and single replica memory writes to be in 2.6 release. Archival storage is in the final stages of getting ready for branch-2 merge as Nicholas has already indicated on the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both of these features are being in development for sometime. On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang andrew.w...@cloudera.com wrote: Hey Arun, Maybe we could do a quick run through of the Roadmap wiki and add/retarget things accordingly? I think the KMS and transparent encryption are ready to go. We've got a very few further bug fixes pending, but that's it. Two HDFS things that I think probably won't make the end of the week are archival storage (HDFS-6584) and single replica memory writes (HDFS-6581), which I believe are under the HSM banner. HDFS-6484 was just merged to trunk and I think needs a little more work before it goes into branch-2. HDFS-6581 hasn't even been merged to trunk yet, so seems a bit further off yet. Just my 2c as I did not work directly on these features. I just generally shy away from shipping bits quite this fresh. Thanks, Andrew On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy a...@hortonworks.com wrote: Looks like most of the content is in and hadoop-2.6 is shaping up nicely. I'll create branch-2.6 by end of the week and we can go from there to stabilize it - hopefully in the next few weeks. Thoughts? thanks, Arun On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, With hadoop-2.5 nearly done, it's time to start thinking ahead to hadoop-2.6. Currently, here is the Roadmap per the wiki: • HADOOP • Credential provider HADOOP-10607 • HDFS • Heterogeneous storage (Phase 2) - Support APIs for using storage tiers by the applications HDFS-5682 • Memory as storage tier HDFS-5851 • YARN • Dynamic Resource Configuration YARN-291 • NodeManager Restart YARN-1336 • ResourceManager HA Phase 2 YARN-556 • Support for admin-specified labels in YARN YARN-796 • Support for automatic, shared cache for YARN application artifacts YARN-1492 • Support NodeGroup layer topology on YARN YARN-18 • Support for Docker containers in YARN YARN-1964 • YARN service registry YARN-913 My suspicion is, as is normal, some will make the cut and some won't. Please do add/subtract from the list as appropriate. Ideally, it would be good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep up a cadence. More importantly, as we discussed previously, we'd like hadoop-2.6 to be the *last* Apache Hadoop 2.x release which support JDK6. I'll start a discussion with other communities (HBase, Pig, Hive, Oozie etc.) and see how they feel about this. thanks, Arun -- -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- http://hortonworks.com/download/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Failed: YARN-2581 PreCommit Build #5088
Jira: https://issues.apache.org/jira/browse/YARN-2581 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5088/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3663 lines...] -rw-r--r-- 1 jenkins jenkins 1366 Sep 4 06:01 README.txt drwxr-xr-x 2 jenkins jenkins 4096 Sep 19 01:20 dev-support drwxr-xr-x 5 jenkins jenkins 4096 Sep 23 22:17 hadoop-assemblies drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 22:17 hadoop-client drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 22:10 hadoop-common-project drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 22:17 hadoop-dist drwxr-xr-x 6 jenkins jenkins 4096 Sep 23 22:12 hadoop-hdfs-project drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 22:15 hadoop-mapreduce-project drwxr-xr-x 5 jenkins jenkins 4096 Sep 23 22:17 hadoop-maven-plugins drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 22:17 hadoop-minicluster drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 22:08 hadoop-project drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 22:08 hadoop-project-dist drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 22:15 hadoop-tools drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 22:14 hadoop-yarn-project drwxr-xr-x 2 jenkins jenkins 4096 Sep 23 22:26 patchprocess -rw-r--r-- 1 jenkins jenkins 22465 Sep 5 22:03 pom.xml drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 22:20 target total 112 drwxr-xr-x 18 jenkins jenkins 4096 Sep 5 21:43 HADOOP2_Release_Artifacts_Builder drwxr-xr-x 11 jenkins jenkins 4096 Sep 19 17:33 HBase-0.94-JDK7 drwxr-xr-x 22 jenkins jenkins 4096 Aug 28 18:29 HBase-0.98 drwxr-xr-x 4 jenkins jenkins 4096 Sep 19 09:01 Hadoop-Common-0.23-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 22 04:12 Hadoop-Common-2-Commit drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 09:01 Hadoop-Common-trunk drwxr-xr-x 16 jenkins jenkins 4096 Sep 11 10:32 Hadoop-Yarn-trunk drwxr-xr-x 3 jenkins jenkins 4096 Sep 22 06:10 PreCommit-Admin drwxr-xr-x 4 jenkins jenkins 4096 Sep 19 09:00 PreCommit-Falcon-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 20:12 PreCommit-HADOOP-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 2 18:13 PreCommit-HADOOP-Build@2 drwxr-xr-x 22 jenkins jenkins 4096 Sep 16 08:00 PreCommit-HBASE-Build drwxr-xr-x 19 jenkins jenkins 4096 Sep 23 21:40 PreCommit-HDFS-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 29 00:39 PreCommit-HDFS-Build@2 drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 21:34 PreCommit-MAPREDUCE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 23:03 PreCommit-YARN-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 29 05:08 PreCommit-YARN-Build@2 drwxr-xr-x 3 jenkins jenkins 4096 Sep 11 10:10 PreCommit-ZOOKEEPER-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 26 16:43 Samza-0.7.0 drwxr-xr-x 18 jenkins jenkins 4096 Sep 15 02:01 Tajo-0.8.1-nightly drwxr-xr-x 19 jenkins jenkins 4096 Sep 11 01:59 Tajo-master-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 6 02:01 Tajo-master-nightly drwxr-xr-x 16 jenkins jenkins 4096 Sep 21 22:49 Tez-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 08:31 ZooKeeper_branch34 drwxr-xr-x 16 jenkins jenkins 4096 Sep 11 12:18 bookkeeper-trunk drwxr-xr-x 21 jenkins jenkins 4096 Sep 6 00:11 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 5 22:31 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Aug 26 00:45 oozie-trunk-precommit-build@2 mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-YARN-Build #4749 Archived 25 artifacts Archive block size is 32768 Received 0 blocks and 9071091 bytes Compression is 0.0% Took 5.6 sec [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
Re: Thinking ahead to hadoop-2.6
I would like to see the shared cache (YARN-1492) make it in as well. We are going through the final review process (with Karthik and Vinod) and should be fairly close to complete. This is another feature that has been under development for a while. Thanks, Chris On Tue, Sep 23, 2014 at 4:00 PM, Suresh Srinivas sur...@hortonworks.com wrote: I actually would like to see both archival storage and single replica memory writes to be in 2.6 release. Archival storage is in the final stages of getting ready for branch-2 merge as Nicholas has already indicated on the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both of these features are being in development for sometime. On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang andrew.w...@cloudera.com wrote: Hey Arun, Maybe we could do a quick run through of the Roadmap wiki and add/retarget things accordingly? I think the KMS and transparent encryption are ready to go. We've got a very few further bug fixes pending, but that's it. Two HDFS things that I think probably won't make the end of the week are archival storage (HDFS-6584) and single replica memory writes (HDFS-6581), which I believe are under the HSM banner. HDFS-6484 was just merged to trunk and I think needs a little more work before it goes into branch-2. HDFS-6581 hasn't even been merged to trunk yet, so seems a bit further off yet. Just my 2c as I did not work directly on these features. I just generally shy away from shipping bits quite this fresh. Thanks, Andrew On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy a...@hortonworks.com wrote: Looks like most of the content is in and hadoop-2.6 is shaping up nicely. I'll create branch-2.6 by end of the week and we can go from there to stabilize it - hopefully in the next few weeks. Thoughts? thanks, Arun On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, With hadoop-2.5 nearly done, it's time to start thinking ahead to hadoop-2.6. Currently, here is the Roadmap per the wiki: • HADOOP • Credential provider HADOOP-10607 • HDFS • Heterogeneous storage (Phase 2) - Support APIs for using storage tiers by the applications HDFS-5682 • Memory as storage tier HDFS-5851 • YARN • Dynamic Resource Configuration YARN-291 • NodeManager Restart YARN-1336 • ResourceManager HA Phase 2 YARN-556 • Support for admin-specified labels in YARN YARN-796 • Support for automatic, shared cache for YARN application artifacts YARN-1492 • Support NodeGroup layer topology on YARN YARN-18 • Support for Docker containers in YARN YARN-1964 • YARN service registry YARN-913 My suspicion is, as is normal, some will make the cut and some won't. Please do add/subtract from the list as appropriate. Ideally, it would be good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep up a cadence. More importantly, as we discussed previously, we'd like hadoop-2.6 to be the *last* Apache Hadoop 2.x release which support JDK6. I'll start a discussion with other communities (HBase, Pig, Hive, Oozie etc.) and see how they feel about this. thanks, Arun -- -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- http://hortonworks.com/download/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Thinking ahead to hadoop-2.6
I think we should delay cutting the branch beyond the end of the week then, seeing as the HDFS-6581 merge vote (assuming it goes through) doesn't close until 12am on Tuesday next week. Archival storage is also still finishing up user APIs and I had a few other q's posted on HDFS-7081, so again might not make it by the end of the week. I don't think it'd be too bad to retarget these to 2.7 though, since Arun said he plans to roll 2.7 shortly after 2.6. I think a few more weeks would really seal the deal. Thanks, Andrew On Tue, Sep 23, 2014 at 4:00 PM, Suresh Srinivas sur...@hortonworks.com wrote: I actually would like to see both archival storage and single replica memory writes to be in 2.6 release. Archival storage is in the final stages of getting ready for branch-2 merge as Nicholas has already indicated on the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both of these features are being in development for sometime. On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang andrew.w...@cloudera.com wrote: Hey Arun, Maybe we could do a quick run through of the Roadmap wiki and add/retarget things accordingly? I think the KMS and transparent encryption are ready to go. We've got a very few further bug fixes pending, but that's it. Two HDFS things that I think probably won't make the end of the week are archival storage (HDFS-6584) and single replica memory writes (HDFS-6581), which I believe are under the HSM banner. HDFS-6484 was just merged to trunk and I think needs a little more work before it goes into branch-2. HDFS-6581 hasn't even been merged to trunk yet, so seems a bit further off yet. Just my 2c as I did not work directly on these features. I just generally shy away from shipping bits quite this fresh. Thanks, Andrew On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy a...@hortonworks.com wrote: Looks like most of the content is in and hadoop-2.6 is shaping up nicely. I'll create branch-2.6 by end of the week and we can go from there to stabilize it - hopefully in the next few weeks. Thoughts? thanks, Arun On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, With hadoop-2.5 nearly done, it's time to start thinking ahead to hadoop-2.6. Currently, here is the Roadmap per the wiki: • HADOOP • Credential provider HADOOP-10607 • HDFS • Heterogeneous storage (Phase 2) - Support APIs for using storage tiers by the applications HDFS-5682 • Memory as storage tier HDFS-5851 • YARN • Dynamic Resource Configuration YARN-291 • NodeManager Restart YARN-1336 • ResourceManager HA Phase 2 YARN-556 • Support for admin-specified labels in YARN YARN-796 • Support for automatic, shared cache for YARN application artifacts YARN-1492 • Support NodeGroup layer topology on YARN YARN-18 • Support for Docker containers in YARN YARN-1964 • YARN service registry YARN-913 My suspicion is, as is normal, some will make the cut and some won't. Please do add/subtract from the list as appropriate. Ideally, it would be good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep up a cadence. More importantly, as we discussed previously, we'd like hadoop-2.6 to be the *last* Apache Hadoop 2.x release which support JDK6. I'll start a discussion with other communities (HBase, Pig, Hive, Oozie etc.) and see how they feel about this. thanks, Arun -- -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- http://hortonworks.com/download/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have
Re: Thinking ahead to hadoop-2.6
Looking at the patches, we might be able to get most of YARN-1492 in by the end of next week. There would be a couple of security items still remaining, but we can may be call the feature alpha-ready without them. On Tue, Sep 23, 2014 at 4:25 PM, Chris Trezzo ctre...@gmail.com wrote: I would like to see the shared cache (YARN-1492) make it in as well. We are going through the final review process (with Karthik and Vinod) and should be fairly close to complete. This is another feature that has been under development for a while. Thanks, Chris On Tue, Sep 23, 2014 at 4:00 PM, Suresh Srinivas sur...@hortonworks.com wrote: I actually would like to see both archival storage and single replica memory writes to be in 2.6 release. Archival storage is in the final stages of getting ready for branch-2 merge as Nicholas has already indicated on the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both of these features are being in development for sometime. On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang andrew.w...@cloudera.com wrote: Hey Arun, Maybe we could do a quick run through of the Roadmap wiki and add/retarget things accordingly? I think the KMS and transparent encryption are ready to go. We've got a very few further bug fixes pending, but that's it. Two HDFS things that I think probably won't make the end of the week are archival storage (HDFS-6584) and single replica memory writes (HDFS-6581), which I believe are under the HSM banner. HDFS-6484 was just merged to trunk and I think needs a little more work before it goes into branch-2. HDFS-6581 hasn't even been merged to trunk yet, so seems a bit further off yet. Just my 2c as I did not work directly on these features. I just generally shy away from shipping bits quite this fresh. Thanks, Andrew On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy a...@hortonworks.com wrote: Looks like most of the content is in and hadoop-2.6 is shaping up nicely. I'll create branch-2.6 by end of the week and we can go from there to stabilize it - hopefully in the next few weeks. Thoughts? thanks, Arun On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, With hadoop-2.5 nearly done, it's time to start thinking ahead to hadoop-2.6. Currently, here is the Roadmap per the wiki: • HADOOP • Credential provider HADOOP-10607 • HDFS • Heterogeneous storage (Phase 2) - Support APIs for using storage tiers by the applications HDFS-5682 • Memory as storage tier HDFS-5851 • YARN • Dynamic Resource Configuration YARN-291 • NodeManager Restart YARN-1336 • ResourceManager HA Phase 2 YARN-556 • Support for admin-specified labels in YARN YARN-796 • Support for automatic, shared cache for YARN application artifacts YARN-1492 • Support NodeGroup layer topology on YARN YARN-18 • Support for Docker containers in YARN YARN-1964 • YARN service registry YARN-913 My suspicion is, as is normal, some will make the cut and some won't. Please do add/subtract from the list as appropriate. Ideally, it would be good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep up a cadence. More importantly, as we discussed previously, we'd like hadoop-2.6 to be the *last* Apache Hadoop 2.x release which support JDK6. I'll start a discussion with other communities (HBase, Pig, Hive, Oozie etc.) and see how they feel about this. thanks, Arun -- -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- http://hortonworks.com/download/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended
Failed: YARN-2161 PreCommit Build #5089
Jira: https://issues.apache.org/jira/browse/YARN-2161 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5089/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3222 lines...] drwxr-xr-x 2 jenkins jenkins 4096 Sep 18 20:40 dev-support drwxr-xr-x 5 jenkins jenkins 4096 Sep 23 23:36 hadoop-assemblies drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 23:36 hadoop-client drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 23:29 hadoop-common-project drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 23:36 hadoop-dist drwxr-xr-x 6 jenkins jenkins 4096 Sep 23 23:31 hadoop-hdfs-project drwxr-xr-x 10 jenkins jenkins 4096 Sep 23 23:34 hadoop-mapreduce-project drwxr-xr-x 5 jenkins jenkins 4096 Sep 23 23:36 hadoop-maven-plugins drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 23:36 hadoop-minicluster drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 23:28 hadoop-project drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 23:28 hadoop-project-dist drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 23:35 hadoop-tools drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 23:33 hadoop-yarn-project drwxr-xr-x 2 jenkins jenkins 4096 Sep 23 23:38 patchprocess -rw-r--r-- 1 jenkins jenkins 22465 Sep 6 03:11 pom.xml drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 23:37 target total 116 drwxr-xr-x 29 jenkins jenkins 4096 Sep 3 00:32 Apache-falcon drwxr-xr-x 12 jenkins jenkins 4096 Sep 13 07:42 HBase-0.94 drwxr-xr-x 22 jenkins jenkins 4096 Sep 23 17:05 HBase-TRUNK drwxr-xr-x 4 jenkins jenkins 4096 Jul 8 23:10 Hadoop-Common-0.23-Commit drwxr-xr-x 16 jenkins jenkins 4096 Aug 29 11:32 Hadoop-Hdfs-trunk drwxr-xr-x 4 jenkins jenkins 4096 Sep 19 06:00 Hadoop-trunk-java7 drwxr-xr-x 4 jenkins jenkins 4096 Sep 16 23:17 Pig-trunk-commit drwxr-xr-x 3 jenkins jenkins 4096 Sep 10 16:51 PreCommit-Admin drwxr-xr-x 4 jenkins jenkins 4096 Sep 17 18:52 PreCommit-Falcon-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 19 23:08 PreCommit-HADOOP-Build drwxr-xr-x 23 jenkins jenkins 4096 Sep 18 11:57 PreCommit-HBASE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 21:35 PreCommit-HDFS-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 02:54 PreCommit-HDFS-Build@2 drwxr-xr-x 19 jenkins jenkins 4096 Sep 22 17:00 PreCommit-MAPREDUCE-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 26 00:34 PreCommit-MAPREDUCE-Build@2 drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 23:42 PreCommit-YARN-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 31 01:57 PreCommit-YARN-Build@2 drwxr-xr-x 3 jenkins jenkins 4096 Sep 19 22:30 PreCommit-ZOOKEEPER-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 02:00 Tajo-0.8.1-nightly drwxr-xr-x 19 jenkins jenkins 4096 Sep 16 14:16 Tajo-block_iteration-branch-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 23 15:05 Tajo-master-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 9 02:01 Tajo-master-nightly drwxr-xr-x 17 jenkins jenkins 4096 Sep 19 07:42 Tez-Build drwxr-xr-x 4 jenkins jenkins 4096 Sep 22 08:30 ZooKeeper_branch34 drwxr-xr-x 10 jenkins jenkins 4096 Sep 20 14:06 cayenne-31 drwxr-xr-x 3 jenkins jenkins 4096 Jul 21 14:17 commons-vfs-trunk drwxr-xr-x 21 jenkins jenkins 4096 Sep 8 20:27 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 9 00:17 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Aug 26 00:36 oozie-trunk-precommit-build@2 mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-YARN-Build #4749 Archived 15 artifacts Archive block size is 32768 Received 0 blocks and 8542661 bytes Compression is 0.0% Took 2.8 sec [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
Re: Thinking ahead to hadoop-2.6
+1 for end of next week. We have got all the patches for YARN-1051 committed to the branch. We are currently fixing test-patch plan to call a merge vote soon. Hopefully it will go through by end of next week. Thanks, Subru On Tue, Sep 23, 2014 at 4:31 PM, Karthik Kambatla ka...@cloudera.com wrote: Looking at the patches, we might be able to get most of YARN-1492 in by the end of next week. There would be a couple of security items still remaining, but we can may be call the feature alpha-ready without them. On Tue, Sep 23, 2014 at 4:25 PM, Chris Trezzo ctre...@gmail.com wrote: I would like to see the shared cache (YARN-1492) make it in as well. We are going through the final review process (with Karthik and Vinod) and should be fairly close to complete. This is another feature that has been under development for a while. Thanks, Chris On Tue, Sep 23, 2014 at 4:00 PM, Suresh Srinivas sur...@hortonworks.com wrote: I actually would like to see both archival storage and single replica memory writes to be in 2.6 release. Archival storage is in the final stages of getting ready for branch-2 merge as Nicholas has already indicated on the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both of these features are being in development for sometime. On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang andrew.w...@cloudera.com wrote: Hey Arun, Maybe we could do a quick run through of the Roadmap wiki and add/retarget things accordingly? I think the KMS and transparent encryption are ready to go. We've got a very few further bug fixes pending, but that's it. Two HDFS things that I think probably won't make the end of the week are archival storage (HDFS-6584) and single replica memory writes (HDFS-6581), which I believe are under the HSM banner. HDFS-6484 was just merged to trunk and I think needs a little more work before it goes into branch-2. HDFS-6581 hasn't even been merged to trunk yet, so seems a bit further off yet. Just my 2c as I did not work directly on these features. I just generally shy away from shipping bits quite this fresh. Thanks, Andrew On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy a...@hortonworks.com wrote: Looks like most of the content is in and hadoop-2.6 is shaping up nicely. I'll create branch-2.6 by end of the week and we can go from there to stabilize it - hopefully in the next few weeks. Thoughts? thanks, Arun On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, With hadoop-2.5 nearly done, it's time to start thinking ahead to hadoop-2.6. Currently, here is the Roadmap per the wiki: • HADOOP • Credential provider HADOOP-10607 • HDFS • Heterogeneous storage (Phase 2) - Support APIs for using storage tiers by the applications HDFS-5682 • Memory as storage tier HDFS-5851 • YARN • Dynamic Resource Configuration YARN-291 • NodeManager Restart YARN-1336 • ResourceManager HA Phase 2 YARN-556 • Support for admin-specified labels in YARN YARN-796 • Support for automatic, shared cache for YARN application artifacts YARN-1492 • Support NodeGroup layer topology on YARN YARN-18 • Support for Docker containers in YARN YARN-1964 • YARN service registry YARN-913 My suspicion is, as is normal, some will make the cut and some won't. Please do add/subtract from the list as appropriate. Ideally, it would be good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep up a cadence. More importantly, as we discussed previously, we'd like hadoop-2.6 to be the *last* Apache Hadoop 2.x release which support JDK6. I'll start a discussion with other communities (HBase, Pig, Hive, Oozie etc.) and see how they feel about this. thanks, Arun -- -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this
Re: Thinking ahead to hadoop-2.6
I am also +1 for the end of next week. YARN-1492 (shared cache) is near the final stages of the review, and with a little more time it would make it easier to put it in. On Tue, Sep 23, 2014 at 4:48 PM, Subramaniam V K subru...@gmail.com wrote: +1 for end of next week. We have got all the patches for YARN-1051 committed to the branch. We are currently fixing test-patch plan to call a merge vote soon. Hopefully it will go through by end of next week. Thanks, Subru On Tue, Sep 23, 2014 at 4:31 PM, Karthik Kambatla ka...@cloudera.com wrote: Looking at the patches, we might be able to get most of YARN-1492 in by the end of next week. There would be a couple of security items still remaining, but we can may be call the feature alpha-ready without them. On Tue, Sep 23, 2014 at 4:25 PM, Chris Trezzo ctre...@gmail.com wrote: I would like to see the shared cache (YARN-1492) make it in as well. We are going through the final review process (with Karthik and Vinod) and should be fairly close to complete. This is another feature that has been under development for a while. Thanks, Chris On Tue, Sep 23, 2014 at 4:00 PM, Suresh Srinivas sur...@hortonworks.com wrote: I actually would like to see both archival storage and single replica memory writes to be in 2.6 release. Archival storage is in the final stages of getting ready for branch-2 merge as Nicholas has already indicated on the dev mailing list. Hopefully HDFS-6581 gets ready sooner. Both of these features are being in development for sometime. On Tue, Sep 23, 2014 at 3:27 PM, Andrew Wang andrew.w...@cloudera.com wrote: Hey Arun, Maybe we could do a quick run through of the Roadmap wiki and add/retarget things accordingly? I think the KMS and transparent encryption are ready to go. We've got a very few further bug fixes pending, but that's it. Two HDFS things that I think probably won't make the end of the week are archival storage (HDFS-6584) and single replica memory writes (HDFS-6581), which I believe are under the HSM banner. HDFS-6484 was just merged to trunk and I think needs a little more work before it goes into branch-2. HDFS-6581 hasn't even been merged to trunk yet, so seems a bit further off yet. Just my 2c as I did not work directly on these features. I just generally shy away from shipping bits quite this fresh. Thanks, Andrew On Tue, Sep 23, 2014 at 3:03 PM, Arun Murthy a...@hortonworks.com wrote: Looks like most of the content is in and hadoop-2.6 is shaping up nicely. I'll create branch-2.6 by end of the week and we can go from there to stabilize it - hopefully in the next few weeks. Thoughts? thanks, Arun On Tue, Aug 12, 2014 at 1:34 PM, Arun C Murthy a...@hortonworks.com wrote: Folks, With hadoop-2.5 nearly done, it's time to start thinking ahead to hadoop-2.6. Currently, here is the Roadmap per the wiki: * HADOOP * Credential provider HADOOP-10607 * HDFS * Heterogeneous storage (Phase 2) - Support APIs for using storage tiers by the applications HDFS-5682 * Memory as storage tier HDFS-5851 * YARN * Dynamic Resource Configuration YARN-291 * NodeManager Restart YARN-1336 * ResourceManager HA Phase 2 YARN-556 * Support for admin-specified labels in YARN YARN-796 * Support for automatic, shared cache for YARN application artifacts YARN-1492 * Support NodeGroup layer topology on YARN YARN-18 * Support for Docker containers in YARN YARN-1964 * YARN service registry YARN-913 My suspicion is, as is normal, some will make the cut and some won't. Please do add/subtract from the list as appropriate. Ideally, it would be good to ship hadoop-2.6 in a 6-8 weeks (say, October) to keep up a cadence. More importantly, as we discussed previously, we'd like hadoop-2.6 to be the *last* Apache Hadoop 2.x release which support JDK6. I'll start a discussion with other communities (HBase, Pig, Hive, Oozie etc.) and see how they feel about this. thanks, Arun -- -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity
Failed: YARN-2056 PreCommit Build #5091
Jira: https://issues.apache.org/jira/browse/YARN-2056 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5091/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3193 lines...] -rw-r--r-- 1 jenkins jenkins 1366 Sep 4 06:01 README.txt drwxr-xr-x 2 jenkins jenkins 4096 Sep 19 01:20 dev-support drwxr-xr-x 5 jenkins jenkins 4096 Sep 24 00:27 hadoop-assemblies drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:27 hadoop-client drwxr-xr-x 10 jenkins jenkins 4096 Sep 24 00:20 hadoop-common-project drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:27 hadoop-dist drwxr-xr-x 6 jenkins jenkins 4096 Sep 24 00:22 hadoop-hdfs-project drwxr-xr-x 10 jenkins jenkins 4096 Sep 24 00:25 hadoop-mapreduce-project drwxr-xr-x 5 jenkins jenkins 4096 Sep 24 00:27 hadoop-maven-plugins drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:27 hadoop-minicluster drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:19 hadoop-project drwxr-xr-x 3 jenkins jenkins 4096 Sep 24 00:19 hadoop-project-dist drwxr-xr-x 16 jenkins jenkins 4096 Sep 24 00:25 hadoop-tools drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:24 hadoop-yarn-project drwxr-xr-x 2 jenkins jenkins 4096 Sep 24 00:28 patchprocess -rw-r--r-- 1 jenkins jenkins 22465 Sep 5 22:03 pom.xml drwxr-xr-x 3 jenkins jenkins 4096 Sep 24 00:28 target total 112 drwxr-xr-x 18 jenkins jenkins 4096 Sep 5 21:43 HADOOP2_Release_Artifacts_Builder drwxr-xr-x 11 jenkins jenkins 4096 Sep 19 17:33 HBase-0.94-JDK7 drwxr-xr-x 22 jenkins jenkins 4096 Aug 28 18:29 HBase-0.98 drwxr-xr-x 4 jenkins jenkins 4096 Sep 19 09:01 Hadoop-Common-0.23-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 22 04:12 Hadoop-Common-2-Commit drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 09:01 Hadoop-Common-trunk drwxr-xr-x 16 jenkins jenkins 4096 Sep 11 10:32 Hadoop-Yarn-trunk drwxr-xr-x 3 jenkins jenkins 4096 Sep 22 06:10 PreCommit-Admin drwxr-xr-x 4 jenkins jenkins 4096 Sep 19 09:00 PreCommit-Falcon-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 20:12 PreCommit-HADOOP-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 2 18:13 PreCommit-HADOOP-Build@2 drwxr-xr-x 22 jenkins jenkins 4096 Sep 16 08:00 PreCommit-HBASE-Build drwxr-xr-x 19 jenkins jenkins 4096 Sep 23 21:40 PreCommit-HDFS-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 29 00:39 PreCommit-HDFS-Build@2 drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 21:34 PreCommit-MAPREDUCE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 24 00:51 PreCommit-YARN-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 29 05:08 PreCommit-YARN-Build@2 drwxr-xr-x 3 jenkins jenkins 4096 Sep 11 10:10 PreCommit-ZOOKEEPER-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 26 16:43 Samza-0.7.0 drwxr-xr-x 18 jenkins jenkins 4096 Sep 15 02:01 Tajo-0.8.1-nightly drwxr-xr-x 19 jenkins jenkins 4096 Sep 11 01:59 Tajo-master-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 6 02:01 Tajo-master-nightly drwxr-xr-x 16 jenkins jenkins 4096 Sep 21 22:49 Tez-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 08:31 ZooKeeper_branch34 drwxr-xr-x 16 jenkins jenkins 4096 Sep 11 12:18 bookkeeper-trunk drwxr-xr-x 21 jenkins jenkins 4096 Sep 6 00:11 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 5 22:31 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Aug 26 00:45 oozie-trunk-precommit-build@2 mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-YARN-Build #4749 Archived 15 artifacts Archive block size is 32768 Received 0 blocks and 8598379 bytes Compression is 0.0% Took 2.2 sec [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
Failed: YARN-2179 PreCommit Build #5092
Jira: https://issues.apache.org/jira/browse/YARN-2179 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5092/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3408 lines...] drwxr-xr-x 10 jenkins jenkins 4096 Sep 24 01:00 hadoop-common-project drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 01:08 hadoop-dist drwxr-xr-x 6 jenkins jenkins 4096 Sep 24 01:02 hadoop-hdfs-project drwxr-xr-x 10 jenkins jenkins 4096 Sep 24 01:05 hadoop-mapreduce-project drwxr-xr-x 5 jenkins jenkins 4096 Sep 24 01:07 hadoop-maven-plugins drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 01:07 hadoop-minicluster drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:59 hadoop-project drwxr-xr-x 3 jenkins jenkins 4096 Sep 24 00:59 hadoop-project-dist drwxr-xr-x 16 jenkins jenkins 4096 Sep 24 01:06 hadoop-tools drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 01:04 hadoop-yarn-project drwxr-xr-x 2 jenkins jenkins 4096 Sep 24 01:13 patchprocess -rw-r--r-- 1 jenkins jenkins 22465 Sep 8 15:02 pom.xml drwxr-xr-x 3 jenkins jenkins 4096 Sep 24 01:11 target total 128 drwxr-xr-x 31 jenkins jenkins 4096 Sep 11 00:30 Apache-falcon drwxr-xr-x 12 jenkins jenkins 4096 Aug 25 19:32 HBase-0.94 drwxr-xr-x 12 jenkins jenkins 4096 Sep 10 02:36 HBase-0.94-JDK7 drwxr-xr-x 22 jenkins jenkins 4096 Sep 19 17:28 HBase-0.98 drwxr-xr-x 22 jenkins jenkins 4096 Sep 19 19:12 HBase-TRUNK drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 09:02 Hadoop-Common-0.23-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 01:33 Hadoop-Common-2-Build drwxr-xr-x 16 jenkins jenkins 4096 Sep 13 11:31 Hadoop-Hdfs-trunk drwxr-xr-x 16 jenkins jenkins 4096 Sep 13 10:31 Hadoop-Yarn-trunk drwxr-xr-x 4 jenkins jenkins 4096 Aug 28 05:02 Hadoop-branch2 drwxr-xr-x 4 jenkins jenkins 4096 Sep 3 06:02 Hadoop-trunk-java7 drwxr-xr-x 4 jenkins jenkins 4096 Aug 27 01:19 Pig-trunk-commit drwxr-xr-x 3 jenkins jenkins 4096 Sep 22 07:00 PreCommit-Admin drwxr-xr-x 4 jenkins jenkins 4096 Sep 23 07:12 PreCommit-Falcon-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 28 14:22 PreCommit-Falcon-Build@2 drwxr-xr-x 19 jenkins jenkins 4096 Sep 18 22:33 PreCommit-HADOOP-Build drwxr-xr-x 23 jenkins jenkins 4096 Sep 17 22:12 PreCommit-HBASE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 24 00:21 PreCommit-HDFS-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 29 01:29 PreCommit-HDFS-Build@2 drwxr-xr-x 18 jenkins jenkins 4096 Sep 17 23:44 PreCommit-MAPREDUCE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 24 01:13 PreCommit-YARN-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 29 21:58 PreCommit-YARN-Build@2 drwxr-xr-x 3 jenkins jenkins 4096 Sep 23 00:22 PreCommit-ZOOKEEPER-Build drwxr-xr-x 17 jenkins jenkins 4096 Sep 17 23:45 Samza-0.7.0 drwxr-xr-x 18 jenkins jenkins 4096 Sep 13 02:02 Tajo-0.8.1-nightly drwxr-xr-x 19 jenkins jenkins 4096 Sep 19 09:00 Tajo-master-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 22 02:01 Tajo-master-nightly drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 05:50 Tez-Build drwxr-xr-x 16 jenkins jenkins 4096 Sep 23 16:01 bookkeeper-trunk drwxr-xr-x 21 jenkins jenkins 4096 Sep 8 19:31 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 8 23:13 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Aug 26 01:35 oozie-trunk-precommit-build@2 mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-YARN-Build #4749 Archived 30 artifacts Archive block size is 32768 Received 0 blocks and 9072510 bytes Compression is 0.0% Took 2.6 sec [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
Failed: YARN-668 PreCommit Build #5090
Jira: https://issues.apache.org/jira/browse/YARN-668 Build: https://builds.apache.org/job/PreCommit-YARN-Build/5090/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 4135 lines...] drwxr-xr-x 2 jenkins jenkins 4096 Sep 18 20:40 dev-support drwxr-xr-x 5 jenkins jenkins 4096 Sep 24 00:26 hadoop-assemblies drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:27 hadoop-client drwxr-xr-x 10 jenkins jenkins 4096 Sep 24 00:19 hadoop-common-project drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:27 hadoop-dist drwxr-xr-x 6 jenkins jenkins 4096 Sep 24 00:21 hadoop-hdfs-project drwxr-xr-x 10 jenkins jenkins 4096 Sep 24 00:24 hadoop-mapreduce-project drwxr-xr-x 5 jenkins jenkins 4096 Sep 24 00:26 hadoop-maven-plugins drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:27 hadoop-minicluster drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:18 hadoop-project drwxr-xr-x 3 jenkins jenkins 4096 Sep 24 00:18 hadoop-project-dist drwxr-xr-x 16 jenkins jenkins 4096 Sep 24 00:25 hadoop-tools drwxr-xr-x 4 jenkins jenkins 4096 Sep 24 00:24 hadoop-yarn-project drwxr-xr-x 2 jenkins jenkins 4096 Sep 24 01:14 patchprocess -rw-r--r-- 1 jenkins jenkins 22465 Sep 6 03:11 pom.xml drwxr-xr-x 3 jenkins jenkins 4096 Sep 24 00:32 target total 116 drwxr-xr-x 29 jenkins jenkins 4096 Sep 3 00:32 Apache-falcon drwxr-xr-x 12 jenkins jenkins 4096 Sep 13 07:42 HBase-0.94 drwxr-xr-x 22 jenkins jenkins 4096 Sep 23 17:05 HBase-TRUNK drwxr-xr-x 4 jenkins jenkins 4096 Jul 8 23:10 Hadoop-Common-0.23-Commit drwxr-xr-x 16 jenkins jenkins 4096 Aug 29 11:32 Hadoop-Hdfs-trunk drwxr-xr-x 4 jenkins jenkins 4096 Sep 19 06:00 Hadoop-trunk-java7 drwxr-xr-x 4 jenkins jenkins 4096 Sep 16 23:17 Pig-trunk-commit drwxr-xr-x 3 jenkins jenkins 4096 Sep 10 16:51 PreCommit-Admin drwxr-xr-x 4 jenkins jenkins 4096 Sep 17 18:52 PreCommit-Falcon-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 19 23:08 PreCommit-HADOOP-Build drwxr-xr-x 23 jenkins jenkins 4096 Sep 18 11:57 PreCommit-HBASE-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 21:35 PreCommit-HDFS-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 23 02:54 PreCommit-HDFS-Build@2 drwxr-xr-x 19 jenkins jenkins 4096 Sep 22 17:00 PreCommit-MAPREDUCE-Build drwxr-xr-x 4 jenkins jenkins 4096 Aug 26 00:34 PreCommit-MAPREDUCE-Build@2 drwxr-xr-x 18 jenkins jenkins 4096 Sep 24 01:17 PreCommit-YARN-Build drwxr-xr-x 17 jenkins jenkins 4096 Aug 31 01:57 PreCommit-YARN-Build@2 drwxr-xr-x 3 jenkins jenkins 4096 Sep 19 22:30 PreCommit-ZOOKEEPER-Build drwxr-xr-x 18 jenkins jenkins 4096 Sep 23 02:00 Tajo-0.8.1-nightly drwxr-xr-x 19 jenkins jenkins 4096 Sep 16 14:16 Tajo-block_iteration-branch-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 23 15:05 Tajo-master-build drwxr-xr-x 19 jenkins jenkins 4096 Sep 9 02:01 Tajo-master-nightly drwxr-xr-x 17 jenkins jenkins 4096 Sep 19 07:42 Tez-Build drwxr-xr-x 4 jenkins jenkins 4096 Sep 22 08:30 ZooKeeper_branch34 drwxr-xr-x 10 jenkins jenkins 4096 Sep 20 14:06 cayenne-31 drwxr-xr-x 3 jenkins jenkins 4096 Jul 21 14:17 commons-vfs-trunk drwxr-xr-x 21 jenkins jenkins 4096 Sep 8 20:27 oozie-trunk-find-patches-available drwxr-xr-x 22 jenkins jenkins 4096 Sep 9 00:17 oozie-trunk-precommit-build drwxr-xr-x 22 jenkins jenkins 4096 Aug 26 00:36 oozie-trunk-precommit-build@2 mv: cannot stat '/home/jenkins/jenkins-slave/workspace/PreCommit-YARN-Build/../patchprocess': No such file or directory Build step 'Execute shell' marked build as failure Archiving artifacts Sending artifact delta relative to PreCommit-YARN-Build #4749 Archived 45 artifacts Archive block size is 32768 Received 0 blocks and 9501968 bytes Compression is 0.0% Took 2.8 sec [description-setter] Could not determine description. Recording test results Email was triggered for: Failure Sending email for trigger: Failure ### ## FAILED TESTS (if any) ## No tests ran.
[jira] [Created] (YARN-2593) Many tests get failed on trunk
Junping Du created YARN-2593: Summary: Many tests get failed on trunk Key: YARN-2593 URL: https://issues.apache.org/jira/browse/YARN-2593 Project: Hadoop YARN Issue Type: Bug Reporter: Junping Du Assignee: Junping Du Priority: Blocker From YARN-668, we can see many test failures there. I already verified that trunk branch can repro these failures. org.apache.hadoop.mapreduce.v2.app.rm.TestRMContainerAllocator org.apache.hadoop.yarn.client.TestResourceTrackerOnHA org.apache.hadoop.yarn.client.TestApplicationMasterServiceOnHA org.apache.hadoop.yarn.client.api.impl.TestAMRMClientOnRMRestart org.apache.hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart org.apache.hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA org.apache.hadoop.yarn.server.resourcemanager.TestApplicationCleanup org.apache.hadoop.yarn.server.resourcemanager.TestContainerResourceUsage -- This message was sent by Atlassian JIRA (v6.3.4#6332)
