Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

Allen, can we bump up the maven surefire heap size to max (if it already is not) for the branch-2 nightly build and see if it helps? Thanks, Subru On Tue, Oct 24, 2017 at 4:22 PM, Allen Wittenauer wrote: > > > On Oct 24, 2017, at 4:10 PM, Andrew Wang

[jira] [Created] (HADOOP-14977) Xenial dockerfile needs ant

Allen Wittenauer created HADOOP-14977: - Summary: Xenial dockerfile needs ant Key: HADOOP-14977 URL: https://issues.apache.org/jira/browse/HADOOP-14977 Project: Hadoop Common Issue Type:

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

> On Oct 24, 2017, at 4:10 PM, Andrew Wang wrote: > > FWIW we've been running branch-3.0 unit tests successfully internally, though > we have separate jobs for Common, HDFS, YARN, and MR. The failures here are > probably a property of running everything in the same

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

FWIW we've been running branch-3.0 unit tests successfully internally, though we have separate jobs for Common, HDFS, YARN, and MR. The failures here are probably a property of running everything in the same JVM, which I've found problematic in the past due to OOMs. On Tue, Oct 24, 2017 at 4:04

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

My plan is currently to: * switch some of Hadoop’s Yetus jobs over to my branch with the YETUS-561 patch to test it out. * if the tests work, work on getting YETUS-561 committed to yetus master * switch jobs back to ASF yetus master either post-YETUS-561 or without it if it doesn’t work * go

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

+1 (binding) Thanks a lot, Junping! I built and installed the source on a 6-node pseudo cluster. I simple sleep and streaming jobs that exercised intra-queue and inter-queue preemption, and used user weights. -Eric From: Junping Du To:

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

Sean/Junping- Ignoring the epistemology, it's a problem. Let's figure out what's causing memory to balloon and then we can work out the appropriate remedy. Is this reproducible outside the CI environment? To Junping's point, would YETUS-561 provide more detailed information to aid debugging? -C

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

Just curious, Junping what would "solid evidence" look like? Is the supposition here that the memory leak is within HDFS test code rather than library runtime code? How would such a distinction be shown? On Tue, Oct 24, 2017 at 4:06 PM, Junping Du wrote: > Allen, > Do

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

Allen, Do we have any solid evidence to show the HDFS unit tests going through the roof are due to serious memory leak by HDFS? Normally, I don't expect memory leak are identified in our UTs - mostly, it (test jvm gone) is just because of test or deployment issues. Unless there is

dependency on hadoop-annotations

For a project that depends on hadoop-common what is a reason to have hadoop-annotations as a dependency (compile or runtime)? Thank you, Vlad - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

Thanks for all your hard work Junping! * Checked signature. * Ran a sleep job. * Checked NN File browser UI works. +1 (binding) Cheers Ravi On Tue, Oct 24, 2017 at 12:26 PM, Rakesh Radhakrishnan wrote: > Thanks Junping for getting this out. > > +1 (non-binding) > > *

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

Thanks Junping for getting this out. +1 (non-binding) * Built from source on CentOS 7.3.1611, jdk1.8.0_111 * Deployed 3 node cluster * Ran some sample jobs * Ran balancer * Operate HDFS from command line: ls, put, dfsadmin etc * HDFS Namenode UI looks good Thanks, Rakesh On Fri, Oct 20, 2017

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/567/ [Oct 23, 2017 4:43:41 PM] (epayne) YARN-4163: Audit getQueueInfo and getApplications calls [Oct 23, 2017 5:47:16 PM] (arp) HDFS-12683. DFSZKFailOverController re-order logic for logging [Oct 23, 2017

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

+1 (non-binding) - Build from source 1.8.0_111 - Deployed on 3 node secure setup - Ran few mapreduce jobs with multiple users. - Verified basic resource localization - Failover of Resource Manager. - Log aggregation verification - Sanity check of JHS Thanks Bibin

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

+1 (non-binding) - Verified all hashes and checksums - Built from source on macOS 10.12.6, Java 1.8.0u65 - Deployed a pseudo cluster - Ran some example jobs Thanks, Eric On Tue, Oct 24, 2017 at 12:59 AM, Mukul Kumar Singh wrote: > Thanks Junping, > > +1 (non-binding)

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

> On Oct 23, 2017, at 12:50 PM, Allen Wittenauer > wrote: > > > > With no other information or access to go on, my current hunch is that one of > the HDFS unit tests is ballooning in memory size. The easiest way to kill a > Linux machine is to eat all of the

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/10/ [Oct 23, 2017 4:58:04 PM] (epayne) YARN-4163: Audit getQueueInfo and getApplications calls [Oct 23, 2017 5:47:35 PM] (arp) HDFS-12683. DFSZKFailOverController re-order logic for logging [Oct 23, 2017

[jira] [Created] (HADOOP-14975) S3AInputStream/OutputStream statistics aren't getting into StorageStatistics

Steve Loughran created HADOOP-14975: --- Summary: S3AInputStream/OutputStream statistics aren't getting into StorageStatistics Key: HADOOP-14975 URL: https://issues.apache.org/jira/browse/HADOOP-14975

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

Thanks Junping, +1 (non-binding) I built from source on Mac OS X 10.12.6 Java 1.8.0_111 - Deployed on a single node cluster. - Deployed a ViewFS cluster with two hdfs mount points. - Performed basic sanity checks. - Performed basic DFS operations. Thanks, Mukul On 20/10/17, 6:12 AM,