Re: [VOTE] Release Apache Hadoop 3.3.2 - RC2

+1 (binding) - Built from source - Brought up a non-secure virtual cluster w/ NN, 1 DN, RM, AHS, JHS, and 3 NMs - Validated inter- and intra-queue preemption - Validated exclusive node labels Thanks a lot Chao for your diligence and hard work on this release. Eric On

[jira] [Reopened] (MAPREDUCE-7203) TestRuntimeEstimators fails intermittent

[ https://issues.apache.org/jira/browse/MAPREDUCE-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne reopened MAPREDUCE-7203: --- Sorry! I was in the wrong window! > TestRuntimeEstimators fails intermitt

[jira] [Resolved] (MAPREDUCE-7203) TestRuntimeEstimators fails intermittent

[ https://issues.apache.org/jira/browse/MAPREDUCE-7203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne resolved MAPREDUCE-7203. --- Fix Version/s: 3.3.2 3.2.3 2.10.2

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

+1 (binding) - Built from source - Brought up cluster - Tested streaming and sleep jobs -Eric Payne On Wednesday, December 9, 2020, 11:01:38 AM CST, Xiaoqiao He wrote: Hi folks, The release candidate (RC4) for Hadoop-3.2.2 is available now. There are 10 commits[1] differences between RC4

Re: [VOTE] Release Apache Hadoop 2.10.1 (RC0)

Masatake, Thank you for the good work on creating this release! +1 I downloaded and built the source. I ran a one-node cluster with 6 NMs. I manually ran apps in the Capacity Scheduler to test labels and capacity assignments. -Eric On Monday, September 14, 2020, 12:59:17 PM CDT, Masatake

Re: [ANNOUNCE] New Apache Hadoop Committer - Tao Yang

Congratulations Tao! Well deserved! On Monday, July 15, 2019, 4:54:10 AM CDT, Weiwei Yang wrote: Hi Dear Apache Hadoop Community It's my pleasure to announce that Tao Yang has been elected as an Apache Hadoop committer, this is to recognize his contributions to Apache Hadoop YARN

[ANNOUNCE] Eric Badger is now a committer!

It is my pleasure to announce that Eric Badger has accepted an invitation to become a Hadoop Core committer. Congratulations, Eric! This is well-deserved! -Eric Payne

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

+1 (binding) - RM refresh updates values as expected - Streaming jobs complete successfully - Moving apps between queues succeeds - Inter-queue preemption works as expected - Successfully ran selected yarn unit tests. === Eric Payne === On Tuesday, January 8, 2019, 5:42:46 AM CST

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

a blocker for release. I'm on the fence. Thanks, -Eric On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne wrote: Sunil, So, the basic symptoms are that if preemption is enabled on any queue, the preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all of the

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

CST, Eric Payne wrote: Sunil, thanks for all of the hard work on this release. I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM. I am still

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

Sunil, thanks for all of the hard work on this release. I have discovered that queue refresh doesn't work in some cases. For example, when I change yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take effect unless I restart the RM. I am still investigating, but I thought I

Re: [VOTE] Release Apache Hadoop 2.9.2 (RC0)

+1 (binding) -- Built from source-- Installed on 6-node pseudo cluster-- Tested intra- inter-queue preemption, user weights-- Ran streaming jobs, word count, and tara gen/sort tests Thanks Akira for all of the hard work.-Eric Payne On Tuesday, November 13, 2018, 7:02:51 PM CST, Akira

Re: [VOTE] Release Apache Hadoop 2.8.5 (RC0)

Thanks a lot Junping! +1 (binding) Tested the following: - Built from source - Installed on a 7 node, multi-tenant, insecure pseudo cluster, running YARN capacity scheduler - Added a queue via refresh - Verified various GUI pages - Streaming jobs - Cross-queue (Inter) preemption - In-queue

Re: [VOTE] Release Apache Hadoop 3.1.1 - RC0

Thanks Wangda for creating this release. +1 (binding) Tested: - Built from source - Deployed to 6-node, multi-tennant, unsecured pseudo cluster with hierarchical queue structure (CS) - Refreshed queue (CS) properties - Intra-queue preemption (CS) - inter-queue preemption (CS) - User weights (CS)

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

Sorry, Yongjun. My +1 is also binding+1 (binding)-Eric Payne On Friday, June 1, 2018, 12:25:36 PM CDT, Eric Payne wrote: Thanks a lot, Yongjun, for your hard work on this release. +1 - Built from source - Installed on 6 node pseudo cluster Tested the following in the Capacity

Re: [VOTE] Release Apache Hadoop 3.0.3 (RC0)

Thanks a lot, Yongjun, for your hard work on this release. +1 - Built from source - Installed on 6 node pseudo cluster Tested the following in the Capacity Scheduler: - Verified that running apps in labelled queues restricts tasks to the labelled nodes. - Verified that various queue config

Re: Apache Hadoop 3.0.3 Release plan

anches if the branch is cut too early. My 2 cents... Thanks, -Eric Payne On Monday, May 7, 2018, 12:09:00 AM CDT, Yongjun Zhang <yjzhan...@apache.org> wrote: Hi All, > We have released Apache Hadoop 3.0.2 in April of this year [1]. Since then, there are quite some commits don

Re: RE: [VOTE] Release Apache Hadoop 2.9.1 (RC0)

line - intra-queue preemption - inter-queue preemption- verified preemption properties are refreshable Thanks, Eric Payne On Wednesday, April 25, 2018, 12:12:24 AM CDT, Chen, Sammi <sammi.c...@intel.com> wrote: Paste the links here, The artifacts are available here: 

Re: [VOTE] Release Apache Hadoop 3.0.2 (RC1)

, with and without size-based weight - tested user weights Thanks! Eric Payne On Monday, April 16, 2018, 7:00:03 PM CDT, Lei Xu <l...@apache.org> wrote: Hi, All I've created release candidate RC-1 for Apache Hadoop 3.0.2, to address missing source jars in the maven repository in RC-0.

Re: [VOTE] Release Apache Hadoop 3.0.2 (RC0)

weights with FifoOrderingPolicy to ensure that weights were assigned to users as expected. Eric Payne On Friday, April 6, 2018, 1:17:10 PM CDT, Lei Xu <l...@apache.org> wrote: Hi, All I've created release candidate RC-0 for Apache Hadoop 3.0.2. Please note: this is an ame

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC1)

- Tested simple inter-queue preemption - Tested priority first intra-queue preemption - Tested userlimit first intra-queue preemption Thanks,Eric Payne === On Thursday, March 29, 2018, 11:15:51 PM CDT, Wangda Tan <wheele...@gmail.

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

Thanks for working on this release! +1 (binding) I tested the following: - yarn distributed shell job - yarn streaming job - inter-queue preemption - compared behavior of fair and fifo ordering policy - both userlimit_first mode and priority_first mode of intra-queue preemption Eric Payne

Re: [VOTE] Release Apache Hadoop 2.7.5 (RC1)

Thanks for the hard work on this release, Konstantin. +1 (binding) - Built from source - Verified that refreshing of queues works as expected. - Verified can run multiple users in a single queue - Ran terasort test - Verified that cross-queue preemption works as expected Thanks. Eric Payne

Re: [VOTE] Release Apache Hadoop 2.8.3 (RC0)

. Eric Payne From: Junping Du <j...@hortonworks.com> To: "common-...@hadoop.apache.org" <common-...@hadoop.apache.org>; "hdfs-...@hadoop.apache.org" <hdfs-...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <mapreduce-dev@hado

Re: [VOTE] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk

this feature. Huge thanks to everyone who helped with reviews, commits, guidance, and technical discussion/design, including Wangda Tan, Vinod Vavilapalli, Rohith Sharma K S, Eric Payne . [1] : http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201711.mbox/%3CCACYiTuhKhF1JCtR7ZFuZSEKQ4s

Re: [DISCUSS] Merge Absolute resource configuration support in Capacity Scheduler (YARN-5881) to trunk

Thanks Sunil for the great work on this feature. I looked through the design document, reviewed the code, and tested out branch YARN-5881. The design makes sense and the code looks like it is implementing the desing in a sensible way. However, I have encountered a couple of bugs. I opened

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

during In-queue preemption o Users with different weights are assigned resources proportional to their weights. o User weights are refreshable, and in-queue preemption works to honor the post-refresh weights Thanks, -Eric Payne From: Andrew Wang <andrew.w...@cloudera.com> To: &

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

the command line o Users with different weights are assigned resources proportional to their weights. Thanks,-Eric Payne From: Arun Suresh <asur...@apache.org> To: yarn-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; Hadoop Common <common-...@hadoop.apache.org>; Hd

Re: [VOTE] Release Apache Hadoop 3.0.0-beta1 RC0

) preemption, both USERFIRST and PRIORITYFIRST o User weights not equals to 1 o User weights in conjunction with in-queue preemption. -Eric Payne From: Andrew Wang <andrew.w...@cloudera.com> To: "common-...@hadoop.apache.org" <common-...@hadoop.apache.org>; "hdfs-.

Re: [DISCUSS] Branches and versions for Hadoop 3

+1 for this branching proposal.-Eric From: Andrew Wang To: "common-...@hadoop.apache.org" ; "mapreduce-dev@hadoop.apache.org" ; "hdfs-...@hadoop.apache.org" ;

Re: [VOTE] Release Apache Hadoop 2.7.4 (RC0)

+1 (binding) Tested the following: - Application History Server -- Apps can be observed from UI -- App and container metadata can be retrieved via REST APIs - RM UI -- Can kill an app from the RM UI - Apps run in different frameworks. Frameworks tested: MR and yarn shell -- In yarn

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha4-RC0

Thanks Andrew. I downloaded the source, built it, and installed it onto a pseudo distributed 4-node cluster. I ran mapred and streaming test cases, including sleep and wordcount. +1 (non-binding) -Eric From: Andrew Wang To: "common-...@hadoop.apache.org"

Pre commit builds seem broken

Hey does anyone know why the precommit builds seem broken? I try to kick one and it hangs for a long time and then produces the following: #1​62​88 (pending—H14 doesn’t have label Hadoop&&!H9&&!H5&&!H6; H15 doesn’t have label Hadoop&&!H9&&!H5&&!H6; H16 doesn’t have label Hadoop&&!H9&&!H5&&!H6;

[jira] [Created] (MAPREDUCE-6812) Capacity Scheduler: Support user-specific minimum user limit percent

Eric Payne created MAPREDUCE-6812: - Summary: Capacity Scheduler: Support user-specific minimum user limit percent Key: MAPREDUCE-6812 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6812 Project

Re: Updated 2.8.0-SNAPSHOT artifact

How do we come to a resolution regarding whether or not re-cut branch-2.8 or release it as it is (after fixing blockers)? There are some things in branch-2 that I would like to pull back into branch-2.8, so a resolution to this question will affect how I proceed. Thanks,-Eric From: Karthik

Re: [VOTE] Release Apache Hadoop 3.0.0-alpha1 RC0

Thank you very much, Andrew. +1 (non-binding) - Downloaded source and built native - Installed on 3-node, non-secure cluster - Ran sleep jobs - Ensured preemption works as expected -Eric Payne - Original Message - From: Andrew Wang <andrew.w...@cloudera.com> To: &

Re: [VOTE] Release Apache Hadoop 2.7.3 RC2

mpted as I thought should have been, but once the other underserved app began to run, it stopped preempting. Also, it didn't preempt between 2 queues with the same partition label. Partition preemption may not be supported in 2.7, so this is probably also okay. Thanks!

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

jobs are running in labelled queues (YARN-4751). - Ensure that a yarn distributed shell application can be launched and complete successfully. Eric Payne From: Vinod Kumar Vavilapalli <vino...@apache.org> To: "common-...@hadoop.apache.org" <common-...@hadoop

Re: [VOTE] Release Apache Hadoop 2.7.2 RC2

Vinod, I have opened https://issues.apache.org/jira/browse/YARN-4751 to cover this issue.It happens in 2.7, not 2.8.Thanks,-Eric From: Vinod Kumar Vavilapalli <vino...@apache.org> To: hdfs-...@hadoop.apache.org; Eric Payne <eric.payne1...@yahoo.com> Cc: Hadoop Co

Re: [VOTE] Release Apache Hadoop 2.6.4 RC0

, which is the behavior I would expect if no node had the specified label. I will also add that this procedure works fine in 2.7. Thanks,-Eric Payne From: Junping Du <j...@hortonworks.com> To: "hdfs-...@hadoop.apache.org" <hdfs-...@hadoop.apache.org>; "yarn-.

Re: [VOTE] Release Apache Hadoop 2.6.4 RC0

com> To: mapreduce-dev@hadoop.apache.org; Eric Payne <eric.payne1...@yahoo.com> Cc: "common-...@hadoop.apache.org" <common-...@hadoop.apache.org>; "hdfs-...@hadoop.apache.org" <hdfs-...@hadoop.apache.org>; "yarn-...@hadoop.apache.org" <yarn-...@hadoop.apach

Re: [VOTE] Release Apache Hadoop 2.7.2 RC2

for that queue in the "Application Queues" section. - Tested to make sure application data is preserved in the generic application history server when the AM crashes during startup (e.g., exceeds max splits). -Eric Payne From: Vinod Kumar Vavilap

[jira] [Resolved] (MAPREDUCE-2011) Reduce number of getFileStatus call made from every task(TaskDistributedCache) setup

[ https://issues.apache.org/jira/browse/MAPREDUCE-2011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne resolved MAPREDUCE-2011. --- Resolution: Won't Fix [~knoguchi], here are [~jlowe]'s comments from an offline

Re: [VOTE] Release Apache Hadoop 2.6.2

aggressively given back to preempted queue (see YARN-3769). - Verified resource metrics on Application UI were correct by comparing against time spent in each container as calculated by the timeline server. - Wordcount test -Eric Payne From: Sangjin Lee <sj...@apache.org> To: &

Re: [VOTE] Release Apache Hadoop 2.6.1 RC1

container, and verified that it would in fact keep the containers running across AM restarts.- sucessfully ran wordcount jobs. Thank you, -Eric Payne From: Vinod Kumar Vavilapalli <vino...@apache.org> To: common-...@hadoop.apache.org; yarn-...@hadoop.apache.org; hdfs-...@hadoop.apac

Re: [VOTE] Release Apache Hadoop 2.7.1 RC0

+1 (non-binding) + Downloaded and built source+ Installed on one-node cluster+ Ran simple manual tests+ spot-checked unit tests Thanks, Vinod, for managing this release!-Eric Payne   From: Vinod Kumar Vavilapalli vino...@apache.org To: common-...@hadoop.apache.org; hdfs

Re: Bug Bash Post-mortem

Thank you Allen and Barbara for organizing the bug bash and for the post-mortem. I would definitely like to have another one in the fall.   From: Allen Wittenauer a...@altiscale.com To: common-...@hadoop.apache.org common-...@hadoop.apache.org Cc: mapreduce-dev@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.7.0 RC0

, and TestWebHdfsFileSystemContract#testGetFileBlockLocations.   Thank you,-Eric Payne From: Vinod Kumar Vavilapalli vino...@apache.org To: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org Cc: vino...@apache.org Sent: Friday, April 10, 2015 6:44 PM

[jira] [Resolved] (MAPREDUCE-3286) Unit tests for MAPREDUCE-3186 - User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed.

[ https://issues.apache.org/jira/browse/MAPREDUCE-3286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne resolved MAPREDUCE-3286. --- Resolution: Invalid Target Version/s: (was: ) Release Note

[jira] [Created] (MAPREDUCE-6174) Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput.

Eric Payne created MAPREDUCE-6174: - Summary: Combine common stream code into parent class for InMemoryMapOutput and OnDiskMapOutput. Key: MAPREDUCE-6174 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6174

Re: [VOTE] Release Apache Hadoop 2.6.0

you,-Eric Payne   From: Arun C Murthy a...@hortonworks.com To: common-...@hadoop.apache.org common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org hdfs-...@hadoop.apache.org; yarn-...@hadoop.apache.org yarn-...@hadoop.apache.org; mapreduce-dev@hadoop.apache.org mapreduce-dev

Re: [VOTE] Release Apache Hadoop 2.6.0

the containers start. Enabled the preemption feature and verified containers were preempted and queues were levelized. Ran unit tests for hadoop-yarn-server-resourcemanagerRan unit tests for hadoop-hdfs  Thank you,-Eric Payne From: Arun C Murthy a...@hortonworks.com To: common

[jira] [Created] (MAPREDUCE-3975) Default value not set for Configuration parameter mapreduce.job.local.dir

/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1, 0.23.2 Reporter: Eric Payne Assignee: Eric Payne Priority: Blocker mapreduce.job.local.dir (formerly job.local.dir in 0.20) is not set by default. This is a regression from

[jira] [Created] (MAPREDUCE-3856) Instances of RunningJob class givs incorrect job tracking urls when mutiple jobs are submitted from same client jvm.

: https://issues.apache.org/jira/browse/MAPREDUCE-3856 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1, 0.23.2 Reporter: Eric Payne Assignee: Eric Payne Priority: Critical When multiple jobs

[jira] [Created] (MAPREDUCE-3456) $HADOOP_PREFIX/bin/yarn should set defaults for $HADOOP_*_HOME

Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Eric Payne If the $HADOOP_PREFIX/hadoop-dist/target/hadoop-0.23.0-SNAPSHOT.tar.gz tarball is used to distribute hadoop, all of the HADOOP components (HDFS, MAPRED, COMMON) are all under one directory

[jira] [Created] (MAPREDUCE-3286) Unit tests for MAPREDUCE-3186 - User jobs are getting hanged if the Resource manager process goes down and comes up while job is getting executed.

. -- Key: MAPREDUCE-3286 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3286 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Environment: linux Reporter: Eric Payne