Re: [DISCUSS] Treating LimitedPrivate({"MapReduce"}) as Public APIs for YARN applications

2016-05-10 Thread Hitesh Shah
quires all apps built against it to go through a full re-compile as well as downtime as existing apps may no longer work out of the box, lets call it out very explicitly in the Release notes. — Hitesh > On May 10, 2016, at 9:24 AM, Allen Wittenauer > wrote: > > >> On May

[DISCUSS] Treating LimitedPrivate({"MapReduce"}) as Public APIs for YARN applications

2016-05-10 Thread Hitesh Shah
There have been various discussions on various JIRAs where upstream projects such as YARN apps ( Tez, Slider, etc ) are called out for using the above so-called Private APIs. A lot of YARN applications that have been built out have picked up various bits and pieces of implementation from MapRedu

Re: [VOTE] Release Apache Hadoop 2.7.2 RC2

2016-01-21 Thread Hitesh Shah
+1 (binding) Did the basic checks and ran against a single node local setup. — Hitesh On Jan 14, 2016, at 8:57 PM, Vinod Kumar Vavilapalli wrote: > Hi all, > > I've created an updated release candidate RC2 for Apache Hadoop 2.7.2. > > As discussed before, this is the next maintenance rele

Re: Planning Hadoop 2.6.1 release

2015-08-11 Thread Hitesh Shah
Jason Lowe just filed YARN-4047 ( patch available too ) which probably is also a good candidate for 2.6.1 or 2.6.2 if the list is frozen. I have labelled the jira as a 2.6.1 candidate. Feel free to switch to 2.6.2 if needed. thanks — Hitesh On Aug 10, 2015, at 4:15 PM, Vinod Kumar Vavilapalli

Re: [DISCUSS] Release numbering for stable 2.8 and beyond

2015-04-26 Thread Hitesh Shah
There are a couple of different approaches we could take. How about publishing/releasing bits such as “2.8.0-RC0”. Downstream projects can depend on these and use them normally similar to the approach that they would have taken with release 2.8.0 or 2.8.0-alpha. After some baking ( or more RC

Re: [VOTE] Release Apache Hadoop 2.6.0

2014-11-14 Thread Hitesh Shah
+1 (binding) Built Hadoop from source, compiled Tez against the hadoop jars pushed to staging repo and ran a few example Tez jobs on a single node cluster. — HItesh On Nov 13, 2014, at 3:08 PM, Arun C Murthy wrote: > Folks, > > I've created another release candidate (rc1) for hadoop-2.6.0

Re: [VOTE] Migration from subversion to git for version control

2014-08-14 Thread Hitesh Shah
+1 — Hitesh On Aug 8, 2014, at 7:57 PM, Karthik Kambatla wrote: > I have put together this proposal based on recent discussion on this topic. > > Please vote on the proposal. The vote runs for 7 days. > > 1. Migrate from subversion to git for version control. > 2. Force-push to be disab

Re: [VOTE] Change by-laws on release votes: 5 days instead of 7

2014-06-24 Thread Hitesh Shah
+1 (binding) — Hitesh On Jun 24, 2014, at 1:53 AM, Arun C Murthy wrote: > Folks, > > As discussed, I'd like to call a vote on changing our by-laws to change > release votes from 7 days to 5. > > I've attached the change to by-laws I'm proposing. > > Please vote, the vote will the usual pe

[jira] [Resolved] (MAPREDUCE-5633) Can Hadoop use multi-cores of a processor under single machine

2013-11-20 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-5633. Resolution: Invalid Please ask such questions on the user list. http

Re: [VOTE] Release Apache Hadoop 2.2.0

2013-10-13 Thread Hitesh Shah
+1. Built from source and ran distributed shell along with simple MR jobs on a single node cluster. -- HItesh On Oct 7, 2013, at 12:00 AM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc0) for hadoop-2.2.0 that I would like to > get released - this release fixes a sm

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-08-21 Thread Hitesh Shah
+1 (binding) Verified checksums, built from source and ran basic MR jobs on a single-node cluster. -- Hitesh On Aug 15, 2013, at 2:15 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc2) for hadoop-2.1.0-beta that I would > like to get released - this fixes the bugs

[jira] [Created] (MAPREDUCE-5416) hadoop-mapreduce-client-common depends on hadoop-yarn-server-common

2013-07-24 Thread Hitesh Shah (JIRA)
Hitesh Shah created MAPREDUCE-5416: -- Summary: hadoop-mapreduce-client-common depends on hadoop-yarn-server-common Key: MAPREDUCE-5416 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5416

[jira] [Resolved] (MAPREDUCE-5399) Large number of map tasks cause slow sort at reduce phase, invariant to amount of data to sort

2013-07-17 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-5399. Resolution: Invalid If this is indeed an issue with Apache Hadoop-1.x, please feel

[jira] [Resolved] (MAPREDUCE-5325) ClientRMProtocol.getAllApplications should accept ApplicationType as a parameter---MR changes

2013-07-09 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-5325. Resolution: Fixed Fix Version/s: 2.1.0-beta Committed to trunk, branch-2

Re: [VOTE] Release Apache Hadoop 2.1.0-beta

2013-06-28 Thread Hitesh Shah
Hi Arun, From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and ma

[jira] [Created] (MAPREDUCE-5324) Admin-provided user environment can be overridden by user provided values for the AM

2013-06-14 Thread Hitesh Shah (JIRA)
Hitesh Shah created MAPREDUCE-5324: -- Summary: Admin-provided user environment can be overridden by user provided values for the AM Key: MAPREDUCE-5324 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5324

Re: [VOTE] Release Apache Hadoop 0.23.8

2013-06-03 Thread Hitesh Shah
+1 (non-binding) Built from source, ran some basic tests on a single node cluster. -- Hitesh On May 28, 2013, at 9:00 AM, Thomas Graves wrote: > > I've created a release candidate (RC0) for hadoop-0.23.8 that I would like > to release. > > This release is a sustaining release with several im

[jira] [Resolved] (MAPREDUCE-5095) TestShuffleExceptionCount#testCheckException fails occasionally with JDK7

2013-05-20 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-5095. Resolution: Fixed Release Note: Thanks Arpit. Committed to branch-1

[jira] [Created] (MAPREDUCE-5179) Change TestHSWebServices to do string equal check on hadoop build version similar to YARN-605

2013-04-24 Thread Hitesh Shah (JIRA)
Hitesh Shah created MAPREDUCE-5179: -- Summary: Change TestHSWebServices to do string equal check on hadoop build version similar to YARN-605 Key: MAPREDUCE-5179 URL: https://issues.apache.org/jira/browse

[jira] [Created] (MAPREDUCE-5178) Fix use of BuilderUtils#newApplicationReport as a result of YARN-577.

2013-04-24 Thread Hitesh Shah (JIRA)
Hitesh Shah created MAPREDUCE-5178: -- Summary: Fix use of BuilderUtils#newApplicationReport as a result of YARN-577. Key: MAPREDUCE-5178 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5178

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-16 Thread Hitesh Shah
+1 (non-binding). Downloaded from source, tested distributed shell and simple MR jobs on a single-node cluster. -- Hitesh On Apr 12, 2013, at 2:56 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (RC2) for hadoop-2.0.4-alpha that I would > like to release. > > The RC

Re: [VOTE] Release Apache Hadoop 0.23.7

2013-04-16 Thread Hitesh Shah
+1. Downloaded source, built and ran a couple of sample jobs on a single node cluster. -- Hitesh On Apr 11, 2013, at 12:55 PM, Thomas Graves wrote: > I've created a release candidate (RC0) for hadoop-0.23.7 that I would like > to release. > > This release is a sustaining release with severa

[jira] [Created] (MAPREDUCE-5142) MR AM unregisters with state KILLED when an error causes dispatcher to shutdown

2013-04-10 Thread Hitesh Shah (JIRA)
Hitesh Shah created MAPREDUCE-5142: -- Summary: MR AM unregisters with state KILLED when an error causes dispatcher to shutdown Key: MAPREDUCE-5142 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5142

[jira] [Resolved] (MAPREDUCE-5083) MiniMRCluster should use a random component when creating an actual cluster

2013-04-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-5083. Resolution: Fixed Fix Version/s: (was: 2.0.5-beta

[jira] [Reopened] (MAPREDUCE-5083) MiniMRCluster should use a random component when creating an actual cluster

2013-04-04 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reopened MAPREDUCE-5083: > MiniMRCluster should use a random component when creating an actual clus

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-07 Thread Hitesh Shah
+1 (non-binding) Downloaded ( verified checksums ) and built from source, deployed and successfully ran both MR and distributed shell examples. -- Hitesh On Feb 6, 2013, at 7:59 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I would

[jira] [Created] (MAPREDUCE-4578) Handle container requests that request more resources than available in the cluster

2012-08-22 Thread Hitesh Shah (JIRA)
Hitesh Shah created MAPREDUCE-4578: -- Summary: Handle container requests that request more resources than available in the cluster Key: MAPREDUCE-4578 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4578

[jira] [Created] (MAPREDUCE-3941) Improve distributed shell example to work on a secure cluster

2012-02-29 Thread Hitesh Shah (Created) (JIRA)
: Improvement Components: mrv2 Reporter: Hitesh Shah Assignee: Hitesh Shah Improve the distributed shell application to be able to work on a secure cluster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact

[jira] [Created] (MAPREDUCE-3891) NM should handle cleaning up containers when it shuts down ( and kill containers from an earlier instance when it comes back up after an unclean shutdown )

2012-02-21 Thread Hitesh Shah (Created) (JIRA)
) --- Key: MAPREDUCE-3891 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3891 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0 Reporter: Hitesh Shah

[jira] [Created] (MAPREDUCE-3796) Scheduler.normalizeRequest does not account for allocation requests that exceed maximumAllocation limits

2012-02-02 Thread Hitesh Shah (Created) (JIRA)
/jira/browse/MAPREDUCE-3796 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Critical -- This message is automatically

[jira] [Created] (MAPREDUCE-3775) Change MiniYarnCluster to escape special chars in testname

2012-01-31 Thread Hitesh Shah (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor When using MiniYarnCluster with the testname set to a nested classname, the "$" within the class name creates issues with the

[jira] [Created] (MAPREDUCE-3765) FifoScheduler does not respect yarn.scheduler.fifo.minimum-allocation-mb setting

2012-01-30 Thread Hitesh Shah (Created) (JIRA)
: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor FifoScheduler uses default min 1 GB regardless of the configuration value set for minimum

[jira] [Created] (MAPREDUCE-3739) Document all missing default configuration values in the relevant *default.xml files.

2012-01-26 Thread Hitesh Shah (Created) (JIRA)
Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Fix For: 0.23.1, 0.24.0 There seem to be quite a few configuration settings that are used in the code but missing from the

[jira] [Resolved] (MAPREDUCE-3654) MiniMRYarnCluster should set MASTER_ADDRESS to "local"

2012-01-19 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3654. Resolution: Invalid Looking more into the hbase test, a couple of issues were found

[jira] [Resolved] (MAPREDUCE-3420) [Umbrella ticket] Make uber jobs functional

2012-01-09 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3420. Resolution: Fixed > [Umbrella ticket] Make uber jobs functio

[jira] [Resolved] (MAPREDUCE-3425) uber job hangs and fails to get killed properly on a job kill signal

2012-01-09 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3425. Resolution: Cannot Reproduce Could not reproduce a second time after the patch

[jira] [Resolved] (MAPREDUCE-3421) Error in MR AM shutdown when running an uber-job

2011-12-12 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3421. Resolution: Duplicate Assignee: Hitesh Shah > Error in MR AM shutdown w

[jira] [Created] (MAPREDUCE-3523) Exclude mrapp-generated-classpath from published jars

2011-12-08 Thread Hitesh Shah (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Critical Fix For: 0.23.1, 0.24.0 Downstream projects will be generating their own classpath file to allow yarn to launch containers correctly in a unit-test env which may potentially

[jira] [Created] (MAPREDUCE-3485) DISKS_FAILED -101 error code should be defined in same location as ABORTED_CONTAINER_EXIT_STATUS

2011-11-29 Thread Hitesh Shah (Created) (JIRA)
/MAPREDUCE-3485 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Fix For: 0.23.1 With MAPREDUCE-3121, it is defined in ContainerExecutor as part of yarn-nodemanager which

[jira] [Created] (MAPREDUCE-3465) org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin fails on 0.23

2011-11-23 Thread Hitesh Shah (Created) (JIRA)
Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 Running org.apache.hadoop.yarn.util.TestLinuxResourceCalculatorPlugin Tests run: 2

[jira] [Created] (MAPREDUCE-3444) trunk/0.23 builds broken

2011-11-21 Thread Hitesh Shah (Created) (JIRA)
Reporter: Hitesh Shah Fix For: 0.23.1 https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/208/ https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1310/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact

[jira] [Created] (MAPREDUCE-3434) Nightly build broken

2011-11-18 Thread Hitesh Shah (Created) (JIRA)
Reporter: Hitesh Shah Priority: Blocker Fix For: 0.23.1 https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk/901/ Results : Failed tests: testSleepJob(org.apache.hadoop.mapreduce.v2.TestMRJobs) testRandomWriter(org.apache.hadoop.mapreduce.v2

[jira] [Created] (MAPREDUCE-3426) uber-jobs tried to write outputs into wrong dir

2011-11-17 Thread Hitesh Shah (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Fix For: 0.23.1 Incorrect setup of the uber tasks causes tasks to try to write intermidiate outputs into dirs that the user does not have permissions to write to on a secure cluster. -- This message is

[jira] [Created] (MAPREDUCE-3425) uber job hangs and fails to get killed properly on a job kill signal

2011-11-17 Thread Hitesh Shah (Created) (JIRA)
Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0, 0.23.1 Reporter: Hitesh Shah Fix For: 0.24.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https

[jira] [Resolved] (MAPREDUCE-3117) Uber jobs are hanging

2011-11-16 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3117. Resolution: Not A Problem > Uber jobs are hang

[jira] [Created] (MAPREDUCE-3421) Error in MR AM shutdown when running an uber-job

2011-11-16 Thread Hitesh Shah (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Fix For: 0.23.1 Exception thrown during the un-registration process in the ContainerAllocator#stop. Exception trace in next comment. -- This message is automatically generated by JIRA. If you think it

[jira] [Created] (MAPREDUCE-3420) Make uber jobs functional

2011-11-16 Thread Hitesh Shah (Created) (JIRA)
Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.1 Umbrella jira for getting uber jobs to work correctly with YARN/MRv2 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact

[jira] [Created] (MAPREDUCE-3407) Wrong jar getting used in TestMR*Jobs* for MiniMRYarnCluster

2011-11-15 Thread Hitesh Shah (Created) (JIRA)
: Bug Components: mrv2 Affects Versions: 0.23.0, 0.24.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Fix For: 0.23.1 pom for mapreduce-client-jobclient sets system property to incorrect jar name. -- This message is

[jira] [Resolved] (MAPREDUCE-3027) MR-279: Completed container exit state needs to be enhanced to differentiate between container aborts/failures and actual application process exit codes

2011-11-15 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3027. Resolution: Invalid > MR-279: Completed container exit state needs to be enhan

[jira] [Resolved] (MAPREDUCE-3225) Killing an unkown job throws NPE.

2011-11-09 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3225. Resolution: Cannot Reproduce No longer reproducible. > Killing

[jira] [Created] (MAPREDUCE-3332) contrib/raid compile breaks due to changes in hdfs/protocol/datatransfer/Sender#writeBlock related to checksum handling

2011-11-01 Thread Hitesh Shah (Created) (JIRA)
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3332 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/raid Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial

[jira] [Resolved] (MAPREDUCE-3218) ant test TestTokenCache failing on trunk

2011-10-31 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3218. Resolution: Duplicate Being addressed as part of MAPREDUCE-3321

[jira] [Resolved] (MAPREDUCE-3216) ant test TestNoDefaultsJobConf fails on trunk

2011-10-31 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3216. Resolution: Duplicate Being addressed as part of MAPREDUCE-3321

[jira] [Created] (MAPREDUCE-3321) Disable some failing legacy tests for MRv2 builds to go through

2011-10-31 Thread Hitesh Shah (Created) (JIRA)
Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Attachments: MR-3321.1.patch By-product of MR-3214. Disable tests for the short term until fixes are available for all tests

[jira] [Created] (MAPREDUCE-3318) TestSeveral timing out in commit builds

2011-10-31 Thread Hitesh Shah (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For

[jira] [Created] (MAPREDUCE-3314) Support AM specific blacklisting of nodes on the RM

2011-10-30 Thread Hitesh Shah (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.1 The AM blacklists certain nodes due to task failures but the RM does not know of this and can allocate blacklisted nodes to the AM again and again. RM should support a protocol that allows AM

Re: Howto kill a container?

2011-10-25 Thread Hitesh Shah
Hi Chris The AM should send a stop container request to the ContainerManager to kill the container in question. This will trigger a kill event to the running container and eventually, when the AM talks to the RM, the killed container would be part of the completed containers list in allocate re

[jira] [Resolved] (MAPREDUCE-3260) Yarn app stuck in KILL_WAIT state

2011-10-25 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3260. Resolution: Duplicate Duplicate of MR-3084. Being fixed as part of MR-3240. Will push

[jira] [Created] (MAPREDUCE-3262) Handle missing events in failure scenarios

2011-10-25 Thread Hitesh Shah (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.0 Need to handle kill container event in localization failed state. Need to handle resource localized in localization failed state. -- This message is automatically generated by JIRA. If you think it was sent

[jira] [Resolved] (MAPREDUCE-3084) race when KILL_CONTAINER is received for a LOCALIZED container

2011-10-24 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3084. Resolution: Duplicate Fix Version/s: 0.23.0 Target Version/s: 0.23.0

[jira] [Created] (MAPREDUCE-3222) ant test TestTaskContext failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testContextStatus took 29.977 sec FAILED null expected: sort]> but was: junit.framework.ComparisonFailure: null expected: sort]> but was:

[jira] [Created] (MAPREDUCE-3221) ant test TestSubmitJob failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testJobWithInvalidMemoryReqs took 2.588 sec Testcase: testSecureJobExecution took 4.089 sec FAILED java.io.IOException: org.apache.hadoop.ipc.RPC$VersionMismatch

[jira] [Created] (MAPREDUCE-3220) ant test TestCombineOutputCollector failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testsuite: org.apache.hadoop.mapred.TestCombineOutputCollector Tests run: 2, Failures: 2, Errors: 0, Time elapsed: 1.591 sec Testcase: testCustomCollect took

[jira] [Created] (MAPREDUCE-3219) ant test TestDelegationToken failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testDelegationToken took 2.043 sec Caused an ERROR Client Hitesh tries to renew a token with renewer specified as alice at

[jira] [Created] (MAPREDUCE-3218) ant test TestTokenCache failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testTokenCache took 11.607 sec Testcase: testLocalJobTokenCache took 12.224 sec Testcase: testGetTokensForNamenodes took 0.009 sec Testcase: testGetTokensForHftpFS took

[jira] [Created] (MAPREDUCE-3217) ant test TestAuditLogger fails on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testKeyValLogFormat took 0.096 sec Testcase: testAuditLoggerWithoutIP took 0.005 sec Testcase: testAuditLoggerWithIP took 0.417 sec Caused an ERROR java.io.IOException

[jira] [Created] (MAPREDUCE-3216) ant test TestNoDefaultsJobConf fails on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testNoDefaults took 4.703 sec Caused an ERROR Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond

[jira] [Created] (MAPREDUCE-3215) org.apache.hadoop.mapreduce.TestNoJobSetupCleanup failing on trunk

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Testcase: testNoJobSetupCleanup took 13.271 sec FAILED Number of part-files is 0 and not 1

[jira] [Created] (MAPREDUCE-3214) ant mapreduce tests failing

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
Reporter: Hitesh Shah Fix For: 0.23.0 Umbrella jira for various test failures -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators

[jira] [Created] (MAPREDUCE-3213) AM should kill all running tasks on a node when it gets marked as blacklisted

2011-10-19 Thread Hitesh Shah (Created) (JIRA)
Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https

[jira] [Created] (MAPREDUCE-3207) TestMRCLI failing on trunk

2011-10-18 Thread Hitesh Shah (Created) (JIRA)
Reporter: Hitesh Shah Priority: Minor Fix For: 0.23.0 Failing tests: 7: Archive: Deleting a file in archive 8: Archive: Renaming a file in archive -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your

[jira] [Created] (MAPREDUCE-3198) Change mode for hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/mock-container-executor to 755

2011-10-17 Thread Hitesh Shah (Created) (JIRA)
- Key: MAPREDUCE-3198 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3198 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah

Re: trying to build MR examples from trunk fails

2011-10-15 Thread Hitesh Shah
The errors (atleast the local framework one) seem to be changes in the past few days so if you haven't tried this already, re-compiling your local source tree if needed to install the jars locally and purging away your ivy cache before re-trying should likely fix things. -- Hitesh On Oct 15,

[jira] [Created] (MAPREDUCE-3183) hadoop-assemblies/src/main/resources/assemblies/hadoop-mapreduce-dist.xml missing license header

2011-10-13 Thread Hitesh Shah (Created) (JIRA)
/MAPREDUCE-3183 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA

Re: Trunk commit builds.

2011-10-10 Thread Hitesh Shah
Opened MAPREDUCE-3158 for fixing the builds. Seems like the change for default value of framework name to yarn from classic broke a couple of the MRv1 tests. -- Hitesh On Oct 9, 2011, at 11:11 PM, Mahadev Konar wrote: > Looks like the trunk commit builds are failing/crashing. > > https://bui

[jira] [Created] (MAPREDUCE-3158) Fix trunk build failures

2011-10-10 Thread Hitesh Shah (Created) (JIRA)
Reporter: Hitesh Shah Assignee: Hitesh Shah Fix For: 0.23.0 https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Commit/1060/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA

[jira] [Resolved] (MAPREDUCE-3033) JobClient requires mapreduce.jobtracker.address config even when mapreduce.framework.name is set to yarn

2011-10-10 Thread Hitesh Shah (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah resolved MAPREDUCE-3033. Resolution: Fixed Will open a new jira for the mrv1 test failures

[jira] [Reopened] (MAPREDUCE-3033) JobClient requires mapreduce.jobtracker.address config even when mapreduce.framework.name is set to yarn

2011-10-10 Thread Hitesh Shah (Reopened) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reopened MAPREDUCE-3033: Re-opening issue to address MRv1 commit build test failures > JobCli

[jira] [Created] (MAPREDUCE-3155) Document and likewise implement relevant checks/escape functions for what form of input is acceptable for setting up a container launch context i.e. special chars in

2011-10-07 Thread Hitesh Shah (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.0 What should a user of yarn escape/not escape when passing in input for the container launch contexts - localized resources' names are used to create symlinks.

[jira] [Created] (MAPREDUCE-3137) Fix broken merge of MR-2719 to 0.23 branch for the distributed shell test case

2011-10-03 Thread Hitesh Shah (Created) (JIRA)
: Hadoop Map/Reduce Issue Type: Sub-task Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Trivial Fix For: 0.23.0 -- This message is automatically generated by JIRA. If you think it

[jira] [Created] (MAPREDUCE-3098) Report Application status as well as ApplicationMaster status in GetApplicationReportResponse

2011-09-26 Thread Hitesh Shah (Created) (JIRA)
-3098 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.0 Currently, an application report received by the client from the RM/ASM for a given application returns

[jira] [Created] (MAPREDUCE-3067) Container exit status not set properly to launched process's exit code on successful completion of process

2011-09-21 Thread Hitesh Shah (JIRA)
https://issues.apache.org/jira/browse/MAPREDUCE-3067 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Fix For: 0.23.0 When testing the distributed shell sample app master, the container exit s

Re: YARN, LocalResource, and Classpaths

2011-09-20 Thread Hitesh Shah
Chris, Another small note in case you have not already figured it out. It should likely be possible to debug the application master by running it directly ( instead of the multi-step client job submission approach). The only thing to note is you will need to comment out the code that does the

[jira] [Created] (MAPREDUCE-3055) Simplify parameter passing to Application Master from Client. SImplify approach to pass info such appId, ClusterTimestamp and failcount required by App Master.

2011-09-20 Thread Hitesh Shah (JIRA)
. Key: MAPREDUCE-3055 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3055 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah

Re: ApplicationMaster appId/appAttemptId/clusterTimestamp

2011-09-20 Thread Hitesh Shah
Hi Chris You are right. At the moment, you do need to pass all of them in to your app master for it to register. I will file a jira to simplify this to end up with something along the lines where the RM implicitly passes such information to the app master without the client explicitly setting i

[jira] [Created] (MAPREDUCE-3041) Enhance YARN Client-RM protocol to provide access to information such as cluster's Min/Max Resource capabilities similar to that of AM-RM protocol

2011-09-19 Thread Hitesh Shah (JIRA)
tocol -- Key: MAPREDUCE-3041 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3041 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh

[jira] [Created] (MAPREDUCE-3027) MR-279: Completed container exit state needs to be enhanced to differentiate between container aborts/failures and actual application process exit codes

2011-09-16 Thread Hitesh Shah (JIRA)
Key: MAPREDUCE-3027 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3027 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee

[jira] [Created] (MAPREDUCE-3004) sort example fails in shuffle/reduce stage as it assumes a local job by default

2011-09-13 Thread Hitesh Shah (JIRA)
: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Hitesh Shah Assignee: Hitesh Shah Priority: Minor Log trace when running sort on a single node setup: 11/09/13 17:01:06 INFO mapreduce.Job: map 100