[jira] [Created] (MAPREDUCE-3853) TestLinuxResourceCalculatorPlugin is failing on trunk and 0.23 branch.

2012-02-10 Thread Mahadev konar (Created) (JIRA)
Issue Type: Bug Components: mrv2 Reporter: Mahadev konar Fix For: 0.23.2 Looks like the test is failing: https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-0.23-Build/188/console -- This message is automatically generated by JIRA. If you

[jira] [Created] (MAPREDUCE-3822) TestJobCounters is failing intermittently on trunk and 0.23.

2012-02-06 Thread Mahadev konar (Created) (JIRA)
: Bug Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.1 TestJobCounters fails sometimes on trunk. I have tracked it down to stats issue in FileSystem. Still working on it. -- This message is automatically generated

[jira] [Created] (MAPREDUCE-3774) yarn-default.xml should be moved to hadoop-yarn-common.

2012-01-31 Thread Mahadev konar (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.1 yarn-default.xml right now resides in hadoop-yarn-server-common jars which is not the right thing to do since this jar might not be needed in some

[jira] [Created] (MAPREDUCE-3762) Resource Manager fails to come up with default capacity scheduler configs.

2012-01-30 Thread Mahadev konar (Created) (JIRA)
/Reduce Issue Type: Bug Components: mrv2 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Thanks to [~harip] for pointing out the issue. This is the stack trace for bringing up RM with default CS configs: {code

[jira] [Created] (MAPREDUCE-3743) Documentation for Log Aggregation and log retrieval.

2012-01-27 Thread Mahadev konar (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Priority: Critical Retrieving logs in 0.23 is very different from what 0.20.* does. This is a very new feature which will require good documentation for users to get used to it. Lets make sure we have some

[jira] [Created] (MAPREDUCE-3735) Add distcp jar to the distribution (tar)

2012-01-26 Thread Mahadev konar (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.1 Distcp jar isnt getting added to the tarball as of now. We need to add it along with archives/streaming and others. -- This message is

[jira] [Resolved] (MAPREDUCE-3733) Add Apache License Header to hadoop-distcp/pom.xml

2012-01-26 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3733. -- Resolution: Fixed I went ahead and committed this. > Add Apa

[jira] [Created] (MAPREDUCE-3733) Add Apache License Header to hadoop-distcp/pom.xml

2012-01-26 Thread Mahadev konar (Created) (JIRA)
Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.1 Looks like I missed the Apache Headers in the review. Adding it now. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA

[jira] [Created] (MAPREDUCE-3717) JobClient test jar has missing files to run all the test programs.

2012-01-24 Thread Mahadev konar (Created) (JIRA)
Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.1 Looks like MAPREDUCE-3582 forgot to move couple of files from the ant builds. The current test

[jira] [Resolved] (MAPREDUCE-3525) Shuffle benchmark is nearly 1.5x slower in 0.23

2012-01-22 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3525. -- Resolution: Fixed Fix Version/s: 0.23.1 Fixed via MAPREDUCE-3641. Thanks

[jira] [Created] (MAPREDUCE-3701) Delete HadoopYarnRPC from 0.23 branch.

2012-01-19 Thread Mahadev konar (Created) (JIRA)
Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.1 HadoopYarnRPC file exists in 0.23 (should have been removed with the new HadoopYarnProtoRPC). Trunk does not have this issue. -- This message is automatically generated by JIRA. If you think it was sent

Re: How is MRv2 fundamentally changed?

2012-01-16 Thread Mahadev Konar
Hi Jie, You might want to read through: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/YARN.html and http://developer.yahoo.com/blogs/hadoop/posts/2011/02/mapreduce-nextgen/ for more information on the architecture. Itll help you understand the major differences between

[jira] [Resolved] (MAPREDUCE-3649) Job End notification gives an error on calling back.

2012-01-16 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3649. -- Resolution: Fixed Hadoop Flags: Reviewed I just committed this. Thanks Ravi

[jira] [Created] (MAPREDUCE-3679) AM logs and others should not automatically refresh after every 1 second.

2012-01-16 Thread Mahadev konar (Created) (JIRA)
/Reduce Issue Type: Improvement Affects Versions: 0.23.0 Reporter: Mahadev konar If you are looking through the logs for AM or containers, the page is automatically refreshed after 1 second or so which makes it problematic to search through the page or debug using the

[jira] [Resolved] (MAPREDUCE-3647) Pipes job fails with "Illegal text protocol"

2012-01-16 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3647. -- Resolution: Not A Problem Looks like the issue has been fixed. No longer an issue

Re: Yarn build error

2012-01-12 Thread Mahadev Konar
Milind, The build command has changed for Yarn. Now its uniform all over hadoop. You can run: mvn clean package -DskipTests -Pdist -Dtar -Dmaven.javadoc.skip=true to create tarballs now. mahadev On Thu, Jan 12, 2012 at 2:25 PM, wrote: > Folks, > > I am getting the "Error reading assemblie

[jira] [Created] (MAPREDUCE-3649) Job End notification gives an error on calling back.

2012-01-09 Thread Mahadev konar (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.1 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.1 When calling job end notification for oozie the AM fails with the following trace: {noformat} 2012-01-09 23:45:41,732 WARN

[jira] [Created] (MAPREDUCE-3624) bin/yarn script adds jdk tools.jar to the classpath.

2012-01-05 Thread Mahadev konar (Created) (JIRA)
Reporter: Mahadev konar Assignee: Mahadev konar Thanks to Roman for pointing it out. Looks like we have the following lines in bin/yarn: {code} CLASSPATH=${CLASSPATH}:$JAVA_HOME/lib/tools.jar {code} We dont really have a dependency on the tools jar. We should remove this. -- This

[jira] [Created] (MAPREDUCE-3608) MAPREDUCE-3522 commit causes compilation to fail

2011-12-29 Thread Mahadev konar (Created) (JIRA)
: mrv2 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.1 There are compilation errors after MAPREDUCE-3522 was committed. Some more changes were need to webapps to fix the compilation issue. -- This message is automatically generated by JIRA. If

[jira] [Created] (MAPREDUCE-3587) The deployment tarball should have different directories for yarn jars and mapreduce jars.

2011-12-20 Thread Mahadev konar (Created) (JIRA)
Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Currently all the jars in the mr tarball go to share/hadoop/mapreduce. The jars should be split into: share/hadoop/yarn and share/hadoop/mapreduce for clear

[jira] [Resolved] (MAPREDUCE-3177) mapreduce tar layout does not conform new layout

2011-12-16 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3177. -- Resolution: Duplicate Duplicate of MAPREDUCE-3366. > mapred

[jira] [Created] (MAPREDUCE-3574) Streaming/tools Jar does not get includes in the tarball.

2011-12-16 Thread Mahadev konar (Created) (JIRA)
Components: mrv2 Reporter: Mahadev konar Priority: Blocker Fix For: 0.23.1 The streaming jar used to be available in the mapreduce tarballs before we created the hadoop-tools package. The streaming and tools jars are not being shipped with any tars

[jira] [Created] (MAPREDUCE-3575) Streaming/tools Jar does not get includes in the tarball.

2011-12-16 Thread Mahadev konar (Created) (JIRA)
Components: mrv2 Reporter: Mahadev konar Priority: Blocker Fix For: 0.23.1 The streaming jar used to be available in the mapreduce tarballs before we created the hadoop-tools package. The streaming and tools jars are not being shipped with any tars

[jira] [Created] (MAPREDUCE-3564) TestStagingCleanup and TestJobEndNotifier are failing on trunk.

2011-12-15 Thread Mahadev konar (Created) (JIRA)
Type: Bug Components: mrv2 Reporter: Mahadev konar Priority: Blocker Fix For: 0.23.1 >From recent jenkins test runs: -1 core tests. The patch failed these unit tests: org.apache.hadoop.mapreduce.v2.app.TestStagingClea

[jira] [Created] (MAPREDUCE-3543) Mavenize Gridmix.

2011-12-13 Thread Mahadev konar (Created) (JIRA)
Mavenize Gridmix. - Key: MAPREDUCE-3543 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3543 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Mahadev konar Fix For

Re: Hadoop-Mapreduce-trunk-Commit - Build # 1437 - Still Failing

2011-12-13 Thread Mahadev Konar
Todd, There is a jira for it. https://issues.apache.org/jira/browse/MAPREDUCE-3541 Ravi will be posting a patch ASAP. thanks mahadev On Tue, Dec 13, 2011 at 10:48 AM, Todd Lipcon wrote: > Is anyone working on this? Been failing a few days and my inbox is sad :( > > -Todd > > -- Forwa

[jira] [Created] (MAPREDUCE-3513) Capacity Scheduler web UI has a spelling mistake for Memory.

2011-12-05 Thread Mahadev konar (Created) (JIRA)
: Bug Reporter: Mahadev konar Priority: Trivial The web page for capacity scheduler has a column named "Memopry Total", a spelling mistake which needs to be fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please co

[jira] [Created] (MAPREDUCE-3507) Remove Cluster as a public API on the client side.

2011-12-04 Thread Mahadev konar (Created) (JIRA)
Components: mrv2 Reporter: Mahadev konar Fix For: 0.23.1 This has been a major source of grief for me when fixing MAPREDUCE-3443 and MAPREDUCE-3380. The public API of cluster is very confusing and doesnt add any value to the client side. This has caused more confusion

Re: MapReduce and MPI

2011-12-01 Thread Mahadev Konar
Ralph, I think there is a lot of interest in MPI over Hadoop. If you are counting votes here's my +1. mahadev On Thu, Dec 1, 2011 at 3:31 AM, Ralph Castain wrote: > Hi folks > > I'm a lead developer on the Open MPI project, and recently joined the Hadoop > community to help with Hamster. A cou

[jira] [Created] (MAPREDUCE-3488) Streaming jobs are failing because the main class isnt set in the pom files.

2011-11-30 Thread Mahadev konar (Created) (JIRA)
Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.1 Streaming jobs are failing since the main MANIFEST file isnt being set in the pom files. -- This message is automatically

[jira] [Resolved] (MAPREDUCE-3393) TestMRJobs, TestMROldApiJobs, and TestUberAM failures

2011-11-29 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3393. -- Resolution: Duplicate > TestMRJobs, TestMROldApiJobs, and TestUberAM failu

[jira] [Created] (MAPREDUCE-3443) Oozie jobs are running as oozie user even though they create the jobclient as doAs.

2011-11-21 Thread Mahadev konar (Created) (JIRA)
Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.1 Oozie is having issues with job submission, since it does the

[jira] [Resolved] (MAPREDUCE-3308) MR builds failing due to download failure

2011-10-30 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3308. -- Resolution: Fixed This is no longer an issue. The build works fine. As for the tar

[jira] [Resolved] (MAPREDUCE-3295) TestAMAuthorization failing on branch 0.23.

2011-10-28 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3295. -- Resolution: Fixed > TestAMAuthorization failing on branch 0

Re: Hadoop-Mapreduce-trunk-Commit - Build # 1175 - Still Failing

2011-10-28 Thread Mahadev Konar
Adding Giri. He's fixing some issues with the autoreconf and other software installations. mahadev On Fri, Oct 28, 2011 at 11:12 AM, Todd Lipcon wrote: > Actually, it looks like the libssl0.9.8 package is installed but not > the libssl-dev package. > > Rajiv, would you mind if I installed the li

[jira] [Resolved] (MAPREDUCE-3288) Mapreduce 23 builds failing

2011-10-27 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3288. -- Resolution: Fixed Just fixed this on hudson. A successful build is still dependent

[jira] [Created] (MAPREDUCE-3295) TestAMAuthorization failing on branch 0.23.

2011-10-27 Thread Mahadev konar (Created) (JIRA)
Reporter: Mahadev konar Fix For: 0.23.0 The test seems to fail both on Mac and linux. Trace in the next comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org

[jira] [Created] (MAPREDUCE-3292) In secure mode job submission fails with Provider org.apache.hadoop.mapreduce.security.token.JobTokenIndentifier$Renewer not found.

2011-10-27 Thread Mahadev konar (Created) (JIRA)
-3292 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3292 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority

[jira] [Created] (MAPREDUCE-3242) Trunk compilation broken with bad interaction from MAPREDUCE-3070 and MAPREDUCE-3239.

2011-10-21 Thread Mahadev konar (Created) (JIRA)
Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 Looks like patch command threw away some of the changes when I committed MAPREDUCE

[jira] [Resolved] (MAPREDUCE-3234) Locality scheduling broken due to mismatch between IPs and hosts

2011-10-20 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3234. -- Resolution: Not A Problem > Locality scheduling broken due to mismatch betw

[jira] [Created] (MAPREDUCE-3225) Killing an unkown job throws NPE.

2011-10-19 Thread Mahadev konar (Created) (JIRA)
: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 On a job -kill of an unkown job, the code currently throws a NPE. Stack trace on the next comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly

[jira] [Created] (MAPREDUCE-3203) Fix some javac warnings in MRAppMaster.

2011-10-18 Thread Mahadev konar (Created) (JIRA)
Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 MAPREDUCE-2762 accidentally introduced a couple of javac warning. This jira is to fix some of them in MRAppMaster. We have plenty more to fix but I dont intend to fix them all

Re: Application/Job is hanging! validate

2011-10-17 Thread Mahadev Konar
Kamesh, This is what will happen if you do not have enough resource in the cluster to run a job. Though there could be an improvement that a job fails with not enough resource in the cluster but thats not what happens right now. thanks mahadev On Mon, Oct 17, 2011 at 6:25 AM, Kamesh wrote: > H

[jira] [Created] (MAPREDUCE-3185) RM Web UI does not sort the columns in some cases.

2011-10-13 Thread Mahadev konar (Created) (JIRA)
Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar While running lots of jobs on a MRv2 cluster the RM web UI shows this error on loading the RM web UI: "DataTables warning (table id = 'apps'): Added data (size 8) does n

[jira] [Created] (MAPREDUCE-3170) Trunk builds are failing.

2011-10-11 Thread Mahadev konar (Created) (JIRA)
Trunk builds are failing. - Key: MAPREDUCE-3170 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3170 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Arun C

Re: Yarn NodeManager services

2011-10-11 Thread Mahadev Konar
Shuffle is an auxiliary service in the NM. You might want to take a look at AuxServices.java in the NodeManager codebase. Also, you can add config parameters to yarn.nodemanager.aux-services mapreduce.shuffle (config knob in NodeManagers) to add more configurable services in the NodeMan

[jira] [Created] (MAPREDUCE-3167) container-executor is not being packaged with the assemly target.

2011-10-10 Thread Mahadev konar (Created) (JIRA)
Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.0 Looks like MAPREDUCE-2988 broke this. This is a temporary fix until we get a full fledged maven

Re: Trunk commit builds.

2011-10-10 Thread Mahadev Konar
Thanks Hitesh! mahadev On Mon, Oct 10, 2011 at 11:34 AM, Hitesh Shah wrote: > Opened MAPREDUCE-3158 for fixing the builds. Seems like the change for > default value of framework name to yarn from classic broke a couple of the > MRv1 tests. > > -- Hitesh > > On Oct 9, 2011,

Trunk commit builds.

2011-10-09 Thread Mahadev Konar
Looks like the trunk commit builds are failing/crashing. https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-trunk-Commit/1060/ I suspect its: MAPREDUCE-3033. Hitesh, could you please take a look? We definitely need to mavenize MRv1 as soon as possible. thanks mahadev

Re: A complicated procedure to Run MRV2.

2011-09-29 Thread Mahadev Konar
Nice Prashanth. We've been trying to add documents to make it easier for users to deploy/test/admin mrv2. Here is a doc on setting up a single node cluster. It would be great if you'd want to add/refine it. http://s.apache.org/mrv2_setup Also, there are some more to write: https://issues.apache.o

[jira] [Reopened] (MAPREDUCE-3064) 27 unit test failures with Invalid "mapreduce.jobtracker.address" configuration value for JobTracker: "local"

2011-09-27 Thread Mahadev konar (Reopened) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar reopened MAPREDUCE-3064: -- Reopening the issue, we are still having issues with the build: https

[jira] [Resolved] (MAPREDUCE-3110) Hadoop-Mapreduce-0.23-Build - Build # 29 - Still Failing

2011-09-27 Thread Mahadev konar (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-3110. -- Resolution: Duplicate Duplicate of MAPREDUCE-3054. > Had

[jira] [Created] (MAPREDUCE-3099) Add docs for setting up a single node MRv2 cluster.

2011-09-26 Thread Mahadev konar (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure

[jira] [Created] (MAPREDUCE-3093) Write tests for data locality in MRv2.

2011-09-26 Thread Mahadev konar (JIRA)
Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.0 We should add tests to make sure data locality is in place in MRv2 (with respect to the capacity scheduler and also the matching/ask of containers in the MR AM). -- This

[jira] [Created] (MAPREDUCE-3073) Build failure for MRv1 caused due to changes to MRConstants.

2011-09-22 Thread Mahadev konar (JIRA)
: Bug Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.0 When runnning ant -Dresolvers=internal binary, the build seems to be failing with: [javac] public class JobTracker implements

[jira] [Created] (MAPREDUCE-3018) Streaming jobs with -file option fail to run.

2011-09-15 Thread Mahadev konar (JIRA)
Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.0 Streaming jobs fail to run with the -file option. hadoop jar streaming.jar -input input.txt -output Out -mapper "mapper.sh"

[jira] [Created] (MAPREDUCE-3017) The Web UI shows FINISHED for killed/successful/failed jobs.

2011-09-15 Thread Mahadev konar (JIRA)
: Bug Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.0 The RM web ui shows FINISHED status for all the jobs even if they failed/killed or were successful. This should be fixed. Only the

[jira] [Resolved] (MAPREDUCE-2964) mapreduce trunk build fails with compile-mapred-test ant target

2011-09-12 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2964. -- Resolution: Fixed Just fixed this. No longer an issue on trunk builds

[jira] [Created] (MAPREDUCE-2990) Health Report on Resource Manager UI is null if the NM's are all healthy.

2011-09-12 Thread Mahadev konar (JIRA)
p Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar The web UI on the RM for the link Nodes shows that Health-report as null when the NM is healthy. This is a simple fix where in we can check for null in NodesPage

[jira] [Created] (MAPREDUCE-2975) ResourceManager Delegate is not getting initialized with yarn-site.xml as default configuration.

2011-09-09 Thread Mahadev konar (JIRA)
/MAPREDUCE-2975 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Blocker Fix For: 0.23.0 MAPREDUCE-2937 accidentally changes ResourceMgrDelegate so

[jira] [Created] (MAPREDUCE-2963) TestMRJobs hangs waiting to connect to history server.

2011-09-08 Thread Mahadev konar (JIRA)
Affects Versions: 0.23.0 Reporter: Mahadev konar Priority: Critical Fix For: 0.23.0 TestMRJobs is hanging waiting to connect to history server. I will post the logs next. -- This message is automatically generated by JIRA. For more information on JIRA, see: http

[jira] [Created] (MAPREDUCE-2961) Increase the default threadpool fize for container launching in the application master.

2011-09-08 Thread Mahadev konar (JIRA)
Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 Currently the default threadpool size is 10 for launching containers in

Re: Hadoop Tools Layout (was Re: DistCpV2 in 0.23)

2011-09-07 Thread Mahadev Konar
I like the idea of having tools as a seperate module and I dont think that it will be a dumping ground unless we choose to make one of it. +1 for hadoop tools module under trunk. thanks mahadev On Wed, Sep 7, 2011 at 11:18 AM, Alejandro Abdelnur wrote: > Agreed, we should not have a dumping gro

Re: Mavenization for MR1 - HADOOP-7590

2011-09-07 Thread Mahadev Konar
Yes, I think we need this, both for 0.23 and 0.24. The current form of build is causing a lot of issues. Folks are not aware of checking ant after running maven and unknowingly break the builds. thanks mahadev On Sep 7, 2011, at 7:16 AM, Alejandro Abdelnur wrote: > Do we still want/need to

[jira] [Created] (MAPREDUCE-2937) Errors in Application failures are not shown in the client trace.

2011-09-06 Thread Mahadev konar (JIRA)
Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Priority: Critical Fix For: 0.23.0, 0.24.0 The client side does not show enough information on why the job failed. Here is step to reproduce it: 1) set the scheduler

[jira] [Resolved] (MAPREDUCE-2926) 500 Error in ResourceManager UI

2011-09-06 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2926. -- Resolution: Duplicate I just verified. I dont see the issue anymore. Marking it as

Re: Jenkins's Links to FindBugs warnings not useful

2011-09-02 Thread Mahadev Konar
Thanks Giri! mahadev On Fri, Sep 2, 2011 at 10:42 AM, Giridharan Kesavan wrote: > artifact archive location for mr per-comitt job on jenkins updated. > -Giri > > On Fri, Sep 2, 2011 at 7:33 AM, Vinod Kumar Vavilapalli > wrote: >> None of the links to the warnings related to FindBugs by Jenkins

[jira] [Created] (MAPREDUCE-2916) Ivy build for MRv1 fails with bad organization for common daemon.

2011-08-30 Thread Mahadev konar (JIRA)
Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 This jira is to ignore ivy resolve errors because of bad poms in common daemons. -- This message is automatically generated by JIRA. For more information on JIRA, see: http

[jira] [Created] (MAPREDUCE-2908) Fix findbugs warnings in Map Reduce.

2011-08-29 Thread Mahadev konar (JIRA)
Versions: 0.23.0 Reporter: Mahadev konar Assignee: Mahadev konar Priority: Critical Fix For: 0.23.0 In the current trunk/0.23 codebase there are 5 findbugs warnings which cause the precommit CI builds to -1 the patches. -- This message is automatically

Re: Trunk and 0.23 build failing with clean .m2 directory

2011-08-29 Thread Mahadev Konar
Bobby, You are right. The test-patch uses mvn compile. Please file a jira. It should be a minor change: thanks mahadev On Mon, Aug 29, 2011 at 10:34 AM, Robert Evans wrote: > Thanks Alejandro, > > That really clears things up. Is the a JIRA you know of to change test-patch > to do mvn test -Ds

[jira] [Created] (MAPREDUCE-2886) Fix Javadoc warnings in MapReduce.

2011-08-26 Thread Mahadev konar (JIRA)
Fix Javadoc warnings in MapReduce. -- Key: MAPREDUCE-2886 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2886 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar

Re: DistCpV2 in 0.23

2011-08-25 Thread Mahadev Konar
e rewrite and did not want to remove it until users are familiarized > with new one. > > On 8/26/11 12:51 AM, "Todd Lipcon" wrote: > > Maybe a separate toplevel for hadoop-tools? Stuff like RAID could go > in there as well - ie tools that are downstream of MR and/or H

Fwd: Hadoop-Mapreduce-trunk-Commit - Build # 789 - Still Failing

2011-08-25 Thread Mahadev Konar
Looks like we might need to clean the ivy cache on the build machine? mahadev Begin forwarded message: > From: Apache Jenkins Server > Date: August 25, 2011 12:32:40 PM PDT > To: mapreduce-dev@hadoop.apache.org > Subject: Hadoop-Mapreduce-trunk-Commit - Build # 789 - Still Failing > Reply-To: m

Re: DistCpV2 in 0.23

2011-08-25 Thread Mahadev Konar
+1 for a seperate module in hadoop-mapreduce-project. I think hadoop-mapreduce-client might not be right place for it. We might have to pick a new maven module under hadoop-mapreduce-project that could host streaming/distcp/hadoop archives. thanks mahadev On Thu, Aug 25, 2011 at 11:04 AM, Alejand

[jira] [Created] (MAPREDUCE-2877) Add missing Apache license header in some files in MR and also add the rat plugin to the poms.

2011-08-25 Thread Mahadev konar (JIRA)
-2877 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 Some of the files in MR have a missing Apache header files. We also need to add the apache-rat plugin to be able to run rat

[jira] [Created] (MAPREDUCE-2870) MR tests are failing webapps/hdfs not found in CLASSPATH

2011-08-22 Thread Mahadev konar (JIRA)
Reporter: Mahadev konar Fix For: 0.23.0 While running ant tests, the tests are failing with the following trace: {noformat} webapps/hdfs not found in CLASSPATH java.io.FileNotFoundException: webapps/hdfs not found in CLASSPATH at

[jira] [Resolved] (MAPREDUCE-2848) Upgrade avro to 1.5.2

2011-08-22 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2848. -- Resolution: Fixed checked in as part of MAPREDUCE-2868. > Upgrade avro to 1.

[jira] [Resolved] (MAPREDUCE-2867) Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager.

2011-08-22 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2867. -- Resolution: Fixed Removed the file via svn commit revision 1160340. > Rem

[jira] [Created] (MAPREDUCE-2867) Remove Unused TestApplicaitonCleanup in resourcemanager/applicationsmanager.

2011-08-22 Thread Mahadev konar (JIRA)
Map/Reduce Issue Type: Bug Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 TestApplicationCleanup in resourcemanager/applicationsmanager doesnt do anything. There is already a test in resourcemanager/TestApplicationCleanup which tests

Re: Notes for working on mapreduce trunk after the MR-279 merge.

2011-08-19 Thread Mahadev Konar
Fri, Aug 19, 2011 at 10:28 AM, Todd Lipcon wrote: >> Let me ping Jukka and see if he has any idea why it's not auto-updating >> anymore. >> >> -Todd >> >> On Fri, Aug 19, 2011 at 10:26 AM, Mahadev Konar >> wrote: >>> I am seeing this issue

Re: Notes for working on mapreduce trunk after the MR-279 merge.

2011-08-19 Thread Mahadev Konar
I am seeing this issue again. Todd, do we need to fix something else? I committed 2 patches one yest and one today and I cant get it on the git branch. Something off with git pulling from svn. Any other fix we need to do? thanks mahadev On Aug 18, 2011, at 10:05 AM, Vinod Kumar Vavilapalli wr

[jira] [Created] (MAPREDUCE-2860) Fix log4j logging in the maven test cases.

2011-08-18 Thread Mahadev konar (JIRA)
Reporter: Mahadev konar Assignee: Mahadev konar Fix For: 0.23.0 At present the logging in the new test cases is broken because surefire isnt able to find the log4j properties file. -- This message is automatically generated by JIRA. For more information on JIRA

Re: Building MR2

2011-08-18 Thread Mahadev Konar
Eli, I think its protoc. Do you have protoc installed? mahadev On Thu, Aug 18, 2011 at 10:28 AM, Eli Collins wrote: > Hey gang, > > I'm running mvn clean install from the hadoop-mapreduce directory on > an  svn co of trunk (hadoop-common) and get the following: > > [ERROR] Failed to execute goa

Re: [VOTE] Merge MR-279 to trunk.

2011-08-17 Thread Mahadev Konar
waway branch first (right > now) to ensure we don't mess it up, so please don't be surprised to see us do > this 'twice'. :) > > Arun > > On Aug 16, 2011, at 2:14 PM, Mahadev Konar wrote: > >> Hi all, >> >> We are excited to let you

[VOTE] Merge MR-279 to trunk.

2011-08-16 Thread Mahadev Konar
Hi all, We are excited to let you know that we have MR-279 ready to be merged to trunk. I have uploaded necessary details on https://issues.apache.org/jira/browse/MAPREDUCE-279. Please take a look and vote. Clearly I am +1 on it. thanks mahadev

Re: Can't find main class when run ResourceManager or NodeManager

2011-08-11 Thread Mahadev Konar
or@0.0.0.0:8088 > 2011-08-11 15:25:44,775 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ResourceManager > metrics system... > 2011-08-11 15:25:44,776 INFO > org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping metrics source > QueueMetrics,q0=default > 2011

Re: Can't find main class when run ResourceManager or NodeManager

2011-08-11 Thread Mahadev Konar
Matei, Are you sure you are using the latest common from trunk? The ClientCache was recently added within the last few weeks. Looks like its using some old version of hadoop common. Also the sl4j errors should be fixed in the latest MR-279 branch. thanks mahadev On Aug 11, 2011, at 1:52 P

Re: Not able to execute MRv2

2011-08-09 Thread Mahadev Konar
Hi Praveen, This is an issue because of incompatible versions of avro. This occurs if you have avro 1.3.2 in your cp. We use avro 1.4. For the time being you can remove any occurence of avro 1.3.2. jar in your classpath and that should get you out of the error below. For fixing it in the righ

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

2011-08-05 Thread Mahadev Konar
ime. We hope to start the vote next week sometime. > > > >thanks > >mahadev > > > >On Fri, Jul 29, 2011 at 3:14 PM, Mahadev Konar > >wrote: > > > >> Good point! We are working on this. Hopefully should get this done > >>within a >

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

2011-08-05 Thread Mahadev Konar
Hi all, We are almost done with fixing findbugs issues. Once done, we plan to start a vote to merge MR-279 into trunk. If you plan to review, this would be a good time. We hope to start the vote next week sometime. thanks mahadev On Fri, Jul 29, 2011 at 3:14 PM, Mahadev Konar wrote: > G

Re: trouble building MR-279 branch

2011-08-05 Thread Mahadev Konar
Thanks for pointing out Ravi. We are working on fixing it. The jars that you mention are created as part of MR-279 build. For building MR-279, all you have to do in mvn clean install -DskipTests and that should work (given you have already installed common and hdfs from trunk). thanks mahadev O

[jira] [Resolved] (MAPREDUCE-2776) MR 279: Fix some of the yarn findbug warnings

2011-08-03 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2776. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just pushed this. thanks sid

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

2011-07-29 Thread Mahadev Konar
, Todd Lipcon wrote: > Hey Mahadev, > > Have you been running findbugs/RAT on the branch? Other test-patch like > things? > > -Todd > > On Fri, Jul 29, 2011 at 12:06 AM, Mahadev Konar > wrote: > >> HI all, >> For those working on MR-279. >> >&g

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

2011-07-29 Thread Mahadev Konar
Wed, Jul 27, 2011 at 9:46 AM, Mahadev Konar wrote: > Just an update. > > Some good news. I have a patch for MR-279 to start working with trunk/common > and trunk/hdfs (some minor fixes). I am facing some issues with secuirty > which should be sorted out by EOD today. > We

[jira] [Resolved] (MAPREDUCE-2707) ProtoOverHadoopRpcEngine without using TunnelProtocol over WritableRpc

2011-07-28 Thread Mahadev konar (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar resolved MAPREDUCE-2707. -- Resolution: Fixed Hadoop Flags: [Reviewed] I just committed this to MR-279

Re: Merge MR-279 (was Re: Status on MAPREDUCE-279 and plans forward)

2011-07-27 Thread Mahadev Konar
Just an update. Some good news. I have a patch for MR-279 to start working with trunk/common and trunk/hdfs (some minor fixes). I am facing some issues with secuirty which should be sorted out by EOD today. We should be ready to start the process of merging soon. thanks mahadev On Jul 12, 201

Re: Jiras need review

2011-07-12 Thread Mahadev Konar
Hi Josh, Ill review the jira's by EOD today. We should clean up our PA's (currently we have 102 of them) and get it down to reasonable/maintainable/searchable number. thanks mahadev On Tue, Jul 12, 2011 at 9:13 AM, Josh Wills wrote: > Please add me (jwills) to that query, which should grab MR-26

Re: MR-279 branch: Problem opening files on the namenode web console (need to pick patches from trunk to resolve)

2011-07-08 Thread Mahadev Konar
HI Amhed, We are still working to start using trunk hdfs and trunk common for MR-279 (mapreduce). I think we are almost there and should be ready next week. For now we can probably skip fixing it since we are anyway going to move to trunk hdfs. Does that seem reasonable? thanks mahadev On Fri, J

Re: Building and Deploying MRv2

2011-06-15 Thread Mahadev Konar
I just checked in Tom's patch for updated INSTALL instructions. Hope that helps. thanks mahadev On Wed, Jun 15, 2011 at 7:49 AM, Thomas Graves wrote: > > > On 6/15/11 8:54 AM, "Praveen Sripati" wrote: > >> Hi, >> >> I am trying to build and deploy MRv2 and following the instructions in the >> I

Re: Queries on MRv2

2011-06-14 Thread Mahadev Konar
s the container once the container > is done. > > Thanks, > Praveen > > On Wed, Jun 15, 2011 at 8:59 AM, Mahadev Konar wrote: > >> Praveen, >>  Answers in line: >> >> > >> > Q) What happens if an ApplicationMaster asks a NM to launch a con

  1   2   >