Re: Jiras need review

2011-07-12 Thread Josh Wills
FWIW, I'd still like my JIRAs reviewed. :) On Tue, Jul 12, 2011 at 11:30 AM, Thomas Graves wrote: > Apologies to all - this was my screw up, this went to the wrong mailing > list. > > Please disregard this request. > > Tom > > > On 7/12/11 1:22 PM, "Allen Wittenauer" wrote: > > > > > On Jul 12,

Re: Jiras need review

2011-07-12 Thread Josh Wills
v > > On Tue, Jul 12, 2011 at 9:13 AM, Josh Wills wrote: > > Please add me (jwills) to that query, which should grab MR-2630 and > > MR-2641-44. > > > > On Tue, Jul 12, 2011 at 8:13 AM, Nathan Roberts >wrote: > > > >> I sent this query to Suresh y

Re: Jiras need review

2011-07-12 Thread Josh Wills
Please add me (jwills) to that query, which should grab MR-2630 and MR-2641-44. On Tue, Jul 12, 2011 at 8:13 AM, Nathan Roberts wrote: > I sent this query to Suresh yesterday. He is supposed to be driving this > from Hortonworks side, if this list doesn't look correct, let me know. > > project in

Re: Problem while running eclipse-files for Next Gen Mapreduce branch

2011-07-08 Thread Josh Wills
configuration file could tell Yarn the location of HDFS? This could > be > found on > > http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL > > Thank you. > > Best, > > Haoyuan > > On Fri, Jul 8, 2011 at 10:58 PM, Josh Wills wrote:

Re: Problem while running eclipse-files for Next Gen Mapreduce branch

2011-07-08 Thread Josh Wills
> > --Bobby > > On 7/8/11 9:04 AM, "Josh Wills" wrote: > > You want to generate them using mvn instead. See the mapreduce/yarn/README > file for how to do it. > > On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K wrote: > > > Hi, > > > > > &g

Re: Problem while running eclipse-files for Next Gen Mapreduce branch

2011-07-08 Thread Josh Wills
You want to generate them using mvn instead. See the mapreduce/yarn/README file for how to do it. On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K wrote: > Hi, > > > > I am getting this below errors when I try to generate eclipse files using > eclipse-files target. Can anybody help me? > > > > > > B

[jira] [Created] (MAPREDUCE-2645) Updates to MRv2 INSTALL documentation

2011-07-05 Thread Josh Wills (JIRA)
Reporter: Josh Wills Assignee: Josh Wills Attachments: MAPREDUCE-2645.patch There are a few issues w/the current INSTALL document for MRv2 that I came across when I attempted to get it running: 1) Correct the mvn arg for skipping tests, 2) Add a step to start the yarn

[jira] [Created] (MAPREDUCE-2644) NodeManager fails to create containers when NM_LOG_DIR is not explicitly set in the Configuration

2011-07-05 Thread Josh Wills (JIRA)
/MAPREDUCE-2644 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Reporter: Josh Wills Assignee: Josh Wills If the yarn configuration does not explicitly specify a value for the yarn.server.nodemanager.log.dir property, container

Re: Introduction

2011-07-05 Thread Josh Wills
port issues and fix them! > > Arun > > On Jul 5, 2011, at 8:34 AM, Josh Wills wrote: > > > Hello MR Developers, > > > > My name is Josh, I started working at Cloudera last week as a data > > scientist. I spent my last three years at Google, where I worked on the

Introduction

2011-07-05 Thread Josh Wills
Hello MR Developers, My name is Josh, I started working at Cloudera last week as a data scientist. I spent my last three years at Google, where I worked on the ads system for awhile and then built alot of the data analysis and experiment infrastructure behind Google+. I am really excited about the

[jira] [Created] (MAPREDUCE-2643) Fixing typos/formatting/null checking in v2.app.speculate package

2011-07-05 Thread Josh Wills (JIRA)
Issue Type: Sub-task Components: mrv2 Reporter: Josh Wills Assignee: Josh Wills Priority: Minor No functional changes in this patch: just fixing some typos in the comments, fixing some formatting issues, renaming classes for consistency, and making the

[jira] [Created] (MAPREDUCE-2642) Fix two bugs in v2.app.speculate.DataStatistics

2011-07-05 Thread Josh Wills (JIRA)
Components: mrv2 Reporter: Josh Wills Assignee: Josh Wills Priority: Minor Fixes two bugs in DataStatistics: a divide by zero in the variance calculation when count == 0, and a synchronization issue in how the updateStatistics method was implemented. -- This message

[jira] [Created] (MAPREDUCE-2641) Fix the ExponentiallySmoothedTaskRuntimeEstimator and its unit test

2011-07-05 Thread Josh Wills (JIRA)
Issue Type: Sub-task Components: mrv2 Reporter: Josh Wills Assignee: Josh Wills Priority: Minor Attachments: MAPREDUCE-2641.patch Fixed the ExponentiallySmoothedTaskRuntimeEstimator so that it can run and pass the test defined for it in

[jira] [Created] (MAPREDUCE-2639) MR-279: Fixup the exponentially smoothed runtime estimator, fix a couple of bugs in DataStatistics, and do a little bit of cleanup.

2011-07-03 Thread Josh Wills (JIRA)
-2639 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2639 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Environment: All Reporter: Josh Wills Assignee: Josh Wills Priority: Minor

Re: trouble building MR-279 branch

2011-07-01 Thread Josh Wills
Hey Anupam, I think Jim is asking about building non-MR application masters using the RM/NM. I think Arun has promised to post a design doc soon. In the meantime, I'm just reading through the code and fixing little issues I come across (e.g., https://issues.apache.org/jira/browse/MAPREDUCE-2630 an

[jira] [Created] (MAPREDUCE-2633) MR-279: Add a getCounter(Enum) method to the Counters interface

2011-06-30 Thread Josh Wills (JIRA)
Type: Improvement Components: mrv2 Environment: All Reporter: Josh Wills Priority: Minor I'm fixing a few TODOs I came across in TaskAttemptImpl.java related to the fact that the MRv2 Counters interface don't expose a getCounter(Enum) m

[jira] [Created] (MAPREDUCE-2630) MR-279: refreshQueues leads to NPEs when used w/FifoScheduler

2011-06-28 Thread Josh Wills (JIRA)
: Bug Components: mrv2 Environment: All Reporter: Josh Wills Priority: Minor The RM's admin service exposes a method refreshQueues that is used to update the queue configuration when used with the CapacityScheduler, but if it is used wit

Re: mappers

2011-06-26 Thread Josh Wills
(redirected to mapreduce-user@, mapreduce-dev bcc'd) The param you're referring to controls the maximum number of simultaneously active mappers on a given task tracker, i.e., how many map slots are available on that node. But a single task tracker can be used for multiple MR jobs, so you can't loo

Re: Queries on MRv2

2011-06-15 Thread Josh Wills
Hey Praveen, I'm in the same boat as you re: getting started with the MR2 code. I have a couple of answers and a couple of followup questions for Arun et al. to keep in mind as they're writing a design doc. On Wed, Jun 15, 2011 at 5:27 AM, Praveen Sripati wrote: > Hi, > > - How to specify that a