[jira] [Resolved] (MAPREDUCE-3936) Clients should not enforce counter limits

2015-05-11 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-3936. -- Resolution: Won't Fix > Clients should not enforce counte

Re: Abstraction layer to support both YARN and Mesos

2013-07-31 Thread Tom White
I can see value in this, since it would allow MR programs and libraries to run on either YARN or Mesos with no recompilation. The value here is really in the libraries since it means library maintainers don't have to maintain two versions of their library. Note that there is no extra level of indi

Re: [VOTE] Release Apache Hadoop 2.0.4-alpha

2013-04-19 Thread Tom White
+1 Checked sigs and checksums, source tag, and built from source. Cheers, Tom On Fri, Apr 12, 2013 at 2:56 PM, Arun C Murthy wrote: > Folks, > > I've created a release candidate (RC2) for hadoop-2.0.4-alpha that I would > like to release. > > The RC is available at: > http://people.apache.org

[jira] [Resolved] (MAPREDUCE-5035) Update MR1 memory configuration docs

2013-02-27 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-5035. -- Resolution: Fixed Fix Version/s: 1.2.0 Hadoop Flags: Reviewed I just committed

[jira] [Created] (MAPREDUCE-5035) Update MR1 memory configuration docs

2013-02-27 Thread Tom White (JIRA)
Tom White created MAPREDUCE-5035: Summary: Update MR1 memory configuration docs Key: MAPREDUCE-5035 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5035 Project: Hadoop Map/Reduce Issue

Re: Release numbering for branch-2 releases

2013-02-01 Thread Tom White
On Wed, Jan 30, 2013 at 11:32 PM, Vinod Kumar Vavilapalli wrote: > I still have a list of pending API/protocol cleanup in YARN that need to be > in before we even attempt supporting compatibility further down the road. To let others track these it would be useful if they were tagged in JIRA with

Re: Release numbering for branch-2 releases

2013-02-01 Thread Tom White
Possibly the reason for Stack's consternation is that this is a Hadoop-specific versioning scheme, rather than a standard one like Semantic Versioning (http://semver.org/) which is more widely understood. With that scheme we would have something like 2.0.0-alpha, 2.0.0-alpha.1, 2.0.0-alpha.2, 2

[jira] [Resolved] (MAPREDUCE-4850) Job recovery may fail if staging directory has been deleted

2013-01-09 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-4850. -- Resolution: Fixed Fix Version/s: 1.2.0 Hadoop Flags: Reviewed I ran test-patch

[jira] [Created] (MAPREDUCE-4850) Job recovery may fail if staging directory has been deleted

2012-12-05 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4850: Summary: Job recovery may fail if staging directory has been deleted Key: MAPREDUCE-4850 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4850 Project: Hadoop

[jira] [Created] (MAPREDUCE-4830) Restore JT DelegationTokenSecretManager state on restart

2012-11-29 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4830: Summary: Restore JT DelegationTokenSecretManager state on restart Key: MAPREDUCE-4830 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4830 Project: Hadoop Map

[jira] [Created] (MAPREDUCE-4824) Provide a mechanism for jobs to indicate they should not be recovered on restart

2012-11-27 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4824: Summary: Provide a mechanism for jobs to indicate they should not be recovered on restart Key: MAPREDUCE-4824 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4824

[jira] [Created] (MAPREDUCE-4637) Killing an unassigned task attempt causes the job to fail

2012-09-05 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4637: Summary: Killing an unassigned task attempt causes the job to fail Key: MAPREDUCE-4637 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4637 Project: Hadoop Map

[jira] [Created] (MAPREDUCE-4632) Make sure MapReduce declares correct set of dependencies

2012-09-04 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4632: Summary: Make sure MapReduce declares correct set of dependencies Key: MAPREDUCE-4632 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4632 Project: Hadoop Map

[jira] [Reopened] (MAPREDUCE-4567) Fix failing TestJobKillAndFail in branch-1

2012-08-31 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White reopened MAPREDUCE-4567: -- Reverted and reopened along with MAPREDUCE-4488 - see that issue for details

[jira] [Reopened] (MAPREDUCE-4488) Port MAPREDUCE-463 (The job setup and cleanup tasks should be optional) to branch-1

2012-08-31 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White reopened MAPREDUCE-4488: -- > Port MAPREDUCE-463 (The job setup and cleanup tasks should be optional) to > br

[jira] [Created] (MAPREDUCE-4610) Support deprecated mapreduce.job.counters.limit property in MR2

2012-08-30 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4610: Summary: Support deprecated mapreduce.job.counters.limit property in MR2 Key: MAPREDUCE-4610 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4610 Project

[jira] [Resolved] (MAPREDUCE-4567) Fix failing TestJobKillAndFail in branch-1

2012-08-21 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-4567. -- Resolution: Fixed Hadoop Flags: Reviewed I committed this to branch-1

[jira] [Created] (MAPREDUCE-4567) Fix failing TestJobKillAndFail in branch-1

2012-08-20 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4567: Summary: Fix failing TestJobKillAndFail in branch-1 Key: MAPREDUCE-4567 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4567 Project: Hadoop Map/Reduce

[jira] [Resolved] (MAPREDUCE-4488) Port MAPREDUCE-463 (The job setup and cleanup tasks should be optional) to branch-1

2012-08-13 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-4488. -- Resolution: Fixed Hadoop Flags: Reviewed I just committed this to branch-1. (Ran

[jira] [Created] (MAPREDUCE-4488) Port MAPREDUCE-463 (The job setup and cleanup tasks should be optional) to branch-1

2012-07-26 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4488: Summary: Port MAPREDUCE-463 (The job setup and cleanup tasks should be optional) to branch-1 Key: MAPREDUCE-4488 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4488

[jira] [Created] (MAPREDUCE-4487) Reduce job latency by removing hardcoded sleep statements

2012-07-26 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4487: Summary: Reduce job latency by removing hardcoded sleep statements Key: MAPREDUCE-4487 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4487 Project: Hadoop Map

[jira] [Created] (MAPREDUCE-4463) JobTracker recovery fails with HDFS permission issue

2012-07-19 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4463: Summary: JobTracker recovery fails with HDFS permission issue Key: MAPREDUCE-4463 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4463 Project: Hadoop Map/Reduce

[jira] [Resolved] (MAPREDUCE-4257) Support fair-sharing option within a MR2 Capacity Scheduler queue

2012-07-11 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-4257. -- Resolution: Invalid Yes, I don't think we need this. > Supp

[jira] [Resolved] (MAPREDUCE-3837) Hadoop 22 Job tracker is not able to recover job in case of crash and after that no user can submit job.

2012-07-03 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-3837. -- Resolution: Fixed Fix Version/s: (was: 1.1.1) 1.2.0 I just

[jira] [Created] (MAPREDUCE-4337) Support multiple reducers in 'uber' jobs

2012-06-11 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4337: Summary: Support multiple reducers in 'uber' jobs Key: MAPREDUCE-4337 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4337 Project: Hadoop

[jira] [Created] (MAPREDUCE-4299) Terasort hangs with MR2 FifoScheduler

2012-05-31 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4299: Summary: Terasort hangs with MR2 FifoScheduler Key: MAPREDUCE-4299 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4299 Project: Hadoop Map/Reduce

[jira] [Created] (MAPREDUCE-4252) MR2 job never completes with 1 pending task

2012-05-11 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4252: Summary: MR2 job never completes with 1 pending task Key: MAPREDUCE-4252 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4252 Project: Hadoop Map/Reduce

[jira] [Created] (MAPREDUCE-4230) Ensure framework counter group synchronization is correct for serialization/deserialization

2012-05-07 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4230: Summary: Ensure framework counter group synchronization is correct for serialization/deserialization Key: MAPREDUCE-4230 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4230

[jira] [Created] (MAPREDUCE-4226) ConcurrentModificationException in FileSystemCounterGroup

2012-05-04 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4226: Summary: ConcurrentModificationException in FileSystemCounterGroup Key: MAPREDUCE-4226 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4226 Project: Hadoop Map

[jira] [Resolved] (MAPREDUCE-4217) Task commit waits for up to 3 seconds

2012-05-01 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-4217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-4217. -- Resolution: Duplicate MAPREDUCE-3809 is the same. Thanks for pointing it out Siddharth

[jira] [Created] (MAPREDUCE-4217) Task commit waits for up to 3 seconds

2012-05-01 Thread Tom White (JIRA)
Tom White created MAPREDUCE-4217: Summary: Task commit waits for up to 3 seconds Key: MAPREDUCE-4217 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4217 Project: Hadoop Map/Reduce

[jira] [Created] (MAPREDUCE-4168) Support multiple network interfaces

2012-04-20 Thread Tom White (Created) (JIRA)
Support multiple network interfaces --- Key: MAPREDUCE-4168 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4168 Project: Hadoop Map/Reduce Issue Type: New Feature Reporter: Tom White

[jira] [Created] (MAPREDUCE-4158) Port mapreduce.MapFileOutputFormat to branch-1

2012-04-16 Thread Tom White (Created) (JIRA)
Components: client Affects Versions: 1.0.2 Reporter: Tom White Assignee: Tom White MapFileOutputFormat was missed in MAPREDUCE-3607. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https

[jira] [Resolved] (MAPREDUCE-3378) Create a single 'hadoop-mapreduce' Maven artifact

2012-04-13 Thread Tom White (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-3378. -- Resolution: Won't Fix I've opened HADOOP-8278 to track 1. HADOOP-8009 addressed

[jira] [Created] (MAPREDUCE-4148) MapReduce should not have a compile-time dependency on HDFS

2012-04-12 Thread Tom White (Created) (JIRA)
Reporter: Tom White Assignee: Tom White MapReduce depends on HDFS's DelegationTokenIdentifier (for printing token debug information). We should remove this dependency and MapReduce's compile-time dependency on HDFS. -- This message is automatically generated b

[jira] [Created] (MAPREDUCE-4147) YARN should not have a compile-time dependency on HDFS

2012-04-12 Thread Tom White (Created) (JIRA)
Affects Versions: 0.23.1 Reporter: Tom White Assignee: Tom White Attachments: MAPREDUCE-4147.patch YARN doesn't (and shouldn't) use any HDFS-specific APIs, so it should not declare HDFS as a compile-time dependency. -- This message is automatically generat

[jira] [Created] (MAPREDUCE-4146) Support limits on task status string length and number of block locations in branch-2

2012-04-12 Thread Tom White (Created) (JIRA)
Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Tom White This brings MAPREDUCE-1943 to branch-2. Counter limits were introduced in MAPREDUCE-901. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your

[jira] [Created] (MAPREDUCE-4138) Reduce memory usage of counters due to non-static nested classes

2012-04-11 Thread Tom White (Created) (JIRA)
Type: Improvement Reporter: Tom White Assignee: Tom White FrameworkCounter is a non-static nested class of FrameworkCounterGroup which means it retains a reference to the outer class, which isn't really needed. -- This message is automatically generated by JIRA. I

[jira] [Resolved] (MAPREDUCE-1572) MapReduceDriver combiner functionality

2012-03-19 Thread Tom White (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-1572. -- Resolution: Duplicate > MapReduceDriver combiner functional

[jira] [Resolved] (MAPREDUCE-2835) Make per-job counter limits configurable

2012-03-14 Thread Tom White (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-2835. -- Resolution: Fixed Fix Version/s: 1.1.0 Hadoop Flags: Reviewed I've

[jira] [Created] (MAPREDUCE-3936) Improve counter limits behaviour in 1.x

2012-02-28 Thread Tom White (Created) (JIRA)
Reporter: Tom White The code for enforcing counter limits (from MAPREDUCE-1943) creates a static JobConf instance to load the limits, which may throw an exception if the client limit is set to be lower than the limit on the cluster (perhaps because the cluster limit was raised from the

[jira] [Created] (MAPREDUCE-3935) Annotate Counters.Counter and Counters.Group as @Public

2012-02-28 Thread Tom White (Created) (JIRA)
Components: client Reporter: Tom White Assignee: Tom White For clarity these inner classes should be marked as public stable. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators

[jira] [Created] (MAPREDUCE-3871) Allow symlinking in LocalJobRunner DistributedCache

2012-02-16 Thread Tom White (Created) (JIRA)
Components: distributed-cache Reporter: Tom White Assignee: Tom White Attachments: MAPREDUCE-3871.patch Currently the LocalJobRunner doesn't create symlinks for files in the DistributedCache. It is safe to create symlinks if files of the same name don

[jira] [Created] (MAPREDUCE-3858) Task attempt failure during commit results in task never completing

2012-02-13 Thread Tom White (Created) (JIRA)
Issue Type: Bug Components: mrv2 Reporter: Tom White Assignee: Tom White Priority: Critical Attachments: MAPREDUCE-3858.patch On a terasort job a task attempt failed during the commit phase. Another attempt was rescheduled, but when it tried

[jira] [Created] (MAPREDUCE-3854) Reinstate environment variable tests in TestMiniMRChildTask

2012-02-10 Thread Tom White (Created) (JIRA)
: Test Components: mrv2 Reporter: Tom White Fix For: 0.23.2 MAPREDUCE-3716 reinstated one of the tests in TestMiniMRChildTask, but there are two more which should be run. -- This message is automatically generated by JIRA. If you think it was sent incorrectly

[jira] [Created] (MAPREDUCE-3794) Support mapred.Task.Counter and mapred.JobInProgress.Counter enums for compatibility

2012-02-02 Thread Tom White (Created) (JIRA)
Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.0 Reporter: Tom White Assignee: Tom White The new counters are mapreduce.TaskCounter and mapreduce.JobCounter, but we should support the old ones too since they are public in Hadoop 1.x

[jira] [Created] (MAPREDUCE-3750) ConcurrentModificationException in counter groups

2012-01-27 Thread Tom White (Created) (JIRA)
Reporter: Tom White Priority: Blocker Iterating over a counter's groups while adding more groups results in a ConcurrentModificationException. This was discovered while running Hive unit tests on a recent 0.23 version of Hadoop. -- This message is automatically generated by JIRA. I

[jira] [Created] (MAPREDUCE-3749) ConcurrentModificationException in counter groups

2012-01-27 Thread Tom White (Created) (JIRA)
Versions: 0.23.0 Reporter: Tom White Priority: Blocker Iterating over a counter's groups while adding more groups will cause a ConcurrentModificationException. This was found while running Hive unit tests against a recent 0.23 version. -- This message is automatically gene

[jira] [Resolved] (MAPREDUCE-3607) Port missing new API mapreduce lib classes to 1.x

2012-01-24 Thread Tom White (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-3607. -- Resolution: Fixed Fix Version/s: 1.0.1 Hadoop Flags: Reviewed Thanks for the

[jira] [Created] (MAPREDUCE-3684) LocalDistributedCacheManager does not shut down its thread pool

2012-01-17 Thread Tom White (Created) (JIRA)
Type: Bug Components: client Affects Versions: 0.23.0 Reporter: Tom White Assignee: Tom White This was observed by running a Hive job in local mode. The job completed but the client process did not exit for 60 seconds. -- This message is automatically

[jira] [Created] (MAPREDUCE-3636) Apply audience and stability annotations to classes in MapReduce for 1.x

2012-01-06 Thread Tom White (Created) (JIRA)
Issue Type: Improvement Components: documentation Reporter: Tom White Port MAPREDUCE-1623 to branch-1. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org

[jira] [Created] (MAPREDUCE-3607) Port missing new API mapreduce lib classes to 1.x

2011-12-28 Thread Tom White (Created) (JIRA)
Components: client Affects Versions: 1.0.0 Reporter: Tom White Assignee: Tom White There are a number of classes under mapreduce.lib that are not present in the 1.x series. Including these would help users and downstream projects using the new MapReduce API migrate to

[jira] [Created] (MAPREDUCE-3595) Add missing TestCounters#testCounterValue test from branch 1 to 0.23

2011-12-22 Thread Tom White (Created) (JIRA)
Issue Type: Test Components: test Reporter: Tom White Assignee: Tom White Attachments: MAPREDUCE-3595.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https

[jira] [Resolved] (MAPREDUCE-3467) Mavenizing har

2011-12-22 Thread Tom White (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-3467. -- Resolution: Duplicate This was fixed in HADOOP-7810. > Mavenizing

[jira] [Created] (MAPREDUCE-3542) Support "FileSystemCounter" legacy counter group name for compatibility

2011-12-13 Thread Tom White (Created) (JIRA)
Map/Reduce Issue Type: Bug Affects Versions: 0.23.0 Reporter: Tom White Assignee: Tom White The group name changed from "FileSystemCounter" to "org.apache.hadoop.mapreduce.FileSystemCounter", but we should support the old one for compati

[jira] [Created] (MAPREDUCE-3527) Fix minor API incompatibilities between 1.0 and 0.23

2011-12-09 Thread Tom White (Created) (JIRA)
Versions: 0.23.0 Reporter: Tom White Assignee: Tom White There are a few minor incompatibilities that were found in HADOOP-7738 and are straightforward to fix. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your

[jira] [Resolved] (MAPREDUCE-3428) MR AppMaster CLASSPATH is dependent on the compile-time environment

2011-12-02 Thread Tom White (Resolved) (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-3428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-3428. -- Resolution: Duplicate > MR AppMaster CLASSPATH is dependent on the compile-t

[jira] [Created] (MAPREDUCE-3479) JobClient#getJob cannot find local jobs

2011-11-28 Thread Tom White (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Tom White Assignee: Tom White The problem is that JobClient#submitJob doesn't pass the Cluster object to Job for the submission process, which means that two Cluster objects and two LocalJobRunner objects are created. LocalJobRunner kee

[jira] [Created] (MAPREDUCE-3433) Finding counters by legacy group name returns empty counters

2011-11-18 Thread Tom White (Created) (JIRA)
: Sub-task Components: client, mrv2 Reporter: Tom White Assignee: Tom White Attempting to find counters with a legacy group name (e.g. org.apache.hadoop.mapred.Task$Counter rather than the new org.apache.hadoop.mapreduce.TaskCounter) returns empty counters. This

[jira] [Created] (MAPREDUCE-3422) Counter display names are not being picked up

2011-11-16 Thread Tom White (Created) (JIRA)
Affects Versions: 0.23.0 Reporter: Tom White When running a job I see "MAP_INPUT_RECORDS" rather than "Map input records" for the counter name. To fix this the resource bundle properties files need to be moved to the src/main/resources tree. -- This mess

[jira] [Created] (MAPREDUCE-3416) Rename {start,stop}-all.sh to {start,stop}-yarn.sh for consistency with HDFS

2011-11-16 Thread Tom White (Created) (JIRA)
Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Tom White There are already {start,stop}-all.sh scripts for starting and stopping all Hadoop daemons which conflict with the YARN {start,stop}-all.sh scripts. The latter

[jira] [Created] (MAPREDUCE-3378) Create a single 'hadoop-mapreduce' Maven artifact

2011-11-08 Thread Tom White (Created) (JIRA)
mprovement Components: build Affects Versions: 0.23.0 Reporter: Tom White In 0.23.0 there are multiple artifacts (hadoop-mapreduce-client-app, hadoop-mapreduce-client-common, hadoop-mapreduce-client-core, etc). It would be simpler for users to declare a dependency on hadoop-mapre

Re: hadoop-0.23.0 jiras

2011-10-31 Thread Tom White
Hi Arun, A couple more issues that I think are blockers: HADOOP-7782 (Aggregate project javadocs) should be a blocker since otherwise private APIs are published as a part of the public API. MAPREDUCE-3237 (Move LocalJobRunner to hadoop-mapreduce-client-core module) is also a blocker since without

[jira] [Created] (MAPREDUCE-3237) Move LocalJobRunner to hadoop-mapreduce-client-core module

2011-10-20 Thread Tom White (Created) (JIRA)
: Improvement Components: client Reporter: Tom White Fix For: 0.23.0 LocalJobRunner works independently of MR1 (jobtracker and tasktrackers) and MR2 (YARN). The MR1 directory is being kept around only to support unit tests, so LocalJobRunner should be moved out to

[jira] [Created] (MAPREDUCE-3003) Publish Yarn and MapReduce artifacts to Maven snapshot repository

2011-09-13 Thread Tom White (JIRA)
Issue Type: Bug Components: build Reporter: Tom White Assignee: Tom White Currently this is failing since no distribution management section is defined in the POM. https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Common-trunk-Commit/883/consoleFull

Re: Problem getting the code from SVN

2011-08-22 Thread Tom White
On Sun, Aug 21, 2011 at 11:13 AM, Konstantin Boudnik wrote: > Seems like it has been broken in the recent re-shuffle of the workspace due to > the maven changes. This external now lives under hadoop-common, apparently. It's actually still under common, however it's no longer needed for HDFS since

Re: Mavenizing the HDFS build

2011-08-19 Thread Tom White
soon (e.g. HADOOP-7498, HADOOP-7496, MAPREDUCE-2856), but these shouldn't stop folks from doing development as usual. Thanks to everyone who helped with this! Cheers, Tom On Thu, Aug 18, 2011 at 11:30 AM, Tom White wrote: > Now that MR-279 has been merged into trunk, I plan to commit

Re: Mavenizing the HDFS build

2011-08-18 Thread Tom White
k' and > should be done by tmrw. > > Also, can you guys please ensure that secure HDFS works after mvn'ization? > > thanks, > Arun > > On Aug 13, 2011, at 9:39 PM, Tom White wrote: > >> Hi Arun, >> >> I'm fine with that. When do you expect

Re: [VOTE] Merge MR-279 to trunk.

2011-08-17 Thread Tom White
Excellent work. +1 Tom On Tue, Aug 16, 2011 at 2:14 PM, Mahadev Konar wrote: > Hi all, > >  We are excited to let you know that we have MR-279 ready to be merged to > trunk. I have uploaded necessary details on > https://issues.apache.org/jira/browse/MAPREDUCE-279. > >  Please take a look and

Re: Mavenizing the HDFS build

2011-08-13 Thread Tom White
are very close to getting the > merge done. We should call a vote asap. By holding off it the mvn patch it > will save us a bit of time - we spent at more than a couple of days on > resolving after the common mvn'ization. > >  Thanks for understanding. > > Arun > > On Au

[jira] [Created] (MAPREDUCE-2835) Make per-job counter limits configurable

2011-08-11 Thread Tom White (JIRA)
: 0.20.204.0 Reporter: Tom White Assignee: Tom White Fix For: 0.20.205.0 The per-job counter limits introduced in MAPREDUCE-1943 are fixed, except for the total number allowed per job (mapreduce.job.counters.limit). It would be useful to make them all configurable

[jira] [Resolved] (MAPREDUCE-2741) Make ant build system work with hadoop-common JAR generated by Maven

2011-08-02 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-2741. -- Resolution: Fixed Hadoop Flags: [Reviewed] I've committed this. Thanks, Alej

Re: MR1 next steps

2011-07-15 Thread Tom White
+1 for #2 as long as the user-level MR API remains compatible. Cheers, Tom On Thu, Jul 7, 2011 at 9:58 AM, Eli Collins wrote: > Hey gang, > > Had some discussion about what to do with MR1 with Arun at the summit, > wanted to move it on-list.. Was thinking we should sort these out some > on mr-de

[jira] [Created] (MAPREDUCE-2638) Create a simple stress test for the fair scheduler

2011-07-01 Thread Tom White (JIRA)
Components: contrib/fair-share Reporter: Tom White Assignee: Tom White This would be a test that runs against a cluster, typically with settings that allow preemption to be exercised. -- This message is automatically generated by JIRA. For more information on JIRA, see: http

[jira] [Created] (MAPREDUCE-2560) Support specification of codecs by name

2011-06-02 Thread Tom White (JIRA)
Support specification of codecs by name --- Key: MAPREDUCE-2560 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2560 Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Tom White

[jira] [Created] (MAPREDUCE-2455) Remove deprecated JobTracker.State in favour of JobTrackerStatus

2011-04-26 Thread Tom White (JIRA)
Type: Sub-task Reporter: Tom White Assignee: Tom White Fix For: 0.23.0 MAPREDUCE-2337 deprecated getJobTrackerState() on ClusterStatus, this issue is to remove the getter (in favour of getJobTrackerStatus(), which will remain) so there is no longer a direct

[jira] [Created] (MAPREDUCE-2428) start-mapred.sh script fails if HADOOP_HOME is not set

2011-04-08 Thread Tom White (JIRA)
Affects Versions: 0.21.0 Reporter: Tom White Assignee: Tom White Priority: Blocker Fix For: 0.22.0 MapReduce portion of HADOOP-6953 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software

[jira] [Created] (MAPREDUCE-2422) Removed unused internal methods from DistributedCache

2011-04-06 Thread Tom White (JIRA)
Components: client Reporter: Tom White Assignee: Tom White DistributedCache has a number of deprecated methods that are no longer used ever since TrackerDistributedCacheManager was introduced in MAPREDUCE-476. Removing these methods (which are not user-facing) will

[jira] [Created] (MAPREDUCE-2421) Remove JobHistory's dependency on JobTracker

2011-04-06 Thread Tom White (JIRA)
nents: client Reporter: Tom White Assignee: Tom White The direct dependency can be removed by introducing a callback inteface. This is for MAPREDUCE-1638. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (MAPREDUCE-2400) Remove Cluster's dependency on JobTracker

2011-03-21 Thread Tom White (JIRA)
lient Reporter: Tom White Assignee: Tom White Introduce a factory using ServiceLoader to remove the direct dependency. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] Resolved: (MAPREDUCE-2392) TaskTracker shutdown in the tests sometimes take 60s

2011-03-17 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-2392. -- Resolution: Fixed Fix Version/s: 0.22.0 Hadoop Flags: [Reviewed] I've

[jira] Created: (MAPREDUCE-2392) TaskTracker shutdown in the tests sometimes take 60s

2011-03-16 Thread Tom White (JIRA)
Reporter: Tom White Assignee: Tom White There are a lot of the following in the test logs: {noformat} 2011-03-16 13:47:02,267 INFO mapred.TaskTracker (TaskTracker.java:shutdown(1275)) - Shutting down StatusHttpServer 2011-03-16 13:48:02,349 ERROR mapred.TaskTracker

[jira] Created: (MAPREDUCE-2351) mapred.job.tracker.history.completed.location should support an arbitrary filesystem URI

2011-03-01 Thread Tom White (JIRA)
Project: Hadoop Map/Reduce Issue Type: Improvement Reporter: Tom White Assignee: Tom White Currently, mapred.job.tracker.history.completed.location is resolved relative to the default filesystem. If not set it defaults to history/done in the local log

[jira] Created: (MAPREDUCE-2338) Deprecate field selection methods on JobConf

2011-02-18 Thread Tom White (JIRA)
: Tom White KeyFieldBasedComparator and KeyFieldBasedPartitioner are libraries so should not appear as getters and setters in JobConf. For the new API there are getters and setters on KeyFieldBasedComparator and KeyFieldBasedPartitioner themselves - we should add the equivalent to the old API

[jira] Created: (MAPREDUCE-2337) Remove dependence of public MapReduce API on classes in server package

2011-02-17 Thread Tom White (JIRA)
Issue Type: Improvement Reporter: Tom White Assignee: Tom White Cluster#getJobTrackerState() returns a org.apache.hadoop.mapreduce.server.jobtracker.State enum, which makes the API in o.a.h.mapreduce have a dependency on the server package. It would be better to

[jira] Created: (MAPREDUCE-2336) Tool-related packages should be in the Tool javadoc group

2011-02-17 Thread Tom White (JIRA)
Components: documentation Reporter: Tom White Assignee: Tom White Fix For: 0.22.0 Some of the tool packages are mistakenly in the general group. -- This message is automatically generated by JIRA. - For more information on JIRA, see: http

[jira] Created: (MAPREDUCE-2282) MapReduce tests don't compile following HDFS-1561

2011-01-21 Thread Tom White (JIRA)
: Bug Reporter: Tom White Priority: Blocker TestMRServerPorts depends on TestHDFSServerPorts which was changed by HDFS-1561, resulting in a compilation failure. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Resolved: (MAPREDUCE-2273) TaskLogServlet does not set content type

2011-01-19 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-2273. -- Resolution: Duplicate > TaskLogServlet does not set content t

[jira] Created: (MAPREDUCE-2273) TaskLogServlet does not set content type

2011-01-19 Thread Tom White (JIRA)
Reporter: Tom White Priority: Blocker TaskLogServlet has never set the content type, even though it can serve plain text or HTML, but since HADOOP-7093 the problem has been highlighted since it serves HTML as "text/plain" (before it was serving plain text as

Re: Trunk build failing with ivy errors

2010-11-10 Thread Tom White
I just manually pushed new snapshots and HDFS and MapReduce trunk now resolve artifacts and compile correctly. So it looks like Hudson is breaking the poms somehow. Tom On Wed, Nov 10, 2010 at 4:16 PM, Todd Lipcon wrote: > We did a bit more sleuthing and figured out that the pom for the most rec

[jira] Resolved: (MAPREDUCE-2086) CHANGES.txt does not reflect the release of version 0.21.0.

2010-10-05 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-2086. -- Resolution: Fixed Assignee: Tom White I've fixed this. > CHANGES.txt

[jira] Resolved: (MAPREDUCE-2092) Trunk can't be compiled

2010-09-24 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-2092. -- Resolution: Fixed HADOOP-6951 has now been reverted and I confirmed MapReduce trunk now

[jira] Created: (MAPREDUCE-2084) Deprecated org.apache.hadoop.util package in MapReduce produces deprecations in Common classes in Eclipse

2010-09-22 Thread Tom White (JIRA)
/jira/browse/MAPREDUCE-2084 Project: Hadoop Map/Reduce Issue Type: Bug Components: documentation Affects Versions: 0.21.0 Reporter: Tom White Priority: Blocker Fix For: 0.21.1 As reported in [this thread|http://mail

[jira] Resolved: (MAPREDUCE-2014) Remove task-controller from 0.21 branch

2010-08-16 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-2014. -- Hadoop Flags: [Reviewed] Resolution: Fixed I committed this to the 0.21 branch

[jira] Created: (MAPREDUCE-2014) Remove task-controller from 0.21 branch

2010-08-16 Thread Tom White (JIRA)
Reporter: Tom White Assignee: Tom White Fix For: 0.21.0 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Created: (MAPREDUCE-1929) Allow artifacts to be published to the staging Apache Nexus Maven Repository

2010-07-09 Thread Tom White (JIRA)
Map/Reduce Issue Type: Bug Components: build Reporter: Tom White Assignee: Giridharan Kesavan Priority: Blocker Fix For: 0.21.0 MapReduce companion issue to HADOOP-6847. -- This message is automatically generated by JIRA. - You

[jira] Created: (MAPREDUCE-1926) MapReduce distribution is missing build-utils.xml

2010-07-08 Thread Tom White (JIRA)
Components: build Reporter: Tom White Priority: Blocker Fix For: 0.21.0 The tarball should be able to build itself. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.

[jira] Resolved: (MAPREDUCE-1885) Trunk compilation is broken because of FileSystem api change in HADOOP-6826

2010-06-22 Thread Tom White (JIRA)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White resolved MAPREDUCE-1885. -- Hadoop Flags: [Reviewed] Fix Version/s: 0.21.0 (was: 0.22.0

[jira] Created: (MAPREDUCE-1870) Harmonize MapReduce JAR library versions with Common and HDFS

2010-06-15 Thread Tom White (JIRA)
: Bug Components: build Reporter: Tom White Assignee: Tom White Priority: Blocker Fix For: 0.21.0 MapReduce part of HADOOP-6800. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the

Re: building 0.21

2010-06-08 Thread Tom White
Perhaps you're not getting up-to-date libraries? Try common> ant clean jar mvn-install hdfs> ant veryclean jar mvn-install -Dresolvers=internal mapreduce> ant veryclean jar -Dresolvers=internal This works for me with the 0.21 branches. Cheers, Tom On Tue, Jun 8, 2010 at 9:48 AM, Torsten Curdt

  1   2   >