[jira] [Created] (MAPREDUCE-5162) Add ability to change the queue a job has been assigned to with JobClient

2013-04-17 Thread Jeff Lord (JIRA) 
Jeff Lord created MAPREDUCE-5162:


 Summary: Add ability to change the queue a job has been assigned 
to with JobClient
 Key: MAPREDUCE-5162
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5162
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: client
Affects Versions: 0.20.2
Reporter: Jeff Lord


Currently the job client does not provide a facility to change the particular 
queue a job is assigned to. It would be nice to be able to do something like 
this:
mapred job  -setqueue 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (MAPREDUCE-4789) Map Reduce Counters are 0 on the Job Tracker job details page

2012-11-12 Thread Jeff Lord (JIRA) 
Jeff Lord created MAPREDUCE-4789:


 Summary: Map Reduce Counters are 0 on the Job Tracker job details 
page
 Key: MAPREDUCE-4789
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4789
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv1
Affects Versions: 2.0.0-alpha
Reporter: Jeff Lord


Counters on the jobtracker details page are 0 for map and reduce columns.
This is reproduced using a simple wordcount. Of note is that customer counters 
show properly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (MAPREDUCE-4357) Snappy Codec does not load properly when m/r job is run in "uber" mode

2012-06-20 Thread Jeff Lord (JIRA) 
Jeff Lord created MAPREDUCE-4357:


 Summary: Snappy Codec does not load properly when m/r job is run 
in "uber" mode
 Key: MAPREDUCE-4357
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4357
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Jeff Lord


# sudo -u hdfs hadoop jar 
/usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-jobclient-2.0.0-cdh4.0.0-tests.jar
 TestDFSIO -write
12/06/01 18:17:11 INFO fs.TestDFSIO: TestDFSIO.0.0.6
12/06/01 18:17:11 INFO fs.TestDFSIO: nrFiles = 1
12/06/01 18:17:11 INFO fs.TestDFSIO: fileSize (MB) = 1.0
12/06/01 18:17:11 INFO fs.TestDFSIO: bufferSize = 100
12/06/01 18:17:11 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO
12/06/01 18:17:11 INFO fs.TestDFSIO: creating control file: 1048576 bytes, 1 
files
12/06/01 18:17:12 INFO fs.TestDFSIO: created control files for: 1 files
12/06/01 18:17:12 INFO mapred.FileInputFormat: Total input paths to process : 1
12/06/01 18:17:12 INFO mapreduce.JobSubmitter: number of splits:1
12/06/01 18:17:12 WARN conf.Configuration: mapred.jar is deprecated. Instead, 
use mapreduce.job.jar
12/06/01 18:17:12 WARN conf.Configuration: mapred.reduce.tasks is deprecated. 
Instead, use mapreduce.job.reduces
12/06/01 18:17:12 WARN conf.Configuration: mapred.output.value.class is 
deprecated. Instead, use mapreduce.job.output.value.class
12/06/01 18:17:12 WARN conf.Configuration: mapred.job.name is deprecated. 
Instead, use mapreduce.job.name
12/06/01 18:17:12 WARN conf.Configuration: mapred.input.dir is deprecated. 
Instead, use mapreduce.input.fileinputformat.inputdir
12/06/01 18:17:12 WARN conf.Configuration: mapred.output.dir is deprecated. 
Instead, use mapreduce.output.fileoutputformat.outputdir
12/06/01 18:17:12 WARN conf.Configuration: mapred.map.tasks is deprecated. 
Instead, use mapreduce.job.maps
12/06/01 18:17:12 WARN conf.Configuration: mapred.output.key.class is 
deprecated. Instead, use mapreduce.job.output.key.class
12/06/01 18:17:12 WARN conf.Configuration: mapred.working.dir is deprecated. 
Instead, use mapreduce.job.working.dir
12/06/01 18:17:12 INFO mapred.ResourceMgrDelegate: Submitted application 
application_1338599410922_0004 to ResourceManager at /0.0.0.0:8032
12/06/01 18:17:12 INFO mapreduce.Job: The url to track the job: 
http://localhost:8088/proxy/application_1338599410922_0004/
12/06/01 18:17:12 INFO mapreduce.Job: Running job: job_1338599410922_0004
12/06/01 18:17:17 INFO mapreduce.Job: Job job_1338599410922_0004 running in 
uber mode : true
12/06/01 18:17:17 INFO mapreduce.Job: map 0% reduce 0%
12/06/01 18:17:17 INFO mapreduce.Job: Job job_1338599410922_0004 failed with 
state FAILED due to: 
12/06/01 18:17:17 INFO mapreduce.Job: Counters: 11
Job Counters 
Failed map tasks=1
Failed reduce tasks=1
Launched map tasks=1
Launched reduce tasks=1
Other local map tasks=1
Total time spent by all maps in occupied slots (ms)=2456
Total time spent by all reduces in occupied slots (ms)=136
TOTAL_LAUNCHED_UBERTASKS=2
NUM_UBER_SUBMAPS=1
NUM_UBER_SUBREDUCES=1
NUM_FAILED_UBERTASKS=2
java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:883)
at org.apache.hadoop.fs.TestDFSIO.runIOTest(TestDFSIO.java:340)
at org.apache.hadoop.fs.TestDFSIO.writeTest(TestDFSIO.java:321)
at org.apache.hadoop.fs.TestDFSIO.run(TestDFSIO.java:520)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.TestDFSIO.main(TestDFSIO.java:445)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at 
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
at org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:112)
at org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:120)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)

Attaching my mapred-site.xml as well.
I have snappy compression enabled but I never see it get loaded in the logs.

cat /etc/hadoop/conf.pseudo/mapred-site.xml 






mapred.job.tracker
localhost:8021



mapreduce.framework.name
yarn



To set the value of tmp directory for map and reduce 
tasks.
mapreduce.task.tmp.dir
/var/lib/hadoop-mapreduce/cache/${user.name}/tasks



mapreduce.job.ubertask.enable
true
Run very small j