Unable to apply patch to contrib/mumak
Hi guys ! I was trying to use capacity scheduler with mumak simulator. hadoop version : haddop-0.21 I am able to run mumak simulator as per slides of Hong Tong http://www.slideshare.net/hadoopusergroup/mumak As per https://issues.apache.org/jira/browse/MAPREDUCE-1253 i need to apply the patch. i have tried like this : 1 > copied patch file to HADOOP_HOME/mapreduce/src/ contrib/mumak 2 > $cd HADOOP_HOME/mapreduce/src/contrib/mumak 3 > $patch -p0 < patch_file_name but i get ERROR 3 out of 3 HUNK failed Is it the right way to apply patches ? If not what is the general way ? Thanks, Arun
Re: Running example application with capacity scheduler ?
Hi all ! Thanks Thomas ! it's working in terminal. I saw the queues in web UI of JT. when i try to run normally again (default) i get this error : i tried formatting namenode and making safemode off and restart but didn't work. hduser@arun-Presario-C500-RU914PA-ACJ:/usr/local/hadoop$ bin/hadoop jar hadoop*examples*.jar wordcount /user/hduser/wcinput /user/hduser/wcoutput6 java.io.IOException: Call to localhost/127.0.0.1:54311 failed on local exception: java.io.IOException: Connection reset by peer The log of JobTracker shows : 2011-09-15 12:46:13,346 INFO org.apache.hadoop.mapred.JobTracker: JobTracker up at: 54311 2011-09-15 12:46:13,347 INFO org.apache.hadoop.mapred.JobTracker: JobTracker webserver: 50030 2011-09-15 12:46:13,634 INFO org.apache.hadoop.mapred.JobTracker: Cleaning up the system directory 2011-09-15 12:46:13,646 INFO org.apache.hadoop.mapred.JobTracker: problem cleaning system directory: hdfs://localhost:54310/app/hadoop/tmp/mapred/system org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /app/hadoop/tmp/mapred/system. Name node is in safe mode. Thanks, Arun On Wed, Sep 14, 2011 at 7:46 PM, Thomas Graves wrote: > I believe it defaults to submit a job to the default queue if you don't > specify it. You don't have the default queue defined in your list of > mapred.queue.names. So add -Dmapred.job.queue.name=myqueue1 (or another > queue you have defined) to the wordcount command like: > > bin/hadoop jar > > hadoop*examples*.jar wordcount -Dmapred.job.queue.name=myqueue1 > /user/hduser/wcinput /user/hduser/wcoutput5 > > Tom > > > On 9/14/11 5:57 AM, "arun k" wrote: > > > Hi ! > > > > I have set up single-node cluster using > > > http://www.google.co.in/url?sa=t&source=web&cd=1&ved=0CB0QFjAA&url=http%3A%2F% > > 2Fwww.michael-noll.com > %2Ftutorials%2Frunning-hadoop-on-ubuntu-linux-single-nod > > > e-cluster%2F&rct=j&q=michael%20noll%20single%20node&ei=b4ZwTvrCLsOrrAei-N32Bg& > > > usg=AFQjCNGhuvv0tNdvPj4u23bbj-qXJDlixg&sig2=7ij8Dy7aQZUkBwhTnS1rLw&cad=rja > > and could run wordcount example application. > > I was trying to run this application using capacity scheduler. > > As per > http://hadoop.apache.org/common/docs/current/capacity_scheduler.htmli > > have done : > > 1.Copied the hadoop-capacity-scheduler-*.jar from * > > contrib/capacity-scheduler* directory to HADOOP_HOME/lib > > 2.Set mapred.jobtracker.taskScheduler > > 3.Set *mapred.queue.names to myqueue1,myqueue2. > > 4.Set *mapred.capacity-scheduler.queue..capacity to 30 and 70 > > for two queues. > > > > When i run i get the error : > > hduser@arun-Presario-C500-RU914PA-ACJ:/usr/local/hadoop$ bin/hadoop jar > > hadoop*examples*.jar wordcount /user/hduser/wcinput > /user/hduser/wcoutput5 > > 11/09/14 16:00:56 INFO input.FileInputFormat: Total input paths to > process : > > 4 > > org.apache.hadoop.ipc.RemoteException: java.io.IOException: Queue > "default" > > does not exist > > at > org.apache.hadoop.mapred.JobTracker.submitJob(JobTracker.java:2998) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.j > > ava:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959) > > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953) > > > > at org.apache.hadoop.ipc.Client.call(Client.java:740) > > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) > > at org.apache.hadoop.mapred.$Proxy0.submitJob(Unknown Source) > > at > > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:800) > > at org.apache.hadoop.mapreduce.Job.submit(Job.java:432) > > at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447) > > at org.apache.hadoop.examples.WordCount.main(WordCount.java:67) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.Delegatin
Remote exception and IO exception when running a job with capacity scheduler
Hi !
I have set up hadoop-0.20.203 in local mode as per
http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/
I have run the wordcount example Job and have all daemons running.
Then to run this job with capacity scheduler i have done the following :
1. Added
properties value
mapred.jobtracker.taskScheduler
org.apache.hadoop.mapred.CapacityTaskScheduler
mapred.queue.names myqueue1,myqueue2
mapred.capacity-scheduler.queue.myqueue1.capacity 25
mapred.capacity-scheduler.queue.myqueue1.capacity 75
Already present property
mapred.job.trackerlocalhost:54311
2.${HADOOP_HOME}$ bin/stop-all.sh
3.${HADOOP_HOME}$ bin/start-all.sh
4.$jps shows all daemons
5.${HADOOP_HOME}$ bin/hadoop jar hadoop*examples*.jar wordcount -
Dmapred.job.queue.name=myqueue1 /user/hduser/wcinput /user/hduser/wcoutput
I get the error:
java.io.IOException: Call to localhost/127.0.0.1:54311 failed on local
exception: java.io.IOException: Connection reset by peer
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
at org.apache.hadoop.ipc.Client.call(Client.java:1033)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
...
When i give
$jps
32463 NameNode
32763 SecondaryNameNode
32611 DataNode
931 Jps
The jobracker log gives info
~
2011-09-16 00:21:42,012 INFO org.apache.hadoop.mapred.JobTracker: Cleaning
up the system directory
2011-09-16 00:21:42,014 INFO org.apache.hadoop.mapred.JobTracker: problem
cleaning system directory:
hdfs://localhost:54310/app203/hadoop203/tmp/mapred/system
org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete
/app203/hadoop203/tmp/mapred/system. Name node is in safe mode.
The ratio of reported blocks 1. has reached the threshold 0.9990. Safe
mode will be turned off automatically in 6 seconds.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.deleteInternal(FSNamesystem.java:1851)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:1831)
2011-09-16 00:21:52,321 FATAL org.apache.hadoop.mapred.JobTracker:
java.io.IOException: Queue 'myqueue1' doesn't have configured capacity!
at
org.apache.hadoop.mapred.CapacityTaskScheduler.parseQueues(CapacityTaskScheduler.java:905)
at
org.apache.hadoop.mapred.CapacityTaskScheduler.start(CapacityTaskScheduler.java:822)
at
org.apache.hadoop.mapred.JobTracker.offerService(JobTracker.java:2563)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4957)
2011-09-16 00:21:52,322 INFO org.apache.hadoop.mapred.JobTracker:
SHUTDOWN_MSG:
~~~
Even if i submit the job to "myqueue2" i see the same error of "myqueue1"
2011-09-16 00:21:52,321 FATAL org.apache.hadoop.mapred.JobTracker:
java.io.IOException: Queue 'myqueue1' doesn't have configured capacity!
Thanks,
Arun
Re: Running example application with capacity scheduler ?
Hi all !
Harsh ! Namenode appears to be out of safe mode :
In http://nn-host:50070 i see in time
T1>Safe mode is ON. The ratio of reported blocks 0. has not reached the
threshold 0.9990. Safe mode will be turned off automatically.
7 files and directories, 1 blocks = 8 total. Heap Size is 15.06 MB / 966.69
MB (1%)
T2>Safe mode is ON. The ratio of reported blocks 1. has reached the
threshold 0.9990. Safe mode will be turned off automatically in 17 seconds.
7 files and directories, 1 blocks = 8 total. Heap Size is 15.06 MB / 966.69
MB (1%)
T3>9 files and directories, 3 blocks = 12 total. Heap Size is 15.06 MB /
966.69 MB (1%)
Added properties :
mapred.jobtracker.taskScheduler org.apache.hadoopertiep.mapred.CTS
mapred.queue.names myqueue1,myqueue2
mapred.capacity-scheduler.queue.myqueue1.capacity 25
mapred.capacity-scheduler.queue.myqueue1.capacity 75
${HADOOP_HOME}$ bin/hadoop jar hadoop*examples*.jar wordcount -
Dmapred.job.queue.name <http://dmapred.job.queue.name/>=
myqueue1 /user/hduser/wcinput /user/hduser/wcoutput
I get the error:
java.io.IOException: Call to localhost/127.0.0.1:54311 failed on local
exception: java.io.IOException: Connection reset by peer
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
at org.apache.hadoop.ipc.Client.call(Client.java:1033)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
...
When i give
$jps
32463 NameNode
32763 SecondaryNameNode
32611 DataNode
931 Jps
The jobracker log gives info
~
2011-09-16 00:21:42,012 INFO org.apache.hadoop.mapred.JobTracker: Cleaning
up the system directory
2011-09-16 00:21:42,014 INFO org.apache.hadoop.mapred.JobTracker: problem
cleaning system directory:
hdfs://localhost:54310/app203/hadoop203/tmp/mapred/system
org.apache.hadoop.ipc.RemoteException:
org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete
/app203/hadoop203/tmp/mapred/system. Name node is in safe mode.
The ratio of reported blocks 1. has reached the threshold 0.9990. Safe
mode will be turned off automatically in 6 seconds.
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.deleteInternal(FSNamesystem.java:1851)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:1831)
2011-09-16 00:21:52,321 FATAL org.apache.hadoop.mapred.JobTracker:
java.io.IOException: Queue 'myqueue1' doesn't have configured capacity!
at
org.apache.hadoop.mapred.CapacityTaskScheduler.parseQueues(CapacityTaskScheduler.java:905)
at
org.apache.hadoop.mapred.CapacityTaskScheduler.start(CapacityTaskScheduler.java:822)
at
org.apache.hadoop.mapred.JobTracker.offerService(JobTracker.java:2563)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4957)
2011-09-16 00:21:52,322 INFO org.apache.hadoop.mapred.JobTracker:
SHUTDOWN_MSG:
~~~
Even if i submit the job to "myqueue2" i see the same error of "myqueue1"
2011-09-16 00:21:52,321 FATAL org.apache.hadoop.mapred.JobTracker:
java.io.IOException: Queue 'myqueue1' doesn't have configured capacity!
Thanks,
Arun
On Thu, Sep 15, 2011 at 5:23 PM, Harsh J wrote:
> Hello Arun,
>
> To me it looks like your HDFS isn't setup properly, in this case. Can
> you ensure all DNs are properly up? Your NN appears to have gotten
> stuck somehow into a safemode. Check out your http://nn-host:50070
> page for more details on why.
>
> Your JT won't come up until the NN is properly up and out of safemode
> (for which it needs the DNs). And once it comes up, I think you should
> be good to go, keeping in mind the changes Thomas mentioned earlier.
>
> On Thu, Sep 15, 2011 at 3:58 PM, arun k wrote:
> > Hi all !
> >
> > Thanks Thomas ! it's working in terminal.
> > I saw the queues in web UI of JT.
> > when i try to run normally again (default) i get this error :
> > i tried formatting namenode and making safemode off and restart but
> didn't
> > work.
> >
> > hduser@arun-Presario-C500-RU914PA-ACJ:/usr/local/hadoop$ bin/hadoop jar
> > hadoop*examples*.jar wordcount /user/hduser/wcinput
> /user/hduser/wcoutput6
> > java.io.IOException: Call to localhost/127.0.0.1:54311 failed on local
> > exception: java.io.IOException: Connection reset by peer
> >
> > The log of JobTracker shows :
> > 2011-09-15 12:46:13,346 INFO org.apache.hadoop.mapred.JobTracker:
> JobTracker
> > up at: 54311
> > 2011-09-15 12:46:13,347 INFO org.apache.hadoop.mapred.JobTracker:
> JobTracker
> > webserver: 50030
> > 2011-09-15 12:46:13,634 INFO org.apache.hadoop.mapred.JobTracker
[no subject]
Hi !
Thanks Harsh !
The problem was that i have set up queue info in mapred-site.xml instead of
capacity-scheduler.xml .
Arun
On Fri, Sep 16, 2011 at 10:52 AM, Harsh J wrote:
> Arun,
>
> Please do not cross-post to multiple lists. Lets continue this on
> mapreduce-user@ alone.
>
> Your problem isn't the job submission here, but your Capacity
> Scheduler configuration. For every queue you configure, you need to
> add in capacities: Please see the queue properties documentation at
>
> http://hadoop.apache.org/common/docs/current/capacity_scheduler.html#Queue+properties
> for the vital configs required in additional to mapred.queue.names.
> Once done, you should have a fully functional JobTracker!
>
> On Fri, Sep 16, 2011 at 10:17 AM, arun k wrote:
> > Hi all !
> >
> > Harsh ! Namenode appears to be out of safe mode :
> > In http://nn-host:50070 i see in time
> >
> > T1>Safe mode is ON. The ratio of reported blocks 0. has not reached
> the
> > threshold 0.9990. Safe mode will be turned off automatically.
> > 7 files and directories, 1 blocks = 8 total. Heap Size is 15.06 MB /
> 966.69
> > MB (1%)
> >
> > T2>Safe mode is ON. The ratio of reported blocks 1. has reached the
> > threshold 0.9990. Safe mode will be turned off automatically in 17
> seconds.
> > 7 files and directories, 1 blocks = 8 total. Heap Size is 15.06 MB /
> 966.69
> > MB (1%)
> >
> > T3>9 files and directories, 3 blocks = 12 total. Heap Size is 15.06 MB /
> > 966.69 MB (1%)
> >
> > Added properties :
> >
> > mapred.jobtracker.taskScheduler org.apache.hadoopertiep.mapred.CTS
> >
> > mapred.queue.names myqueue1,myqueue2
> > mapred.capacity-scheduler.queue.myqueue1.capacity 25
> > mapred.capacity-scheduler.queue.myqueue1.capacity 75
> > ${HADOOP_HOME}$ bin/hadoop jar hadoop*examples*.jar wordcount
> > -Dmapred.job.queue.name=
> > myqueue1 /user/hduser/wcinput /user/hduser/wcoutput
> >
> > I get the error:
> > java.io.IOException: Call to localhost/127.0.0.1:54311 failed on local
> > exception: java.io.IOException: Connection reset by peer
> > at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
> > at org.apache.hadoop.ipc.Client.call(Client.java:1033)
> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
> > ...
> >
> > When i give
> > $jps
> > 32463 NameNode
> > 32763 SecondaryNameNode
> > 32611 DataNode
> > 931 Jps
> >
> >
> > The jobracker log gives info
> > ~
> > 2011-09-16 00:21:42,012 INFO org.apache.hadoop.mapred.JobTracker:
> Cleaning
> > up the system directory
> > 2011-09-16 00:21:42,014 INFO org.apache.hadoop.mapred.JobTracker: problem
> > cleaning system directory:
> > hdfs://localhost:54310/app203/hadoop203/tmp/mapred/system
> > org.apache.hadoop.ipc.RemoteException:
> > org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete
> > /app203/hadoop203/tmp/mapred/system. Name node is in safe mode.
> > The ratio of reported blocks 1. has reached the threshold 0.9990.
> Safe
> > mode will be turned off automatically in 6 seconds.
> > at
> >
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.deleteInternal(FSNamesystem.java:1851)
> > at
> >
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.delete(FSNamesystem.java:1831)
> > 2011-09-16 00:21:52,321 FATAL org.apache.hadoop.mapred.JobTracker:
> > java.io.IOException: Queue 'myqueue1' doesn't have configured capacity!
> > at
> >
> org.apache.hadoop.mapred.CapacityTaskScheduler.parseQueues(CapacityTaskScheduler.java:905)
> > at
> >
> org.apache.hadoop.mapred.CapacityTaskScheduler.start(CapacityTaskScheduler.java:822)
> > at
> > org.apache.hadoop.mapred.JobTracker.offerService(JobTracker.java:2563)
> > at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4957)
> >
> > 2011-09-16 00:21:52,322 INFO org.apache.hadoop.mapred.JobTracker:
> > SHUTDOWN_MSG:
> > ~~~
> >
> > Even if i submit the job to "myqueue2" i see the same error of "myqueue1"
> > 2011-09-16 00:21:52,321 FATAL org.apache.hadoop.mapred.JobTracker:
> > java.io.IOException: Queue 'myqueue1' doesn't have configured capacity!
> >
> > Thanks,
> > Arun
> >
> >
> >
> > On Thu, Sep 15, 2011 a
Re: Running Jobs with capacity scheduler on hadoop in eclipse
Hi !
I have setup hadoop0.20.2 on eclipse Helios and able to run the Example
wordcount using ExampleDriver class as mentioned by Faraz in
http://lucene.472066.n3.nabble.com/HELP-configuring-hadoop-on-ECLIPSE-td1086829.html#a2241534
Two questions ?
1. I am unable to see the jobtracker & others in browser at http addr
mentioned in mapred- default.xml. i have not edited any site.xml files.
I have tried to edit site.xml files as per Michael noll site but that
didn't help.
2.Capacity Scheduler :I see the capacity -*- jar in lib folder. I have
modified mapred-site.xml and capacity-scheduler.xml as required. How do i
run some application jobs by submitting a job to a queue in this case ?
I have tried to run:
Program & Args as : wordcount -Dmapred.job.queue.name=myqueue1
input_file_loc output_file_loc
But i get error :
Exception in thread "main" java.lang.Error: Unresolved compilation
problems:
ProgramDriver cannot be resolved to a type
ProgramDriver cannot be resolved to a type
DistributedPentomino cannot be resolved to a type
.
Thanks,
Arun
On Fri, Sep 16, 2011 at 12:46 PM, Harsh J wrote:
> Arun,
>
> Good to know. Happy Hadoopin'!
>
> On Fri, Sep 16, 2011 at 12:34 PM, arun k wrote:
> > Hi !
> > Thanks Harsh !
> > The problem was that i have set up queue info in mapred-site.xml instead
> of
> > capacity-scheduler.xml .
> > Arun
> >
> > On Fri, Sep 16, 2011 at 10:52 AM, Harsh J wrote:
> >>
> >> Arun,
> >>
> >> Please do not cross-post to multiple lists. Lets continue this on
> >> mapreduce-user@ alone.
> >>
> >> Your problem isn't the job submission here, but your Capacity
> >> Scheduler configuration. For every queue you configure, you need to
> >> add in capacities: Please see the queue properties documentation at
> >>
> >>
> http://hadoop.apache.org/common/docs/current/capacity_scheduler.html#Queue+properties
> >> for the vital configs required in additional to mapred.queue.names.
> >> Once done, you should have a fully functional JobTracker!
> >>
> >> On Fri, Sep 16, 2011 at 10:17 AM, arun k wrote:
> >> > Hi all !
> >> >
> >> > Harsh ! Namenode appears to be out of safe mode :
> >> > In http://nn-host:50070 i see in time
> >> >
> >> > T1>Safe mode is ON. The ratio of reported blocks 0. has not
> reached
> >> > the
> >> > threshold 0.9990. Safe mode will be turned off automatically.
> >> > 7 files and directories, 1 blocks = 8 total. Heap Size is 15.06 MB /
> >> > 966.69
> >> > MB (1%)
> >> >
> >> > T2>Safe mode is ON. The ratio of reported blocks 1. has reached
> the
> >> > threshold 0.9990. Safe mode will be turned off automatically in 17
> >> > seconds.
> >> > 7 files and directories, 1 blocks = 8 total. Heap Size is 15.06 MB /
> >> > 966.69
> >> > MB (1%)
> >> >
> >> > T3>9 files and directories, 3 blocks = 12 total. Heap Size is 15.06 MB
> /
> >> > 966.69 MB (1%)
> >> >
> >> > Added properties :
> >> >
> >> > mapred.jobtracker.taskScheduler org.apache.hadoopertiep.mapred.CTS
> >> >
> >> > mapred.queue.names myqueue1,myqueue2
> >> > mapred.capacity-scheduler.queue.myqueue1.capacity 25
> >> > mapred.capacity-scheduler.queue.myqueue1.capacity 75
> >> > ${HADOOP_HOME}$ bin/hadoop jar hadoop*examples*.jar wordcount
> >> > -Dmapred.job.queue.name=
> >> > myqueue1 /user/hduser/wcinput /user/hduser/wcoutput
> >> >
> >> > I get the error:
> >> > java.io.IOException: Call to localhost/127.0.0.1:54311 failed on
> local
> >> > exception: java.io.IOException: Connection reset by peer
> >> > at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
> >> > at org.apache.hadoop.ipc.Client.call(Client.java:1033)
> >> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
> >> > ...
> >> >
> >> > When i give
> >> > $jps
> >> > 32463 NameNode
> >> > 32763 SecondaryNameNode
> >> > 32611 DataNode
> >> > 931 Jps
> >> >
> >> >
> >> > The jobracker log gives info
> >> > ~
> >> > 2011-09-16 00:21:42,012 INFO org.apache.
Re: Running Jobs with capacity scheduler on hadoop in eclipse
Hi ! I am using hadoop on eclipse. I am able to run jobs. My question was how can i run some application with capacity scheduler ? Also I am unable to see Jobtracker in Web GUI at default Jobtracker address. I have not edited any *-site.xmls. Thanks, Arun On Fri, Sep 16, 2011 at 3:56 PM, Swathi V wrote: > It might be in safemode. turn off the safemode by using > *bin/hadoop dfsadmin safemode -leave* > to see the task tracker. > > > On Fri, Sep 16, 2011 at 2:09 PM, arun k wrote: > >> Hi ! >> >> I have setup hadoop0.20.2 on eclipse Helios and able to run the Example >> wordcount using ExampleDriver class as mentioned by Faraz in >> http://lucene.472066.n3.nabble.com/HELP-configuring-hadoop-on-ECLIPSE-td1086829.html#a2241534 >> >> Two questions ? >> 1. I am unable to see the jobtracker & others in browser at http addr >> mentioned in mapred- default.xml. i have not edited any site.xml files. >> I have tried to edit site.xml files as per Michael noll site but that >> didn't help. >> >> 2.Capacity Scheduler :I see the capacity -*- jar in lib folder. I have >> modified mapred-site.xml and capacity-scheduler.xml as required. How do i >> run some application jobs by submitting a job to a queue in this case ? >> I have tried to run: >> Program & Args as : wordcount -Dmapred.job.queue.name=myqueue1 >> input_file_loc output_file_loc >> But i get error : >> Exception in thread "main" java.lang.Error: Unresolved compilation >> problems: >> ProgramDriver cannot be resolved to a type >> ProgramDriver cannot be resolved to a type >> DistributedPentomino cannot be resolved to a type >> ..... >> >> Thanks, >> Arun >> >> >> On Fri, Sep 16, 2011 at 12:46 PM, Harsh J wrote: >> >>> Arun, >>> >>> Good to know. Happy Hadoopin'! >>> >>> On Fri, Sep 16, 2011 at 12:34 PM, arun k wrote: >>> > Hi ! >>> > Thanks Harsh ! >>> > The problem was that i have set up queue info in mapred-site.xml >>> instead of >>> > capacity-scheduler.xml . >>> > Arun >>> > >>> > On Fri, Sep 16, 2011 at 10:52 AM, Harsh J wrote: >>> >> >>> >> Arun, >>> >> >>> >> Please do not cross-post to multiple lists. Lets continue this on >>> >> mapreduce-user@ alone. >>> >> >>> >> Your problem isn't the job submission here, but your Capacity >>> >> Scheduler configuration. For every queue you configure, you need to >>> >> add in capacities: Please see the queue properties documentation at >>> >> >>> >> >>> http://hadoop.apache.org/common/docs/current/capacity_scheduler.html#Queue+properties >>> >> for the vital configs required in additional to mapred.queue.names. >>> >> Once done, you should have a fully functional JobTracker! >>> >> >>> >> On Fri, Sep 16, 2011 at 10:17 AM, arun k wrote: >>> >> > Hi all ! >>> >> > >>> >> > Harsh ! Namenode appears to be out of safe mode : >>> >> > In http://nn-host:50070 i see in time >>> >> > >>> >> > T1>Safe mode is ON. The ratio of reported blocks 0. has not >>> reached >>> >> > the >>> >> > threshold 0.9990. Safe mode will be turned off automatically. >>> >> > 7 files and directories, 1 blocks = 8 total. Heap Size is 15.06 MB / >>> >> > 966.69 >>> >> > MB (1%) >>> >> > >>> >> > T2>Safe mode is ON. The ratio of reported blocks 1. has reached >>> the >>> >> > threshold 0.9990. Safe mode will be turned off automatically in 17 >>> >> > seconds. >>> >> > 7 files and directories, 1 blocks = 8 total. Heap Size is 15.06 MB / >>> >> > 966.69 >>> >> > MB (1%) >>> >> > >>> >> > T3>9 files and directories, 3 blocks = 12 total. Heap Size is 15.06 >>> MB / >>> >> > 966.69 MB (1%) >>> >> > >>> >> > Added properties : >>> >> > >>> >> > mapred.jobtracker.taskScheduler org.apache.hadoopertiep.mapred.CTS >>> >> > >>> >> > mapred.queue.names myqueue1,myqueue2 >>> >> > mapred.capacity-scheduler.queue.my
Building modified capacity scheduler and seeing console ouputs.
Hi ! I have set up hadoop as per http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ I am able to run jobs with capacity scheduler. I am interested in extending the capacity scheduler. So first i wanted to see whether i can build it after making changes. i have tried this : >modified mapre-site.xml and capcity-sche.xml accordingly. > Put some println statements in CapacityTaskScheduler.java >In src/contrib/capacity_scheduler$ ant BUILD was successful. >Copied the hadoop-capacity-*-jar in build/contrib to lib/ >ran some job with capacity scheduler by submitting to a queue But i don't see any console output and i couldn't find print statements in any logs too. How do i go about it ? Thanks, Arun
Re: RE: Making Mumak work with capacity scheduler
Hi !
I have changed he permissions for hadoop extract and /jobstory and
/history/done dir recursively:
$chmod -R 777 branch-0.22
$chmod -R logs
$chmod -R jobracker
but still i get the same problem.
The permissions are like this <http://pastebin.com/sw3UPM8t>
The log is here <http://pastebin.com/CztUPywB>.
I am able to run as sudo.
Arun
On Thu, Sep 22, 2011 at 7:19 PM, Uma Maheswara Rao G 72686 <
mahesw...@huawei.com> wrote:
> Yes Devaraj,
> From the logs, looks it failed to create /jobtracker/jobsInfo
>
>
>
> code snippet:
>
> if (!fs.exists(path)) {
>if (!fs.mkdirs(path, new
> FsPermission(JOB_STATUS_STORE_DIR_PERMISSION))) {
> throw new IOException(
> "CompletedJobStatusStore mkdirs failed to create "
> + path.toString());
>}
>
> @ Arun, Can you check, you have correct permission as Devaraj said?
>
>
> 2011-09-22 15:53:57.598::INFO: Started
> SelectChannelConnector@0.0.0.0:50030
> 11/09/22 15:53:57 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId=
> 11/09/22 15:53:57 WARN conf.Configuration: mapred.task.cache.levels is
> deprecated. Instead, use mapreduce.jobtracker.taskcache.levels
> 11/09/22 15:53:57 WARN mapred.SimulatorJobTracker: Error starting tracker:
> java.io.IOException: CompletedJobStatusStore mkdirs failed to create
> /jobtracker/jobsInfo
>at
> org.apache.hadoop.mapred.CompletedJobStatusStore.(CompletedJobStatusStore.java:83)
>at org.apache.hadoop.mapred.JobTracker.(JobTracker.java:4684)
>at
> org.apache.hadoop.mapred.SimulatorJobTracker.(SimulatorJobTracker.java:81)
>at
> org.apache.hadoop.mapred.SimulatorJobTracker.startTracker(SimulatorJobTracker.java:100)
>at
> org.apache.hadoop.mapred.SimulatorEngine.init(SimulatorEngine.java:210)
>at
> org.apache.hadoop.mapred.SimulatorEngine.init(SimulatorEngine.java:184)
>at
> org.apache.hadoop.mapred.SimulatorEngine.run(SimulatorEngine.java:292)
>at
> org.apache.hadoop.mapred.SimulatorEngine.run(SimulatorEngine.java:323)
>
> I cc'ed to Mapreduce user mailing list as well.
>
> Regards,
> Uma
>
> - Original Message -
> From: Devaraj K
> Date: Thursday, September 22, 2011 6:01 pm
> Subject: RE: Making Mumak work with capacity scheduler
> To: common-u...@hadoop.apache.org
>
> > Hi Arun,
> >
> >I have gone through the logs. Mumak simulator is trying to
> > start the job
> > tracker and job tracking is failing to start because it is not able to
> > create "/jobtracker/jobsinfo" directory.
> >
> > I think the directory doesn't have enough permissions. Please check
> > thepermissions or any other reason why it is failing to create the
> > dir.
> >
> >
> > Devaraj K
> >
> >
> > -Original Message-
> > From: arun k [mailto:arunk...@gmail.com]
> > Sent: Thursday, September 22, 2011 3:57 PM
> > To: common-u...@hadoop.apache.org
> > Subject: Re: Making Mumak work with capacity scheduler
> >
> > Hi Uma !
> >
> > u got me right !
> > >Actually without any patch when i modified appropriate mapred-
> > site.xml and
> > capacity-scheduler.xml and copied capaciy jar accordingly.
> > I am able to see see queues in Jobracker GUI but both the queues
> > show same
> > set of job's execution.
> > I ran with trace and topology files from test/data :
> > $bin/mumak.sh trace_file topology_file
> > Is it because i am not submitting jobs to a particular queue ?
> > If so how can i do it ?
> >
> > >Got hadoop-0.22 from
> > http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.22/
> > builded all three components but when i give
> > arun@arun-Presario-C500-RU914PA-ACJ:~/hadoop22/branch-
> > 0.22/mapreduce/src/contrib/mumak$
> > bin/mumak.sh src/test/data/19-jobs.trace.json.gz
> > src/test/data/19-jobs.topology.json.gz
> > it gets stuck at some point. Log is here
> > <http://pastebin.com/9SNUHLFy>
> > Thanks,
> > Arun
> >
> >
> >
> >
> >
> > On Wed, Sep 21, 2011 at 2:03 PM, Uma Maheswara Rao G 72686 <
> > mahesw...@huawei.com> wrote:
> >
> > >
> > > Hello Arun,
> > > If you want to apply MAPREDUCE-1253 on 21 version,
> > > applying patch directly using commands may not work because of
> > codebase> changes.
> > >
> > > So, you take the patch and apply the lines in your code base
> > manually. I
> > > am not sure any otherway for this.
> > >
> > > D
Fwd: RE: Making Mumak work with capacity scheduler
Hi guys !
I have run mumak as sudo. It works fine.
i am trying to run jobtrace in test/data with capacity scheduler.
I have done :
1> Build contrib/capacity-scheduler
2>Copied hadoop-*-capacity-jar from build/contrib/capacity_scheduler to lib/
3>added mapred.jobtracker.taskScheduler and mapred.queue.names in
mapred-site.xml
4>In conf/capacity-scheduler
set the propoery value for 2 queues
mapred.capacity-scheduler.queue.default.capacity 20
mapred.capacity-scheduler.queue.myqueue2.capacity 80
When i run mumak.sh
i see in console
11/09/23 11:51:19 INFO mapred.QueueHierarchyBuilder: No capacity specified
for queue default
11/09/23 11:51:19 INFO mapred.QueueHierarchyBuilder: Created a jobQueue
default and added it as a child to
11/09/23 11:51:19 INFO mapred.QueueHierarchyBuilder: No capacity specified
for queue myqueue2
11/09/23 11:51:19 INFO mapred.QueueHierarchyBuilder: Created a jobQueue
myqueue2 and added it as a child to
11/09/23 11:51:19 INFO mapred.AbstractQueue: Total capacity to be
distributed among the others are 100.0
11/09/23 11:51:19 INFO mapred.AbstractQueue: Capacity share for un
configured queue default is 50.0
11/09/23 11:51:19 INFO mapred.AbstractQueue: Capacity share for un
configured queue myqueue2 is 50.0
11/09/23 11:51:19 INFO mapred.CapacityTaskScheduler: Capacity scheduler
started successfully
2 Q's :
1> In web GUI of Jobtracker i see both he queues but "CAPACITIES ARE
REFLECTED"
2> All the jobs by defaul are submitted to "default" queue. How can i submit
jobs to various queues in mumak ?
Regards,
Arun
On Fri, Sep 23, 2011 at 10:12 AM, arun k wrote:
> Hi !
>
> I have changed he permissions for hadoop extract and /jobstory and
> /history/done dir recursively:
> $chmod -R 777 branch-0.22
> $chmod -R logs
> $chmod -R jobracker
> but still i get the same problem.
> The permissions are like this <http://pastebin.com/sw3UPM8t>
> The log is here <http://pastebin.com/CztUPywB>.
> I am able to run as sudo.
>
> Arun
>
> On Thu, Sep 22, 2011 at 7:19 PM, Uma Maheswara Rao G 72686 <
> mahesw...@huawei.com> wrote:
>
>> Yes Devaraj,
>> From the logs, looks it failed to create /jobtracker/jobsInfo
>>
>>
>>
>> code snippet:
>>
>> if (!fs.exists(path)) {
>>if (!fs.mkdirs(path, new
>> FsPermission(JOB_STATUS_STORE_DIR_PERMISSION))) {
>> throw new IOException(
>> "CompletedJobStatusStore mkdirs failed to create "
>> + path.toString());
>>}
>>
>> @ Arun, Can you check, you have correct permission as Devaraj said?
>>
>>
>> 2011-09-22 15:53:57.598::INFO: Started
>> SelectChannelConnector@0.0.0.0:50030
>> 11/09/22 15:53:57 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>> processName=JobTracker, sessionId=
>> 11/09/22 15:53:57 WARN conf.Configuration: mapred.task.cache.levels is
>> deprecated. Instead, use mapreduce.jobtracker.taskcache.levels
>> 11/09/22 15:53:57 WARN mapred.SimulatorJobTracker: Error starting tracker:
>> java.io.IOException: CompletedJobStatusStore mkdirs failed to create
>> /jobtracker/jobsInfo
>>at
>> org.apache.hadoop.mapred.CompletedJobStatusStore.(CompletedJobStatusStore.java:83)
>>at org.apache.hadoop.mapred.JobTracker.(JobTracker.java:4684)
>>at
>> org.apache.hadoop.mapred.SimulatorJobTracker.(SimulatorJobTracker.java:81)
>>at
>> org.apache.hadoop.mapred.SimulatorJobTracker.startTracker(SimulatorJobTracker.java:100)
>>at
>> org.apache.hadoop.mapred.SimulatorEngine.init(SimulatorEngine.java:210)
>>at
>> org.apache.hadoop.mapred.SimulatorEngine.init(SimulatorEngine.java:184)
>>at
>> org.apache.hadoop.mapred.SimulatorEngine.run(SimulatorEngine.java:292)
>>at
>> org.apache.hadoop.mapred.SimulatorEngine.run(SimulatorEngine.java:323)
>>
>> I cc'ed to Mapreduce user mailing list as well.
>>
>> Regards,
>> Uma
>>
>> - Original Message -
>> From: Devaraj K
>> Date: Thursday, September 22, 2011 6:01 pm
>> Subject: RE: Making Mumak work with capacity scheduler
>> To: common-u...@hadoop.apache.org
>>
>> > Hi Arun,
>> >
>> >I have gone through the logs. Mumak simulator is trying to
>> > start the job
>> > tracker and job tracking is failing to start because it is not able to
>> > create "/jobtracker/jobsinfo" directory.
>> >
>> > I think the directory doesn't have enough permissions. Please check
>> > thepermissions or any other reason why it is failing to create the
>> > dir.
>> >
>> &g
Re: RE: Making Mumak work with capacity scheduler
Sorry ,
1Q: In web GUI of Jobtracker i see both he queues but "CAPACITIES ARE NOT
REFLECTED"
2Q:All the jobs by defaul are submitted to "default" queue. How can i submit
jobs to various queues in mumak ?
regards,
Arun
On Fri, Sep 23, 2011 at 11:57 AM, arun k wrote:
> Hi guys !
>
> I have run mumak as sudo. It works fine.
> i am trying to run jobtrace in test/data with capacity scheduler.
> I have done :
> 1> Build contrib/capacity-scheduler
> 2>Copied hadoop-*-capacity-jar from build/contrib/capacity_scheduler to
> lib/
> 3>added mapred.jobtracker.taskScheduler and mapred.queue.names in
> mapred-site.xml
> 4>In conf/capacity-scheduler
> set the propoery value for 2 queues
> mapred.capacity-scheduler.queue.default.capacity 20
> mapred.capacity-scheduler.queue.myqueue2.capacity 80
>
> When i run mumak.sh
> i see in console
> 11/09/23 11:51:19 INFO mapred.QueueHierarchyBuilder: No capacity specified
> for queue default
> 11/09/23 11:51:19 INFO mapred.QueueHierarchyBuilder: Created a jobQueue
> default and added it as a child to
> 11/09/23 11:51:19 INFO mapred.QueueHierarchyBuilder: No capacity specified
> for queue myqueue2
> 11/09/23 11:51:19 INFO mapred.QueueHierarchyBuilder: Created a jobQueue
> myqueue2 and added it as a child to
> 11/09/23 11:51:19 INFO mapred.AbstractQueue: Total capacity to be
> distributed among the others are 100.0
> 11/09/23 11:51:19 INFO mapred.AbstractQueue: Capacity share for un
> configured queue default is 50.0
> 11/09/23 11:51:19 INFO mapred.AbstractQueue: Capacity share for un
> configured queue myqueue2 is 50.0
> 11/09/23 11:51:19 INFO mapred.CapacityTaskScheduler: Capacity scheduler
> started successfully
>
> 2 Q's :
>
> 1> In web GUI of Jobtracker i see both he queues but "CAPACITIES ARE
> REFLECTED"
> 2> All the jobs by defaul are submitted to "default" queue. How can i
> submit jobs to various queues in mumak ?
>
> Regards,
> Arun
>
>
> On Fri, Sep 23, 2011 at 10:12 AM, arun k wrote:
>
>> Hi !
>>
>> I have changed he permissions for hadoop extract and /jobstory and
>> /history/done dir recursively:
>> $chmod -R 777 branch-0.22
>> $chmod -R logs
>> $chmod -R jobracker
>> but still i get the same problem.
>> The permissions are like this <http://pastebin.com/sw3UPM8t>
>> The log is here <http://pastebin.com/CztUPywB>.
>> I am able to run as sudo.
>>
>> Arun
>>
>> On Thu, Sep 22, 2011 at 7:19 PM, Uma Maheswara Rao G 72686 <
>> mahesw...@huawei.com> wrote:
>>
>>> Yes Devaraj,
>>> From the logs, looks it failed to create /jobtracker/jobsInfo
>>>
>>>
>>>
>>> code snippet:
>>>
>>> if (!fs.exists(path)) {
>>>if (!fs.mkdirs(path, new
>>> FsPermission(JOB_STATUS_STORE_DIR_PERMISSION))) {
>>> throw new IOException(
>>> "CompletedJobStatusStore mkdirs failed to create "
>>> + path.toString());
>>>}
>>>
>>> @ Arun, Can you check, you have correct permission as Devaraj said?
>>>
>>>
>>> 2011-09-22 15:53:57.598::INFO: Started
>>> SelectChannelConnector@0.0.0.0:50030
>>> 11/09/22 15:53:57 INFO jvm.JvmMetrics: Initializing JVM Metrics with
>>> processName=JobTracker, sessionId=
>>> 11/09/22 15:53:57 WARN conf.Configuration: mapred.task.cache.levels is
>>> deprecated. Instead, use mapreduce.jobtracker.taskcache.levels
>>> 11/09/22 15:53:57 WARN mapred.SimulatorJobTracker: Error starting
>>> tracker: java.io.IOException: CompletedJobStatusStore mkdirs failed to
>>> create /jobtracker/jobsInfo
>>>at
>>> org.apache.hadoop.mapred.CompletedJobStatusStore.(CompletedJobStatusStore.java:83)
>>>at
>>> org.apache.hadoop.mapred.JobTracker.(JobTracker.java:4684)
>>>at
>>> org.apache.hadoop.mapred.SimulatorJobTracker.(SimulatorJobTracker.java:81)
>>>at
>>> org.apache.hadoop.mapred.SimulatorJobTracker.startTracker(SimulatorJobTracker.java:100)
>>>at
>>> org.apache.hadoop.mapred.SimulatorEngine.init(SimulatorEngine.java:210)
>>>at
>>> org.apache.hadoop.mapred.SimulatorEngine.init(SimulatorEngine.java:184)
>>>at
>>> org.apache.hadoop.mapred.SimulatorEngine.run(SimulatorEngine.java:292)
>>>at
>>> org.apache.hadoop.mapred.SimulatorEngine.run(SimulatorEngine.java:323)
>>>
>>> I cc'ed to Mapreduce user mailing list as well.
>>>
&
Modifyiing Scheduler code in Mumak
Hi guys ! I have gone thru Mumak code. I ran mumak.sh with given Job and Topology trace files .In my understanding i see that when a job is fetched from JobStoryProducer an event is asscoiated with it and the listener / node where it is assigned is fixed when these events are created. I have not understood how *mumak is simulating scheduling exactly*.Can anyone help me in this regard ? I am interested in coding a simple scheduler in Mumak. 1> Which files do i need to modify if i want to change the way the events are allocated to a particular node/Taskracker ? 2> How do i compare throughput with existing schedulers ? 3> Do i need to have my own Job trace file ? Thanks, Arun
How is data of each job assigned to nodes in Mumak ?
Hi guys ! Q> How can i assign data of each job in mumak nodes and what else i need to do ? In general how can i use the pluggable block-placement for HDFS in Mumak ? Meaning in my context i am using 19-jobs-trace json file and modified topology json file consisting of say 4 nodes. Since the number of tasks(map & reduce) are fixed for these jobs i want to assign the input data( with their replication) of each job to some particular nodes, so that i can use this info in my scheduler . Does the code for "ObtainLocalMapTask" / "ObtainNonlocalMaptask" in Scheduler need to be changed if i have to use this data placement info ? Q>If i have to add some fields to jobs in job trace file , how do i add it to job trace and access in my scheduler code for scheduling in Mumak. Which classes do i need to modify for this? Thanks, Arun
How does splitting and choosing hosts for splits is done in Mumak ?
Hi ! I see that InputFormat, FileSplit and FileInputFormat files are involved in creating the splits of the input data for jobs. I am interested in knowing how this splitting and choosing hosts where splits are stored is managed in Mumak. I see that SimulatorJobInProgress is involved in this but the metainfo like split location has empty value. where does mumak assign split location's or does it do it at all ? Can anyone help me in this regard ? Thanks, Arun
Creating a custom trace file using Rumen
Hi guys ! I was trying to use Rumen to generate trace files. I have few queries : Q1> Is there any way to create a new trace file from job history logs with custom set of split locations ? Q2> Can we create new trace files from existing trace files with new values for the attributes like preferred locations ? Q3> How can i add new attributes / fields (which are not in job history logs) to the job or the tasks in the trace ? (or) Is there any way to generate trace with extra fields ? Can anyone help me in this regard ? Thanks, Arun
Capturing Map and Reduce I/O time
Hi guys ! I see that hadoop doesn't capture the Map task I/O time and Reduce task I/O time and captures only map runtime and reduce runtime. Am i right ? By I/O time for map task i meant time taken by the map task to read the input chunk allocated to it for processing and the time for it to write the O/P data to the local disk. By I/O time for Reduce task i meant time for reduce task to transfer map O/Ps to reduce task(shuffle phase) and writing reduce O/Ps to DFS. > What does that runtime of Map and reduce tasks mean ? Does it mean time taken to execute the Map method and reduce method respectively ? (or) Does it mean time taken from the start of the Map/Reduce task to the completion of the Map/Reduce task(i.e including time to read,sort ,compute map or reduce ,merge,etc.) ? > Which files do i need to look at and modify in Hadoop if i want to capture the map and reduce I/O time's ? > If i want to capture these values for few jobs of applications like wordcount,sort,etc. what is the best way to do ? Can anyone guide me in this regard ? Thanks, Arun
Availability of Job traces
Hi guys ! Apart from generating the job traces from RUMEN , can i get logs or job traces of varied sizes from some organizations. How can i make sure that the rumen generates only say 25 jobs,50 jobs or so ? Thanks, Arun
Re: Capturing Map/reduce task run times and bytes read
Harsh,
Sorry for creating confusion.
The question is if i have a single node setup and i give Sysout statements
in maptask.java and reducetask.java.
{HADOOP_HOME}$ant build
{HADOOP_HOME}$start all daemons
{HADOOP_HOME}$ run wordcount example
Yes i am able to see o/p in *.out files of tasktrackers.
Q>Does the map/reduce task run time displayed in web GUI is decent/accurate
enough ?
Q>If i want to do find the IO rate of a task, will the task run time
divided by total number of FIle bytes and HDFS bytes read/written give it
approximately ?
Q>Does the FILE Bytes read for the reduce task include the map output
record bytes read non-locally over network or the bytes read locally from
the map output records after they are copied locally ?
Thanks,
Arun
Re: Capturing Map/reduce task run times and bytes read
Harsh, I wanted to conform about it b'coz in case if it doesn't i want to write code to capture it. Does it make sense to classify a map/reduce task as I/O bound or cpu bound based on its I/O rate ? Arun On Sat, Dec 3, 2011 at 2:43 PM, Harsh J wrote: > Arun, > > Inline again. > > On 03-Dec-2011, at 12:39 PM, arun k wrote: > > > Q>Does the map/reduce task run time displayed in web GUI is > decent/accurate enough ? > > > Don't see why not. We only display what's been genuinely collected. What > you get out of an API on the CLI is absolutely the same thing. Or perhaps I > do not understand your question completely here - what's led you to ask > this? > > Q>If i want to do find the IO rate of a task, will the task run time > divided by total number of FIle bytes and HDFS bytes read/written give it > approximately ? > > > Yes, that should give you a stop-watch measure. Task start -> Task end, > and the counters the task puts up for itself. > > Q>Does the FILE Bytes read for the reduce task include the map output > record bytes read non-locally over network or the bytes read locally from > the map output records after they are copied locally ? > > > FILE counters are from whatever is read off a local filesystem (file:///), > so would mean the latter. If you look again, you will notice another > counter named "Reduce shuffle bytes" that gives you the former count - > separately. >
Are the task run times displayed in Web GUI accurate ?
Harsh, I had a doubt regarding task runtimes displayed in Web GUI b'coz the Web GUI shows only task run times in seconds and not in milliseconds. Can i make it display nanotime or atleast milliseconds ? (or) Can i capture the time in some other way ? Arun On Sat, Dec 3, 2011 at 8:00 PM, arun k wrote: > Harsh, > > I wanted to conform about it b'coz in case if it doesn't i want to write > code to capture it. > > Does it make sense to classify a map/reduce task as I/O bound or cpu bound > based on its I/O rate ? > > Arun > > On Sat, Dec 3, 2011 at 2:43 PM, Harsh J wrote: > >> Arun, >> >> Inline again. >> >> On 03-Dec-2011, at 12:39 PM, arun k wrote: >> >> >> Q>Does the map/reduce task run time displayed in web GUI is >> decent/accurate enough ? >> >> >> Don't see why not. We only display what's been genuinely collected. What >> you get out of an API on the CLI is absolutely the same thing. Or perhaps I >> do not understand your question completely here - what's led you to ask >> this? >> >> Q>If i want to do find the IO rate of a task, will the task run time >> divided by total number of FIle bytes and HDFS bytes read/written give it >> approximately ? >> >> >> Yes, that should give you a stop-watch measure. Task start -> Task end, >> and the counters the task puts up for itself. >> >> Q>Does the FILE Bytes read for the reduce task include the map output >> record bytes read non-locally over network or the bytes read locally from >> the map output records after they are copied locally ? >> >> >> FILE counters are from whatever is read off a local filesystem (file:///), >> so would mean the latter. If you look again, you will notice another >> counter named "Reduce shuffle bytes" that gives you the former count - >> separately. >> > >
Which Hadoop version has Adaptive scheduler ?
Hi guys ! In which Hadoop Version can i find the adaptive scheduler of https://issues.apache.org/jira/browse/MAPREDUCE-1380 Can anyone tell me the difference between Dynamic scheduler and Adaptive scheduler ? Arun
Re: Which Hadoop version has Adaptive scheduler ?
Thanks Praveen ! Arun On Wed, Dec 7, 2011 at 10:57 PM, Praveen Sripati wrote: > The resolution of the JIRA says unresolved, so it's not yet in any of the > release. Best bet is to download the patch attached with the JIRA and see > the code changes if interested. > > Regards, > Praveen > > On Wed, Dec 7, 2011 at 8:06 PM, arun k wrote: > >> Hi guys ! >> >> In which Hadoop Version can i find the adaptive scheduler of >> https://issues.apache.org/jira/browse/MAPREDUCE-1380 >> Can anyone tell me the difference between Dynamic scheduler and Adaptive >> scheduler ? >> >> >> Arun >> >> >
Checksum error during trace generation using Rumen
Hi guys !
I was trying to generate job trace and topology trace.
I have hadoop set up for hduser at /usr/local/hadoop and ran wordcount
program as hduser .
I have mapreduce component set up in eclipse for user "arun".
I set for a configuration :
Class: org.apache.hadoop.tools.rumen.TraceBuilder
Args : /home/arun/Documents/jh.json /home/arun/Documents/top.json
/usr/local/hadoop/logs/history
When i run it i get the following error even when i have given :
${HADOOP_HOME} chmod -R 777 /logs
11/12/08 11:55:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
11/12/08 11:55:18 WARN fs.FSInputChecker: Problem opening checksum file:
file:/usr/local/hadoop/logs/history/job_201109191122_0002_1316411924369_hduser_word+count.
Ignoring exception: java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:180)
at java.io.DataInputStream.readFully(DataInputStream.java:152)
at
org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.(ChecksumFileSystem.java:145)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:311)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:534)
at
org.apache.hadoop.tools.rumen.PossiblyDecompressedInputStream.(PossiblyDecompressedInputStream.java:42)
at
org.apache.hadoop.tools.rumen.DefaultInputDemuxer.bindTo(DefaultInputDemuxer.java:42)
at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:225)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:142)
Any help ?
Arun
Re: Checksum error during trace generation using Rumen
Amar,
I didn't modfiy Jobhistory file but i deleted hidden checksumfiels and it
worked.
I have a 3 node cluster set up.
For generating a single job i am running say wordcount. Is there any way
where i can generate good number of jobs(say 50,100,..) at a time ?
How can i generate jobs trace for 50, 100, so on by keeping two nodes under
one rack and the other in second rack in my cluster ?
HOw do i control the placement of nodes in different rack and generate job
and topology trace ?
Arun
On Thu, Dec 8, 2011 at 12:11 PM, Amar Kamat wrote:
> Arun,
> Did you modify the job history file manually? Looks like HDFS is not able
> to match the job history file contents to its checksum. Try deleting the
> checksum file. Note that the checksum file is a hidden file.
> Amar
>
>
>
> On 12/8/11 12:03 PM, "arun k" wrote:
>
> Hi guys !
>
> I was trying to generate job trace and topology trace.
> I have hadoop set up for hduser at /usr/local/hadoop and ran wordcount
> program as hduser .
> I have mapreduce component set up in eclipse for user "arun".
> I set for a configuration :
> Class: org.apache.hadoop.tools.rumen.TraceBuilder
> Args : /home/arun/Documents/jh.json /home/arun/Documents/top.json
> /usr/local/hadoop/logs/history
>
> When i run it i get the following error even when i have given :
> ${HADOOP_HOME} chmod -R 777 /logs
>
> 11/12/08 11:55:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 11/12/08 11:55:18 WARN fs.FSInputChecker: Problem opening checksum file:
> file:/usr/local/hadoop/logs/history/job_201109191122_0002_1316411924369_hduser_word+count.
> Ignoring exception: java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at java.io.DataInputStream.readFully(DataInputStream.java:152)
> at
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.(ChecksumFileSystem.java:145)
> at
> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:311)
> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:534)
> at
> org.apache.hadoop.tools.rumen.PossiblyDecompressedInputStream.(PossiblyDecompressedInputStream.java:42)
> at
> org.apache.hadoop.tools.rumen.DefaultInputDemuxer.bindTo(DefaultInputDemuxer.java:42)
> at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:225)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
> at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:142)
>
> Any help ?
>
> Arun
>
>
>
>
Re: Checksum error during trace generation using Rumen
Amar,
Job trace and topology trace are generated but the TraceBuilder program
terminates after displaying
11/12/08 11:55:17 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
It doesn't indicate whether it was a success and doesn't give any other
info. Is it Ok ?
I have used the generated trace with Mumak and it works fine.
Arun
On Thu, Dec 8, 2011 at 7:43 PM, arun k wrote:
> Amar,
>
> I didn't modfiy Jobhistory file but i deleted hidden checksumfiels and it
> worked.
> I have a 3 node cluster set up.
> For generating a single job i am running say wordcount. Is there any way
> where i can generate good number of jobs(say 50,100,..) at a time ?
> How can i generate jobs trace for 50, 100, so on by keeping two nodes
> under one rack and the other in second rack in my cluster ?
> HOw do i control the placement of nodes in different rack and generate job
> and topology trace ?
>
>
> Arun
>
>
> On Thu, Dec 8, 2011 at 12:11 PM, Amar Kamat wrote:
>
>> Arun,
>> Did you modify the job history file manually? Looks like HDFS is not able
>> to match the job history file contents to its checksum. Try deleting the
>> checksum file. Note that the checksum file is a hidden file.
>> Amar
>>
>>
>>
>> On 12/8/11 12:03 PM, "arun k" wrote:
>>
>> Hi guys !
>>
>> I was trying to generate job trace and topology trace.
>> I have hadoop set up for hduser at /usr/local/hadoop and ran wordcount
>> program as hduser .
>> I have mapreduce component set up in eclipse for user "arun".
>> I set for a configuration :
>> Class: org.apache.hadoop.tools.rumen.TraceBuilder
>> Args : /home/arun/Documents/jh.json /home/arun/Documents/top.json
>> /usr/local/hadoop/logs/history
>>
>> When i run it i get the following error even when i have given :
>> ${HADOOP_HOME} chmod -R 777 /logs
>>
>> 11/12/08 11:55:17 WARN util.NativeCodeLoader: Unable to load
>> native-hadoop library for your platform... using builtin-java classes where
>> applicable
>> 11/12/08 11:55:18 WARN fs.FSInputChecker: Problem opening checksum file:
>> file:/usr/local/hadoop/logs/history/job_201109191122_0002_1316411924369_hduser_word+count.
>> Ignoring exception: java.io.EOFException
>> at java.io.DataInputStream.readFully(DataInputStream.java:180)
>> at java.io.DataInputStream.readFully(DataInputStream.java:152)
>> at
>> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.(ChecksumFileSystem.java:145)
>> at
>> org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:311)
>> at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:534)
>> at
>> org.apache.hadoop.tools.rumen.PossiblyDecompressedInputStream.(PossiblyDecompressedInputStream.java:42)
>> at
>> org.apache.hadoop.tools.rumen.DefaultInputDemuxer.bindTo(DefaultInputDemuxer.java:42)
>> at org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:225)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
>> at org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:142)
>>
>> Any help ?
>>
>> Arun
>>
>>
>>
>>
>
Benchmarks for IO and CPU Intensive workload
Hi guys ! I want to see the behavior of a single node of Hadoop cluster when IO intensive / CPU intensive workload and mix of both is submitted to the single node alone. These workloads must stress the nodes. I see that TestDFSIO benchmark is good for IO intensive workload. Which benchmarks do i need to use for this ? What amount of input data will be fair enough for seeing the behavior under these workloads for each type of boxes if i have boxes with :- B1: 4 GB RAM, Dual core ,150-250 GB DISK , B2 : 1GB RAM, 50-80 GB Disk. Arun
Accessing Job counters displayed in WEB GUI in Hadoop Code
Hai guys ! Can i access the Job counters displayed in WEB GUI in Hadoop code when the job finished their execution ? If so, how can i access the values like "average task run time" and counters "FILE/HDFS BYTES READ/WRITTEN" immediately after the job has completed in JobQueueTaskScheduler Code or in some other code file ? Thanks, Arun
Where do i see Sysout statements after building example ?
HI guys ! I have a single node set up as per http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ 1>I have put some sysout statements in Jobtracker and wordcount (src/examples/org/..) code 2>ant build 3>Ran example jar with wordcount Where do i find the sysout statements ? i have seen in logs/ datanode,tasktracker,*.out files. Can anyone help me out ? Arun
Analysing Completed Job info programmatically apart from Jobtracker GUI
Hi Guys ! I want to analyse the completed Job counters like FILE/HDFS BYTES READ/WRITTEN along with other values like average map/reduce task run time. I see that Jobtracker GUI has this info but i want to programmatically retrieve these values instead of manually noting down these values and do some analysis. Can i do it in a simple/easier way ? I also see that Cloudera's HUE is good for this but is there anything equivalent in Hadoop. Can anyone guide me in this regard ? Arun
Re: Generating job and topology traces from history folder of multinode cluster using Rumen
Ravi,
Thanks for the info.
Arun
On Fri, Dec 16, 2011 at 12:27 PM, Ravi Gummadi wrote:
> Amar is working on this issue MAPREDUCE-3349. The patch is not comiited to
> trunk yet. Feel free to try it out while it gets reviewed and committed.
>
> -Ravi
> ____
> From: arun k [arunk...@gmail.com]
> Sent: Friday, December 16, 2011 12:22 PM
> To: mapreduce-user@hadoop.apache.org
> Subject: Generating job and topology traces from history folder of
> multinode cluster using Rumen
>
> Hai guys !
>
> I have set up 5 node cluster with each of them in different racks.
>
> I have hadoop-0.20.2 set up on my Eclipse Helios. So, i ran Tracebuilder
> using
> Main Class: org.apache.hadoop.tools.rumen.TraceBuilder
> I ran some jobs on cluster and used copy of
> /usr/local/hadoop/logs/history folder of master node as input to
> TraceBuilder.
> Arguments: /home/arun/job.json /home/arun/topology.json
> /home/ubuntu/Documents/testlog
>
> But i get
> 11/12/16 12:02:37 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 11/12/16 12:02:38 WARN rumen.TraceBuilder: TraceBuilder got an error while
> processing the [possibly virtual] file
> master_1324011575958_job_201112161029_0001_hduser_word+count within Path
> file:/home/ubuntu/Documents/testlog/master_1324011575958_job_201112161029_0001_hduser_word+count
> java.lang.NullPointerException
>at
> org.apache.hadoop.tools.rumen.JobBuilder.processTaskAttemptFinishedEvent(JobBuilder.java:492)
>at
> org.apache.hadoop.tools.rumen.JobBuilder.process(JobBuilder.java:149)
>at
> org.apache.hadoop.tools.rumen.TraceBuilder.processJobHistory(TraceBuilder.java:310)
>at
> org.apache.hadoop.tools.rumen.TraceBuilder.run(TraceBuilder.java:264)
>at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
>at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
>at
> org.apache.hadoop.tools.rumen.TraceBuilder.main(TraceBuilder.java:142)
> 11/12/16 12:02:39 WARN rumen.TraceBuilder: TraceBuilder got an error while
> processing the [possibly virtual] file
> master_1324011575958_job_201112161029_0004_hduser_word+count within Path
> file:/home/ubuntu/Documents/testlog/master_1324011575958_job_201112161029_0004_hduser_word+count
> java.lang.NullPointerException
>at
> org.apache.hadoop.tools.rumen.JobBuilder.processTaskAttemptFinishedEvent(JobBuilder.java:492)
>at org.apache.hadoop.tools.rumen.JobBuilder.process(JobBuilde
> .
>
>
> It generates job trace json file but the topology trace json file doesn't
> have 5 node's info and is like this :
>
> {
> "name" : "",
> "children" : [ ]
> }
>
> > Which history folder do i need to use ? masters or slaves ? how do i
> use it ?
> > I have a single node cluster from which i got logs and gave input
> TraceBuilder and it works.
> > How do i run tracebuilder from terminal and do i need to use only from
> version 21 ?
>
> Can anyone help me out ?
>
>
> Thanks,
> Arun
>
Can reduce phase be CPU/IO bound ?
Hi guys ! If we neglect the shuffle part, can reduce phase be CPU/IO bound ? Can anyone suggest some benchmark or example where we can see this ? Arun
