This the content of the jobtracker log file : 2013-03-23 12:06:48,912 INFO org.apache.hadoop.mapred.JobInProgress: Input size for job job_201303231139_0001 = 6950001. Number of splits = 7 2013-03-23 12:06:48,925 INFO org.apache.hadoop.mapred.JobInProgress: tip:task_201303231139_0001_m_000000 has split on node:/default-rack/hadoop0.novalocal 2013-03-23 12:06:48,927 INFO org.apache.hadoop.mapred.JobInProgress: tip:task_201303231139_0001_m_000001 has split on node:/default-rack/hadoop0.novalocal 2013-03-23 12:06:48,930 INFO org.apache.hadoop.mapred.JobInProgress: tip:task_201303231139_0001_m_000002 has split on node:/default-rack/hadoop0.novalocal 2013-03-23 12:06:48,931 INFO org.apache.hadoop.mapred.JobInProgress: tip:task_201303231139_0001_m_000003 has split on node:/default-rack/hadoop0.novalocal 2013-03-23 12:06:48,933 INFO org.apache.hadoop.mapred.JobInProgress: tip:task_201303231139_0001_m_000004 has split on node:/default-rack/hadoop0.novalocal 2013-03-23 12:06:48,934 INFO org.apache.hadoop.mapred.JobInProgress: tip:task_201303231139_0001_m_000005 has split on node:/default-rack/hadoop0.novalocal 2013-03-23 12:06:48,939 INFO org.apache.hadoop.mapred.JobInProgress: tip:task_201303231139_0001_m_000006 has split on node:/default-rack/hadoop0.novalocal 2013-03-23 12:06:48,950 INFO org.apache.hadoop.mapred.JobInProgress: job_201303231139_0001 LOCALITY_WAIT_FACTOR=0.5 2013-03-23 12:06:48,978 INFO org.apache.hadoop.mapred.JobInProgress: Job job_201303231139_0001 initialized successfully with 7 map tasks and 1 reduce tasks. 2013-03-23 12:06:50,855 INFO org.apache.hadoop.mapred.JobTracker: Adding task (JOB_SETUP) 'attempt_201303231139_0001_m_000008_0' to tip task_201303231139_0001_m_000008, for tracker 'tracker_hadoop0.novalocal:hadoop0.novalocal/127.0.0.1:44879' 2013-03-23 12:08:00,340 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201303231139_0001_m_000008_0' has completed task_201303231139_0001_m_000008 successfully. 2013-03-23 12:08:00,538 WARN org.apache.hadoop.mapred.JobInProgress: No room for map task. Node hadoop0.novalocal has 8791543808 bytes free; but we expect map to take 1317624576693539401 2013-03-23 12:08:00,543 WARN org.apache.hadoop.mapred.JobInProgress: No room for map task. Node hadoop0.novalocal has 8791543808 bytes free; but we expect map to take 1317624576693539401 2013-03-23 12:08:00,544 WARN org.apache.hadoop.mapred.JobInProgress: No room for map task. Node hadoop0.novalocal has 8791543808 bytes free; but we expect map to take 1317624576693539401 2013-03-23 12:08:00,544 WARN org.apache.hadoop.mapred.JobInProgress: No room for map task. Node hadoop0.novalocal has 8791543808 bytes free; but we expect map to take 1317624576693539401 2013-03-23 12:08:01,264 WARN org.apache.hadoop.mapred.JobInProgress: No room for map task. Node hadoop1.novalocal has 8807518208 bytes free; but we expect map to take 1317624576693539401
The value in we excpect map to take is too huge 1317624576693539401 bytes !!!!!!! On Sat, Mar 23, 2013 at 11:37 AM, Redwane belmaati cherkaoui < reduno1...@googlemail.com> wrote: > The estimated value that the hadoop compute is too huge for the simple > example that i am running . > > ---------- Forwarded message ---------- > From: Redwane belmaati cherkaoui <reduno1...@googlemail.com> > Date: Sat, Mar 23, 2013 at 11:32 AM > Subject: Re: About running a simple wordcount mapreduce > To: Abdelrahman Shettia <ashet...@hortonworks.com> > Cc: user@hadoop.apache.org, reduno1985 <reduno1...@gmail.com> > > > This the output that I get I am running two machines as you can see do u > see anything suspicious ? > Configured Capacity: 21145698304 (19.69 GB) > Present Capacity: 17615499264 (16.41 GB) > DFS Remaining: 17615441920 (16.41 GB) > DFS Used: 57344 (56 KB) > DFS Used%: 0% > Under replicated blocks: 0 > Blocks with corrupt replicas: 0 > Missing blocks: 0 > > ------------------------------------------------- > Datanodes available: 2 (2 total, 0 dead) > > Name: 11.1.0.6:50010 > Decommission Status : Normal > Configured Capacity: 10572849152 (9.85 GB) > DFS Used: 28672 (28 KB) > Non DFS Used: 1765019648 (1.64 GB) > DFS Remaining: 8807800832(8.2 GB) > DFS Used%: 0% > DFS Remaining%: 83.31% > Last contact: Sat Mar 23 11:30:10 CET 2013 > > > Name: 11.1.0.3:50010 > Decommission Status : Normal > Configured Capacity: 10572849152 (9.85 GB) > DFS Used: 28672 (28 KB) > Non DFS Used: 1765179392 (1.64 GB) > DFS Remaining: 8807641088(8.2 GB) > DFS Used%: 0% > DFS Remaining%: 83.3% > Last contact: Sat Mar 23 11:30:08 CET 2013 > > > On Fri, Mar 22, 2013 at 10:19 PM, Abdelrahman Shettia < > ashet...@hortonworks.com> wrote: > >> Hi Redwane, >> >> Please run the following command as hdfs user on any datanode. The output >> will be something like this. Hope this helps >> >> hadoop dfsadmin -report >> Configured Capacity: 81075068925 (75.51 GB) >> Present Capacity: 70375292928 (65.54 GB) >> DFS Remaining: 69895163904 (65.09 GB) >> DFS Used: 480129024 (457.89 MB) >> DFS Used%: 0.68% >> Under replicated blocks: 0 >> Blocks with corrupt replicas: 0 >> Missing blocks: 0 >> >> Thanks >> -Abdelrahman >> >> >> On Fri, Mar 22, 2013 at 12:35 PM, reduno1985 >> <reduno1...@googlemail.com>wrote: >> >>> >>> I have my hosts running on openstack virtual machine instances each >>> instance has 10gb hard disc . Is there a way too see how much space is in >>> the hdfs without web ui . >>> >>> >>> Sent from Samsung Mobile >>> >>> Serge Blazhievsky <hadoop...@gmail.com> wrote: >>> Check web ui how much space you have on hdfs??? >>> >>> Sent from my iPhone >>> >>> On Mar 22, 2013, at 11:41 AM, Abdelrahman Shettia < >>> ashet...@hortonworks.com> wrote: >>> >>> Hi Redwane , >>> >>> It is possible that the hosts which are running tasks are do not have >>> enough space. Those dirs are confiugred in mapred-site.xml >>> >>> >>> >>> On Fri, Mar 22, 2013 at 8:42 AM, Redwane belmaati cherkaoui < >>> reduno1...@googlemail.com> wrote: >>> >>>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: Redwane belmaati cherkaoui <reduno1...@googlemail.com> >>>> Date: Fri, Mar 22, 2013 at 4:39 PM >>>> Subject: About running a simple wordcount mapreduce >>>> To: mapreduce-iss...@hadoop.apache.org >>>> >>>> >>>> Hi >>>> I am trying to run a wordcount mapreduce job on several files (<20 mb) >>>> using two machines . I get stuck on 0% map 0% reduce. >>>> The jobtracker log file shows the following warning: >>>> WARN org.apache.hadoop.mapred.JobInProgress: No room for map task. >>>> Node hadoop0.novalocal has 8791384064 bytes free; but we expect map to take >>>> 1317624576693539401 >>>> >>>> Please help me , >>>> Best Regards, >>>> >>>> >>> >> > >
