Can Hadoop handle many appliations
Hello Everybody, I have a few questions regarding running applications on Hadoop. 1) Can a single Hadoop cluster simultaneously handle applications of same type ( or of different type). 2) What are the challenges in running applications of different types in Hadoop. How Does it differ from running applications of same type. Thanking you, Regards, Ashish.
Re: Hadoop error help- file system closed, could only be replicated to 0 nodes, instead of 1
HI , What seems from your details is that datanode is not running.can you run *bin/hadoop dfsadmin -report* and find out whether your datanodes are up ? then post your observation and it would be better if you post even your hadoop-site.xml file deatils also. Regards, Ashish. On Fri, Jun 19, 2009 at 3:16 AM, wrote: > Hi, > > I am extremely new to Hadoop and have come across a few errors that I'm not > sure how to fix. I am running Hadoop version 0.19.0 from an image through > Elasticfox and S3. I am on windows and use puTTY as my ssh. I am trying to > run a wordcount with 5 slaves. This is what I do so far: > > 1. boot up the instance through ElasticFox > 2. cd /usr/local/hadoop-0.19.0 > 3. bin/hadoop namenode -format > 4. bin/start-all.sh > 5. jps --( shows jps, jobtracker, secondarynamenode) > 6.bin/stop-all.sh > 7. ant examples > 8. bin/start-all.sh > 9. bin/hadoop jar build/hadoop-0.19.0-examples.jar pi 0 100 > > Then I get this error trace: > > Number of Maps = 0 Samples per Map = 100 > Starting Job > 09/06/18 17:31:25 INFO hdfs.DFSClient: > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File > /mnt/hadoop/mapred/system/job_200906181730_0001/job.jar could only be > replicated to 0 nodes, instead of 1 >at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1270) >at > org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:351) >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >at java.lang.reflect.Method.invoke(Method.java:597) >at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452) >at org.apache.hadoop.ipc.Server$Handler.run(Server.java:892) > >at org.apache.hadoop.ipc.Client.call(Client.java:696) >at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216) >at $Proxy0.addBlock(Unknown Source) >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >at java.lang.reflect.Method.invoke(Method.java:597) >at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) >at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) >at $Proxy0.addBlock(Unknown Source) >at > org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2815) >at > org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2697) >at > org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:1997) >at > org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2183) > > 09/06/18 17:31:25 WARN hdfs.DFSClient: NotReplicatedYetException sleeping > /mnt/hadoop/mapred/system/job_200906181730_0001/job.jar retries left 4 > 09/06/18 17:31:25 INFO hdfs.DFSClient: > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File > /mnt/hadoop/mapred/system/job_200906181730_0001/job.jar could only be > replicated to 0 nodes, instead of 1 >at > org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1270) >at > org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:351) >at sun.reflec,t.NativeMethodAccessorImpl.invoke0(Native Method) >at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >at java.lang.reflect.Method.invoke(Method.java:597) >at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:452) >at org.apache.hadoop.ipc.Server$Handler.run(Server.java:892) > >at org.apache.hadoop.ipc.Client.call(Client.java:696) >at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216) >at $Proxy0.addBlock(Unknown Source) >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >at java.lang.reflect.Method.invoke(Method.java:597) >at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82) >at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59) >at $Proxy0.addBlock(Unknown Source) >at > org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2815) >at > org.apac
Re: Need help
Hello Everybody, How can we handle different applications having different requirement being run on the same hadoop cluster ? What are the various approaches to solve such problem.. if possible please mention some of those ideas. Does such implementation exists ? Thanks , Regards, Ashish On Thu, Jun 18, 2009 at 9:36 PM, jason hadoop wrote: > For me, I like to have one configuration file that I distribute to all of > the machines in my cluster via rsync. > > In there are things like the number of tasks per node to run, and where to > store dfs data and local temporary data, and the limits to storage for the > machines. > > If the machines are very different, it becomes important to tailor the > configuration file per machine or type of machine. > > At this point, you are pretty much going to have to spend the time, reading > through the details of configuring a hadoop cluster. > > > On Thu, Jun 18, 2009 at 8:33 AM, ashish pareek > wrote: > > > Can you tell few of the challenges in configuring heterogeneous > > cluster...or > > can pass on some link where I would get some information regarding > > challenges in running Hadoop on heterogeneous hardware > > > > One more things is How about running different applications on the same > > Hadoop cluster?and what challenges are involved in it ? > > > > Thanks, > > Regards, > > Ashish > > > > > > On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop > >wrote: > > > > > I don't know anyone who has a completely homogeneous cluster. > > > > > > So hadoop is scalable across heterogeneous environments. > > > > > > I stated that configuration is simpler if the machines are similar > (There > > > are optimizations in configuration for near homogeneous machines.) > > > > > > On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek > > > wrote: > > > > > > > Does that mean hadoop is not scalable wrt heterogeneous environment? > > and > > > > one > > > > more question is can we run different application on the same hadoop > > > > cluster > > > > . > > > > > > > > Thanks. > > > > Regards, > > > > Ashish > > > > > > > > On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop < > jason.had...@gmail.com > > > > >wrote: > > > > > > > > > Hadoop has always been reasonably agnostic wrt hardware and > > > homogeneity. > > > > > There are optimizations in configuration for near homogeneous > > machines. > > > > > > > > > > > > > > > > > > > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek < > pareek...@gmail.com> > > > > > wrote: > > > > > > > > > > > Hello, > > > > > >I am doing my master my final year project is on > Hadoop > > > > ...so > > > > > I > > > > > > would like to know some thing about Hadoop cluster i.e, Do new > > > version > > > > of > > > > > > Hadoop are able to handle heterogeneous hardware.If you have any > > > > > > informantion regarding these please mail me as my project is in > > > > > > heterogenous > > > > > > environment. > > > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > Reagrds, > > > > > > Ashish Pareek > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Pro Hadoop, a book to guide you from beginner to hadoop mastery, > > > > > http://www.amazon.com/dp/1430219424?tag=jewlerymall > > > > > www.prohadoopbook.com a community for Hadoop Professionals > > > > > > > > > > > > > > > > > > > > > -- > > > Pro Hadoop, a book to guide you from beginner to hadoop mastery, > > > http://www.amazon.com/dp/1430219424?tag=jewlerymall > > > www.prohadoopbook.com a community for Hadoop Professionals > > > > > > > > > -- > Pro Hadoop, a book to guide you from beginner to hadoop mastery, > http://www.amazon.com/dp/1430219424?tag=jewlerymall > www.prohadoopbook.com a community for Hadoop Professionals >
Re: Need help
Can you tell few of the challenges in configuring heterogeneous cluster...or can pass on some link where I would get some information regarding challenges in running Hadoop on heterogeneous hardware One more things is How about running different applications on the same Hadoop cluster?and what challenges are involved in it ? Thanks, Regards, Ashish On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop wrote: > I don't know anyone who has a completely homogeneous cluster. > > So hadoop is scalable across heterogeneous environments. > > I stated that configuration is simpler if the machines are similar (There > are optimizations in configuration for near homogeneous machines.) > > On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek > wrote: > > > Does that mean hadoop is not scalable wrt heterogeneous environment? and > > one > > more question is can we run different application on the same hadoop > > cluster > > . > > > > Thanks. > > Regards, > > Ashish > > > > On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop > >wrote: > > > > > Hadoop has always been reasonably agnostic wrt hardware and > homogeneity. > > > There are optimizations in configuration for near homogeneous machines. > > > > > > > > > > > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek > > > wrote: > > > > > > > Hello, > > > >I am doing my master my final year project is on Hadoop > > ...so > > > I > > > > would like to know some thing about Hadoop cluster i.e, Do new > version > > of > > > > Hadoop are able to handle heterogeneous hardware.If you have any > > > > informantion regarding these please mail me as my project is in > > > > heterogenous > > > > environment. > > > > > > > > > > > > Thanks! > > > > > > > > Reagrds, > > > > Ashish Pareek > > > > > > > > > > > > > > > > -- > > > Pro Hadoop, a book to guide you from beginner to hadoop mastery, > > > http://www.amazon.com/dp/1430219424?tag=jewlerymall > > > www.prohadoopbook.com a community for Hadoop Professionals > > > > > > > > > -- > Pro Hadoop, a book to guide you from beginner to hadoop mastery, > http://www.amazon.com/dp/1430219424?tag=jewlerymall > www.prohadoopbook.com a community for Hadoop Professionals >
Re: Need help
Does that mean hadoop is not scalable wrt heterogeneous environment? and one more question is can we run different application on the same hadoop cluster . Thanks. Regards, Ashish On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop wrote: > Hadoop has always been reasonably agnostic wrt hardware and homogeneity. > There are optimizations in configuration for near homogeneous machines. > > > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek > wrote: > > > Hello, > >I am doing my master my final year project is on Hadoop ...so > I > > would like to know some thing about Hadoop cluster i.e, Do new version of > > Hadoop are able to handle heterogeneous hardware.If you have any > > informantion regarding these please mail me as my project is in > > heterogenous > > environment. > > > > > > Thanks! > > > > Reagrds, > > Ashish Pareek > > > > > > -- > Pro Hadoop, a book to guide you from beginner to hadoop mastery, > http://www.amazon.com/dp/1430219424?tag=jewlerymall > www.prohadoopbook.com a community for Hadoop Professionals >
Fwd: Need help
Hello, I am doing my master my final year project is on Hadoop ...so I would like to know some thing about Hadoop cluster i.e, Do new version of Hadoop are able to handle heterogeneous hardware.If you have any informantion regarding these please mail me as my project is in heterogenous environment. Thanks! Reagrds, Ashish Pareek
Problem in viewing WEB UI
Hi, When I run command *bin/hadoop dfsadmin -report *it shows that 2 datanodes are alive but when i try to http://hadoopmster:50070/ but the problem is that it opens doesnot opne http://hadoopmaster:50070/dfshealth.jsp page and throws *error HTTP: 404 . So why is't happening like this? * Regards, Ashish Pareek On Wed, Jun 17, 2009 at 10:06 AM, Sugandha Neaolekar < sugandha@gmail.com> wrote: > Well, You just have to specify the address in the URL address bar as:: > http://hadoopmaster:50070 U'll be able to see the web UI..! > > > On Tue, Jun 16, 2009 at 7:17 PM, ashish pareek wrote: > >> HI Sugandha, >>Hmmm your suggestion helped and Now I am able >> to run two datanode one on the same machine as name node and other on >> the different machine Thanks a lot :) >> >> But the problem is now I am not able to see web UI . >> for both datanode and as well as name node >> should I have to consider some more things in the site.xml ? if so please >> help... >> >> Thanking you again, >> regards, >> Ashish Pareek. >> >> On Tue, Jun 16, 2009 at 3:10 PM, Sugandha Naolekar < >> sugandha@gmail.com> wrote: >> >>> hi,,! >>> >>> >>> First of all, get your concepts clear of hadoop. >>> You can refer to the following >>> >>> site:: >>> http://www.google.co.in/url?sa=t&source=web&ct=res&cd=1&url=http%3A%2F%2Fwww.michael-noll.com%2Fwiki%2FRunning_Hadoop_On_Ubuntu_Linux_(Single-Node_Cluster)&ei=lGU3Spv2FZbLjAe19KmiDQ&usg=AFQjCNFbmVGsoChOSMzCB3tRhoV0ylHOzA&sig2=t2AJ_nf24SFtveN4PHS_TA<http://www.google.co.in/url?sa=t&source=web&ct=res&cd=1&url=http%3A%2F%2Fwww.michael-noll.com%2Fwiki%2FRunning_Hadoop_On_Ubuntu_Linux_%28Single-Node_Cluster%29&ei=lGU3Spv2FZbLjAe19KmiDQ&usg=AFQjCNFbmVGsoChOSMzCB3tRhoV0ylHOzA&sig2=t2AJ_nf24SFtveN4PHS_TA> >>> >>> >>> I have small doubt whether in the mater.xml and slave.xml we can have >>> same port numbers to both of them like >>> >>> >>> for slave : >>> >>> >>> fs.default.name >>> hdfs://hadoopslave: >>>> >>>> 9000 >>>> >>>> >>>> >>>> for master::: >>>> >>>> >>>> fs.default.name >>>> hdfs://hadoopmaster:9000 >>>> >>>> >>>> >>> >>> Well, any two daemons or services can run on the same port unless, they >>> are not run on the same machine.If you wish to run DN and NN on the same >>> machine, their port numbers have to be different. >>> >>> >>> >>> >>> On Tue, Jun 16, 2009 at 2:55 PM, ashish pareek wrote: >>> >>>> HI sugandha, >>>> >>>> >>>> >>>> and one more thing can we have in slave::: >>>> >>>> >>>> dfs.datanode.address>>>> >>>>> name> >>>>> hadoopmaster:9000 >>>>> hadoopslave:9001 >>>>> >>>>> >>>> >>> >>> Also, fs,default.name is the tag which specifies the default filesystem. >>> And generaLLY, it is run on namenode. So, it;s value has to be a namenode's >>> address only and not slave's. >>> >>> >>>> >>>> Else if you have complete procedure for installing and running Hadoop in >>>> cluster can you please send it to me .. I need to step up hadoop with >>>> in >>>> two days and show it to my guide.Currently I am doing my masters. >>>> >>>> Thanks for your spending time >>> >>> >>> Try for the above, and this should work! >>> >>>> >>>> >>>> regards, >>>> Ashish Pareek >>>> >>>> >>>> On Tue, Jun 16, 2009 at 2:33 PM, Sugandha Naolekar < >>>> sugandha@gmail.com> wrote: >>>> >>>>> Following changes are to be done:: >>>>> >>>>> Under master folder:: >>>>> >>>>> -> put slaves address as well under the values of >>>>> tag(dfs.datanode.address) >>>>> >>>>> -> You want to make namenode as datanode as well. As per your config >>>>> file, you have specified hadoopmaster in your slave file. If you don't >>>>> want >>
org.apache.hadoop.ipc.client : trying connect to server failed
HI , I am trying to step up a hadoop cluster on 3GB machine and using hadoop 0.18.3 and have followed procedure given in apache hadoop site for hadoop cluster. In conf/slaves I have added two datanode i.e including the namenode vitrual machine and other machine virtual machine (datanode) . and have set up passwordless ssh between both virtual machines . But now problem is when I run command : bin/hadoop start-all.sh It start only one datanode on the same namenode vitrual machine but it doesn't start the datanode on other machine. in logs/hadoop-datanode.log i get message INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop1/192.168.1.28:9000. Already tried 1 time(s). 2009-05-09 18:35:14,266 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop1/192.168.1.28:9000. Already tried 2 time(s). 2009-05-09 18:35:14,266 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop1/192.168.1.28:9000. Already tried 3 time(s). . . . . . . . . . . . . I have tried formatting and start the cluster again .but still I get the same error. So can any one help in solving this problem. :) Thanks Regards Ashish Pareek
Re: org.apache.hadoop.ipc.client : trying connect to server failed
Yes I am able to ping and ssh between two virtual machine and even i have set ip address of both the virtual machines in their respective /etc/hosts file ... thanx for reply .. if you suggest some other thing which i could have missed or any remedy Regards, Ashish Pareek. On Fri, May 29, 2009 at 10:04 AM, Pankil Doshi wrote: > make sure u can ping that data node and ssh it. > > > On Thu, May 28, 2009 at 12:02 PM, ashish pareek > wrote: > > > HI , > > I am trying to step up a hadoop cluster on 512 MB machine and using > > hadoop 0.18 and have followed procedure given in apache hadoop site for > > hadoop cluster. > > I included in conf/slaves two datanode i.e including the namenode > > vitrual machine and other machine virtual machine . and have set up > > passwordless ssh between both virtual machines. But now problem is > when > > is run command >> > > > > bin/hadoop start-all.sh > > > > It start only one datanode on the same namenode vitrual machine but it > > doesn't start the datanode on other machine. > > > > in logs/hadoop-datanode i get message > > > > > > INFO org.apache.hadoop.ipc.Client: Retrying > > connect to server: hadoop1/192.168.1.28:9000. Already > > tried 1 time(s). > > 2009-05-09 18:35:14,266 INFO org.apache.hadoop.ipc.Client: Retrying > > connect to server: hadoop1/192.168.1.28:9000. Already tried 2 time(s). > > 2009-05-09 18:35:14,266 INFO org.apache.hadoop.ipc.Client: Retrying > > connect to server: hadoop1/192.168.1.28:9000. Already tried 3 time(s). > > > > . > > . > > . > > . > > . > > . > > . > > . > > . > > > > . > > . > > > > . > > > > > > So can any one help in solving this problem. :) > > > > Thanks > > > > Regards > > Ashish Pareek > > >
Re: org.apache.hadoop.ipc.client : trying connect to server failed
Hi some help me out On Thu, May 28, 2009 at 10:32 PM, ashish pareek wrote: > HI , > I am trying to step up a hadoop cluster on 512 MB machine and using > hadoop 0.18 and have followed procedure given in apache hadoop site for > hadoop cluster. > I included in conf/slaves two datanode i.e including the namenode > vitrual machine and other machine virtual machine . and have set up > passwordless ssh between both virtual machines. But now problem is when > is run command >> > > bin/hadoop start-all.sh > > It start only one datanode on the same namenode vitrual machine but it > doesn't start the datanode on other machine. > > in logs/hadoop-datanode i get message > > > INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: hadoop1/192.168.1.28:9000. Already > tried 1 time(s). > > 2009-05-09 18:35:14,266 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: hadoop1/192.168.1.28:9000. Already tried 2 time(s). > 2009-05-09 18:35:14,266 INFO org.apache.hadoop.ipc.Client: Retrying > connect to server: hadoop1/192.168.1.28:9000. Already tried 3 time(s). > > . > . > . > . > . > . > . > . > . > > . > . > > . > > > So can any one help in solving this problem. :) > > Thanks > > Regards > > Ashish Pareek > > >
org.apache.hadoop.ipc.client : trying connect to server failed
HI , I am trying to step up a hadoop cluster on 512 MB machine and using hadoop 0.18 and have followed procedure given in apache hadoop site for hadoop cluster. I included in conf/slaves two datanode i.e including the namenode vitrual machine and other machine virtual machine . and have set up passwordless ssh between both virtual machines. But now problem is when is run command >> bin/hadoop start-all.sh It start only one datanode on the same namenode vitrual machine but it doesn't start the datanode on other machine. in logs/hadoop-datanode i get message INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop1/192.168.1.28:9000. Already tried 1 time(s). 2009-05-09 18:35:14,266 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop1/192.168.1.28:9000. Already tried 2 time(s). 2009-05-09 18:35:14,266 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: hadoop1/192.168.1.28:9000. Already tried 3 time(s). . . . . . . . . . . . . So can any one help in solving this problem. :) Thanks Regards Ashish Pareek
Re: Could only be replicated to 0 nodes, instead of 1
Hi , I have two suggestion i)Choose a right version ( Hadoop- 0.18 is good) ii)replication should be 3 as ur having 3 modes.( Indirectly see to it that ur configuration is correct !!) Hey even i am just suggesting this as i am also a new to hadoop Ashish Pareek On Thu, May 21, 2009 at 2:41 PM, Stas Oskin wrote: > Hi. > > I'm testing Hadoop in our lab, and started getting the following message > when trying to copy a file: > Could only be replicated to 0 nodes, instead of 1 > > I have the following setup: > > * 3 machines, 2 of them with only 80GB of space, and 1 with 1.5GB > * Two clients are copying files all the time (one of them is the 1.5GB > machine) > * The replication is set on 2 > * I let the space on 2 smaller machines to end, to test the behavior > > Now, one of the clients (the one located on 1.5GB) works fine, and the > other > one - the external, unable to copy and displays the error + the exception > below > > Any idea if this expected on my scenario? Or how it can be solved? > > Thanks in advance. > > > > 09/05/21 10:51:03 WARN dfs.DFSClient: NotReplicatedYetException sleeping > /test/test.bin retries left 1 > > 09/05/21 10:51:06 WARN dfs.DFSClient: DataStreamer Exception: > org.apache.hadoop.ipc.RemoteException: java.io.IOException: File > /test/test.bin could only be replicated to 0 nodes, instead of 1 > >at > > org.apache.hadoop.dfs.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1123 > ) > >at org.apache.hadoop.dfs.NameNode.addBlock(NameNode.java:330) > >at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) > >at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25 > ) > >at java.lang.reflect.Method.invoke(Method.java:597) > >at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:481) > >at org.apache.hadoop.ipc.Server$Handler.run(Server.java:890) > > > >at org.apache.hadoop.ipc.Client.call(Client.java:716) > >at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216) > >at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source) > >at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39 > ) > >at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25 > ) > >at java.lang.reflect.Method.invoke(Method.java:597) > >at > > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82 > ) > >at > > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59 > ) > >at org.apache.hadoop.dfs.$Proxy0.addBlock(Unknown Source) > >at > > org.apache.hadoop.dfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2450 > ) > >at > > org.apache.hadoop.dfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2333 > ) > >at > > org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1800(DFSClient.java:1745 > ) > >at > > org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1922 > ) > > > > 09/05/21 10:51:06 WARN dfs.DFSClient: Error Recovery for block null bad > datanode[0] > > java.io.IOException: Could not get block locations. Aborting... > >at > > org.apache.hadoop.dfs.DFSClient$DFSOutputStream.processDatanodeError(DFSClient.java:2153 > ) > >at > > org.apache.hadoop.dfs.DFSClient$DFSOutputStream.access$1400(DFSClient.java:1745 > ) > >at > > org.apache.hadoop.dfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:1899 > ) >
TASKS KILLED WHEN RUNNING : bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'
hELLO TO EVERY BODY I AM A NEW HAOODP USER I STARTED RUNNING HADOOP USING SITE http://hadoop.apache.org/core/docs/current/quickstart.html BUT WHEN I RUN COMMAND " bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+' ", IN PSEDUO DISTRIBUTED MODE I GET ERROR LIKE ::: Task task_200801251420_0007_m_06_0 failed to report status for 601 seconds. Killing! Task task_200801251420_0007_m_07_0 failed to report status for 602 seconds. Killing! AND SO ON... THEN ALL THE TAKED GETS KILLED BUT STILL DATANODE IS ALIVE... I HAVE BEEN RUNNING HADOOP IN VMWARE AND ON 512MB RAM MACHINE.. SO PLZ HELP ME IN SOLVING THIS PROBLEM... THANKS IN ADVANCE. REGARDS, ASHISH
