Re: is jdk required to run hadoop or jre alone is sufficient
In general to run just hadoop without using any other tools like sqoop, do we need jdk ? On Wed, Oct 16, 2013 at 1:08 PM, Harsh J ha...@cloudera.com wrote: You will need a JDK. Certain tools like Sqoop, etc. have a dependency on JDK for compiling generated code at runtime, and will not function without a JDK. On Wed, Oct 16, 2013 at 10:38 AM, oc tsdb oc.t...@gmail.com wrote: HI , I would like to know if JRE alone is sufficient to run HADOOP services or JDK is required ? we are planning to install latest stable version of hadoop Thanks, Oc.tsdb -- Harsh J
CDH4.4 and HBASE-8912 issue
Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912 issue. Suggested solution was to update hbase to version 0.94.13, which is absent in cloudera distribution. Is it possible to run pure hbase over cloudera hadoop? Or how can i find if this bug is present in previous versions of cdh? -- Best regards, Boris Emelyanov.
Re: is jdk required to run hadoop or jre alone is sufficient
No, you do not need JDK to just run Apache Hadoop. On Thu, Oct 17, 2013 at 2:16 PM, oc tsdb oc.t...@gmail.com wrote: In general to run just hadoop without using any other tools like sqoop, do we need jdk ? On Wed, Oct 16, 2013 at 1:08 PM, Harsh J ha...@cloudera.com wrote: You will need a JDK. Certain tools like Sqoop, etc. have a dependency on JDK for compiling generated code at runtime, and will not function without a JDK. On Wed, Oct 16, 2013 at 10:38 AM, oc tsdb oc.t...@gmail.com wrote: HI , I would like to know if JRE alone is sufficient to run HADOOP services or JDK is required ? we are planning to install latest stable version of hadoop Thanks, Oc.tsdb -- Harsh J -- Harsh J
Re: CDH4.4 and HBASE-8912 issue
If I read Lars' comment on the JIRA correctly, HBASE-8912's target was moved to 0.94.13 It is still open. Meaning, if there is no patch, the target may move to next release. Cheers On Oct 17, 2013, at 2:25 AM, Boris Emelyanov emelya...@post.km.ru wrote: Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912 issue. Suggested solution was to update hbase to version 0.94.13, which is absent in cloudera distribution. Is it possible to run pure hbase over cloudera hadoop? Or how can i find if this bug is present in previous versions of cdh? -- Best regards, Boris Emelyanov.
Re: CDH4.4 and HBASE-8912 issue
Moving to u...@hbase.apache.org. HBASE-8912 is still unresolved, and 0.94.13 is just the targeted version presently. Are you certain this is the exact issue you're hitting? I believe you can workaround this by removing the specific bad znode in ZK or so. When starting up after your major upgrade, did you ensure cleaning your ZK /hbase znode? On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanov emelya...@post.km.ru wrote: Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with hbase-0.94.6 and have faced https://issues.apache.org/jira/browse/HBASE-8912 issue. Suggested solution was to update hbase to version 0.94.13, which is absent in cloudera distribution. Is it possible to run pure hbase over cloudera hadoop? Or how can i find if this bug is present in previous versions of cdh? -- Best regards, Boris Emelyanov. -- Harsh J
Re: CDH4.4 and HBASE-8912 issue
On 17.10.2013 13:43, Harsh J wrote: Moving tou...@hbase.apache.org. HBASE-8912 is still unresolved, and 0.94.13 is just the targeted version presently. Are you certain this is the exact issue you're hitting? I believe you can workaround this by removing the specific bad znode in ZK or so. When starting up after your major upgrade, did you ensure cleaning your ZK /hbase znode? On Thu, Oct 17, 2013 at 2:55 PM, Boris Emelyanovemelya...@post.km.ru wrote: Hello! I've just upgraded my hadoop test-cluster from cdh3 to cdh4.4 with hbase-0.94.6 and have facedhttps://issues.apache.org/jira/browse/HBASE-8912 issue. Suggested solution was to update hbase to version 0.94.13, which is absent in cloudera distribution. Is it possible to run pure hbase over cloudera hadoop? Or how can i find if this bug is present in previous versions of cdh? -- Best regards, Boris Emelyanov. Are you certain this is the exact issue you're hitting? I gues so. Description suits just fine! I believe you can workaround this by removing the specific bad znode in ZK or so. When I disable loadbalancer just after master startup (balance_switch false), cluster works fine. But when loadbalancer is enabled and it starts, it allways fails with exception described in issue, allways with different regions. When starting up after your major upgrade, did you ensure cleaning your ZK /hbase znode? Yes, of course. -- Best regards, Boris Emelyanov.
Is hadoop 1.x series deprecated now ?
Hi all, As the version 2.2.0 is marked stable and is available for general release, is version 1.x series deprecated now ? As a Hadoop user, should I assume that the version 1.x series is not going to be updated with newer updates e.g., newer versions of HDFS. If that is going to be the case, can it be marked as deprecated in the website. Personally, I would have preferred the continuation of version 1.x series due to its simplicity, as YARN tries to encompass things beyond mapreduce and Hadoop has so far been about running mapreduce jobs. Also, is there any possibility that there would be resource management features added to the Hadoop 1.x series with its jobtracker/tasktracker model (maybe backported from YARN) or is the 1.x series not even considered for those types of features and now has been completely abandoned. Please excuse me if some of the points are not relevant or important. I was just curious about the status of 1.x series as I have been using it for quite a while now. thanks, Hari
Cannot start resourcemanager
Hi Team, trying to start resourcemanager in the latest hadoop-2.2.0 stable release. It throws following error. Please help 2013-10-17 10:01:51,230 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ResourceManager metrics system...2013-10-17 10:01:51,230 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system stopped.2013-10-17 10:01:51,231 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete.2013-10-17 10:01:51,232 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManagerjava.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1752) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:263) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:249) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:871)Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not foundat org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1720) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1744) ... 5 moreCaused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1718) ... 6 more2013-10-17 10:01:51,239 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUTDOWN_MSG: /SHUTDOWN_MSG: Shutting down ResourceManager at node1/192.168.147.101 ** Cheers !!! Siddharth Tiwari Have a refreshing day !!! Every duty is holy, and devotion to duty is the highest form of worship of God.” Maybe other people will try to limit me but I don't limit myself
Re: Cannot start resourcemanager
What command did you use to start the RM? On Oct 17, 2013, at 10:18 AM, Siddharth Tiwari siddharth.tiw...@live.com wrote: Hi Team, trying to start resourcemanager in the latest hadoop-2.2.0 stable release. It throws following error. Please help 2013-10-17 10:01:51,230 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ResourceManager metrics system... 2013-10-17 10:01:51,230 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system stopped. 2013-10-17 10:01:51,231 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete. 2013-10-17 10:01:51,232 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1752) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:263) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:249) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:871) Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1720) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1744) ... 5 more Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1718) ... 6 more 2013-10-17 10:01:51,239 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUTDOWN_MSG: / SHUTDOWN_MSG: Shutting down ResourceManager at node1/192.168.147.101 ** Cheers !!! Siddharth Tiwari Have a refreshing day !!! Every duty is holy, and devotion to duty is the highest form of worship of God.” Maybe other people will try to limit me but I don't limit myself -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
RE: Querying cluster nodes list
Ah, never mind.. it is getNodeReports() john From: John Lilley [mailto:john.lil...@redpoint.net] Sent: Thursday, October 17, 2013 1:44 PM To: user@hadoop.apache.org Subject: Querying cluster nodes list I thought mistakenly that getClusterMetrics() would return information about the cluster's node, or about a queue's nodes, but this doesn't seem to be true - it is only a count. How can a YARN application query the available node list on a cluster and what resources are configured on each node? Thanks john
Re: Cannot start resourcemanager
Hi, in capacity-scheduler.xml what value you have set it for below property? property nameyarn.scheduler.capacity.resource-calculator/name valueorg.apache.hadoop.yarn.util.resource.DefaultResourceCalculator/value description The ResourceCalculator implementation to be used to compare Resources in the scheduler. The default i.e. DefaultResourceCalculator only uses Memory while DominantResourceCalculator uses dominant-resource to compare multi-dimensional resources such as Memory, CPU etc. /description /property Thanks, Omkar Joshi *Hortonworks Inc.* http://www.hortonworks.com On Thu, Oct 17, 2013 at 12:48 PM, Arun C Murthy a...@hortonworks.com wrote: What command did you use to start the RM? On Oct 17, 2013, at 10:18 AM, Siddharth Tiwari siddharth.tiw...@live.com wrote: Hi Team, trying to start resourcemanager in the latest hadoop-2.2.0 stable release. It throws following error. Please help 2013-10-17 10:01:51,230 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ResourceManager metrics system... 2013-10-17 10:01:51,230 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system stopped. 2013-10-17 10:01:51,231 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete. 2013-10-17 10:01:51,232 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1752) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:263) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:249) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:871) Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1720) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1744) ... 5 more Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1718) ... 6 more 2013-10-17 10:01:51,239 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUTDOWN_MSG: / SHUTDOWN_MSG: Shutting down ResourceManager at node1/192.168.147.101 **** *Cheers !!!* *Siddharth Tiwari* Have a refreshing day !!! *Every duty is holy, and devotion to duty is the highest form of worship of God.” * *Maybe other people will try to limit me but I don't limit myself* -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Hadoop HBase Pseudo mode - RegionServer disconnects after some time
Hi - Please find the below log of HBase-master. I have tried all sorts of fixes mentioned in various threads yet I could not overcome this issue. I made sure I dont have 127.0.1.1 in /etc/hosts file. I pinged my localhost (hostname) which gives back the actual IP and not 127.0.0.1 using ping -c 1 localhost. I have 'localhost' in my /etc/hostname and actual IP address mapped to localhost.localdomain and localhost as alias - something like /etc/hosts - 192.***.*.*** localhost.localdomain localhost /etc/hostname - localhost I am using *Hadoop 0.20.205.0 and HBase 0.90.6 in Pseudo mode* for storing crawled data from a crawler - Apache Nutch 2.2.1. I can start Hadoop and HBase and when I do jps it shows all good, then after that when I start Nutch crawl after about 40mins of crawling or so, I can see Nutch hanging up while in about 4th iteration of parsing and at the same time when I do jps in HBase, I can see everything except HRegionServer. Below is the log. I tried all possible ways but couldn't overcome this issue. I really need someone from HBase list to help me with this issue. 2013-10-15 02:02:08,285 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Pushed=56 entries from hdfs://localhost:8020/hbase/.logs/127.0.0.1,60020,1381814216471/ 127.0.0.1%3A60020.1381816329235 2013-10-15 02:02:08,285 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Splitting hlog 28 of 29: hdfs://localhost:8020/hbase/.logs/127.0.0.1,60020,1381814216471/ 127.0.0.1%3A60020.1381816367672, length=64818440 2013-10-15 02:02:08,285 WARN org.apache.hadoop.hbase.util.FSUtils: Running on HDFS without append enabled may result in data loss 2013-10-15 02:02:08,554 DEBUG org.apache.*hadoop.hbase.master.HMaster: Not running balancer because processing dead regionserver(s): [127.0.0.1,60020* ,1381814216471] 2013-10-15 02:02:08,556 INFO org.apache.hadoop.hbase.catalo*g.CatalogTracker: Failed verification of .META.,,1 at address=127.0.0.1:60020; java.net.ConnectException: Connection refused* 2013-10-15 02:02:08,559 INFO org.apache.hadoop.hbase.catalog.*CatalogTracker: Current cached META location is not valid*, resetting 2013-10-15 02:02:08,601 WARN org.apache.hadoop.*hbase.master.CatalogJanitor: Failed scan of catalog table org.apache.hadoop.hbase.NotAllMetaRegionsOnlineException: Timed out (2147483647ms)* at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMeta(CatalogTracker.java:390) at org.apache.hadoop.hbase.catalog.CatalogTracker.waitForMetaServerConnectionDefault(CatalogTracker.java:422) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:255) at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:237) at org.apache.hadoop.hbase.master.CatalogJanitor.scan(CatalogJanitor.java:120) at org.apache.hadoop.hbase.master.CatalogJanitor.chore(CatalogJanitor.java:88) at org.apache.hadoop.hbase.Chore.run(Chore.java:66) 2013-10-15 02:02:08,842 INFO org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogWriter: syncFs -- HDFS-200 -- not available, dfs.support.append=false 2013-10-15 02:02:08,842 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Creating writer path=hdfs://localhost:8020/hbase/1_webpage/853ef78be7c0853208e865a9ff13d5fb/recovered.edits/0001556.temp region=853ef78be7c0853208e865a9ff13d5fb 2013-10-15 02:02:09,443 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Pushed=39 entries from hdfs://localhost:8020/hbase/.logs/127.0.0.1,60020,1381814216471/ 127.0.0.1%3A60020.1381816367672 2013-10-15 02:02:09,444 DEBUG org.apache.hadoop.hbase.regionserver.wal.HLogSplitter: Splitting hlog 29 of 29: hdfs://localhost:8020/hbase/.logs/127.0.0.1,60020,1381814216471/ 127.0.0.1%3A60020.1381816657239, length=0 Thanks for your help!
Querying cluster nodes list
I thought mistakenly that getClusterMetrics() would return information about the cluster's node, or about a queue's nodes, but this doesn't seem to be true - it is only a count. How can a YARN application query the available node list on a cluster and what resources are configured on each node? Thanks john
Re: Cannot start resourcemanager
Hi, could you check environment variables(e.g. HADOOP_COMMON_HOME/HADOOP_HDFS_HOME/HADOOP_MAPRED_HOME/HADOOP_CONF_DIR) and send us the contents of etc/yarn-site.conf? In my environment, I cannot reproduce your problem with 2.2.0 tar ball. Thanks, Tsuyoshi On Thu, Oct 17, 2013 at 10:18 AM, Siddharth Tiwari siddharth.tiw...@live.com wrote: Hi Team, trying to start resourcemanager in the latest hadoop-2.2.0 stable release. It throws following error. Please help 2013-10-17 10:01:51,230 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ResourceManager metrics system... 2013-10-17 10:01:51,230 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system stopped. 2013-10-17 10:01:51,231 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ResourceManager metrics system shutdown complete. 2013-10-17 10:01:51,232 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1752) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getResourceCalculator(CapacitySchedulerConfiguration.java:333) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:263) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:249) at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:871) Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1720) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1744) ... 5 more Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.yarn.server.resourcemanager.resource.DefaultResourceCalculator not found at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1718) ... 6 more 2013-10-17 10:01:51,239 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUTDOWN_MSG: / SHUTDOWN_MSG: Shutting down ResourceManager at node1/192.168.147.101 ** Cheers !!! Siddharth Tiwari Have a refreshing day !!! Every duty is holy, and devotion to duty is the highest form of worship of God.” Maybe other people will try to limit me but I don't limit myself -- - Tsuyoshi
Re: Yarn never use TeraSort#TotalOrderPartitioner when run TeraSort job?
It's really weird and confusing me. Anyone can help this question? Thanks! 2013/10/16 sam liu samliuhad...@gmail.com Hi Experts, In Hadoop-2.0.4, the TeraSort leverage TeraSort#TotalOrderPartitioner as its Partitioner: 'job.setPartitionerClass(TotalOrderPartitioner.class);'. However, seems Yarn did not execute the methods of TeraSort#TotalOrderPartitioner at all. I did some tests to verify it as below: Test 1: Add some code in the method readPartitions() and setConf() in TeraSort#TotalOrderPartitioner to print some words and write some word to a file. Expected Result: Some words should be printed and wrote into a file Actual Result: No word was printed and wrote into a file at all Test 2: Remove all existing methods in TeraSort#TotalOrderPartitioner, but only remaining some necessary but empty methods in it Expected Result: TeraSort job will ocurr some exception, as the specified Partitioner is not implemented at all Actual Result: TeraSort job completed successfully without any exception Above tests confused me a lot, because seems Yarn never use specified partitioner TeraSort#TotalOrderPartitioner at all during job execution. Any one can help provide the reasons? Thanks very much!
RE: Intermittent DataStreamer Exception while appending to file inside HDFS
Hi Arinto, You can check 3rd DN logs. Whether any space issues so that node was not selected for write etc. Does it mean that one of the datanodes was unreachable when we try to append into the files? It did not select for write itself. If it failed after selected for write means you should have get this error while recovery itself. Regards, Uma From: Arinto Murdopo [mailto:ari...@gmail.com] Sent: 11 October 2013 08:48 To: user@hadoop.apache.org Subject: Re: Intermittent DataStreamer Exception while appending to file inside HDFS Thank you for the comprehensive answer, When I inspect our NameNode UI, I see there are 3 datanodes are up. However, as you mentioned, the log only showed 2 datanodes are up. Does it mean that one of the datanodes was unreachable when we try to append into the files? Best regards, Arinto www.otnira.comhttp://www.otnira.com On Thu, Oct 10, 2013 at 4:57 PM, Uma Maheswara Rao G mahesw...@huawei.commailto:mahesw...@huawei.com wrote: Hi Arinto, Please disable this feature with smaller clusters. dfs.client.block.write.replace-datanode-on-failure.policy Reason for this exception is, you have replication set to 3 and looks like you have only 2 nodes in the cluster from the logs. When you first time created pipeline we will not do any verification i.e, whether pipeline DNs met the replication or not. Above property says only replace DN on failure. But here additionally we took advantage of verifying this condition when we reopen the pipeline for append. So, here unfortunately it will not meet the replication with existing DNs and it will try to add another node. Since you are not having any extra nodes in cluster other than selected nodes, it will fail. With the current configurations you can not append. Also please take a look at default configuration description: namedfs.client.block.write.replace-datanode-on-failure.enable/name valuetrue/value description If there is a datanode/network failure in the write pipeline, DFSClient will try to remove the failed datanode from the pipeline and then continue writing with the remaining datanodes. As a result, the number of datanodes in the pipeline is decreased. The feature is to add new datanodes to the pipeline. This is a site-wide property to enable/disable the feature. When the cluster size is extremely small, e.g. 3 nodes or less, cluster administrators may want to set the policy to NEVER in the default configuration file or disable this feature. Otherwise, users may experience an unusually high rate of pipeline failures since it is impossible to find new datanodes for replacement. See also dfs.client.block.write.replace-datanode-on-failure.policy /description Make this configuration false at your client side. Regards, Uma From: Arinto Murdopo [mailto:ari...@gmail.commailto:ari...@gmail.com] Sent: 10 October 2013 13:02 To: user@hadoop.apache.orgmailto:user@hadoop.apache.org Subject: Intermittent DataStreamer Exception while appending to file inside HDFS Hi there, I have this following exception while I'm appending existing file in my HDFS. This error appears intermittently. If the error does not show up, I can append the file successfully. If the error appears, I could not append the file. Here is the error: https://gist.github.com/arinto/d37a56f449c61c9d1d9c For your convenience, here it is: 13/10/10 14:17:30 WARN hdfs.DFSClient: DataStreamer Exception java.io.IOException: Failed to add a datanode. User may turn off this feature by setting dfs.client.block.write.replace-datanode-on-failure.policy in configuration, where the current policy is DEFAULT. (Nodes: current=[10.0.106.82:50010http://10.0.106.82:50010, 10.0.106.81:50010http://10.0.106.81:50010], original=[10.0.106.82:50010http://10.0.106.82:50010, 10.0.106.81:50010http://10.0.106.81:50010]) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:778) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:838) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:934) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461) Some configuration files: 1. hdfs-site.xml: https://gist.github.com/arinto/f5f1522a6f6994ddfc17#file-hdfs-append-datastream-exception-hdfs-site-xml 2. core-site.xml: https://gist.github.com/arinto/0c6f40872181fe26f8b1#file-hdfs-append-datastream-exception-core-site-xml So, any idea how to solve this issue? Some links that I've found (but unfortunately they do not help) 1. StackOverflowhttp://stackoverflow.com/questions/15347799/java-io-ioexception-failed-to-add-a-datanode-hdfs-hadoop, our replication factor is 3 and we've never changed the replication factor since we setup the cluster. 2. Impala-User mailing
