Re: Error starting MRAppMaster
Seems you are using Pig with Hadoop 0.23 or 2.0 version. Can you quickly recompile pig with 23 option and try this again. On Thu, Jun 21, 2012 at 10:02 AM, Prajakta Kalmegh pkalm...@gmail.comwrote: Hi I am getting the following error while trying to execute any example (wordcount, terasort etc): 12/06/21 09:52:39 INFO mapreduce.Job: Running job: job_1340251923324_0001 12/06/21 09:52:45 INFO mapreduce.Job: Job job_1340251923324_0001 running in uber mode : false 12/06/21 09:52:45 INFO mapreduce.Job: map 0% reduce 0% 12/06/21 09:52:45 INFO mapreduce.Job: Job job_1340251923324_0001 failed with state FAILED due to: Application application_1340251923324_0001 failed 1 times due to AM Container for appattempt_1340251923324_0001_01 exited with exitCode: 1 due to: .Failing this attempt.. Failing the application. 12/06/21 09:52:45 INFO mapreduce.Job: Counters: 0 - *The contents of containers logs are pasted below:* 2012-06-21 09:52:43,856 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1340251923324_0001_01 2012-06-21 09:52:44,625 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.Error: Unresolved compilation problems: The method register(Class? extends Enum, EventHandler) in the type Dispatcher is not applicable for the arguments (ClassEventType, EventHandlerJobHistoryEvent) org.apache.hadoop.mapreduce.jobhistory.EventType cannot be resolved to a type at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.init(MRAppMaster.java:261) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1049) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1046) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1010) 2012-06-21 09:52:44,628 INFO [Thread-0] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster received a signal. Signaling RMCommunicator and JobHistoryEventHandler. -- Any idea what would be going wrong? Regards, Prajakta
Re: Permission request from Translation for education
Hello, The documentation is released under Apache 2.0 You are free to modify and translate. Just go through the below *Redistribution *section also. http://www.apache.org/licenses/LICENSE-2.0.html Regards, Jagat Singh On Tue, Jun 19, 2012 at 10:04 AM, vseslava.kavch...@gmail.com wrote: Hey there, I am a student at the Department of Foreign Languages and at the same time a volunteer at an organization named “Translation for Education”. I love surfing on the Internet and being informed about the latest happenings around me. Unfortunately, most of my fellow citizens don’t know English, so that makes them feel somehow excluded from all this variety of useful info from the Internet. That was the reason that made me decide to start being a volunteer at such an NGO. So I created a blog, where I post translations of some of the texts that really caught my attention. Sometimes it’s a simple description of some organization, and sometimes it might be a narrowly specialized scientific article. Don’t you think that I do that without authors’ permission! I do ask them if they let me translate and then post the translation on my blog. And if they don’t, then they don’t. So I am asking you about the same, actually. Could you provide me the permission to translate the article on page http://hadoop.apache.org/** common/docs/stable/hdfs_user_**guide.htmlhttp://hadoop.apache.org/common/docs/stable/hdfs_user_guide.htmlinto , on condition that the translation is absolutely non-commercial and I will mention and credit you as the author of the article, and put the link to your source next to my translation too. Hoping for your understanding and waiting for your answer! Cheers,
Re: namenode startup error
Just try the steps which i mentioned in your earlier question. And this would go.
Re: Yahoo Hadoop Tutorial with new APIs?
Hi , Can you update this link in Jira which Robert created so that we can merge what ever we can into code examples. Thanks, Jagat Singh --- Sent from Mobile , short and crisp. On 19-Jun-2012 8:34 AM, JAGANADH G jagana...@gmail.com wrote: On Mon, Jun 4, 2012 at 8:35 PM, Robert Evans ev...@yahoo-inc.com wrote: I am happy to announce that I was able to get the license on the Yahoo! Hadoop tutorial updated from Creative Commons Attribution 3.0 Unported License to Apache 2.0. I have filed HADOOP-8477 https://issues.apache.org/jira/browse/HADOOP-8477 to pull the tutorial into the Hadoop project, and to update it accordingly. I am going to be very busy the next little while and I am hoping that those in the community that want this can help drive it and possibly break it down into subtasks to get the tutorial up to date. I am very happy to help out, but like I said it may be a while before I am able to do much on this. Hi I Just started rewriting the code . The code is available at https://bitbucket.org/jaganadhg/yahoohadooptutorialcoderw Best regards -- ** JAGANADH G http://jaganadhg.in *ILUGCBE* http://ilugcbe.org.in
Re: Yahoo Hadoop Tutorial with new APIs?
Thank you, Would you be willing to volunteer for editing some document also along with code? If you go through the links in Jira you can have idea about work done till now and then you can join accordingly. Just email me. Thanks, Jagat Singh --- Sent from Mobile , short and crisp. On 19-Jun-2012 9:15 AM, JAGANADH G jagana...@gmail.com wrote: On Tue, Jun 19, 2012 at 8:44 AM, Jagat Singh jagatsi...@gmail.com wrote: Hi , Can you update this link in Jira which Robert created so that we can merge what ever we can into code examples. Hi Jagat Done Best regards -- ** JAGANADH G http://jaganadhg.in *ILUGCBE* http://ilugcbe.org.in
Re: set the mapred.map.tasks.speculative.execution=false, but it is not useful.
Besides Speculative execution , Tasks can be attempted multiple times due to failures. So you can see 3 attempt there. On Tue, Jun 12, 2012 at 5:08 PM, 林育智 mylinyu...@gmail.com wrote: hi all: I set the mapred.map.tasks.speculative.execution=false,but in the userlogs,you could found 3 attempt map task's log. have i miss something? expect your help. thanks.
Re: Hadoop command not found:hdfs and yarn
Hello , Can you quickly review your hadoop install with below page may be you get some hints to install. http://jugnu-life.blogspot.in/2012/05/hadoop-20-install-tutorial-023x.html The depreciated warning is correct as hadoop jobs have been divided now. Regards, Jagat Singh On Fri, Jun 8, 2012 at 2:56 PM, Prajakta Kalmegh prkal...@in.ibm.comwrote: Hi I am trying to execute the following commands for setting up Hadoop: # Format the namenode hdfs namenode -format # Start the namenode hdfs namenode # Start a datanode hdfs datanode yarn resourcemanager yarn nodemanager It gives me a Hadoop Command not found. error for all the commands. When I try to use hadoop namenode -format instead, it gives me a deprecated command warning. Can someone please tell me if I am missing including any env variables? I have included HADOOP_COMMON_HOME, HADOOP_HDFS_HOME, HADOOP_MAPRED_HOME, YARN_HOME, HADOOP_CONF_DIR, YARN_CONF_DIR, HADOOP_PREFIX in my path (apart from java etc). I am following the instructions for setting up Hadoop with Eclipse given in - http://wiki.apache.org/hadoop/HowToSetupYourDevelopmentEnvironment - http://hadoop.apache.org/common/docs/r2.0.0-alpha/hadoop-yarn/hadoop-yarn-site/SingleCluster.html Regards, Prajakta
Re: MapReduce - Libjars
Hello Karan,
Did you read this article
http://www.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/
You can have external jars in lib directory in your jar while packaging
your jar
or you can use
DistributedCachehttp://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/filecache/DistributedCache.html
.
The syntax for doing with libjars is something like this
hadoop jar myjar.jar com.driver.ClassName -libjars
/home/thirdparty/grearJar.jar
It would be available at ${mapred.local.dir}/taskTracker/archive/${user.name
}/distcache/
Read that Cloudera article it would give you idea about number of ways of
doing it.
Besides this one previous discussion can also be helpful.
http://search-hadoop.com/m/gKLlP1hMjp21
Regards,
Jagat Singh
On Thu, Jun 7, 2012 at 10:36 AM, karanveer.si...@barclays.com wrote:
Hi,
Within my map reduce programs; I am using an external JAVA library to help
parse my raw files. When I am submitting my map reduce program, I am
getting errors as the external class being referenced is not identified.
Later, I explicitly specified the external jar being referenced with the
libjars option. But I still continue to face problems as below. Any
inputs to resolve the issue will be helpful.
12/06/06 13:36:37 ERROR security.UserGroupInformation:
PriviledgedActionException as:hadoop
cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not
exist: hdfs://ldtdsrbd02.etf.barcapetf.com:8020/user/hadoop/-libjars
Exception in thread main org.apache.hadoop.mapred.InvalidInputException:
Input path does not exist: hdfs://
ldtdsrbd02.etf.barcapetf.com:8020/user/hadoop/-libjars
Regards,
Karan
This e-mail and any attachments are confidential and intended
solely for the addressee and may also be privileged or exempt from
disclosure under applicable law. If you are not the addressee, or
have received this e-mail in error, please notify the sender
immediately, delete it from your system and do not copy, disclose
or otherwise act upon any part of this e-mail or its attachments.
Internet communications are not guaranteed to be secure or
virus-free.
The Barclays Group does not accept responsibility for any loss
arising from unauthorised access to, or interference with, any
Internet communications by any third party, or from the
transmission of any viruses. Replies to this e-mail may be
monitored by the Barclays Group for operational or business
reasons.
Any opinion or other information in this e-mail or its attachments
that does not relate to the business of the Barclays Group is
personal to the sender and is not given or endorsed by the Barclays
Group.
Barclays Bank PLC. Registered in England and Wales (registered no.
1026167).
Registered Office: 1 Churchill Place, London, E14 5HP, United
Kingdom.
Barclays Bank PLC is authorised and regulated by the Financial
Services Authority.
Re: Yahoo Hadoop Tutorial with new APIs?
Hello Marcos Yes , Yahoo tutorials are pretty old but still they explain the concepts of Map Reduce , HDFS beautifully. The way in which tutorials have been defined into sub sections , each builing on previous one is awesome. I remember when i started i was digged in there for many days. The tutorials are lagging now from new API point of view. Lets have some documentation session one day , I would love to Volunteer to update those tutorials if people at Yahoo take input from outside world :) Regards, Jagat - Original Message - From: Marcos Ortiz Sent: 04/04/12 08:32 AM To: common-user@hadoop.apache.org, 'hdfs-u...@hadoop.apache.org', mapreduce-u...@hadoop.apache.org Subject: Yahoo Hadoop Tutorial with new APIs? Regards to all the list. There are many people that use the Hadoop Tutorial released by Yahoo at http://developer.yahoo.com/hadoop/tutorial/ http://developer.yahoo.com/hadoop/tutorial/module4.html#chaining The main issue here is that, this tutorial is written with the old APIs? (Hadoop 0.18 I think). Is there a project for update this tutorial to the new APIs? to Hadoop 1.0.2 or YARN (Hadoop 0.23) Best wishes -- Marcos Luis Ortíz Valmaseda (@marcosluis2186) Data Engineer at UCI http://marcosluis2186.posterous.com http://www.uci.cu/
