WholeFileInputFormat in hadoop

2014-06-28 Thread unmesha sreeveni
Hi A small clarification: WholeFileInputFormat takes the entire input file as input or each record(input split) as whole? -- *Thanks Regards * *Unmesha Sreeveni U.B* *Hadoop, Bigdata Developer* *Center for Cyber Security | Amrita Vishwa Vidyapeetham*

Re: Partitioning and setup errors

2014-06-28 Thread Chris MacKenzie
HI Chris, I¹m away from my books for the weekend. Is that call (extends Configurable implements Tool) a Hadoop 2 call. Would that mean that I am better sticking with Hadoop 1.x ? Warm regards. Chris http://www.chrismackenziephotography.co.uk/ Expert in all aspects of photography telephone: 0131

Re: WholeFileInputFormat in hadoop

2014-06-28 Thread Shahab Yunus
I think it takes the entire file as input. Otherwise it won't be any different from the normal line/record-based input format. Regards, Shahab On Jun 28, 2014 3:28 AM, unmesha sreeveni unmeshab...@gmail.com wrote: Hi A small clarification: WholeFileInputFormat takes the entire input file

Re: WholeFileInputFormat in hadoop

2014-06-28 Thread Mohit Anchlia
It takes entire file as input. There is a method in the class isSplittable in this input format class which is set to false. This method determines if file can be split in multiple chunks. On Sat, Jun 28, 2014 at 5:38 AM, Shahab Yunus shahab.yu...@gmail.com wrote: I think it takes the entire

Re: Partitioning and setup errors

2014-06-28 Thread Vinod Kumar Vavilapalli
What is happening is the client is not able to pick up the right jar to push to the cluster. It looks in the class-path for the jar that contains the class ParallelGeneticAlignment. How are you packaging your code? How are your running your job - paste the command line? +Vinod On Jun 27,

Hadoop 2.2.0 : Job not showing on resource management web UI

2014-06-28 Thread Shing Hing Man
Hi, I have set up a single node Hadoop 2.2.0 . I can submit wordcount hadoop job to YARN from the command line. It run successfully and I checked the result is in hdfs. matmsh@gauss:~ hadoop jar ~/installed/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount input

Re: Hadoop 2.2.0 : Job not showing on resource management web UI

2014-06-28 Thread Zhijie Shen
You were still running the local MR job, which is why you didn't see the job on RM web UI: property valuemapreduce.framework.name/value nameyarn/name /property This config is not in bad format. Please change to *name*- mapreduce.framework.name and *value*-yarn. On Sat, Jun 28, 2014 at 8:54 PM,