Data Staging using Rest API

Dear friends, Please share your exp if you have worked in Data staging using REST API for hadoop project. your help is highly appreciated. Thanks br Sridevi/

Jobtracker could only be replicated to 0 nodes instead of 1

Hello friends, I got the above error jobtarcker.info could only be replicated to 0 nodes instead of 1 Tried different Solutions found on web : * Formatted namenode * removed tmp Folder * cleaned uncessary logs just to have more space But still no success . What other Solutions could it be ?

Re: Jobtracker could only be replicated to 0 nodes instead of 1

you have set replication factor to 1, I am assuming its running a single node cluster. i would recommend you to check the datanode logs to see if it was able to connect with namenode successfully. On Fri, Aug 15, 2014 at 1:58 PM, sindhu hosamane sindh...@gmail.com wrote: Hello friends, I

Re: Test read caching

You can check the response of your command. For example, you can execute hdfs dfsadmin -report and you will get reply like following and can ensure the space of cache used and remaining is reasonable. Configured Cache Capacity: 64000 (62.50 KB) Cache Used: 4096 (4 KB) Cache Remaining: 59904

Re: hadoop/yarn and task parallelization on non-hdfs filesystems

Your FileSystem implementation should provide specific tuning parameters for IO. For example, in the GlusterFileSystem, we have a buffer parameter that is typically embedded into the core-site.xml.

RE: hadoop/yarn and task parallelization on non-hdfs filesystems

The split configurations in FIF mentioned earlier would work for local files as well. They aren't deemed unsplitable, just considered as one single block. If the FS in use has its advantages it's better to implement a proper interface to it making use of them, than to rely on the LFS by mounting

Don't want to read during namenode is in safemode

Hi Users, Usually read operation is allowed when namenode is in safemode, due to this my application is showing faulty behaviour. I don't want to do even read operation during safemode state of namenode, but for this i have to check status of namenode before reading. I couldn't find any

Re: hadoop/yarn and task parallelization on non-hdfs filesystems

Thanks for the responses! To clarify, I'm not using any special FileSystem implementation. An example input parameter to a MapReduce job would be something like -input file:///scratch/data. Thus I think (any clarification would be helpful) Hadoop is then utilizing LocalFileSystem

RE: hadoop/yarn and task parallelization on non-hdfs filesystems

Interesting to know that. I also want to know what underline logic holding the force to only generate 25-35 parallelized containers, instead of up to 1300. Another suggestion I can give is following: 1) In your driver, generate a text file, including all your 1300 bz2 file names with absolute

Re: Started learning Hadoop. Which distribution is best for native install in pseudo distributed mode?

Actually there was another thread about using MR for ML but I didn't see many responses. I use Octave or R for this but it would be useful to know how this is solved using Hadoop. The closest community that has an interest in this could be H2o but they have implemented MR for their engine to

Re: Don't want to read during namenode is in safemode

You can use org.apache.hadoop.hdfs.DistributedFileSystem#isInSafeMode() API call to check if the NameNode is in safe-mode. It returns a simple boolean result. On Fri, Aug 15, 2014 at 10:26 PM, Satyam Singh satyam.si...@ericsson.com wrote: Hi Users, Usually read operation is allowed when

Problems with the Fedarated name node configuration

Hello, I am a HDFS newbie I am using Hadoop version 2.4.1 And following instructions for cluster set-up from http://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/ClusterSetup.html and for namenode federation from