Re: How to debug why example not finishing (or even starting)

2015-02-01 Thread Frank Lanitz
Am 28.01.2015 um 14:39 schrieb Frank Lanitz: Hi, I've got a simple 3-node-setup where I wanted to test the grep function based on some examples. So $ hadoop fs -put /home/hadoop/hadoop/etc/hadoop hadoop-config $ hadoop jar hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar

RE: About HDFS's single-writer, multiple-reader model, any use case?

2015-02-01 Thread hadoop.support
Hello Dongzhe Ma, Yes HDFS employs Single writer, multiple reader model. This means : WRITE • HDFS client maintains a lease on files it opened for write (for entire file not for block) • Only one client can hold a lease on a single file • For each block of data, setup

Failed to run distcp against ftp server installed on Windows.

2015-02-01 Thread sam liu
Hi Experts, I could run distcp against ftp server installed on Linux, but could NOT run distcp against ftp server installed on Windows. Below are the steps. Is this a DistCp bug? Any comments? [Scenario 1] I installed a BI cluster using trunk build on HadoopNode1, and then could copy file from

RE: set default queue for user

2015-02-01 Thread Naganarasimha G R (Naga)
Hi Vikas, To a small scale user to queue mapping was supported as part of YARN-2411, Please refer the following configurations : property nameyarn.scheduler.capacity.queue-mappings/name value/value description A list of mappings that will be used to assign jobs to queues

Re: Multiple separate Hadoop clusters on same physical machines

2015-02-01 Thread daemeon reiydelle
Make virtualization an option. Federation will NOT solve your problems. *...* *“Life should not be a journey to the grave with the intention of arriving safely in apretty and well preserved body, but rather to skid in broadside in a cloud of smoke,thoroughly used up, totally worn out,

Re: Multiple separate Hadoop clusters on same physical machines

2015-02-01 Thread Alexander Alten-Lorenz
I see no possibility how federation may help to have different Clusters on _same_ machines. On top, federation isn’t production ready, since the NN can have massively issues with GC on high loaded systems, which will be the case here. To have multiple, maybe single node, clusters the best way

Re: Multiple separate Hadoop clusters on same physical machines

2015-02-01 Thread Alexander Pivovarov
start several vms and install hadoop on each vm keywords: kvm, QEMU On Mon, Jan 26, 2015 at 1:18 AM, Harun Reşit Zafer harun.za...@tubitak.gov.tr wrote: Hi everyone, We have set up and been playing with Hadoop 1.2.x and its friends (Hbase, pig, hive etc.) on 7 physical servers. We want to

About HDFS's single-writer, multiple-reader model, any use case?

2015-02-01 Thread Dongzhe Ma
We know that HDFS employs a single-writer, multiple-reader model, which means that there could be only one process writing to a file at the same time, but multiple readers can also work in parallel and new readers can even observe the new content. The reason for this design is to simplify

set default queue for user

2015-02-01 Thread Vikas Parashar
Hello folks, Could any body please let us know how can i assign a particular queue to my user. My scenario is Scheduler - Capacity scheduler Having 3 queue:- A, B and C Users - user1, user2 and user3 Now, what i am looking from above scenario. - if user1 submit any job that should