RE: how to add a shareable node label?

2016-10-06 Thread Frank Luo
Sunil, You confirmed my understanding. I got the understanding by reading the docs and haven’t really tried 2.8 or 3.0-alphal. My situation is that I am in a multi-tenant env, and got several very powerful machines with expensive licenses to run a particular linux job, let’s say Job_A. But th

Re: how to add a shareable node label?

2016-10-06 Thread Sunil Govind
HI Frank Ideally those containers will be preempted if there are unsatisfied demand for "configured label". I could explain this: "labelA" has few empty resources. All nodes under "default" label is used. Hence a new application which is submitted to "default" label has to wait. But if "labelA"

RE: how to add a shareable node label?

2016-10-06 Thread Frank Luo
Thanks Sunil. Ø 3. If there is any future ask for those resources , we will preempt the non labeled apps and give them back to labeled apps. Unfortunately, I am still not able to use it, because of the preemptive behavior. The jobs that steals labelled resources are not preemptable, and I’d

Hadoop: precomputing data

2016-10-06 Thread venito camelas
I'm designing a prototype using *Hadoop* for video processing to do face recognition. I thought of 2 ways of doing it. *Approach 1:* I was thinking of doing something in 2 steps: 1. A map that receives frames and if a face is found it gets stored for the next step. 2. A map that receive

S3AFileSystem & read-after-write consistency

2016-10-06 Thread Dave Maughan
Hi, I'm investigating S3's read-after-write consistency model with S3AFileSystem and something is not quite clear to me, so I'm hoping someone more knowledgeable can clarify it for me. Amazon state ( http://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html): *"Amazon S3 provides read

Block Replace Operation in Mover/Balancer

2016-10-06 Thread Senthil Kumar
Hi All , Recently we have upgraded our Cluster to Hadoop 2.7.1 version. When i try to run Mover , it throws below exception : 2016-10-06 00:24:44,643 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: opCopyBlock BP-1267816582-10.114.118.11-1392844883031:blk_12046985669_160995799 recei

Newbie Ambari Question

2016-10-06 Thread Deepak Goel
Hey Namaskara~Nalama~Guten Tag~Bonjour Sorry, Is this the right forum for asking a question "Ambari Hadoop Installation" from Hortonworks? Thanks Deepak -- Keigu Deepak 73500 12833 www.simtree.net, dee...@simtree.net deic...@gmail.com LinkedIn: www.linkedin.com/in/deicool Skype: thumsupdeic