Question about namenode HA

2014-12-05 Thread Alexandru Pacurar
Hello, I'm trying to configure HA for the HDFS namenode with QJM following the instructions form here http://hadoop.apache.org/docs/r2.2.0/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailabilityWithQJM.html. My setup is the following : Ubuntu 12.04.5 LTS on all the nodes, Hadoop 2.4.1 installed,

Scheduler questions

2014-12-05 Thread Oleg Sukhoroslov
Hi, We have a Hadoop cluster which is used by many users (students) competing for resources. We would like to prioritize jobs based on resource usage history per user. Such functionality is common among classic HPC schedulers, e.g.

Re: Query about Speculative Execution

2014-12-05 Thread Srinivas Chamarthi
Hi, may I know where should I find the sources related to speculative scheduling happens ? and also how do we discard the output coming from already completed mapper ? I am actually trying to do something similar, like spawn map tasks redundantly but not for speculative reasons but for each

Unsubscribe me from the list

2014-12-05 Thread Sri kripa
Unsubscribe me from the list

Unsubscribe me from the list

2014-12-05 Thread Dani, Vaibhav
Unsubscribe me from the list Sincerely Dani

Re: Unsubscribe me from the list

2014-12-05 Thread Alec Ten Harmsel
Please email user-unsubscr...@hadoop.apache.org to unsubscribe. Regards, Alec On 12/05/2014 07:15 AM, Dani, Vaibhav wrote: Unsubscribe me from the list Sincerely / /*/ Dani/*

to all this unsubscribe sender

2014-12-05 Thread Aleks Laz
Dear wished unsubscriber Is it really that hard to start brain vX.X? You or anyone which have used your email account, have subscribed to this list, as described here. http://hadoop.apache.org/mailing_lists.html You or this Person must unsubscribe at the same way as the subscription was,

Re: to all this unsubscribe sender

2014-12-05 Thread Chana
THANK YOU for posting this Very rude indeed to not learn *for oneself* - as opposed to expecting to be spoon fed information like a small child. On Fri, Dec 5, 2014 at 7:00 AM, Aleks Laz al-userhad...@none.at wrote: Dear wished unsubscriber Is it really that hard to start brain vX.X?

Re: to all this unsubscribe sender

2014-12-05 Thread mark charts
Not rude at all. We were not born with any knowledge. We learn as we go. We learn by asking. Capiche? Mark Charts On Friday, December 5, 2014 8:13 AM, Chana chana.ole...@gmail.com wrote: THANK YOU for posting this Very rude indeed to not learn for oneself - as opposed to

Re: Unsubscribe me from the list

2014-12-05 Thread Jagannath Naidu
Dear List, Just of curios, need to ask one thing. With in last two weeks, many users are sending mails to unsubscribe from list. So why so many ?? Is sky falling down?? On 5 December 2014 at 17:49, Alec Ten Harmsel a...@alectenharmsel.com wrote: Please email

When schedulers consider x% of resources what do they mean?

2014-12-05 Thread Chris Mawata
Hi all, when you divide up resources e.g. on CapacityScheduler or FairScheduler etc., what does x% or resources mean? So, for example, a guranteed 70% meant to indicate you can have up to 70% of the containers clusterwide irrespective of size of container 70% of the containers on each node

Re: to all this unsubscribe sender

2014-12-05 Thread Chana
Actually, it's capisce. And, obviously, we are all here because we are learning. But, now that instructions have been posted again - I would expect that people would not expect to have their hands held to do something so simple as unsubscribe. They are supposedly technologists. I would think they

Re: Unsubscribe me from the list

2014-12-05 Thread mark charts
My take on this item is that folks are too preoccupied with existence to be bothered with a mail list that does not suit their specific needs. Or maybe too impatient for knowledge. No sky is not falling-not yet anyway. On Friday, December 5, 2014 8:39 AM, Jagannath Naidu

Re: to all this unsubscribe sender

2014-12-05 Thread mark charts
Trained as an electrical engineer I learned again and again from my instructors: man is inherently lazy. Maybe that's the case. Also, some people have complicated lives and need hand holding. It is best to accept people as they are. On Friday, December 5, 2014 8:40 AM, Chana

Re: Unsubscribe me from the list

2014-12-05 Thread Jagannath Naidu
*Hmmm...* *too many impatient or bothered all at once.* *Isn't that weird * *Or may be I am missing some important news, in which may be you can help.* On 5 December 2014 at 19:15, mark charts mcha...@yahoo.com wrote: My take on this item is that folks are too preoccupied with

Re: Unsubscribe me from the list

2014-12-05 Thread Anil Jagtap
Possibly all have got their PHD degrees in hand on Hadoop and they don’t need their group for knowledge sharing.. I think the best way to keep all the mates busy is asking the experts to share their experiences, use cases.. help the new-comers… So far i have seen many emails from Indians… why

Re: to all this unsubscribe sender

2014-12-05 Thread Niels Basjes
Yes, I agree. We should accept people as they are. So perhaps we should increase the hurdle to subscribe in the first place? Something like adding a question like What do you do if you want to unsubscribe from a mailing list? That way the people who are lazy or need hand holding are simply unable

Re: to all this unsubscribe sender

2014-12-05 Thread Amjad Syed
My take on this is simple. The owner/maintainer / administrator of the list can implement a filter where if the subject of the email has word unsubscribe . That email gets blocked and sender gets automated reply stating that if you want to unsubscribe use the unsubscribe list. On 5 Dec 2014

Unsubscribe me from the list

2014-12-05 Thread Joginder Sethi
UNSUBSCRIBE From: Sri kripa [mailto:srikripa2...@gmail.com] Sent: Friday, December 05, 2014 5:38 AM To: user@hadoop.apache.org Subject: Unsubscribe me from the list Unsubscribe me from the list

Re: to all this unsubscribe sender

2014-12-05 Thread mark charts
I concur. Good idea. On Friday, December 5, 2014 10:10 AM, Amjad Syed amjad...@gmail.com wrote: My take on this is simple. The owner/maintainer / administrator of the list can implement a filter where if the subject of the email has word unsubscribe . That email gets blocked and

Re: to all this unsubscribe sender

2014-12-05 Thread Aleks Laz
+1 Am 05-12-2014 16:05, schrieb Niels Basjes: Yes, I agree. We should accept people as they are. So perhaps we should increase the hurdle to subscribe in the first place? Something like adding a question like What do you do if you want to unsubscribe from a mailing list? That way

Re: to all this unsubscribe sender

2014-12-05 Thread Aleks Laz
+1 Am 05-12-2014 16:12, schrieb mark charts: I concur. Good idea. On Friday, December 5, 2014 10:10 AM, Amjad Syed amjad...@gmail.com wrote: My take on this is simple. The owner/maintainer / administrator of the list can implement a filter where if the subject of the email has

Fwd: No FileSystem for scheme: hdfs

2014-12-05 Thread Vineet Mishra
+user@hadoop.apache.org -- Forwarded message -- From: Vineet Mishra clearmido...@gmail.com Date: Fri, Dec 5, 2014 at 8:51 PM Subject: No FileSystem for scheme: hdfs To: cdh-u...@cloudera.org Hi All, I am trying to invoke hadoop FileUtil.copyMerge from a piece of java code but

Re: to all this unsubscribe sender

2014-12-05 Thread Ted Yu
+1 On Fri, Dec 5, 2014 at 7:22 AM, Aleks Laz al-userhad...@none.at wrote: +1 Am 05-12-2014 16:12, schrieb mark charts: I concur. Good idea. On Friday, December 5, 2014 10:10 AM, Amjad Syed amjad...@gmail.com wrote: My take on this is simple. The owner/maintainer / administrator

Re: to all this unsubscribe sender

2014-12-05 Thread Niels Basjes
+1 On Fri, Dec 5, 2014 at 4:28 PM, Ted Yu yuzhih...@gmail.com wrote: +1 On Fri, Dec 5, 2014 at 7:22 AM, Aleks Laz al-userhad...@none.at wrote: +1 Am 05-12-2014 16:12, schrieb mark charts: I concur. Good idea. On Friday, December 5, 2014 10:10 AM, Amjad Syed amjad...@gmail.com

Re: to all this unsubscribe sender

2014-12-05 Thread Rich Haase
+1 Sent from my iPhone On Dec 5, 2014, at 08:29, Ted Yu yuzhih...@gmail.commailto:yuzhih...@gmail.com wrote: +1 On Fri, Dec 5, 2014 at 7:22 AM, Aleks Laz al-userhad...@none.atmailto:al-userhad...@none.at wrote: +1 Am 05-12-2014 16:12, schrieb mark charts: I concur. Good idea. On Friday,

Re: to all this unsubscribe sender

2014-12-05 Thread Jagannath Naidu
Open source On 05-Dec-2014 8:59 PM, Ted Yu yuzhih...@gmail.com wrote: +1 On Fri, Dec 5, 2014 at 7:22 AM, Aleks Laz al-userhad...@none.at wrote: +1 Am 05-12-2014 16:12, schrieb mark charts: I concur. Good idea. On Friday, December 5, 2014 10:10 AM, Amjad Syed amjad...@gmail.com

Re: When schedulers consider x% of resources what do they mean?

2014-12-05 Thread Vinod Kumar Vavilapalli
Resources can mean memory-only (by default) or memory + CPU etc across the _entire_ cluster. So 70% of cluster resources for a queue means that 70% of the total memory set for Hadoop in the cluster are available for all applications in that queue. Heap sizes are part of the memory

Re: When schedulers consider x% of resources what do they mean?

2014-12-05 Thread Chris Mawata
Thanks -- so this would be enforced at the ResourceManager level and not the NodeManager level. On Fri, Dec 5, 2014 at 2:22 PM, Vinod Kumar Vavilapalli vino...@apache.org wrote: Resources can mean memory-only (by default) or memory + CPU etc across the _entire_ cluster. So 70% of cluster