Capacity Scheduler APIs
Hi all, are there any capacity scheduler apis that I can use? e.g. adding, removing queues, tuning properties on the fly and so on. Any help is appreciated. Thanks Harshad
Re: Capacity Scheduler APIs
Does anybody have an answer to this question? Harshad On Wed, Mar 14, 2012 at 1:51 PM, hdev ml wrote: > Hi all, > > are there any capacity scheduler apis that I can use? > > e.g. adding, removing queues, tuning properties on the fly and so on. > > Any help is appreciated. > > Thanks > > Harshad >
Re: Capacity Scheduler APIs
Thanks for the email Shailesh. I am looking for some Java API to manage queues. I have already defined queues in the capacity-scheduler.xml and everything works fine. But my question is, can the same thing be done without restarting the cluster or namenode? The only option I see is Java API, hence the question. Please let me know. Harshad On Thu, Mar 15, 2012 at 10:33 AM, Shailesh wrote: > Hi Harshad, > have you looked into the file conf/capacity-scheduler.xml? you can assign > and change parameters like capacity of each queue, reclaim time and job > priorities. Is that what you're looking for? > > Shailesh > > On Thu, Mar 15, 2012 at 12:57 PM, hdev ml wrote: > > > Does anybody have an answer to this question? > > > > Harshad > > > > On Wed, Mar 14, 2012 at 1:51 PM, hdev ml wrote: > > > > > Hi all, > > > > > > are there any capacity scheduler apis that I can use? > > > > > > e.g. adding, removing queues, tuning properties on the fly and so on. > > > > > > Any help is appreciated. > > > > > > Thanks > > > > > > Harshad > > > > > >
Re: Capacity Scheduler APIs
Thanks Shailesh/Harsh, I will try the hadoop command first and then the internal code. Thanks again. Harshad. On Thu, Mar 15, 2012 at 12:06 PM, Harsh J wrote: > To refresh your queues, you may do, as your MR admin user: > > $ hadoop mradmin -refreshQueues > > Am not sure if this covers CS config refreshes, but let us know if it does. > The above command is present in Apache Hadoop 1.x. > > On Fri, Mar 16, 2012 at 12:08 AM, Shailesh >wrote: > > > Hi Harshad, > > > > Have you looked into CapacitySchedulerConf.java class? > > > > > http://www.java2s.com/Open-Source/Java/Database-DBMS/hadoop-0.20.1/org/apache/hadoop/mapred/CapacitySchedulerConf.java.htm > > > > I don't know whether it can be done without restarting the cluster or > > namenode. > > > > On Thu, Mar 15, 2012 at 2:03 PM, hdev ml wrote: > > > > > Thanks for the email Shailesh. > > > > > > I am looking for some Java API to manage queues. > > > > > > I have already defined queues in the capacity-scheduler.xml and > > everything > > > works fine. > > > > > > But my question is, can the same thing be done without restarting the > > > cluster or namenode? The only option I see is Java API, hence the > > question. > > > > > > Please let me know. > > > > > > Harshad > > > > > > On Thu, Mar 15, 2012 at 10:33 AM, Shailesh > > >wrote: > > > > > > > Hi Harshad, > > > > have you looked into the file conf/capacity-scheduler.xml? you can > > assign > > > > and change parameters like capacity of each queue, reclaim time and > job > > > > priorities. Is that what you're looking for? > > > > > > > > Shailesh > > > > > > > > On Thu, Mar 15, 2012 at 12:57 PM, hdev ml wrote: > > > > > > > > > Does anybody have an answer to this question? > > > > > > > > > > Harshad > > > > > > > > > > On Wed, Mar 14, 2012 at 1:51 PM, hdev ml wrote: > > > > > > > > > > > Hi all, > > > > > > > > > > > > are there any capacity scheduler apis that I can use? > > > > > > > > > > > > e.g. adding, removing queues, tuning properties on the fly and so > > on. > > > > > > > > > > > > Any help is appreciated. > > > > > > > > > > > > Thanks > > > > > > > > > > > > Harshad > > > > > > > > > > > > > > > > > > > > > > > > -- > Harsh J >
Split brain - is it possible in hadoop?
All hadoop contributors/experts, I am trying to simulate split brain in our installation. There are a few things we want to know 1. Does data corruption happen? 2. If Yes in #1, how to recover from it. 3. What are the corrective steps to take in this situation e.g. killing one namenode etc So to simulate this I took following steps. 1. We already have a healthy test cluster, consisting of 4 machines. One machine runs namenode and a datanode, other machine runs secondarynamenode and a datanode, 3rd runs jobtracker and a datanode, and 4th one just a datanode. 2. Copied the hadoop installation folder to a new location in the datanode. 3. Kept all configurations same in hdfs-site and core-site xmls, except renamed the fs.default.name to a different URI 4. The namenode directory - dfs.name.dir was pointing to the same shared NFS mounted directory to which the main namenode points to. I started this standby namenode using following command bin/hadoop-daemon.sh --config conf --hosts slaves start namenode It errored out saying that "the directory is already locked", which is an expected behaviour. The directory has been locked by the original namenode. So I changed the dfs.name.dir to some other folder, and issued the same command. It fails with message - "namenode has not been formatted", which is also expected. This makes me think - does splitbrain situation really occur in hadoop? My understanding is that split brain happens because of timeouts on the main namenode. The way it happens is, when the timeout occurs, the HA implementation - Be it Linux HA, Veritas etc., thinks that the main namenode has died and tries to start the standby namenode. The standby namenode starts up and then main namenode comes back from the timeout phase and starts functioning as if nothing happened, giving rise to 2 namenodes in the cluster - Split Brain. Considering the error messages and the above understanding, I cannot point 2 different namenodes to same directory, because the main namenode isn't responding but has locked the directory. So can I safely conclude that split brain does not occur in hadoop? Or am I missing any other situation where split brain happens and the namenode directory is not locked, thus allowing the standby namenode also to start up? Has anybody encountered this? Any help is really appreciated. Harshad
Re: Split brain - is it possible in hadoop?
Hello Michael, thanks for responding. At the bottom of the email, I have given the following scenario. This is my understanding of split brain and I am trying to simulate it, which is where I am getting problems. My understanding is that split brain happens because of timeouts on the main namenode. The way it happens is, when the timeout occurs, the HA implementation - Be it Linux HA, Veritas etc., thinks that the main namenode has died and tries to start the standby namenode. The standby namenode starts up and then main namenode comes back from the timeout phase and starts functioning as if nothing happened, giving rise to 2 namenodes in the cluster - Split Brain. On Tue, Jun 19, 2012 at 5:47 AM, Michael Segel wrote: > In your example, you only have one active Name Node. So how would you > encounter a 'split brain' scenario? > Maybe it would be better if you defined what you mean by a split brain? > > -Mike > > On Jun 18, 2012, at 8:30 PM, hdev ml wrote: > > > All hadoop contributors/experts, > > > > I am trying to simulate split brain in our installation. There are a few > > things we want to know > > > > 1. Does data corruption happen? > > 2. If Yes in #1, how to recover from it. > > 3. What are the corrective steps to take in this situation e.g. killing > one > > namenode etc > > > > So to simulate this I took following steps. > > > > 1. We already have a healthy test cluster, consisting of 4 machines. One > > machine runs namenode and a datanode, other machine runs > secondarynamenode > > and a datanode, 3rd runs jobtracker and a datanode, and 4th one just a > > datanode. > > 2. Copied the hadoop installation folder to a new location in the > datanode. > > 3. Kept all configurations same in hdfs-site and core-site xmls, except > > renamed the fs.default.name to a different URI > > 4. The namenode directory - dfs.name.dir was pointing to the same shared > > NFS mounted directory to which the main namenode points to. > > > > I started this standby namenode using following command > > bin/hadoop-daemon.sh --config conf --hosts slaves start namenode > > > > It errored out saying that "the directory is already locked", which is an > > expected behaviour. The directory has been locked by the original > namenode. > > > > So I changed the dfs.name.dir to some other folder, and issued the same > > command. It fails with message - "namenode has not been formatted", which > > is also expected. > > > > This makes me think - does splitbrain situation really occur in hadoop? > > > > My understanding is that split brain happens because of timeouts on the > > main namenode. The way it happens is, when the timeout occurs, the HA > > implementation - Be it Linux HA, Veritas etc., thinks that the main > > namenode has died and tries to start the standby namenode. The standby > > namenode starts up and then main namenode comes back from the timeout > phase > > and starts functioning as if nothing happened, giving rise to 2 namenodes > > in the cluster - Split Brain. > > > > Considering the error messages and the above understanding, I cannot > point > > 2 different namenodes to same directory, because the main namenode isn't > > responding but has locked the directory. > > > > So can I safely conclude that split brain does not occur in hadoop? > > > > Or am I missing any other situation where split brain happens and the > > namenode directory is not locked, thus allowing the standby namenode also > > to start up? > > > > Has anybody encountered this? > > > > Any help is really appreciated. > > > > Harshad > >
Hadoop automated tests
Hi all, Are there automated tests available for testing sanity of hadoop layer and also for negative tests i.e. One Data node going down, HBase Region Server going down, Namenode, Jobtracker etc. By Hadoop Layer I am asking about Hadoop, MapReduce, HBase, Zookeeper. What does hadoop dev team use for this? Any pointers, documentation articles would help a lot. Thanks Harshad
