All JNs are deployed on the same node with DN.
On Mon, Feb 18, 2013 at 3:35 PM, 谢良 <xieli...@xiaomi.com> wrote: > Hi Azuryy, just want to confirm one thing, your JN did not deploy on the > same machines within DN, right ? > > Regards, > Liang > ------------------------------ > *发件人:* Azuryy Yu [azury...@gmail.com] > *发送时间:* 2013年2月18日 15:22 > *收件人:* user@hadoop.apache.org > *主题:* Re: some ideas for QJM and NFS > > Hi, > > I did it on hadoop-2.0.3-alpha without HA as following: > > [root@webdm test]# date +%Y-%m-%d_%H:%M:%S; hdfs dfs -put > testspeed.tar.gz / ; date +%Y-%m-%d_%H:%M:%S > 2013-02-18_15:20:01 > 13/02/18 15:20:02 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 2013-02-18_15:20:30 > > so the performance is a little bit better than hadoop-1.0.4. > > > > On Mon, Feb 18, 2013 at 2:53 PM, Azuryy Yu <azury...@gmail.com> wrote: > >> Oh, yes, you are right, George. I'll probably do it in the next days. >> >> >> On Mon, Feb 18, 2013 at 2:47 PM, George Datskos < >> george.dats...@jp.fujitsu.com> wrote: >> >>> Hi Azuryy, >>> >>> So you have measurements for hadoop-1.0.4 and hadoop-2.0.3+QJM, but I >>> think you should also measure hadoop-2.0.3 _wihout_ QJM so you can know for >>> sure if the performance degrade is actually related to QJM or not. >>> >>> >>> George >>> >>> >>> Hi, >>> >>> HarshJ is a good guy, I've seen this JIRA: >>> https://issues.apache.org/jira/browse/HDFS-4508 >>> >>> I have a test cluster hadoop-1.0.4, I've upgrade to >>> hadoop-2.0.3-alpha. mu cluster is very small, four nodes totally. >>> >>> then I did some test on the original Hadoop and new Hadoop, the >>> testing is very simple: I have a data file with 450MB, I just put it on the >>> HDFS. >>> >>> block size: 128MB, replica: 2 >>> >>> the following is the result: >>> >>> [root@webdm test]# ll testspeed.tar.gz >>> -rw-r--r-- 1 root root 452M Feb 18 13:54 testspeed.tar.gz >>> [root@webdm test]# >>> >>> //On the hadoop-1.0.4 >>> [root@webdm test]# date +%Y-%m-%d_%H:%M:%S; hadoop dfs -put >>> testspeed.tar.gz / ; date +%Y-%m-%d_%H:%M:%S >>> 2013-02-18_13:54:24 >>> Warning: $HADOOP_HOME is deprecated. >>> 2013-02-18_13:54:58 >>> >>> //On the hadoop-2.0.3-alpha with QJM >>> [root@webdm test]# date +%Y-%m-%d_%H:%M:%S; hdfs dfs -put >>> testspeed.tar.gz / ; date +%Y-%m-%d_%H:%M:%S >>> 2013-02-18_14:13:29 >>> 13/02/18 14:13:30 WARN util.NativeCodeLoader: Unable to load >>> native-hadoop library for your platform... using builtin-java classes where >>> applicable >>> 2013-02-18_14:14:33 >>> >>> I do think QJM HA feature affect the performance, because each writer >>> from QJM, it will do: fence old writer; sync in-progress log; start new log >>> segment; then write. only if writer received a successful response from a >>> quorum of JNs, writer finished for this time. >>> >>> But NFS HA just write log segment in the local and NFS, when it >>> receive successful response from NFS, it finished this time. >>> >>> So, I just suggest we always keep these two HA features in future, >>> even in the stable release. which one should be used, which depends on >>> yourself based on your infrastructure. >>> >>> Thanks. >>> >>> >>> >> >
