Re: journal node shared edits directory should be present on HDFS or NAS or anything else?
Hi Brahma Reddy, Thanks for the quick answer. It explains a lot but I have one more question. Maybe it is a stupid question but, required shared storage means active name node will write to its local disk? Do I need to configure or use any shared storage like NAS or SAN array or S3 storage for this purpose? Regards, Chandrash3khar Kotekar Mobile - +91 8600011455 On Thu, Feb 12, 2015 at 5:08 PM, Brahma Reddy Battula brahmareddy.batt...@huawei.com wrote: Hello Chandrashekhar, Active namenode will write to require shared storage and will not write to HDFS.. Please check following docs for reference *When Sharedstorage is Journalnode:* property namedfs.namenode.shared.edits.dir/name valueqjournal://node1.example.com:8485;node2.example.com:8485;node3.example.com:8485/mycluster/value /property http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html *When Sharedstorage is NFS:* property namedfs.namenode.shared.edits.dir/name valuefile:///mnt/filer1/dfs/ha-name-dir-shared/value /property http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html Thanks Regards Brahma Reddy Battula -- *From:* Chandrashekhar Kotekar [shekhar.kote...@gmail.com] *Sent:* Thursday, February 12, 2015 5:01 PM *To:* user@hadoop.apache.org *Subject:* journal node shared edits directory should be present on HDFS or NAS or anything else? Hi, I am trying to configure name node HA and I want to further configure automatic fail over. I am confused about '*dfs.namenode.shared.edits.dir*' configuration. Documentation says that active namde node writes to shared storage. I would like to know if this means that name nodes write it on HDFS or do they require shared storage like NAS or SAN or something else. Regards, Chandrash3khar Kotekar Mobile - +91 8600011455
RE: journal node shared edits directory should be present on HDFS or NAS or anything else?
Hello Chandrashekhar, Active namenode will write to require shared storage and will not write to HDFS.. Please check following docs for reference When Sharedstorage is Journalnode: property namedfs.namenode.shared.edits.dir/name valueqjournal://node1.example.com:8485;node2.example.com:8485;node3.example.com:8485/mycluster/value /property http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html When Sharedstorage is NFS: property namedfs.namenode.shared.edits.dir/name valuefile:///mnt/filer1/dfs/ha-name-dir-shared/value /property http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html Thanks Regards Brahma Reddy Battula From: Chandrashekhar Kotekar [shekhar.kote...@gmail.com] Sent: Thursday, February 12, 2015 5:01 PM To: user@hadoop.apache.org Subject: journal node shared edits directory should be present on HDFS or NAS or anything else? Hi, I am trying to configure name node HA and I want to further configure automatic fail over. I am confused about 'dfs.namenode.shared.edits.dir' configuration. Documentation says that active namde node writes to shared storage. I would like to know if this means that name nodes write it on HDFS or do they require shared storage like NAS or SAN or something else. Regards, Chandrash3khar Kotekar Mobile - +91 8600011455
RE: journal node shared edits directory should be present on HDFS or NAS or anything else?
Hello Chandrashekhar, Yes, you need to configure the shared storage(Active namenode writes to shared storage and Standby NN will read). Please check following mail for configuration..Shared storage can be Journalnode(which is one process and come along with hadoop package, check following reference for more details)/NFS/BOOKKeeper Thanks Regards Brahma Reddy Battula From: Chandrashekhar Kotekar [shekhar.kote...@gmail.com] Sent: Thursday, February 12, 2015 5:31 PM To: user@hadoop.apache.org Subject: Re: journal node shared edits directory should be present on HDFS or NAS or anything else? Hi Brahma Reddy, Thanks for the quick answer. It explains a lot but I have one more question. Maybe it is a stupid question but, required shared storage means active name node will write to its local disk? Do I need to configure or use any shared storage like NAS or SAN array or S3 storage for this purpose? Regards, Chandrash3khar Kotekar Mobile - +91 8600011455 On Thu, Feb 12, 2015 at 5:08 PM, Brahma Reddy Battula brahmareddy.batt...@huawei.commailto:brahmareddy.batt...@huawei.com wrote: Hello Chandrashekhar, Active namenode will write to require shared storage and will not write to HDFS.. Please check following docs for reference When Sharedstorage is Journalnode: property namedfs.namenode.shared.edits.dir/name valueqjournal://node1.example.com:8485http://node1.example.com:8485;node2.example.com:8485http://node2.example.com:8485;node3.example.com:8485/myclusterhttp://node3.example.com:8485/mycluster/value /property http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html When Sharedstorage is NFS: property namedfs.namenode.shared.edits.dir/name valuefile:///mnt/filer1/dfs/ha-name-dir-shared/value /property http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html Thanks Regards Brahma Reddy Battula From: Chandrashekhar Kotekar [shekhar.kote...@gmail.commailto:shekhar.kote...@gmail.com] Sent: Thursday, February 12, 2015 5:01 PM To: user@hadoop.apache.orgmailto:user@hadoop.apache.org Subject: journal node shared edits directory should be present on HDFS or NAS or anything else? Hi, I am trying to configure name node HA and I want to further configure automatic fail over. I am confused about 'dfs.namenode.shared.edits.dir' configuration. Documentation says that active namde node writes to shared storage. I would like to know if this means that name nodes write it on HDFS or do they require shared storage like NAS or SAN or something else. Regards, Chandrash3khar Kotekar Mobile - +91 8600011455tel:%2B91%208600011455
