Hi Talat, I was considering replication, but decided to start with snapshots. Moreover, there are some drawbacks with replication, like propagation of user error, etc. Also I need a secure connection between data-centers, and I can't find information about this.
> On 13 Mar 2015, at 05:45, Talat Uyarer <ta...@uyarer.com> wrote: > > Hi Akmal, > > Why do not you use Cluster Replication ? > > [1] http://hbase.apache.org/book.html#_cluster_replication > On Mar 12, 2015 11:40 PM, "Vladimir Rodionov" <vladrodio...@gmail.com> > wrote: > >> Thanks, Jerry. I think webdfs is preferable as since it is natively >> supported by hdfs (name node and data nodes) and traffic does not pass >> single gateway? >> >> Found this link how to set up webdfs over ssl: >> >> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.7/bk_Security_Guide/content/ch_wire-webhdfs-mr-yarn.html >> >> Cool. If works :). >> >> -Vlad >> >> >>> On Thu, Mar 12, 2015 at 2:24 PM, Jerry He <jerry...@gmail.com> wrote: >>> >>> Hi, Vladimir >>> >>> Hope I understand your question correctly. >>> If both local cluster and remote cluster are Kerberos enabled, >>> ExportSnapshot from local to remote will work as long as both >>> clusters' Kerberos >>> have been set up in a way that they understand each other. >>> If the remote cluster's httpfs/webhdfs port is protected by https >> security, >>> after you set up the certificate on the client side, you will be able to >>> talk to the remote port with SSL protection. >>> >>> Jerry >>> >>> >>> On Thu, Mar 12, 2015 at 1:48 PM, Vladimir Rodionov < >> vladrodio...@gmail.com >>> wrote: >>> >>>>>> You can also specify the remote target with a httpfs or webfdfs url, >>>> which >>>>>> then you can leverage SSL on the transport. >>>> >>>> What if remote cluster has security enabled? Will it work? >>>> >>>> -Vlad >>>> >>>>> On Thu, Mar 12, 2015 at 1:39 PM, Jerry He <jerry...@gmail.com> wrote: >>>>> >>>>> ExportSnapshot does not use DistCp but directly use FileSystem API to >>>> copy, >>>>> as Vladimir mentioned. >>>>> But ExportSnapshot supports exporting to a remote target cluster. >> Give >>>> the >>>>> full hdfs url. >>>>> You can also specify the remote target with a httpfs or webfdfs url, >>>> which >>>>> then you can leverage SSL on the transport. >>>>> >>>>> You also can copy to local cluster and use DistCp to copy to remote >>>>> cluster. >>>>> >>>>> Jerry >>>>> >>>>> On Thu, Mar 12, 2015 at 12:28 PM, Vladimir Rodionov < >>>>> vladrodio...@gmail.com> >>>>> wrote: >>>>> >>>>>> No, ExportSnapshot does not use DistCp it runs its own M/R job to >>> copy >>>>> data >>>>>> over to a new destination. >>>>>> >>>>>> In a map task it uses HDFS API to create/write data to a new >>>> destination. >>>>>> Therefore, the easiest way to secure communication >>>>>> during this operation is to use secure HDFS transport. >> http://www.cloudera.com/content/cloudera/en/documentation/cdh4/v4-3-1/CDH4-Security-Guide/cdh4sg_topic_14_2.html >>>>>> >>>>>> but there is caveat ... >>>>>> >>>>>> ExportSnapshot does not support external cluster configuration - >> you >>>>> can't >>>>>> provide path to external cluster config dir. This seems like a good >>>>> feature >>>>>> request. >>>>>> >>>>>> -Vlad >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Mar 12, 2015 at 10:38 AM, Akmal Abbasov < >>>>> akmal.abba...@icloud.com> >>>>>> wrote: >>>>>> >>>>>>> Hi, I am new to Hadoop Hbase. I have a Hbase cluster in one >>>> datacenter, >>>>>>> and I need to create a backup in the second one. Currently the >>>> second >>>>>>> HBase cluster is ready, and I would like to import data from >> first >>>>>> cluster. >>>>>>> I would like to use exportSnapshot tool for this, I’ve tried it >> one >>>> my >>>>>>> test environment, and it worked well. >>>>>>> But, since know I am going to export to a different cluster in >>>>> different >>>>>>> datacenter, I would like to be sure that my data is secure. So >> how >>> I >>>>> can >>>>>>> make exportSnapshot secure? >>>>>>> As far as I understood exportSnapshot uses distcp tool to copy >>>> snapshot >>>>>> to >>>>>>> destination cluster, so in this case is it enough to configure >>>> distcp? >>>>>>> Thank you! >>
