Hi Akmal, Why do not you use Cluster Replication ?
[1] http://hbase.apache.org/book.html#_cluster_replication On Mar 12, 2015 11:40 PM, "Vladimir Rodionov" <[email protected]> wrote: > Thanks, Jerry. I think webdfs is preferable as since it is natively > supported by hdfs (name node and data nodes) and traffic does not pass > single gateway? > > Found this link how to set up webdfs over ssl: > > http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.7/bk_Security_Guide/content/ch_wire-webhdfs-mr-yarn.html > > Cool. If works :). > > -Vlad > > > On Thu, Mar 12, 2015 at 2:24 PM, Jerry He <[email protected]> wrote: > > > Hi, Vladimir > > > > Hope I understand your question correctly. > > If both local cluster and remote cluster are Kerberos enabled, > > ExportSnapshot from local to remote will work as long as both > > clusters' Kerberos > > have been set up in a way that they understand each other. > > If the remote cluster's httpfs/webhdfs port is protected by https > security, > > after you set up the certificate on the client side, you will be able to > > talk to the remote port with SSL protection. > > > > Jerry > > > > > > On Thu, Mar 12, 2015 at 1:48 PM, Vladimir Rodionov < > [email protected] > > > > > wrote: > > > > > >>You can also specify the remote target with a httpfs or webfdfs url, > > > which > > > >>then you can leverage SSL on the transport. > > > > > > What if remote cluster has security enabled? Will it work? > > > > > > -Vlad > > > > > > On Thu, Mar 12, 2015 at 1:39 PM, Jerry He <[email protected]> wrote: > > > > > > > ExportSnapshot does not use DistCp but directly use FileSystem API to > > > copy, > > > > as Vladimir mentioned. > > > > But ExportSnapshot supports exporting to a remote target cluster. > Give > > > the > > > > full hdfs url. > > > > You can also specify the remote target with a httpfs or webfdfs url, > > > which > > > > then you can leverage SSL on the transport. > > > > > > > > You also can copy to local cluster and use DistCp to copy to remote > > > > cluster. > > > > > > > > Jerry > > > > > > > > On Thu, Mar 12, 2015 at 12:28 PM, Vladimir Rodionov < > > > > [email protected]> > > > > wrote: > > > > > > > > > No, ExportSnapshot does not use DistCp it runs its own M/R job to > > copy > > > > data > > > > > over to a new destination. > > > > > > > > > > In a map task it uses HDFS API to create/write data to a new > > > destination. > > > > > Therefore, the easiest way to secure communication > > > > > during this operation is to use secure HDFS transport. > > > > > > > > > > > > > > > > > > > > http://www.cloudera.com/content/cloudera/en/documentation/cdh4/v4-3-1/CDH4-Security-Guide/cdh4sg_topic_14_2.html > > > > > > > > > > but there is caveat ... > > > > > > > > > > ExportSnapshot does not support external cluster configuration - > you > > > > can't > > > > > provide path to external cluster config dir. This seems like a good > > > > feature > > > > > request. > > > > > > > > > > -Vlad > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Mar 12, 2015 at 10:38 AM, Akmal Abbasov < > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > Hi, I am new to Hadoop Hbase. I have a Hbase cluster in one > > > datacenter, > > > > > > and I need to create a backup in the second one. Currently the > > > second > > > > > > HBase cluster is ready, and I would like to import data from > first > > > > > cluster. > > > > > > I would like to use exportSnapshot tool for this, I’ve tried it > one > > > my > > > > > > test environment, and it worked well. > > > > > > But, since know I am going to export to a different cluster in > > > > different > > > > > > datacenter, I would like to be sure that my data is secure. So > how > > I > > > > can > > > > > > make exportSnapshot secure? > > > > > > As far as I understood exportSnapshot uses distcp tool to copy > > > snapshot > > > > > to > > > > > > destination cluster, so in this case is it enough to configure > > > distcp? > > > > > > Thank you! > > > > > > > > > > > > > > >
