Re: snapshotexport to outside of cluster
What if there is a bad sector on the disk of remote server ? Using hdfs, you're shielded from such scenario. On Dec 1, 2013, at 6:56 AM, oc tsdb oc.t...@gmail.com wrote: We want to use that exported data in case of any disaster happens. Thanks -OC On Sun, Dec 1, 2013 at 2:25 PM, Ted Yu yuzhih...@gmail.com wrote: Since the remote server doesn't have hdfs, what would the backup be used for ? Cheers On Nov 30, 2013, at 10:34 PM, oc tsdb oc.t...@gmail.com wrote: Hi, Can you clarify what 'without copying to local file system' means ? We want to take HBase tables backup.So we periodically take snapshots and export data to backup server which is not in cluster. Currently we are achieving this as mentioned below. 1.Create snapshots from HBase command line lets say we have created snapshot for a table called 'hbase_tabel' and its snapshot name is 'hbase_tabel_snapshot' 2. Then export snapshot data to remote backup server which is not in cluster.We are achieving this in two steps. a. Export snapshot to local file system(sample path /tmp/hbase_backup) from hdfs file system as mentioned below hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hbase_tabel_snapshot -copy-to file:///tmp/hbase_backup - mappers 16 Here we should have enough disc space in local file system to export snapshot from hdfs. b.Transfer(scp) exported snapshot from local file system (/tmp/hbase_backup) to backup server which is not hadoop/HBase running server. That means first we have to export snapshot to local system from hdfs and then transfer ( scp ) to remote server.Instead of doing in two steps can we directly export snapshot to remote server(which is not in cluster) like how it is exporting to another cluster. . Thanks OC On Sat, Nov 30, 2013 at 11:11 AM, Ted Yu yuzhih...@gmail.com wrote: HBASE-4618 is not related to snapshots. Lars did some work in HBASE-5509, a sub-task of HBASE-4618. FYI On Fri, Nov 29, 2013 at 9:31 PM, Yu Li car...@gmail.com wrote: I guess this might be a use case of HBASE-7912https://issues.apache.org/jira/browse/HBASE-7912or HBASE-4618 https://issues.apache.org/jira/browse/HBASE-4618? Maybe some backup/restore tools taking use of snapshot would be nice to have? On 30 November 2013 12:58, Ted Yu yuzhih...@gmail.com wrote: The remote server is in different cluster, right ? Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies data to outputPath which is in the other cluster. Can you clarify what 'without copying to local file system' means ? Cheers On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb oc.t...@gmail.com wrote: Hi, thanks for your replay. We are able to export snapshot to another cluster But this is not what we are looking for. We want to export snapshot to remote server without copying to local file system. Thanks OC On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu yuzhih...@gmail.com wrote: Here is a sample command exporting to another cluster: hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6 On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb oc.t...@gmail.com wrote: Hi, can someone please let us know if we can export snapshots data directly to outside of cluster without copying to local. Please refer us if there are any blog with example. Below command is copying data to local file system under /tmp. hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot mytable -copy-to file:///tmp/hbase_backup -mappers 16 Without copying to local system, we want to export snapshot(eg scp) data to a server which is not part of any cluster but its reachable from cluster. Thanks OC -- Best Regards, Li Yu
Re: snapshotexport to outside of cluster
Hi, Can you clarify what 'without copying to local file system' means ? We want to take HBase tables backup.So we periodically take snapshots and export data to backup server which is not in cluster. Currently we are achieving this as mentioned below. 1.Create snapshots from HBase command line lets say we have created snapshot for a table called 'hbase_tabel' and its snapshot name is 'hbase_tabel_snapshot' 2. Then export snapshot data to remote backup server which is not in cluster.We are achieving this in two steps. a. Export snapshot to local file system(sample path /tmp/hbase_backup) from hdfs file system as mentioned below hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot hbase_tabel_snapshot -copy-to file:///tmp/hbase_backup - mappers 16 Here we should have enough disc space in local file system to export snapshot from hdfs. b.Transfer(scp) exported snapshot from local file system (/tmp/hbase_backup) to backup server which is not hadoop/HBase running server. That means first we have to export snapshot to local system from hdfs and then transfer ( scp ) to remote server.Instead of doing in two steps can we directly export snapshot to remote server(which is not in cluster) like how it is exporting to another cluster. . Thanks OC On Sat, Nov 30, 2013 at 11:11 AM, Ted Yu yuzhih...@gmail.com wrote: HBASE-4618 is not related to snapshots. Lars did some work in HBASE-5509, a sub-task of HBASE-4618. FYI On Fri, Nov 29, 2013 at 9:31 PM, Yu Li car...@gmail.com wrote: I guess this might be a use case of HBASE-7912https://issues.apache.org/jira/browse/HBASE-7912or HBASE-4618 https://issues.apache.org/jira/browse/HBASE-4618? Maybe some backup/restore tools taking use of snapshot would be nice to have? On 30 November 2013 12:58, Ted Yu yuzhih...@gmail.com wrote: The remote server is in different cluster, right ? Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies data to outputPath which is in the other cluster. Can you clarify what 'without copying to local file system' means ? Cheers On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb oc.t...@gmail.com wrote: Hi, thanks for your replay. We are able to export snapshot to another cluster But this is not what we are looking for. We want to export snapshot to remote server without copying to local file system. Thanks OC On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu yuzhih...@gmail.com wrote: Here is a sample command exporting to another cluster: hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6 On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb oc.t...@gmail.com wrote: Hi, can someone please let us know if we can export snapshots data directly to outside of cluster without copying to local. Please refer us if there are any blog with example. Below command is copying data to local file system under /tmp. hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot mytable -copy-to file:///tmp/hbase_backup -mappers 16 Without copying to local system, we want to export snapshot(eg scp) data to a server which is not part of any cluster but its reachable from cluster. Thanks OC -- Best Regards, Li Yu
snapshotexport to outside of cluster
Hi, can someone please let us know if we can export snapshots data directly to outside of cluster without copying to local. Please refer us if there are any blog with example. Below command is copying data to local file system under /tmp. hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot mytable -copy-to file:///tmp/hbase_backup -mappers 16 Without copying to local system, we want to export snapshot(eg scp) data to a server which is not part of any cluster but its reachable from cluster. Thanks OC
Re: snapshotexport to outside of cluster
Here is a sample command exporting to another cluster: hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6 On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb oc.t...@gmail.com wrote: Hi, can someone please let us know if we can export snapshots data directly to outside of cluster without copying to local. Please refer us if there are any blog with example. Below command is copying data to local file system under /tmp. hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot mytable -copy-to file:///tmp/hbase_backup -mappers 16 Without copying to local system, we want to export snapshot(eg scp) data to a server which is not part of any cluster but its reachable from cluster. Thanks OC
Re: snapshotexport to outside of cluster
Hi, thanks for your replay. We are able to export snapshot to another cluster But this is not what we are looking for. We want to export snapshot to remote server without copying to local file system. Thanks OC On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu yuzhih...@gmail.com wrote: Here is a sample command exporting to another cluster: hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6 On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb oc.t...@gmail.com wrote: Hi, can someone please let us know if we can export snapshots data directly to outside of cluster without copying to local. Please refer us if there are any blog with example. Below command is copying data to local file system under /tmp. hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot mytable -copy-to file:///tmp/hbase_backup -mappers 16 Without copying to local system, we want to export snapshot(eg scp) data to a server which is not part of any cluster but its reachable from cluster. Thanks OC
Re: snapshotexport to outside of cluster
The remote server is in different cluster, right ? Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies data to outputPath which is in the other cluster. Can you clarify what 'without copying to local file system' means ? Cheers On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb oc.t...@gmail.com wrote: Hi, thanks for your replay. We are able to export snapshot to another cluster But this is not what we are looking for. We want to export snapshot to remote server without copying to local file system. Thanks OC On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu yuzhih...@gmail.com wrote: Here is a sample command exporting to another cluster: hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6 On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb oc.t...@gmail.com wrote: Hi, can someone please let us know if we can export snapshots data directly to outside of cluster without copying to local. Please refer us if there are any blog with example. Below command is copying data to local file system under /tmp. hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot mytable -copy-to file:///tmp/hbase_backup -mappers 16 Without copying to local system, we want to export snapshot(eg scp) data to a server which is not part of any cluster but its reachable from cluster. Thanks OC
Re: snapshotexport to outside of cluster
I guess this might be a use case of HBASE-7912https://issues.apache.org/jira/browse/HBASE-7912or HBASE-4618 https://issues.apache.org/jira/browse/HBASE-4618? Maybe some backup/restore tools taking use of snapshot would be nice to have? On 30 November 2013 12:58, Ted Yu yuzhih...@gmail.com wrote: The remote server is in different cluster, right ? Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies data to outputPath which is in the other cluster. Can you clarify what 'without copying to local file system' means ? Cheers On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb oc.t...@gmail.com wrote: Hi, thanks for your replay. We are able to export snapshot to another cluster But this is not what we are looking for. We want to export snapshot to remote server without copying to local file system. Thanks OC On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu yuzhih...@gmail.com wrote: Here is a sample command exporting to another cluster: hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6 On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb oc.t...@gmail.com wrote: Hi, can someone please let us know if we can export snapshots data directly to outside of cluster without copying to local. Please refer us if there are any blog with example. Below command is copying data to local file system under /tmp. hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot mytable -copy-to file:///tmp/hbase_backup -mappers 16 Without copying to local system, we want to export snapshot(eg scp) data to a server which is not part of any cluster but its reachable from cluster. Thanks OC -- Best Regards, Li Yu
Re: snapshotexport to outside of cluster
HBASE-4618 is not related to snapshots. Lars did some work in HBASE-5509, a sub-task of HBASE-4618. FYI On Fri, Nov 29, 2013 at 9:31 PM, Yu Li car...@gmail.com wrote: I guess this might be a use case of HBASE-7912https://issues.apache.org/jira/browse/HBASE-7912or HBASE-4618 https://issues.apache.org/jira/browse/HBASE-4618? Maybe some backup/restore tools taking use of snapshot would be nice to have? On 30 November 2013 12:58, Ted Yu yuzhih...@gmail.com wrote: The remote server is in different cluster, right ? Looking at ExportSnapshot#ExportMapper#copyFile(), it directly copies data to outputPath which is in the other cluster. Can you clarify what 'without copying to local file system' means ? Cheers On Fri, Nov 29, 2013 at 8:49 PM, oc tsdb oc.t...@gmail.com wrote: Hi, thanks for your replay. We are able to export snapshot to another cluster But this is not what we are looking for. We want to export snapshot to remote server without copying to local file system. Thanks OC On Fri, Nov 29, 2013 at 8:52 PM, Ted Yu yuzhih...@gmail.com wrote: Here is a sample command exporting to another cluster: hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot gs_1383068263 -copy-to hdfs://experiment-1d:8020/hbase -mappers 6 On Fri, Nov 29, 2013 at 4:02 AM, oc tsdb oc.t...@gmail.com wrote: Hi, can someone please let us know if we can export snapshots data directly to outside of cluster without copying to local. Please refer us if there are any blog with example. Below command is copying data to local file system under /tmp. hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot mytable -copy-to file:///tmp/hbase_backup -mappers 16 Without copying to local system, we want to export snapshot(eg scp) data to a server which is not part of any cluster but its reachable from cluster. Thanks OC -- Best Regards, Li Yu
