Lex: Please also see this thread about s3n versus s3a: http://search-hadoop.com/m/uOzYtE1Fy22eEWfe1&subj=Re+S3+Hadoop+FileSystems
On Wed, May 4, 2016 at 9:01 PM, Matteo Bertozzi <theo.berto...@gmail.com> wrote: > never seen that problem before, but a couple of suggestions you can try. > > Instead of the old s3 driver, you can use s3n or s3a if you have it > available (those are the ones I tested) > and instead of using hbase.root dir use -copy-from > > ExportSnapshot -snapshot SNAPSHOT_NAME -copy-to s3a://BUCKET/NAMESPACE > ExportSnapshot -snapshot SNAPSHOT_NAME -copy-from s3a://BUCKET/NAMESPACE > -copy-to hdfs://HOST/hbase > > you can take a look at some in-progress doc about s3 and snapshot here: > https://issues.apache.org/jira/browse/HBASE-15646 > > Matteo > > > On Wed, May 4, 2016 at 8:53 PM, Lex Toumbourou <l...@scrunch.com> wrote: > > > Hi all, > > > > I'm having a couple of problems with exporting HBase snapshots to S3. I > am > > running HBase version 1.2.0. > > > > I have a table called "domain" > > > > And I have created a snapshot for it: > > > > hbase(main):003:0> snapshot 'domain', 'domain-aws-test' > > 0 row(s) in 0.3310 seconds > > > > --- > > > > I am attempting to export it to S3 using the following command: > > > > hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot > > "domain-aws-test" -copy-to s3://my-hbase-snapshots/domain-aws-test > > > > Now, when I view the snapshot metadata in the S3 bucket, there's nothing > > there: > > > > > aws s3 ls my-hbase-snapshots/domain-aws-snapshots > > > > But there is data under: > > > > > aws s3 ls my-hbase-snapshots/\/domain-aws-test/ > > PRE .hbase-snapshot/ > > 2016-05-05 13:38:12 1 .hbase-snapshot > > > > It seems what's happening is, HBase is creating a directory/prefix with > no > > name and placing the snapshot data under there. > > > > That wouldn't be a problem, except that when I try to import the snapshot > > on my destination cluster, it seems unable to deal with the empty > > directory: > > > > sudo -u hbase hbase snapshot export -D > > hbase.rootdir=s3://my-hbase-snapshots/\/domain-aws-test -snapshot > > my-aws-test -copy-to hdfs://hbaseClusterDNSName:8020/user/hbase -mappers > 2 > > > > Caused by: java.io.FileNotFoundException: No such file or directory > > 's3://my-hbase-snapshots/domain-aws-test/.hbase-snapshot/domain-aws-test' > > > > --- > > > > Any one come across this before? > > > > Lex > > >
