Thank you sooo much guys! I used s3n and that's fixed it. I will try s3a too. Thanks for much for that thread, Ted.
On 5 May 2016 at 14:07, Ted Yu <yuzhih...@gmail.com> wrote: > Lex: > Please also see this thread about s3n versus s3a: > > http://search-hadoop.com/m/uOzYtE1Fy22eEWfe1&subj=Re+S3+Hadoop+FileSystems > > On Wed, May 4, 2016 at 9:01 PM, Matteo Bertozzi <theo.berto...@gmail.com> > wrote: > > > never seen that problem before, but a couple of suggestions you can try. > > > > Instead of the old s3 driver, you can use s3n or s3a if you have it > > available (those are the ones I tested) > > and instead of using hbase.root dir use -copy-from > > > > ExportSnapshot -snapshot SNAPSHOT_NAME -copy-to s3a://BUCKET/NAMESPACE > > ExportSnapshot -snapshot SNAPSHOT_NAME -copy-from s3a://BUCKET/NAMESPACE > > -copy-to hdfs://HOST/hbase > > > > you can take a look at some in-progress doc about s3 and snapshot here: > > https://issues.apache.org/jira/browse/HBASE-15646 > > > > Matteo > > > > > > On Wed, May 4, 2016 at 8:53 PM, Lex Toumbourou <l...@scrunch.com> wrote: > > > > > Hi all, > > > > > > I'm having a couple of problems with exporting HBase snapshots to S3. I > > am > > > running HBase version 1.2.0. > > > > > > I have a table called "domain" > > > > > > And I have created a snapshot for it: > > > > > > hbase(main):003:0> snapshot 'domain', 'domain-aws-test' > > > 0 row(s) in 0.3310 seconds > > > > > > --- > > > > > > I am attempting to export it to S3 using the following command: > > > > > > hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot > > > "domain-aws-test" -copy-to s3://my-hbase-snapshots/domain-aws-test > > > > > > Now, when I view the snapshot metadata in the S3 bucket, there's > nothing > > > there: > > > > > > > aws s3 ls my-hbase-snapshots/domain-aws-snapshots > > > > > > But there is data under: > > > > > > > aws s3 ls my-hbase-snapshots/\/domain-aws-test/ > > > PRE .hbase-snapshot/ > > > 2016-05-05 13:38:12 1 .hbase-snapshot > > > > > > It seems what's happening is, HBase is creating a directory/prefix with > > no > > > name and placing the snapshot data under there. > > > > > > That wouldn't be a problem, except that when I try to import the > snapshot > > > on my destination cluster, it seems unable to deal with the empty > > > directory: > > > > > > sudo -u hbase hbase snapshot export -D > > > hbase.rootdir=s3://my-hbase-snapshots/\/domain-aws-test -snapshot > > > my-aws-test -copy-to hdfs://hbaseClusterDNSName:8020/user/hbase > -mappers > > 2 > > > > > > Caused by: java.io.FileNotFoundException: No such file or directory > > > > 's3://my-hbase-snapshots/domain-aws-test/.hbase-snapshot/domain-aws-test' > > > > > > --- > > > > > > Any one come across this before? > > > > > > Lex > > > > > > -- Lex ToumbourouLead engineer at scrunch.com <http://scrunch.com/>
