14/09/15 23:34:59 DEBUG snapshot.SnapshotManifestV1: Adding reference for file (4/4): hftp:// master42.stg.com:50070/hbase/.hbase-snapshot/msg_snapshot/84f60fc2aa7e96df91e6289e6c19dc25/c/afe341e4149649578c5861e32494dbec
14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed java.io.IOException at org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) at org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) at org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) at org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitReferencedFiles(SnapshotReferenceUtil.java:108) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.getSnapshotFiles(ExportSnapshot.java:479) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.access$200(ExportSnapshot.java:89) at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportSnapshotInputFormat.getSplits(ExportSnapshot.java:600) at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1107) at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1124) at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:178) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1023) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:976) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1554) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:976) at org.apache.hadoop.mapreduce.Job.submit(Job.java:582) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:612) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.runCopyJob(ExportSnapshot.java:751) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.run(ExportSnapshot.java:905) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.innerMain(ExportSnapshot.java:975) at org.apache.hadoop.hbase.snapshot.ExportSnapshot.main(ExportSnapshot.java:979) Caused by: java.io.IOException: read=-1, wanted=4 at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) at org.apache.hadoop.hbase.regionserver.StoreFileInfo.<init>(StoreFileInfo.java:115) at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java:204) at org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.buildManifestFromDisk(SnapshotManifestV1.java:179) at org.apache.hadoop.hbase.snapshot.SnapshotManifestV1$1.call(SnapshotManifestV1.java:131) at org.apache.hadoop.hbase.snapshot.SnapshotManifestV1$1.call(SnapshotManifestV1.java:127) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) On Mon, Sep 15, 2014 at 10:18 PM, Matteo Bertozzi <theo.berto...@gmail.com> wrote: > can you post the full exception and the file path ? > maybe there is a bug in looking up the reference file. > It seems to not be able to find enough data in the file... > > Matteo > > > On Mon, Sep 15, 2014 at 10:08 PM, Gautam <gautamkows...@gmail.com> wrote: > > > Thanks for the reply Matteo. > > > > This is exactly what I did. I modified the source cluster's dir structure > > to mimic that of the 98 cluster. I even got as far as it trying to look > > through the reference files. > > > > I end up with this exception : > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export failed > > java.io.IOException > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > > at > > > > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > > ... > > .. > > Caused by: java.io.IOException: read=-1, wanted=4 > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > > at > > > > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.<init>(StoreFileInfo.java:115) > > at > > > > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > > .. > > > > > > This and Ted's reply about HBASE-7987 leads me to believe that the export > > tool from my distro is in capable of working around the regionManifest > file > > requirement. I'm now left with the option of downgrading my dest cluster > to > > 94, copying data and then upgrading using the upgrade migration tool. > > Wanted to know if others have tried this or there are other things I can > > do. If not, i'l just go ahead and do this :-) > > > > > > Cheers, > > -Gautam. > > > > > > > > On Mon, Sep 15, 2014 at 8:10 PM, Matteo Bertozzi < > theo.berto...@gmail.com> > > wrote: > > > > > 94 and 98 differs in directory layout > > > so 98 is not able to read 94 layout unless you run the migration tool > > > which is basically moving all the data in a "default" namespace > directory > > > e.g. > > > /hbase/table -> /hbase/data/default/table > > > /hbase/.archive/table -> /hbase/archive/default/table > > > > > > Matteo > > > > > > > > > On Mon, Sep 15, 2014 at 6:17 PM, Gautam <gautamkows...@gmail.com> > wrote: > > > > > > > Yep, looks like the CDH distro backports HBASE-7987. Having said > that, > > is > > > > there a transition path for us or are we hosed :-) ? In general, > what's > > > the > > > > recommended way to achieve this, at this point I feel i'm going > around > > > the > > > > system to achieve what I want. If nothing else works with export > > snapshot > > > > should I just downgrade to 94, export snapshot and then upgrade to > 98? > > Is > > > > the upgrade migration path different from what export snapshot does > > (i'd > > > > imagine yes)? > > > > > > > > Cheers, > > > > -Gautam. > > > > > > > > > > > > > > > > > > > > On Mon, Sep 15, 2014 at 5:14 PM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > > > > > bq. 98.1 on dest cluster > > > > > > > > > > Looking at the history for SnapshotManifestV1, it came with > > HBASE-7987 > > > > > which went to 0.99.0 > > > > > > > > > > Perhaps you're using a distro with HBASE-7987 ? > > > > > > > > > > On Mon, Sep 15, 2014 at 4:58 PM, Gautam <gautamkows...@gmail.com> > > > wrote: > > > > > > > > > > > Hello, > > > > > > I'm trying to copy data between Hbase clusters on > > different > > > > > > versions. I am using : > > > > > > > > > > > > /usr/bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot > > > > > > -chuser hbase > > > > > > -chgroup hadoop > > > > > > -snapshot msg_snapshot > > > > > > -mappers 50 > > > > > > -copy-from hftp://src-cluster:50070/hbase > > > > > > -copy-to hdfs:/dest-cluster:8020/hbase > > > > > > > > > > > > > > > > > > Till now, based on various tips from the mailing list, I have > > > modified > > > > > the > > > > > > source cluster data dir paths to mimic the 98 convention > (archive, > > > > table > > > > > > data paths, etc). This helped in jumping some roadblocks but not > > all. > > > > > > > > > > > > This is what I see now : > > > > > > > > > > > > > > > > > > 14/09/15 23:34:59 ERROR snapshot.ExportSnapshot: Snapshot export > > > failed > > > > > > java.io.IOException > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifestV1.loadRegionManifests(SnapshotManifestV1.java:145) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:265) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.snapshot.SnapshotReferenceUtil.visitTableStoreFiles(SnapshotReferenceUtil.java:125) > > > > > > ... > > > > > > .. > > > > > > Caused by: java.io.IOException: read=-1, wanted=4 > > > > > > at org.apache.hadoop.hbase.io.Reference.read(Reference.java:175) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.StoreFileInfo.<init>(StoreFileInfo.java:115) > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.hadoop.hbase.regionserver.HRegionFileSystem.getStoreFiles(HRegionFileSystem.java > > > > > > .. > > > > > > > > > > > > > > > > > > Fails while trying to read refernce hfile. Is this something > folks > > > have > > > > > > done before and/or is possible to do? I'd really like to do this > > > > without > > > > > > having to upgrade my source cluster or downgrade my dest cluster. > > > > > > > > > > > > I'm using 94.6 on source cluster and 98.1 on dest cluster. > > > > > > > > > > > > > > > > > > Cheers, > > > > > > -Gautam. > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > "If you really want something in this life, you have to work for it. > > Now, > > > > quiet! They're about to announce the lottery numbers..." > > > > > > > > > > > > > > > -- > > "If you really want something in this life, you have to work for it. Now, > > quiet! They're about to announce the lottery numbers..." > > > -- "If you really want something in this life, you have to work for it. Now, quiet! They're about to announce the lottery numbers..."
