I can debug this next week when I have access to an HA cluster. Which release of Hadoop are you using ?
Cheers > On Apr 14, 2017, at 9:21 AM, Vasco Pinho <va...@hotjar.com> wrote: > > I'm not sure I follow. This file already exists under the install dir at: > > /home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/hbase-prefix-tree-1.2.4.jar > > The class is also present at: > > /home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/classes/org/apache/hadoop/hbase/codec/prefixtree/PrefixTreeCodec.class > > Why would I rename it? Is my classpath wrong for some reason? In any case, > renaming it to something like > "hbase-prefix-tree/target/hbase-prefix-tree-1.2.4.jar.bak " yields another > exception of the same sort (full trace: https://pastebin.com/mzcPthZR): > > java.io.FileNotFoundException: File does not exist: > hdfs://clusterID/root/.m2/repository/org/apache/htrace/htrace-core/3.1.0-incubating/htrace-core-3.1.0-incubating.jar > > So it looks like something else is wrong with the installation or there is > a bug that is making these searches attempt to look in "hdfs://" instead of > the filesystem? > > Thanks, > Vasco Pinho > >> On Fri, Apr 14, 2017 at 4:32 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> >> Here is how hbase-prefix-tree dependency is detected: >> >> try { >> >> prefixTreeCodecClass = >> >> Class.forName( >> "org.apache.hadoop.hbase.codec.prefixtree.PrefixTreeCodec"); >> >> } catch (ClassNotFoundException e) { >> >> // this will show up in unit tests but should not show in real >> deployments >> >> LOG.warn("The hbase-prefix-tree module jar containing PrefixTreeCodec >> is not present." + >> >> " Continuing without it."); >> >> } >> >> As a workaround, consider temporarily renaming hbase-prefix-tree jar on the >> node where ExportSnapshot is run. >> >> Remember to rename hbase-prefix-tree jar back after the job. >> >> FYI >> >>> On Fri, Apr 14, 2017 at 5:02 AM, Vasco Pinho <va...@hotjar.com> wrote: >>> >>> Sure Ted, >>> >>> Here's the full trace: https://pastebin.com/AGimZ8wZ >>> And here's the hbase-site.xml: https://pastebin.com/yhRWCsaU >>> >>> This is a working Hbase HA setup which has been happily chugging along. 3 >>> node HDFS JN, 3 node zk, 2 of them NN+backup NN. Also two of them >>> HMaster+HMaster backup. Then several data nodes. >>> >>> Thanks for taking a look! >>> >>> Vasco >>> >>>> On Fri, Apr 14, 2017 at 1:41 PM, Ted Yu <yuzhih...@gmail.com> wrote: >>>> >>>> Can you show the complete stack trace ? >>>> >>>> Please pastebin contents of hbase/site.xml >>>> >>>> Thanks >>>> >>>>> On Apr 14, 2017, at 3:51 AM, Vasco Pinho <va...@hotjar.com> wrote: >>>>> >>>>> When running: >>>>> >>>>> bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot >>>>> -Dfs.s3a.buffer.dir=/tmp/hbase_snap_tmp -snapshot >>>> TestTable-20170413-143020 >>>>> -copy-to s3a://bucket-backup/hbase/snapshots/ >>>>> >>>>> >>>>> The operation fails with: >>>>> >>>>> >>>>> 2017-04-13 15:03:24,947 ERROR [main] snapshot.ExportSnapshot: >> Snapshot >>>>> export failed >>>>> java.io.FileNotFoundException: File does not exist: >>>>> hdfs://clusterID/home/ubuntu/hbase-1.2.4/hbase-prefix-tree/ >>>> target/hbase-prefix-tree-1.2.4.jar >>>>> at >>>>> org.apache.hadoop.hdfs.DistributedFileSystem$17. >>>> doCall(DistributedFileSystem.java:1072) >>>>> >>>>> >>>>> I assume it's looking for the >>>>> "/home/ubuntu/hbase-1.2.4/hbase-prefix-tree/target/ >>>> hbase-prefix-tree-1.2.4.jar" >>>>> which does exist but obviously not under hdfs://. I don't know why >> this >>>>> file is needed or why it's being looked for in the hdfs:// instead of >>> the >>>>> local filesystem, where it does exist. I confirmed that it looks >> under >>>>> "fs.defaultFS" + <hbase install path> + <relative path to >>>> hbase-prefix-tree >>>>> jar", although I have no idea how to fix this. Any ideas? >>>>> >>>>> >>>>> Thanks, >>>>> Vasco Pinho >>
