[ https://issues.apache.org/jira/browse/HBASE-16567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15469159#comment-15469159 ]
stack commented on HBASE-16567: ------------------------------- Just a note to say that upgrading our pb to pb3 w/o running protoc3, we do pretty good but fail reading trailers on hfiles. We can't find CellComparator. TODO: check why. {code} 2016-09-06 18:06:42,856 ERROR [RS_OPEN_META-localhost:55129-0] handler.OpenRegionHandler: Failed open of region=hbase:meta,,1.1588230740, starting to roll back the global memstore size. java.io.IOException: java.io.IOException: org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile Trailer from file file:/var/folders/cj/jgfy62h13vz019xgz681df_r0000gp/T/hbase-stack/hbase/data/hbase/meta/1588230740/info/47a6d1f4adb24f6e92de8dbe9e9c144f at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionStores(HRegion.java:831) at org.apache.hadoop.hbase.regionserver.HRegion.initializeRegionInternals(HRegion.java:732) at org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:705) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4750) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4721) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4693) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4649) at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4600) at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:276) at org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:103) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.io.IOException: org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile Trailer from file file:/var/folders/cj/jgfy62h13vz019xgz681df_r0000gp/T/hbase-stack/hbase/data/hbase/meta/1588230740/info/47a6d1f4adb24f6e92de8dbe9e9c144f at org.apache.hadoop.hbase.regionserver.HStore.openStoreFiles(HStore.java:544) at org.apache.hadoop.hbase.regionserver.HStore.loadStoreFiles(HStore.java:499) at org.apache.hadoop.hbase.regionserver.HStore.<init>(HStore.java:267) at org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:3652) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:805) at org.apache.hadoop.hbase.regionserver.HRegion$1.call(HRegion.java:802) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) ... 3 more Caused by: org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile Trailer from file file:/var/folders/cj/jgfy62h13vz019xgz681df_r0000gp/T/hbase-stack/hbase/data/hbase/meta/1588230740/info/47a6d1f4adb24f6e92de8dbe9e9c144f at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:477) at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:505) at org.apache.hadoop.hbase.regionserver.StoreFile$Reader.<init>(StoreFile.java:1033) at org.apache.hadoop.hbase.regionserver.StoreFileInfo.open(StoreFileInfo.java:241) at org.apache.hadoop.hbase.regionserver.StoreFile.open(StoreFile.java:365) at org.apache.hadoop.hbase.regionserver.StoreFile.createReader(StoreFile.java:462) at org.apache.hadoop.hbase.regionserver.HStore.createStoreFileAndReader(HStore.java:629) at org.apache.hadoop.hbase.regionserver.HStore.access$000(HStore.java:123) at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:519) at org.apache.hadoop.hbase.regionserver.HStore$1.call(HStore.java:516) ... 6 more Caused by: java.io.IOException: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.CellComparator$MetaCellComparator at org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.getComparatorClass(FixedFileTrailer.java:581) at org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.deserializeFromPB(FixedFileTrailer.java:300) at org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.deserialize(FixedFileTrailer.java:242) at org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.readFromStream(FixedFileTrailer.java:407) at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:462) ... 15 more Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.CellComparator$MetaCellComparator at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.hfile.FixedFileTrailer.getComparatorClass(FixedFileTrailer.java:579) ... 19 more {code} > Upgrade to protobuf3 > -------------------- > > Key: HBASE-16567 > URL: https://issues.apache.org/jira/browse/HBASE-16567 > Project: HBase > Issue Type: Task > Components: Protobufs > Affects Versions: 2.0.0 > Reporter: stack > Assignee: stack > Priority: Critical > Attachments: HBASE-16567.master.001.patch > > > Move master branch on to protobuf3. See > https://github.com/google/protobuf/releases We'd do it because pb3 saves some > on byte copies can work with offheap buffers -- needed for the off-heap write > path project -- thought read-time is still a TODO. > HBASE-15638 has us first shading protobufs before upgrading. Let us list here > issues just going to pb3 without shading if only for completeness sake; i.e. > do we have to shade? > * pb3 is by default wire compatible with pb2. > * protoc3 run against our .protos works fine except pb3 breaks our > HBaseZeroCopyLiteralByteString hack. > * Starting up a cluster that is all pb3'd seems to work fine. > * A pb2 branch-1 can read and write against the pb3 master cluster. > What will break if we just upgrade to pb3? > * We should be able to write HDFS messages on our AsyncWAL using pb3; the > pb2 HDFS should be able to read them (not tested). Or maybe not. See policy > here: https://github.com/google/protobuf/issues/1852 > * Core Coprocessor Endpoints such as AccessControl seem to just work (their > protos will have been protoc3'd). I did simple test with a server from master > branch up on pb3 and then going against it with a branch-1 client on pb2. I > was able to add grants. > * For non-core CPEPs where the protos are pb2 still, it might just work. To > test. It would not be the end-of-the-world if they did not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)