Have you tried adding path to hbase-protocol jar to spark.driver.extraClassPath and spark.executor.extraClassPath ?
Cheers On Mon, Aug 17, 2015 at 7:51 PM, stark_summer <stark_sum...@qq.com> wrote: > spark vesion:1.4.1 > java version:1.7 > hadoop version: > Hadoop 2.3.0-cdh5.1.0 > > submit spark job to yarn cluster that read hbase data,after job running, it > comes below error : > > 15/08/17 19:28:33 ERROR yarn.ApplicationMaster: User class threw exception: > org.apache.hadoop.hbase.DoNotRetryIOException: > java.lang.IllegalAccessError: > class com.google.protobuf.HBaseZeroCopyByteString ca > nnot access its superclass com.google.protobuf.LiteralByteString > org.apache.hadoop.hbase.DoNotRetryIOException: > java.lang.IllegalAccessError: > class com.google.protobuf.HBaseZeroCopyByteString cannot access its > superclass com.google.protobuf.LiteralByteString > at > > org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:210) > at > > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:121) > at > > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90) > at > > org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:264) > at > > org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:169) > at > org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:164) > at > org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:107) > at > org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:736) > at > org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:178) > at > org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:82) > at > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isTableAvailable(HConnectionManager.java:962) > at > > org.apache.hadoop.hbase.client.HBaseAdmin.isTableAvailable(HBaseAdmin.java:1081) > at > > org.apache.hadoop.hbase.client.HBaseAdmin.isTableAvailable(HBaseAdmin.java:1089) > at com.umeng.dp.yuliang.play.HBaseToES$.main(HBaseToES.scala:28) > at com.umeng.dp.yuliang.play.HBaseToES.main(HBaseToES.scala) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > > org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:483) > Caused by: java.lang.IllegalAccessError: class > com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass > com.google.protobuf.LiteralByteString > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:800) > at > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at > > org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:930) > at > > org.apache.hadoop.hbase.protobuf.RequestConverter.buildScanRequest(RequestConverter.java:434) > at > > org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:297) > at > > org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:157) > at > > org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:57) > at > > org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114) > ... 18 more > > PS: > running hadoop mr on yarn that read hbase data, also have this error, > https://issues.apache.org/jira/browse/HBASE-10304,that is hbase issues , > > when submit hadoop mr, add export > > HADOOP_CLASSPATH="./hbase/hbase-protocol/target/hbase-protocol-0.99.0-SNAPSHOT.jar" > to shell comand > or add export > > HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar > to linux /etc/basrc file,it can work well, > > but submit spark job ,it can not work > > > > > > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/java-lang-IllegalAccessError-class-com-google-protobuf-HBaseZeroCopyByteString-cannot-access-its-supg-tp24303.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
