Sanjay Kumar created HBASE-15225: ------------------------------------ Summary: Running connecting to HBase via newAPIHadoopRDD gives org.apache.hadoop.hbase.client.RetriesExhaustedException Key: HBASE-15225 URL: https://issues.apache.org/jira/browse/HBASE-15225 Project: HBase Issue Type: Bug Environment: spark 1.6.0 , Hbase 0.98, kerberos, hbase.rpc.protection set to authentication. Reporter: Sanjay Kumar Assignee: Sanjay Kumar Priority: Minor
Unable to read HBase table into Spark with hbase security authentication set to kerberos. Seeing the following error. : org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=31, exceptions: Thu Feb 04 22:01:55 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:01:56 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:01:56 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:01:57 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:01:59 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:02:03 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:02:13 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:02:23 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:02:34 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:02:44 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:03:04 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:03:24 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:03:44 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:04:04 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:04:24 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:04:44 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException Thu Feb 04 22:05:04 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.IOException: Connection reset by peer . . . Thu Feb 04 22:09:46 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@395327da, java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:129) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90) at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:282) at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:187) at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:182) at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:109) at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:738) at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:178) at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:82) at org.apache.hadoop.hbase.client.MetaScanner.allTableRegions(MetaScanner.java:282) at org.apache.hadoop.hbase.client.HTable.getRegionLocations(HTable.java:616) at org.apache.hadoop.hbase.util.RegionSizeCalculator.<init>(RegionSizeCalculator.java:79) at org.apache.hadoop.hbase.util.RegionSizeCalculator.<init>(RegionSizeCalculator.java:64) at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:160) at org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:115) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237) at scala.Option.getOrElse(Option.scala:120) at org.apache.spark.rdd.RDD.partitions(RDD.scala:237) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237) at scala.Option.getOrElse(Option.scala:120) at org.apache.spark.rdd.RDD.partitions(RDD.scala:237) at org.apache.spark.rdd.RDD$$anonfun$take$1.apply(RDD.scala:1277) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:147) at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108) at org.apache.spark.rdd.RDD.withScope(RDD.scala:306) at org.apache.spark.rdd.RDD.take(RDD.scala:1272) at org.apache.spark.api.python.SerDeUtil$.pairRDDToPython(SerDeUtil.scala:202) at org.apache.spark.api.python.PythonRDD$.newAPIHadoopRDD(PythonRDD.scala:530) at org.apache.spark.api.python.PythonRDD.newAPIHadoopRDD(PythonRDD.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) at py4j.Gateway.invoke(Gateway.java:259) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:207) at java.lang.Thread.run(Thread.java:744) Caused by: java.io.IOException: Call to d-767tfz1.target.com/10.66.241.13:60020 failed on local exception: java.io.EOFException at org.apache.hadoop.hbase.ipc.RpcClient.wrapException(RpcClient.java:1484) at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1456) at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1656) at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1714) at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:29900) at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:308) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:164) at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:59) at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114) ... 42 more Caused by: java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.java:392) at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1071) at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727) -- This message was sent by Atlassian JIRA (v6.3.4#6332)