Re: Bulk loading Serialized RDD into Hbase throws KryoException - IndexOutOfBoundsException

Put is a type of Mutation so not sure what you mean by if I use mutation. Anyway I registered all 3 classes to kryo. kryo.register(classOf[org.apache.hadoop.hbase.client.Put]) kryo.register(classOf[ImmutableBytesWritable]) kryo.register(classOf[Mutable]) It still fails with the same

Re: Bulk loading Serialized RDD into Hbase throws KryoException - IndexOutOfBoundsException

org.apache.hadoop.hbase.client.{Mutation, Put} org.apache.hadoop.hbase.io.ImmutableBytesWritable if u used mutation, register the above class too > On May 30, 2016, at 08:11, Nirav Patel wrote: > > Sure let me can try that. But from looks of it it seems kryo >

Re: Bulk loading Serialized RDD into Hbase throws KryoException - IndexOutOfBoundsException

Sure let me can try that. But from looks of it it seems kryo kryo. util.MapReferenceResolver.getReadObject trying to access incorrect index (100) On Sun, May 29, 2016 at 5:06 PM, Ted Yu wrote: > Can you register Put with Kryo ? > > Thanks > > On May 29, 2016, at 4:58 PM,

Re: Bulk loading Serialized RDD into Hbase throws KryoException - IndexOutOfBoundsException

Can you register Put with Kryo ? Thanks > On May 29, 2016, at 4:58 PM, Nirav Patel wrote: > > I pasted code snipped for that method. > > here's full def: > > def writeRddToHBase2(hbaseRdd: RDD[(ImmutableBytesWritable, Put)], > tableName: String) { > > > >

Re: Bulk loading Serialized RDD into Hbase throws KryoException - IndexOutOfBoundsException

I pasted code snipped for that method. here's full def: def writeRddToHBase2(hbaseRdd: RDD[(ImmutableBytesWritable, Put)], tableName: String) { hbaseRdd.values.foreachPartition{ itr => val hConf = HBaseConfiguration.create() hConf.setInt("hbase.client.write.buffer",

Re: Bulk loading Serialized RDD into Hbase throws KryoException - IndexOutOfBoundsException

bq. at com.mycorpt.myprojjobs.spark.jobs.hbase.HbaseUtils$$ anonfun$writeRddToHBase2$1.apply(HbaseUtils.scala:80) Can you reveal related code from HbaseUtils.scala ? Which hbase version are you using ? Thanks On Sun, May 29, 2016 at 4:26 PM, Nirav Patel wrote: > Hi, >

Bulk loading Serialized RDD into Hbase throws KryoException - IndexOutOfBoundsException

Hi, I am getting following Kryo deserialization error when trying to buklload Cached RDD into Hbase. It works if I don't cache the RDD. I cache it with MEMORY_ONLY_SER. here's the code snippet: hbaseRdd.values.foreachPartition{ itr => val hConf = HBaseConfiguration.create()