RE: spark kryo serilizable exception

[Sameer Tilak]

Hi,I was able to set this parameter in my application to resolve this issue:
set("spark.kryoserializer.buffer.mb", "256")
Please let me know if this helps.

Date: Mon, 18 Aug 2014 21:50:02 +0800
From: dujinh...@hzduozhun.com
To: user@spark.apache.org
Subject: spark  kryo serilizable  exception


  
    
  
  
    hi all,

    In RDD map , i invoke an object that is
    
    Serialized by java standard ,  and exception ::

    

    

     

    com.esotericsoftware.kryo.KryoException: Buffer overflow. Available:
    0, required: 13

        at com.esotericsoftware.kryo.io.Output.require(Output.java:138)

        at
    com.esotericsoftware.kryo.io.Output.writeAscii_slow(Output.java:446)

        at
    com.esotericsoftware.kryo.io.Output.writeString(Output.java:306)

        at
com.esotericsoftware.kryo.serializers.DefaultSerializers$StringSerializer.write(DefaultSerializers.java:153)

        at
com.esotericsoftware.kryo.serializers.DefaultSerializers$StringSerializer.write(DefaultSerializers.java:146)

        at
    com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568)

        at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:318)

        at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$ObjectArraySerializer.write(DefaultArraySerializers.java:293)

        at
    com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568)

        at
org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:138)

        at
    org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:197)

        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

        at java.lang.Thread.run(Thread.java:745)