com.esotericsoftware.kryo.KryoException: Buffer overflow.

[nitinkak001]

I am running a simple rdd filter command. What does it mean? 
Here is the full stack trace(and code below it): 

com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 0,
required: 133
        at com.esotericsoftware.kryo.io.Output.require(Output.java:138)
        at
com.esotericsoftware.kryo.io.Output.writeString_slow(Output.java:420)
        at com.esotericsoftware.kryo.io.Output.writeString(Output.java:326)
        at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$StringArraySerializer.write(DefaultArraySerializers.java:274)
        at
com.esotericsoftware.kryo.serializers.DefaultArraySerializers$StringArraySerializer.write(DefaultArraySerializers.java:262)
        at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:568)
        at
org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:138)
        at
org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:197)
        at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)

*Here is the code of the main function:*

/String comparisonFieldIndexes = "16,18";
String segmentFieldIndexes = "14,15";
String comparisonFieldWeights = "50, 50";
String delimiter = ""+'\001';
                
PartitionDataOnColumn parOnCol = new PartitionDataOnColumn(70,
comparisonFieldIndexes, comparisonFieldWeights, segmentFieldIndexes,
delimiter);
                
JavaRDD<String> filtered_rdd = origRDD.filter(parOnCol.new
FilterEmptyFields(parOnCol.fieldIndexes, parOnCol.DELIMITER) );
            
parOnCol.printRDD(filtered_rdd);/


*Here is the FilterEmptyFields class:*

        /public class FilterEmptyFields implements Function<String, Boolean>
{
                
                final int[] nonEmptyFields;
                final String DELIMITER;
                
                public FilterEmptyFields(int[] nonEmptyFields, String 
delimiter){
                        this.nonEmptyFields = nonEmptyFields;
                        this.DELIMITER = delimiter;
                }
                
                @Override
                public Boolean call(String s){
                        
                        String[] fields = s.split(DELIMITER);
                        
                        for(int i=0; i<nonEmptyFields.length; i++){
                                if(fields[nonEmptyFields[i]] == null  ||
fields[nonEmptyFields[i]].isEmpty()){
                                        return false;
                                }
                        }
                        
                        return true;
                }
                
        }&lt;/i>

Any suggestions guys?



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/com-esotericsoftware-kryo-KryoException-Buffer-overflow-tp16947.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org