Re: PARSING_ERROR from kryo

2014-09-26 Thread Arun Ahuja 
I am seeing the same error as well since upgrading to Spark1.1:

14/09/26 15:35:05 ERROR executor.Executor: Exception in task 1032.0 in
stage 5.1 (TID 22449)
com.esotericsoftware.kryo.KryoException: java.io.IOException: failed to
uncompress the chunk: PARSING_ERROR(2)
at com.esotericsoftware.kryo.io.Input.fill(Input.java:142)
at com.esotericsoftware.kryo.io.Input.require(Input.java:155)
at com.esotericsoftware.kryo.io.Input.readInt(Input.java:337)
at
com.esotericsoftware.kryo.util.DefaultClassResolver.readClass(DefaultClassResolver.java:109)
at com.esotericsoftware.kryo.Kryo.readClass(Kryo.java:610)
at com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:721)
at
org.apache.spark.serializer.KryoDeserializationStream.readObject(KryoSerializer.scala:133)
at
org.apache.spark.serializer.DeserializationStream$$anon$1.getNext(Serializer.scala:133)
at org.apache.spark.util.NextIterator.hasNext(NextIterator.scala:71)
at
org.apache.spark.storage.BlockManager$LazyProxyIterator$1.hasNext(BlockManager.scala:1082)
at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
at
org.apache.spark.util.CompletionIterator.hasNext(CompletionIterator.scala:30)
at
org.apache.spark.InterruptibleIterator.hasNext(InterruptibleIterator.scala:39)

Out of 6000 tasks 5000 something finish fine, so I don't believe there are
any issues with the serialization and some other datasets everything works
fine. Also the same code, same dataset worked fine with Spark 1.0.2

On Mon, Sep 15, 2014 at 9:57 PM, npanj nitinp...@gmail.com wrote:

 Hi Andrew,

 No I could not figure out the root cause. This seems to be
 non-deterministic
 error... I didn't see same error after rerunning same program. But I
 noticed
 same error on a different program.

 First I thought that this may be related to SPARK-2878, but @Graham replied
 that this looks irrelevant.






 --
 View this message in context:
 http://apache-spark-developers-list.1001551.n3.nabble.com/PARSING-ERROR-from-kryo-tp7944p8433.html
 Sent from the Apache Spark Developers List mailing list archive at
 Nabble.com.

 -
 To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
 For additional commands, e-mail: dev-h...@spark.apache.org

Re: PARSING_ERROR from kryo

2014-09-15 Thread Ankur Dave 
At 2014-09-15 08:59:48 -0700, Andrew Ash and...@andrewash.com wrote:
 I'm seeing the same exception now on the Spark 1.1.0 release.  Did you ever
 get this figured out?

 [...]

 On Thu, Aug 21, 2014 at 2:14 PM, npanj nitinp...@gmail.com wrote:
 I am getting PARSING_ERROR while running my job on the code checked out up
 to commit# db56f2df1b8027171da1b8d2571d1f2ef1e103b6.

The error is because I merged a GraphX PR that introduced a nondeterministic 
bug [1]. I reverted the faulty PR, but it was too late for the 1.1.0 release. 
The problem should go away if you use branch-1.1 or master. Sorry about that...

Ankur

[1] https://issues.apache.org/jira/browse/SPARK-3400

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org

Re: PARSING_ERROR from kryo

2014-09-15 Thread npanj 
Hi Andrew,

No I could not figure out the root cause. This seems to be non-deterministic
error... I didn't see same error after rerunning same program. But I noticed
same error on a different program. 

First I thought that this may be related to SPARK-2878, but @Graham replied
that this looks irrelevant.






--
View this message in context: 
http://apache-spark-developers-list.1001551.n3.nabble.com/PARSING-ERROR-from-kryo-tp7944p8433.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org
For additional commands, e-mail: dev-h...@spark.apache.org