[
https://issues.apache.org/jira/browse/SPARK-7043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14602604#comment-14602604
]
Sean Owen commented on SPARK-7043:
----------------------------------
[~freiss] that's a great analysis. We tried changing {{createSparkContext()}}
to not make a new context if one has been configured already, which seems
reasonable anyway, but didn't change the outcome either. Still looking at it as
this has come up in another case as well.
> KryoSerializer cannot be used with REPL to interpret code in which case class
> definition and its shipping are in the same line
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-7043
> URL: https://issues.apache.org/jira/browse/SPARK-7043
> Project: Spark
> Issue Type: Bug
> Components: Spark Shell
> Affects Versions: 1.3.1
> Environment: Ubuntu 14.04, no hadoop
> Reporter: Peng Cheng
> Priority: Minor
> Labels: classloader, kryo
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> When deploying Spark-shell with
> "spark.serializer=org.apache.spark.serializer.KryoSerializer" option.
> Spark-shell cannot execute the following code (in 1 line):
> case class Foo(i: Int);val ret = sc.parallelize((1 to 100).map(Foo),
> 10).collect()
> This problem won't exist for either JavaSerializer or code splitted into 2
> lines. The only possible explanation is that KryoSerializer is using a
> ClassLoader that is not registered as an subsidiary ClassLoader of the one in
> REPL.
> A "dirty" fix would be just breaking input by semicolon, but its better to
> fix the ClassLoader to avoid other liabilities.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
