[
https://issues.apache.org/jira/browse/SPARK-9002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639940#comment-14639940
]
Randy Kerber commented on SPARK-9002:
-------------------------------------
That's what I was thinking when I created the Issue. Looking at KryoSerializer
object's toRegister(.) method it looked like Array[Int] class was just
inadvertently missed, so I thought this might be a perfect simple fix for a
first contribution.
But then as I continued my attempt to transition from Java serializer to Kryo,
I kept hitting new "Class is not registered" errors, one after one. First
"Array[String]". Then "Array[Map.empty]", then "Array[Seq.empty]", then
"Array[TreeMap]", plus several other flavors of "empty" collection classes,
"Array[Tuple3]", DataFrame, Row, even Array[GenericRowWithSchema]. Like
swatting cockroaches -- no end to it. Started to wonder if this was a futile
process.
I could use some guidance here as to how it makes sense to proceed. Is it
worthwhile to add the 15-20 classes I've found so far, knowing there will
almost certainly be more? Or drop it, because this route cannot possibly be a
complete fix, and/or a more comprehensive solution for Kryo is already in the
works?
> KryoSerializer initialization does not include 'Array[Int]'
> -----------------------------------------------------------
>
> Key: SPARK-9002
> URL: https://issues.apache.org/jira/browse/SPARK-9002
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 1.4.0
> Environment: MacBook Pro, OS X 10.10.4, Spark 1.4.0, master=local[*],
> IntelliJ IDEA.
> Reporter: Randy Kerber
> Priority: Minor
> Labels: easyfix, newbie
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The object KryoSerializer (inside KryoRegistrator.scala) contains a list of
> classes that are automatically registered with Kryo. That list includes:
> Array\[Byte], Array\[Long], and Array\[Short]. Array\[Int] is missing from
> that list. Can't think of any good reason it shouldn't also be included.
> Note: This is first time creating an issue or contributing code to an apache
> project. Apologies if I'm not following the process correct. Appreciate any
> guidance or assistance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
