GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/790
[SPARK-1819] [SQL] Use AllScalaRegistrar for SparkSqlSerializer to register
serializers of ...
...Scala collections.
When I execute `orderBy` or `limit` for `SchemaRDD` including `ArrayType`
or `MapType`, `SparkSqlSerializer` throws the following exception:
```
com.esotericsoftware.kryo.KryoException: Class cannot be created (missing
no-arg constructor): scala.collection.immutable.$colon$colon
```
or
```
com.esotericsoftware.kryo.KryoException: Class cannot be created (missing
no-arg constructor): scala.collection.immutable.Vector
```
or
```
com.esotericsoftware.kryo.KryoException: Class cannot be created (missing
no-arg constructor): scala.collection.immutable.HashMap$HashTrieMap
```
and so on.
This is because registrations of serializers for each concrete collections
are missing in `SparkSqlSerializer`.
I believe it should use `AllScalaRegistrar`.
`AllScalaRegistrar` covers a lot of serializers for concrete classes of
`Seq`, `Map` for `ArrayType`, `MapType`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ueshin/apache-spark issues/SPARK-1845
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/790.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #790
----
commit d1ed9929d8c328ef913c428699cbb63130bb3712
Author: Takuya UESHIN <ues...@happy-camper.st>
Date: 2014-05-15T11:10:37Z
Use AllScalaRegistrar for SparkSqlSerializer to register serializers of
Scala collections.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
