Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/16161
Shall we backport this to branch-2.1? I'd consider this as a bug because of
the following snippet fail in Spark 2.1:
```scala
case class Wrapper1(value: Option[Map[String, String]])
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16161
I'm fine to backport this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wis
Github user liancheng commented on the issue:
https://github.com/apache/spark/pull/16161
Thanks. Backported to branch-2.1.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16161
**[Test build #69696 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69696/consoleFull)**
for PR 16161 at commit
[`699cc75`](https://github.com/apache/spark/commit/6
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16161
**[Test build #69696 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69696/consoleFull)**
for PR 16161 at commit
[`699cc75`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16161
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16161
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69696/
Test FAILed.
---
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/16161
cc @cloud-fan
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if th
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16161
can you highlight your approach to fix this bug? It looks to me that this
is not a general fix, e.g. `mutable.Map` will still fail.
---
If your project is set up for it, you can reply to this ema
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16161
**[Test build #69741 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69741/consoleFull)**
for PR 16161 at commit
[`f325e75`](https://github.com/apache/spark/commit/f
Github user aray commented on the issue:
https://github.com/apache/spark/pull/16161
The approach is to change the deserializer (via
`ScalaReflection#deserializerFor`) to return the more specific type
`scala.collections.immutable.Map` instead of `scala.collections.Map` as it does
now
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16161
**[Test build #69741 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/69741/consoleFull)**
for PR 16161 at commit
[`f325e75`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16161
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69741/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16161
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/16161
OK, it sounds like what you're doing is unsupported at the moment and this
doesn't change that, and you have a workaround. Is this worth merging?
---
If your project is set up for it, you can reply
Github user aray commented on the issue:
https://github.com/apache/spark/pull/16161
Right now it's not supported to have the following:
```
case class Foo(a: Map[Int, Int])
```
(using the scala Predef version of Map)
The
[documented](http://spark.apache.org/doc
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16161
option 1 is better, but this PR only adds the support for
`scala.collection.immutable.Map`, with a very specific way: we can't extend it
to support `scala.collection.mutable.Map` or `List`.
Github user aray commented on the issue:
https://github.com/apache/spark/pull/16161
I would be happy to create a seperate PR for adding support for
`mutable.Map` (and `List`) if that is wanted. But there is no _generic_
solution as there is no type that is assignable to both `mutable.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16161
> there is no type that is assignable to both mutable.Map and immutable.Map
Yea, and a general solution would be, remember the original type and create
that type when converting catalyst v
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16161
**[Test build #70027 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70027/consoleFull)**
for PR 16161 at commit
[`e0b3e76`](https://github.com/apache/spark/commit/e
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/16161
**[Test build #70027 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70027/consoleFull)**
for PR 16161 at commit
[`e0b3e76`](https://github.com/apache/spark/commit/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16161
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70027/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/16161
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
e
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/16161
thanks, merging to master!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wi
24 matches
Mail list logo
