Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21611
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92687/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92687 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92687/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/733/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92687 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92687/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/732/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92670/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92670 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92670/testReport)**
for PR 21611 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21611
LGTM except 2 minor comments
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/720/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92670 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92670/testReport)**
for PR 21611 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92661/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92661 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92661/testReport)**
for PR 21611 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92661 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92661/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/713/
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
Based on the comment, this null check is for preventing encoding a Product
type to a null top-level row. For `Option[Int]`, it is encoded to an int column
in a top-level row.
An example
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21611
In `ExpressionEncoder.apply`, we have
```
val flat = !ScalaReflection.definedByConstructorParams(tpe)
val inputObject = BoundReference(0, ScalaReflection.dataTypeFor[T],
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
ping @cloud-fan for taking a look again.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92464/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92464 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92464/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/577/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92464 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92464/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/574/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92455/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92455 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92455/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/567/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92455 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92455/testReport)**
for PR 21611 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92448/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92448 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92448/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92447/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92447 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92447/testReport)**
for PR 21611 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92448 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92448/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/560/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92447 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92447/testReport)**
for PR 21611 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
A test of `join` is added. The change of Option of product encoder is
removed.
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21611
SGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
I agree with you. So I think this PR should be focused on the first
problem. Is it ok for you?
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21611
There are 2 problems:
1. `Option` is mistakenly mapped to a struct type. It's not only for
aggregate, but join too
```
scala> val a = Seq(Some(1)).toDS
a:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92403/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92403 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92403/testReport)**
for PR 21611 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92403 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92403/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/528/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
This affects the dataset with `Aggregator` of output type `Option`.
For normal dataset schema, it does affect the following case:
```scala
val ds = Seq(Some(1), None,
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21611
does this affect normal dataset schema?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
ping @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
cc @cloud-fan for review.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92202/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92202 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92202/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/402/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92202 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92202/testReport)**
for PR 21611 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21611
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92200 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92200/testReport)**
for PR 21611 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92200/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/400/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21611
**[Test build #92200 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92200/testReport)**
for PR 21611 at commit
80 matches
Mail list logo
