Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62152 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62152/consoleFull)**
for PR 13704 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13704
@cloud-fan I added test cases. Could you review it again?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62102/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62102 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62102/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62102 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62102/consoleFull)**
for PR 13704 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62097/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62097 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62097/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62097 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62097/consoleFull)**
for PR 13704 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62087/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62087 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62087/consoleFull)**
for PR 13704 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62084/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62084 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62084/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62084 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62084/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62087 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62087/consoleFull)**
for PR 13704 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62071/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62071 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62071/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62071 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62071/consoleFull)**
for PR 13704 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13704
@cloud-fan could you please review this? As you pointed, I also changed
code related to `cast`. I added benchmark results, too.
---
If your project is set up for it, you can reply to this email and
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62057/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62057 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62057/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62057 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62057/consoleFull)**
for PR 13704 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62050/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62050 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62050/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #62050 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/62050/consoleFull)**
for PR 13704 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13704
@cloud-fan , regarding a ```cast```, I think that you are correct. When we
insert a ```cast```, I pass information on ```ArrayType.containsNull``` to
```Cast``` now. In this case, it is ```false```.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61677/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #61677 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61677/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #61677 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61677/consoleFull)**
for PR 13704 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61630/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #61630 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61630/consoleFull)**
for PR 13704 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/61629/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #61629 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61629/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #61630 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61630/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #61629 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/61629/consoleFull)**
for PR 13704 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13704
@cloud-fan, I checked the following
>Let me check which code portion inserts Cast in this tree.
IIUC, [this
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13704
Let me check which code portion inserts ```Cast``` in this tree.
> And we do have a chance to optimize it: if the target array type's
element type is primitive and the input array type's
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/13704
> the `Cast` performs conversion from `UnsafeArrayData` to
`GenericArrayData`.
`Cast` is used to cast one type to another, not cast one kind of data
representation to another, so I don't
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13704
Regarding the plan tree printout (I removed my debug information), the
```Cast``` performs conversion from ```UnsafeArrayData``` to
```GenericArrayData``` since the target type of ```Cast``` is
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/13704
From the plan tree given by you, the `Cast` is casting `value#63` to array
type, not `ObjectType(classOf[Array[Double]])`, there should be some other
reason that we need to find out(maybe it's
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13704
I agree that ```value#63``` is double array type. ```value#63``` is stored
as ```UnsafeArrayData```.
`` in ```MapElements , obj#67: double``` is
represented by Java byte code instead of
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/13704
Sorry I should say it more explicitly: `value#63` is already double array
type. `UnsafeArrayData` is an internal data representation of array type, so it
seems weird to have the `Cast` there.
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13704
@cloud-fan I think ```value#63``` is ```UnsafeArrayData```.
When I ran a DataFrame program, I got the following trees. Since operations
for DataFrame access data in UnsafeArrayData, I think that
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/13704
I'm wondering why we have this `Cast` in `DeserializeToObject`. Isn't the
`value#63` already a double array?
---
If your project is set up for it, you can reply to this email and have your
reply
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/13704
@cloud-fan I updated the PR description by adding plan trees.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/13704
can you also put the plan tree of the example program in PR description?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/60636/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/13704
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #60636 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60636/consoleFull)**
for PR 13704 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/13704
**[Test build #60636 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/60636/consoleFull)**
for PR 13704 at commit
59 matches
Mail list logo
