Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21106
mostly LGTM, though people may have better ideas about naming.
cc @hvanhovell
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90546/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90546 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90546/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90545/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90545 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90545/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3172/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90546 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90546/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3171/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90545 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90545/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90490/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90490 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90490/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90485/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90485 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90485/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3123/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
Any more comments on this? @hvanhovell @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90490 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90490/testReport)**
for PR 21106 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
I agree. It is what I saw when I've tried to change it locally.
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21106
I took a deeper look, changing to case class needs a lot of changes, let's
follow your previous approach.
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3118/
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
> I think we have a general problem with passing configs to code
generators. For this PR, let's make it simple and only use the conf in tests.
We are fine in tests because tests create these code
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90485 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90485/testReport)**
for PR 21106 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21106
I think we have a general problem with passing configs to code generators.
For this PR, let's make it simple and only use the conf in tests. We are fine
in tests because tests create these code
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90319/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90319 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90319/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90313/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90313 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90313/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3001/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90319 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90319/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2996/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90313 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90313/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90278/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90278 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90278/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90277/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90277 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90277/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2965/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90278 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90278/testReport)**
for PR 21106 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
@hvanhovell @cloud-fan A SQL config was added to control
codegen/interpreted for test. Please let me know if you have any comment on
current approach. Thanks.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2964/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #90277 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90277/testReport)**
for PR 21106 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
@hvanhovell If you don't have other comment, I will follow @cloud-fan's
suggestion to add a sql conf for it. Thanks.
---
-
To
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
A sql conf sounds good to me. @hvanhovell What do you think?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89882/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #89882 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89882/testReport)**
for PR 21106 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21106
I think `UnsafeProjectionCreator` is only useful for tests, because in
production we should always try codegen first, and fallback to interpretation
if codegen fails. There is no need to disable
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2691/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #89882 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89882/testReport)**
for PR 21106 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
I tried to unify this class with `UnsafeProjectionCreator`. So there is no
more `UnsafeProjectionCreator` trait.
Without the trait `UnsafeProjectionCreator`, one problem is we can't easily
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21106
I think it's very hard to unify the entry point(input type) for all the
code generators. E.g. some use `Seq[Expression]` as input, some use
`Seq[DataType]`.
I'd like to make
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/21106
Yeah, let's move forward slowly. I am still pondering about the what the
right abstraction here would look like; this looks promising though.
Can you try to unify this class with
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
@hvanhovell Any more comments on the current design? If not, I will apply
this to all places that we create unsafe projection.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89615/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #89615 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89615/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2516/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #89615 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89615/testReport)**
for PR 21106 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89604/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #89604 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89604/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2508/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #89604 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89604/testReport)**
for PR 21106 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89603/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #89603 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89603/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2507/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #89603 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89603/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89578/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #89578 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89578/testReport)**
for PR 21106 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2491/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21106
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21106
**[Test build #89578 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89578/testReport)**
for PR 21106 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21106
cc @hvanhovell
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
96 matches
Mail list logo
