Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20856
@HyukjinKwon @cloud-fan Thanks for pinging me, sorry for replying late.
Yeah I looked at the final fixing at #21815, it looks good for a fixing at this
particular problem.
> It seems to me
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/20856
@cloud-fan I do like the idea of generating the code on driver side. The
main issues I see are:
- In the generated code we reference external objects. We have to send
this array of
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20856
@HyukjinKwon good analysis!
Currently Spark is a little messy about what shall be serialized and sent
to executors. Sometimes we just send an entire query tree but only read a few
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20856
cc @hvanhovell too
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/20856
@HyukjinKwon thanks for your great analysis. I agree with you that the
proposed fix is more a "workaround" than a real fix for the issue we have here.
The main problem here as you pointed
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20856
Okay, I was investigating this and the fix itself looks quite inappropriate.
This looks what happened now. I can reproduce this by a bit of messy way:
```diff
diff --git
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93077/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20856
**[Test build #93077 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93077/testReport)**
for PR 20856 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20856
Sure. Let me give a shot to reproduce it.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20856
We are not able to merge it without a valid test case. We need to
understand the root cause why `relation` can be null.
@HyukjinKwon If you can help, please try to create a test case.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/988/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20856
**[Test build #93077 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93077/testReport)**
for PR 20856 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20856
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20856
**[Test build #4171 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4171/testReport)**
for PR 20856 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20856
**[Test build #4171 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4171/testReport)**
for PR 20856 at commit
Github user jaceklaskowski commented on the issue:
https://github.com/apache/spark/pull/20856
BTW, I've just realized that even without the issue it's clear that
creating a new `FileSourceScanExec` will end up with a NPE from the
`supportsBatch` field.
---
Github user jaceklaskowski commented on the issue:
https://github.com/apache/spark/pull/20856
I spent over 2 days applying different modifications to the query hoping I
could cut the number of `CASE WHEN`s and other projections, but noticed no
correlation between the number or their
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20856
What is the test case?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88362/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20856
**[Test build #88362 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88362/testReport)**
for PR 20856 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/20856
I think that this problem does not depend on data. Can you create a test
case by modifying the query slightly with different column names?
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1597/
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20856
**[Test build #88362 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88362/testReport)**
for PR 20856 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/20856
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88359/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20856
**[Test build #88359 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88359/testReport)**
for PR 20856 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20856
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1595/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20856
**[Test build #88359 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88359/testReport)**
for PR 20856 at commit
34 matches
Mail list logo
