Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/20699
This only handles a corner case. This is not what we expect.
---
-
To unsubscribe, e-mail:
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/20699
cc @HyukjinKwon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93449/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20699
**[Test build #93449 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93449/testReport)**
for PR 20699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20699
**[Test build #93449 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93449/testReport)**
for PR 20699 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20699
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/20699
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93426/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20699
**[Test build #93426 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93426/testReport)**
for PR 20699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20699
**[Test build #93426 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93426/testReport)**
for PR 20699 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93057/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20699
**[Test build #93057 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93057/testReport)**
for PR 20699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20699
**[Test build #93057 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93057/testReport)**
for PR 20699 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/20699
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/20699
cc @hvanhovell , @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20699
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88234/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20699
**[Test build #88234 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88234/testReport)**
for PR 20699 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20699
**[Test build #88234 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88234/testReport)**
for PR 20699 at commit
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/20699
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/20699
@hvanhovell , Iâm sorry to delayed answer your question. we have a task
to write with RDD, and do repartition on the memory table and then left join
another table.
Optimized Logical Plan
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20699
I think they are caused by the same issue: an operator(the repartition
operator) can produce exchange operator directly. If `EnsureRequirement` is the
only place to produce/add exchange operator,
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/20699
`EnsureRequirements `can eliminates unnecessary shuffles if child has same
partitioning or compatible child partitionings that same expressions
distribution.
but when child has different
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20699
Sorry I should make the question more specific: `EnsureRequirement#apply`
has a hack to eliminate unnecessary shuffles, do we still need that?
---
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/20699
@cloud-fan ,
`+- Exchange hashpartitioning(b#204, c#205, 5)` it was produced by`
EnsureRequirement#apply`.
so we can't clean up `EnsureRequirement#apply `after this PR.
thanks.
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20699
Can we clean up `EnsureRequirement#apply` after this PR?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user hvanhovell commented on the issue:
https://github.com/apache/spark/pull/20699
A very valid use cases for repartitioning is limiting concurrency. For
example when you have a UDF that calls some service you don't want to DDOS that
service because you make requests from too
Github user heary-cao commented on the issue:
https://github.com/apache/spark/pull/20699
cc @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
32 matches
Mail list logo
