Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95200/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #95200 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95200/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #95200 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95200/testReport)**
for PR 21859 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21859
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95070/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #95070 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95070/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #95070 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95070/testReport)**
for PR 21859 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21859
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #95048 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95048/testReport)**
for PR 21859 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95048/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #95048 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95048/testReport)**
for PR 21859 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21859
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94990/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94990 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94990/testReport)**
for PR 21859 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21859
If this optimization is done more generally, will the implicitly cached
data cause memory pressure on driver, as seems we don't have way to release
them?
---
Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
'The ShuffleWriter should treat RangePartitioner specially and consume the
sampled data in RangePartitioner instead of the input iterator.' This idea is
good, maybe we can cache both the K and V
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94990 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94990/testReport)**
for PR 21859 at commit
Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
I read the source code again.
The RangePartitioner[K, V] in ShuffleExchangeExec is an instance of
RangePartitioner[InternalRow, Null]. RangePartitioner only sample K for getting
the
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21859
I don't think this optimization should be done at SQL layer. The
`ShuffleWriter` should treat `RangePartitioner` specially and consume the
sampled data in `RangePartitioner` instead of the input
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94958/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94958 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94958/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94958 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94958/testReport)**
for PR 21859 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94947/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94947 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94947/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94947 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94947/testReport)**
for PR 21859 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21859
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94939/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94939 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94939/testReport)**
for PR 21859 at commit
Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
We may not know in advance how big this query is. The data at the beginning
is large, but it may be very small after filtering.
I encountered this problem while using thrift server for queries.
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/21859
for small queries, can we just do
```
val df = table.filter(...).cache()
df.sort()
```
We should carefully make trade off between the SQL engine complexity and
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94939 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94939/testReport)**
for PR 21859 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21859
LGTM cc @viirya @cloud-fan @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94931/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94931 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94931/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94931 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94931/testReport)**
for PR 21859 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21859
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94688/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94688 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94688/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94688 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94688/testReport)**
for PR 21859 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21859
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
I think I need another retest . Please help. @viirya
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94662/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94662 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94662/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94662 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94662/testReport)**
for PR 21859 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21859
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
please help retest it . @kiszk @viirya
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94603/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94603 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94603/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94603 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94603/testReport)**
for PR 21859 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94496/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94496 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94496/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94496 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94496/testReport)**
for PR 21859 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21859
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94485/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94485 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94485/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94485 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94485/testReport)**
for PR 21859 at commit
Github user ueshin commented on the issue:
https://github.com/apache/spark/pull/21859
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
@ueshin
please retest it, an unkown error occurred.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
retest this, please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94468/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94468 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94468/testReport)**
for PR 21859 at commit
Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
This optimization is only for SQL, but other places also use
RangePartitioner. What it can affect other places?
The failed UTs are caused by
```
else if (sampleCacheEnabled
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94468 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94468/testReport)**
for PR 21859 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/21859
Good point. [These
failures](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94431/testReport/)
may show that it affects other places.
---
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/21859
This optimization is only for SQL, but other places also use
`RangePartitioner`. What it can affect other places?
---
-
To
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94431/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94431 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94431/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94431 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94431/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94424 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94424/testReport)**
for PR 21859 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94424/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94424 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94424/testReport)**
for PR 21859 at commit
Github user sddyljsx commented on the issue:
https://github.com/apache/spark/pull/21859
@ueshin please test again
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21859
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/94417/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94417 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94417/testReport)**
for PR 21859 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21859
**[Test build #94417 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/94417/testReport)**
for PR 21859 at commit
1 - 100 of 105 matches
Mail list logo
