Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19763
thanks, merging to master!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19763
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19763
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84161/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19763
**[Test build #84161 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84161/testReport)**
for PR 19763 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19763
**[Test build #84161 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84161/testReport)**
for PR 19763 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19763
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19763
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19763
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19763
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84134/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19763
**[Test build #84134 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84134/testReport)**
for PR 19763 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19763
**[Test build #84134 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84134/testReport)**
for PR 19763 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19763
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19763
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gczsjdy commented on the issue:
https://github.com/apache/spark/pull/19763
@cloud-fan Seems Jenkins's not started?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user zsxwing commented on the issue:
https://github.com/apache/spark/pull/19763
> We can shut down the pool after some certain idle time, but not sure if
it's worth the complexity
Yeah, that's just what the cached thread pool does :)
---
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19763
OK to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19763
cc @zsxwing
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gczsjdy commented on the issue:
https://github.com/apache/spark/pull/19763
Actually, the time gap is O(number of mappers * shuffle partitions). In
this case, number of mappers is not very large, while users are more likely to
get slowed down when they run on a big data
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19763
Looks not a significant difference.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/19763
Seems like not a big deal for the end-to-end performance?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user gczsjdy commented on the issue:
https://github.com/apache/spark/pull/19763
This happens a lot in our TPC-DS 100TB test. We have a Intel Xeon CPU
E5-2699 v4 @2.2GHz CPU as master, this will influence the driver's performance.
And we set `spark.sql.shuffle.partitions` to
Github user CodingCat commented on the issue:
https://github.com/apache/spark/pull/19763
my question is "how many times we have seen this operation of collecting
statistics is the bottleneck?"
---
-
To unsubscribe,
22 matches
Mail list logo
