Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19229
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82141/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19229
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19229
**[Test build #82141 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82141/testReport)**
for PR 19229 at commit
Github user WeichenXu123 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140690182
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala ---
@@ -223,20 +223,18 @@ class ImputerModel private[ml] (
Github user WeichenXu123 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140689908
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala ---
@@ -223,20 +223,18 @@ class ImputerModel private[ml] (
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19336
**[Test build #82142 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82142/testReport)**
for PR 19336 at commit
GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/19336
[SPARK-21947][SS] Check and report error when monotonically_increasing_id
is used in streaming query
## What changes were proposed in this pull request?
`monotonically_increasing_id`
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19321
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82140/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19321
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19321
**[Test build #82140 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82140/testReport)**
for PR 19321 at commit
Github user yssharma commented on the issue:
https://github.com/apache/spark/pull/18029
@brkyvz Could you please check this for the last suggestions ?
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19335
ping @listenLearning!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19321
Please update the PR title and description.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19321#discussion_r140684015
--- Diff: docs/sql-programming-guide.md ---
@@ -1553,6 +1553,7 @@ options.
## Upgrading From Spark SQL 2.2 to 2.3
- Since Spark 2.3,
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19335
@listenLearning, If you'd like to ask a question, please ask this to the
mailing list (see https://spark.apache.org/community.html).
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19335
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19335
@listenLearning Close this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user listenLearning opened a pull request:
https://github.com/apache/spark/pull/19335
mapPartitions Api
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19334
@listenLearning, looks mistakenly open. Can you close this please?
---
-
To unsubscribe, e-mail:
GitHub user listenLearning opened a pull request:
https://github.com/apache/spark/pull/19334
Branch 1.6
## What changes were proposed in this pull request?
(Please fill in changes proposed in this fix)
## How was this patch tested?
(Please explain how this
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19334
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19229
**[Test build #82141 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82141/testReport)**
for PR 19229 at commit
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140680794
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2102,6 +2102,55 @@ class Dataset[T] private[sql](
}
/**
Github user ConeyLiu commented on the issue:
https://github.com/apache/spark/pull/19317
Test case:
```scala
test("performance of aggregateByKeyLocally ") {
val random = new Random(1)
val pairs = sc.parallelize(0 until 1000, 20)
.map(p =>
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140680654
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2102,6 +2102,55 @@ class Dataset[T] private[sql](
}
/**
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140680629
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2102,6 +2102,55 @@ class Dataset[T] private[sql](
}
/**
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140680435
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2102,6 +2102,55 @@ class Dataset[T] private[sql](
}
/**
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19290
For more context, I believe `lintr` was initially installed in
https://github.com/apache/spark/commit/004f57374b98c4df32d9f1e19221f68e92639a49.
Upgrade to jimhester/lintr@a769c0b was proposed
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140678630
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2102,6 +2102,55 @@ class Dataset[T] private[sql](
}
/**
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140678574
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala ---
@@ -223,20 +223,18 @@ class ImputerModel private[ml] (
override def
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19321
**[Test build #82140 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82140/testReport)**
for PR 19321 at commit
Github user WeichenXu123 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140675967
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala ---
@@ -223,20 +223,18 @@ class ImputerModel private[ml] (
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19290
Do you maybe have some worries about this? If that worry is quite crucial,
I think we could also consider an option, not upgrading this, leaving
`lint-r.R` script as was, and only fixing the
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19290
@shaneknapp Sure, it was my bad. I will be careful next time.
It is required to fix an actual issue in order to to detect R codes that do
not follow project's R style.
---
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/19290
@HyukjinKwon -- you will absolutely not have builds install packages on the
build system. this is a really bad idea.
is this absolutely required, or just to fix a warning in the build
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19283
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19326
Merged to master and branch-2.2.
@jgoleary, I merged this considering the first contribution but let's do
this in a batch if possible in the future.
---
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19326
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/16548
So there is something similar in the fulltests for R
`./R/pkg/tests/fulltests/test_mllib.R` (found while working on packaging).
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82138/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82138 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82138/testReport)**
for PR 19222 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19326
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82139/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19326
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19326
**[Test build #82139 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82139/testReport)**
for PR 19326 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19326
**[Test build #82139 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82139/testReport)**
for PR 19326 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19326
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82137/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82137 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82137/testReport)**
for PR 19294 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19324
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19324#discussion_r140664581
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala
---
@@ -186,8 +186,7 @@ case class
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19324#discussion_r140664550
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala
---
@@ -328,10 +325,11 @@ case class
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19324#discussion_r140664499
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala
---
@@ -328,10 +325,11 @@ case class
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19222
@hvanhovell @rednaxelafx
After running a benchmark program, I took a polymorphic approach (i.e. each
subclass has `getInt()`/`putInt()` methods. Then, I got better performance than
monomorphic
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82138 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82138/testReport)**
for PR 19222 at commit
Github user jgoleary commented on the issue:
https://github.com/apache/spark/pull/19326
Updated description. The only other mentions of `as()` I can find in the
docs are in Java examples, and the method appears to exist on the Java side.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82136/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82136 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82136/testReport)**
for PR 19294 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82135/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82132/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82132 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82132/testReport)**
for PR 19222 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82135 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82135/testReport)**
for PR 19294 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82134/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82134 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82134/testReport)**
for PR 19294 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82137 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82137/testReport)**
for PR 19294 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82133/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82133 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82133/testReport)**
for PR 19294 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82136 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82136/testReport)**
for PR 19294 at commit
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/19294#discussion_r140658582
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala
---
@@ -130,17 +135,21 @@ class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82130/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82130 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82130/testReport)**
for PR 19294 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82135 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82135/testReport)**
for PR 19294 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19229
@WeichenXu123 Have any more comments on this? Thanks. I think the ML part
is straightforward.
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82134 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82134/testReport)**
for PR 19294 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82133 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82133/testReport)**
for PR 19294 at commit
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/19317
It is better adding more perf test for `OpenHashSet` replacement to avoid
perf regression. And I found `reduceByKeyLocally` also use `JHashSet`, I am not
sure whether there is some special
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82132 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82132/testReport)**
for PR 19222 at commit
Github user ConeyLiu commented on the issue:
https://github.com/apache/spark/pull/19317
OK, just keep it. Does this need more test or more improvements ?
---
-
To unsubscribe, e-mail:
Github user szhem commented on the issue:
https://github.com/apache/spark/pull/19294
@mridulm Updated `FileFormatWriterSuite` [to
cover](https://github.com/apache/spark/pull/19294/files#diff-bc98a3d91cf4f95f4f473146400044aa)
both branches of the [committer
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82131/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82131 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82131/testReport)**
for PR 19294 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82131 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82131/testReport)**
for PR 19294 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19290
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82129/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19290
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19290
**[Test build #82129 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82129/testReport)**
for PR 19290 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19290
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19290
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82128/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19290
**[Test build #82128 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82128/testReport)**
for PR 19290 at commit
Github user sathiyapk commented on the issue:
https://github.com/apache/spark/pull/19295
I pushed a new commit that addresses @wzhfy review comments..
---
-
To unsubscribe, e-mail:
Github user szhem commented on a diff in the pull request:
https://github.com/apache/spark/pull/19294#discussion_r140654204
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala
---
@@ -57,6 +57,11 @@ class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82130 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82130/testReport)**
for PR 19294 at commit
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/19317
@ConeyLiu Yes tree aggregate introduce extra shuffle. But it is possible to
improve perf when driver total collecting data size from executors are large
and there're many partitions.
But I
1 - 100 of 129 matches
Mail list logo
