Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82133/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82135/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82132/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19326
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19326
**[Test build #82139 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82139/testReport)**
for PR 19326 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82133 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82133/testReport)**
for PR 19294 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82134/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82134 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82134/testReport)**
for PR 19294 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82137 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82137/testReport)**
for PR 19294 at commit
Github user kiszk commented on the issue:
https://github.com/apache/spark/pull/19222
@hvanhovell @rednaxelafx
After running a benchmark program, I took a polymorphic approach (i.e. each
subclass has `getInt()`/`putInt()` methods. Then, I got better performance than
monomorphic
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19324#discussion_r140664581
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala
---
@@ -186,8 +186,7 @@ case class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82137/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82137 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82137/testReport)**
for PR 19294 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19326
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19324#discussion_r140664499
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala
---
@@ -328,10 +325,11 @@ case class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82136 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82136/testReport)**
for PR 19294 at commit
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19324#discussion_r140664550
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala
---
@@ -328,10 +325,11 @@ case class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82132 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82132/testReport)**
for PR 19222 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82135 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82135/testReport)**
for PR 19294 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82138 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82138/testReport)**
for PR 19222 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19326
**[Test build #82139 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82139/testReport)**
for PR 19326 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19326
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82139/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82136/
Test FAILed.
---
Github user jgoleary commented on the issue:
https://github.com/apache/spark/pull/19326
Updated description. The only other mentions of `as()` I can find in the
docs are in Java examples, and the method appears to exist on the Java side.
---
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/19324
LGTM
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19321
**[Test build #82140 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82140/testReport)**
for PR 19321 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82138 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82138/testReport)**
for PR 19222 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82138/
Test PASSed.
---
Github user shaneknapp commented on the issue:
https://github.com/apache/spark/pull/19290
@HyukjinKwon -- you will absolutely not have builds install packages on the
build system. this is a really bad idea.
is this absolutely required, or just to fix a warning in the build
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19222
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user holdenk commented on the issue:
https://github.com/apache/spark/pull/16548
So there is something similar in the fulltests for R
`./R/pkg/tests/fulltests/test_mllib.R` (found while working on packaging).
---
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19283
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19326
Merged to master and branch-2.2.
@jgoleary, I merged this considering the first contribution but let's do
this in a batch if possible in the future.
---
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19326
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19290
@shaneknapp Sure, it was my bad. I will be careful next time.
It is required to fix an actual issue in order to to detect R codes that do
not follow project's R style.
---
Github user WeichenXu123 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140675967
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala ---
@@ -223,20 +223,18 @@ class ImputerModel private[ml] (
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19290
Do you maybe have some worries about this? If that worry is quite crucial,
I think we could also consider an option, not upgrading this, leaving
`lint-r.R` script as was, and only fixing the
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140678574
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/Imputer.scala ---
@@ -223,20 +223,18 @@ class ImputerModel private[ml] (
override def
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140678630
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2102,6 +2102,55 @@ class Dataset[T] private[sql](
}
/**
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19290
For more context, I believe `lintr` was initially installed in
https://github.com/apache/spark/commit/004f57374b98c4df32d9f1e19221f68e92639a49.
Upgrade to jimhester/lintr@a769c0b was proposed
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140680435
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2102,6 +2102,55 @@ class Dataset[T] private[sql](
}
/**
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140680629
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2102,6 +2102,55 @@ class Dataset[T] private[sql](
}
/**
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140680654
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2102,6 +2102,55 @@ class Dataset[T] private[sql](
}
/**
Github user ConeyLiu commented on the issue:
https://github.com/apache/spark/pull/19317
Test case:
```scala
test("performance of aggregateByKeyLocally ") {
val random = new Random(1)
val pairs = sc.parallelize(0 until 1000, 20)
.map(p =>
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19229#discussion_r140680794
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ---
@@ -2102,6 +2102,55 @@ class Dataset[T] private[sql](
}
/**
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19229
**[Test build #82141 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82141/testReport)**
for PR 19229 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18747
**[Test build #82127 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82127/testReport)**
for PR 18747 at commit
Github user sathiyapk commented on a diff in the pull request:
https://github.com/apache/spark/pull/19295#discussion_r140652720
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/ExperimentalMethods.scala ---
@@ -44,11 +44,14 @@ class ExperimentalMethods private[sql]() {
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19229
@WeichenXu123 Have any more comments on this? Thanks. I think the ML part
is straightforward.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19326
That's fine. I believe we don't usually need a JIRA for a trivial change
though. Would you mind double checking if there are similar instances in the
PySpark doc?
Also, it'd be great
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user ConeyLiu commented on the issue:
https://github.com/apache/spark/pull/19317
OK, just keep it. Does this need more test or more improvements ?
---
-
To unsubscribe, e-mail:
Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/19294#discussion_r140658582
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala
---
@@ -130,17 +135,21 @@ class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82136 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82136/testReport)**
for PR 19294 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19307
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19290#discussion_r140651399
--- Diff: R/pkg/R/column.R ---
@@ -238,8 +238,10 @@ setMethod("between", signature(x = "Column"),
#' @param x a Column.
#' @param dataType a
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/19290#discussion_r140651387
--- Diff: dev/lint-r.R ---
@@ -24,10 +24,16 @@ if (! library(SparkR, lib.loc = LOCAL_LIB_LOC,
logical.return = TRUE)) {
stop("You should install
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19290
**[Test build #82128 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82128/testReport)**
for PR 19290 at commit
Github user mridulm commented on a diff in the pull request:
https://github.com/apache/spark/pull/19311#discussion_r140651513
--- Diff:
core/src/test/scala/org/apache/spark/storage/MemoryStoreSuite.scala ---
@@ -407,4 +407,119 @@ class MemoryStoreSuite
})
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82134 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82134/testReport)**
for PR 19294 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19332
Merged to master.
Thank you @srowen and @dongjoon-hyun.
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/18747
**[Test build #82127 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82127/testReport)**
for PR 18747 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82130 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82130/testReport)**
for PR 19294 at commit
Github user szhem commented on a diff in the pull request:
https://github.com/apache/spark/pull/19294#discussion_r140654204
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala
---
@@ -57,6 +57,11 @@ class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82133 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82133/testReport)**
for PR 19294 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19293
**[Test build #82126 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82126/testReport)**
for PR 19293 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19293
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82126/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19293
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19290
**[Test build #82129 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82129/testReport)**
for PR 19290 at commit
Github user sathiyapk commented on the issue:
https://github.com/apache/spark/pull/19295
I pushed a new commit that addresses @wzhfy review comments..
---
-
To unsubscribe, e-mail:
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19222
**[Test build #82132 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82132/testReport)**
for PR 19222 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82135 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82135/testReport)**
for PR 19294 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19332
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19184
@viirya @jerryshao To take a step back here.
This specific issue is applicable to window operations and not to shuffle.
In shuffle, you a much larger volume of data written per
Github user sathiyapk commented on the issue:
https://github.com/apache/spark/pull/19295
@gatorsmile thanks for your comments. Here are my thoughts, thanks for
correcting me if i'm wrong. (sorry for the big comment though :))
1. This PR don't change any existing API, it adds a new
Github user szhem commented on a diff in the pull request:
https://github.com/apache/spark/pull/19294#discussion_r140652214
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala
---
@@ -130,17 +135,21 @@ class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19290
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19290
**[Test build #82129 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82129/testReport)**
for PR 19290 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19290
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82129/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19293
**[Test build #82126 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82126/testReport)**
for PR 19293 at commit
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/19324
@juliuszsompolski Thanks for pinging me.
#18931 is an attempt to separate the consume function as it can as
possible. With long chain of any operators, you can have a long consume
function
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/19317
@ConeyLiu Yes tree aggregate introduce extra shuffle. But it is possible to
improve perf when driver total collecting data size from executors are large
and there're many partitions.
But I
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18747
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82127/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/18747
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/19290
Ugh.. it failed to install due to permission issue ...
```
Downloading GitHub repo jimhester/lintr@5431140
from URL https://api.github.com/repos/jimhester/lintr/zipball/5431140
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19290
**[Test build #82128 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82128/testReport)**
for PR 19290 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19290
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82128/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19290
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82131 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82131/testReport)**
for PR 19294 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19294
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82131/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/19294
**[Test build #82131 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82131/testReport)**
for PR 19294 at commit
Github user WeichenXu123 commented on the issue:
https://github.com/apache/spark/pull/19317
It is better adding more perf test for `OpenHashSet` replacement to avoid
perf regression. And I found `reduceByKeyLocally` also use `JHashSet`, I am not
sure whether there is some special
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/19294
@szhem You are correct, currently it fails in the driver itself.
So failures in executor are not seen - since job submission fails.
With this pr, the job submission should succeed - but
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/19277
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
1 - 100 of 129 matches
Mail list logo
