Github user sujith71955 commented on the issue:
https://github.com/apache/spark/pull/20611
@wzhfy @HyukjinKwon @jiangxb1987 @gatorsmile @vinodkc
All test-cases has been passed, please let me know for any other
suggestions.
With current solution following improvement has
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21053
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user felixcheung commented on a diff in the pull request:
https://github.com/apache/spark/pull/21033#discussion_r181627739
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala
---
@@ -495,9
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21053
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89383/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21053
**[Test build #89383 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89383/testReport)**
for PR 21053 at commit
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21053#discussion_r181624253
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,106 @@ case class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21053#discussion_r181623803
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,106 @@ case class
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21053#discussion_r181623729
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,106 @@ case class
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21053#discussion_r181623519
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,106 @@ case class
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21053#discussion_r181622879
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,106 @@ case class
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21031#discussion_r181622107
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
---
@@ -407,6 +407,7 @@ object FunctionRegistry {
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21031#discussion_r181621963
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -3282,6 +3282,14 @@ object functions {
*/
def size(e:
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21024
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21025
@mgaido91 Could you address the conflicts? Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21024
LGTM. Thanks! Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21053#discussion_r181619426
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,106 @@ case class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21053#discussion_r181618055
--- Diff: python/pyspark/sql/functions.py ---
@@ -1846,6 +1846,27 @@ def array_contains(col, value):
return
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21053#discussion_r181617303
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeExtractors.scala
---
@@ -354,3 +336,37 @@ case class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21053#discussion_r181619450
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,106 @@ case class
Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/21074#discussion_r181620108
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -178,7 +178,13 @@ object TypeCoercion {
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21074#discussion_r181619518
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala
---
@@ -178,7 +178,13 @@ object TypeCoercion {
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/21047
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/21047
Merging to master. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21047
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89384/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21047
**[Test build #89384 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89384/testReport)**
for PR 21047 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21047
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21037#discussion_r181614224
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,60 @@ case class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21037#discussion_r181613270
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,60 @@ case class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21037#discussion_r181616266
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala
---
@@ -105,4 +106,26 @@ class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21037#discussion_r181613044
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,60 @@ case class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21037#discussion_r181615751
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,60 @@ case class
Github user ueshin commented on a diff in the pull request:
https://github.com/apache/spark/pull/21037#discussion_r181614388
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,61 @@ case class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21047
**[Test build #89384 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89384/testReport)**
for PR 21047 at commit
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/21047
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21053
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21053
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2334/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21053
**[Test build #89383 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89383/testReport)**
for PR 21053 at commit
Github user jiangxb1987 commented on the issue:
https://github.com/apache/spark/pull/21074
cc @HyukjinKwon @gatorsmile @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user edwinalu commented on a diff in the pull request:
https://github.com/apache/spark/pull/20940#discussion_r181611071
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -234,8 +244,22 @@ private[spark] class EventLoggingListener(
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21053#discussion_r181610494
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala ---
@@ -413,6 +413,78 @@ class DataFrameFunctionsSuite extends QueryTest
Github user edlee123 commented on the issue:
https://github.com/apache/spark/pull/18378
I see the rationale now, thank you everyone
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20535
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20535
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2333/
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20535
**[Test build #89382 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89382/testReport)**
for PR 20535 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20535
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20937
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20937
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89381/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20937
**[Test build #89381 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89381/testReport)**
for PR 20937 at commit
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19868#discussion_r181607994
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/hive/QueryPartitionSuite.scala ---
@@ -70,6 +71,45 @@ class QueryPartitionSuite extends QueryTest
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19868#discussion_r181607863
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -279,6 +293,10 @@ class HadoopRDD[K, V](
case e: IOException if
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19868#discussion_r181607797
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -260,6 +270,10 @@ class HadoopRDD[K, V](
logWarning(s"Skipped
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19868#discussion_r181607746
--- Diff: core/src/main/scala/org/apache/spark/rdd/HadoopRDD.scala ---
@@ -197,17 +200,24 @@ class HadoopRDD[K, V](
val jobConf = getJobConf()
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/18378
It's pretty natural to convert integer type to int32. Although Spark tries
its best to avoid behavior changes, it's allowed to fix some wrong behaviors in
new releases, and I believe it's well
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21060
cc @rdblue and @steveloughran too who I guess should be interested in
setting up a backporting policy.
---
-
To
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21060
I am not saying we shouldn't be careful. I am trying to be careful when I
backport. So, your reasons are:
- any behaviour changes shouldn't be backported and it's the basic backport
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21060
Like what I said above, we need to be very careful when backporting the PR
with the behavior changes, especially when this is **neither a critical issue
nor a regression**. Even if this is a bug
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21075
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21075
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user pmackles opened a pull request:
https://github.com/apache/spark/pull/21075
[SPARK-23988][MESOS] Improve handling of appResource in mesos dispatcher
when using Docker
Improve/fix handling of appResource for mesos dispatcher when using docker
Tested with new unit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21060
This is not just about just inconsistency but a bug. The previous behaivour
doesn't make sense.
Sure, no need to rush.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20611
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20611
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89380/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20611
**[Test build #89380 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89380/testReport)**
for PR 20611 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20937
**[Test build #89381 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89381/testReport)**
for PR 20937 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20937
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89379/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20937
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20937
**[Test build #89379 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89379/testReport)**
for PR 20937 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21073
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21073
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89378/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21073
**[Test build #89378 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89378/testReport)**
for PR 21073 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20937
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20937
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89377/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20937
**[Test build #89377 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89377/testReport)**
for PR 20937 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21074
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21074
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89376/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21074
**[Test build #89376 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89376/testReport)**
for PR 21074 at commit
Github user SimonUzL closed the pull request at:
https://github.com/apache/spark/pull/19526
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20611
**[Test build #89380 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89380/testReport)**
for PR 20611 at commit
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21034
Any other comments?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20938#discussion_r181593152
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +289,160 @@ case class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20937
**[Test build #89379 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89379/testReport)**
for PR 20937 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20937
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89375/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20937
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20937
**[Test build #89375 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89375/testReport)**
for PR 20937 at commit
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21061#discussion_r181590066
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +288,80 @@ case class
Github user kiszk commented on a diff in the pull request:
https://github.com/apache/spark/pull/21070#discussion_r181589911
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java
---
@@ -63,115 +58,139 @@ public final
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21073
**[Test build #89378 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89378/testReport)**
for PR 21073 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21068
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89373/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21068
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21068
**[Test build #89373 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89373/testReport)**
for PR 21068 at commit
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21060
Fixing API inconsistency should not be treated as a bug fix.
Please give me a few days. I need to summarize the Spark 2.3 release and
list all the PRs that were backported to the
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20937
**[Test build #89377 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89377/testReport)**
for PR 20937 at commit
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21060
This is not an new feature addition .. this fixes an exiting functionality
to work as expected and consistently ..
Sure, that'd be great. Will join in the discussion.
---
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21060
> This case specifically collect in PySpark doesn't work alone whereas all
other actions like foreach, show and other cases in other languages works in
all other APIs. Also, that's what a query
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21060
> The behavior consistency among Python/Scala/R/JAVA does not mean a bug,
right?
This case specifically `collect` in PySpark doesn't work alone whereas all
other actions like
Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21060
> The callback works for collect in R and Scala but Python doesn't. I think
we should at least match the behaviour. I wonder why it's hard to say a bug
when collect is detected in some APIs but
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21060
> We need to be very careful when backporting the PR with the behavior
changes, especially when this is neither a critical issue nor a regression.
Thus, I do not think we should backport this
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21074
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/2332/
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/21060
> withCallback was added in Spark 1.6 release
https://issues.apache.org/jira/browse/SPARK-11068 Since then, my understanding
is we never clearly define which should be part of withCallback.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21074
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
1 - 100 of 139 matches
Mail list logo
