Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23121
I had better ask about the target branches. :)
Thanks, @jerryshao . Now, it lands for 2.4.1.
---
-
To unsubscribe,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22979
**[Test build #99211 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99211/testReport)**
for PR 22979 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23080
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23080
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23080
**[Test build #99210 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99210/testReport)**
for PR 23080 at commit
Github user zouzias commented on the issue:
https://github.com/apache/spark/pull/23103
@koeninger, I will make the doc changes asap. FYI, I plan to make changes
on file `structured-streaming-kafka-integration.md` seems the most relevant doc
for this diff.
---
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23121
Merged to `branch-2.4`.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23121
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99206/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23121
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23121
**[Test build #99206 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99206/testReport)**
for PR 23121 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23119
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23119
Merged to master to recover `master` branch Maven testings.
---
-
To unsubscribe, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99205/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23119
**[Test build #99205 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99205/testReport)**
for PR 23119 at commit
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235852779
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayBasedMapBuilder.scala
---
@@ -0,0 +1,118 @@
+/*
+ * Licensed
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23119
Hi, @cloud-fan . Could you review this, please?
[Test build #99205 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99205/testReport)
is almost passed.
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235851923
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala
---
@@ -558,8 +558,11 @@
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235851798
--- Diff: docs/sql-migration-guide-upgrade.md ---
@@ -19,6 +19,8 @@ displayTitle: Spark SQL Upgrading Guide
- In Spark version 2.4 and
Github user dongjoon-hyun commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235851554
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/ArrayBasedMapBuilder.scala
---
@@ -0,0 +1,118 @@
+/*
+ * Licensed
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235849825
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -751,171 +739,46 @@ case class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23124#discussion_r235849697
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/InternalRow.scala ---
@@ -125,22 +125,36 @@ object InternalRow {
* actually
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23124
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23124
**[Test build #99209 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99209/testReport)**
for PR 23124 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23124
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23124
cc @dongjoon-hyun @gatorsmile @viirya @kiszk @mgaido91
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23119
**[Test build #99202 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99202/testReport)**
for PR 23119 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99202/
Test FAILed.
---
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/23124
[SPARK-25829][SQL] remove duplicated map keys with last wins policy
## What changes were proposed in this pull request?
Currently duplicated map keys are not handled consistently. For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23123
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23123
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99208/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23123
**[Test build #99208 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99208/testReport)**
for PR 23123 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23122
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23122
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99207/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23122
**[Test build #99207 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99207/testReport)**
for PR 23122 at commit
Github user guoxiaolongzte commented on the issue:
https://github.com/apache/spark/pull/23104
Yes I tested and understood, you are right. @mgaido91
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23123
**[Test build #99208 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99208/testReport)**
for PR 23123 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23123
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23123
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user zhengruifeng opened a pull request:
https://github.com/apache/spark/pull/23123
[SPARK-26153][ML] GBT & RandomForest avoid unnecessary `first` job to
compute `numFeatures`
## What changes were proposed in this pull request?
use base models' `numFeature` instead of
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23122
**[Test build #99207 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99207/testReport)**
for PR 23122 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23122
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23122
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user zhengruifeng opened a pull request:
https://github.com/apache/spark/pull/23122
[MINOR][ML] add missing params to Instr
## What changes were proposed in this pull request?
add following param to instr:
GBTC: validationTol
GBTR: validationTol,
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21732#discussion_r235840903
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -257,6 +251,11 @@ case class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/21732#discussion_r235840884
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -257,6 +251,11 @@ case class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99203/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23119
**[Test build #99203 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99203/testReport)**
for PR 23119 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23121
**[Test build #99206 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99206/testReport)**
for PR 23121 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23121
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23121
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/23121
[SPARK-24553][UI][FOLLOWUP][2.4 Backport] Fix unnecessary UI redirect
## What changes were proposed in this pull request?
This is a backport PR of #23116 .
This PR is a
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23119
**[Test build #99205 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99205/testReport)**
for PR 23119 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23119
Retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99204/
Test FAILed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23119
**[Test build #99204 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99204/testReport)**
for PR 23119 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23119
**[Test build #99204 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99204/testReport)**
for PR 23119 at commit
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/23116
@dongjoon-hyun , this should also be backported to branch 2.4, let me
create a backport PR.
---
-
To unsubscribe, e-mail:
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23119
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23119
The current one hangs on `BroadcastSuite`.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23120
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99201/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23120
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23120
**[Test build #99201 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99201/testReport)**
for PR 23120 at commit
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23105#discussion_r235834225
--- Diff: core/src/main/scala/org/apache/spark/shuffle/ShuffleManager.scala
---
@@ -48,7 +48,8 @@ private[spark] trait ShuffleManager {
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23105#discussion_r235834136
--- Diff: core/src/main/scala/org/apache/spark/shuffle/metrics.scala ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF)
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23105#discussion_r235834088
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/ShuffleMetricsReporter.scala ---
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23043
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23043
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user koeninger commented on the issue:
https://github.com/apache/spark/pull/23103
@zouzias can you add the new option to
docs/structured-streaming-kafka-integration.md as part of this PR?
Instructions for building docs are in docs/README.md , ping me if you need a
hand.
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22598
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22598
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99197/
Test PASSed.
---
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22598
**[Test build #99197 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99197/testReport)**
for PR 22598 at commit
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23080#discussion_r235830894
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamReader.scala
---
@@ -377,6 +377,8 @@ final class DataStreamReader
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23111
Yea, the improvement looks persistent:
`Tests passed in 1027 seconds`
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/23117
It's not urgent :) so it's okay. Actually i'm on a vacation for a week as
well. Thanks for taking a look @shaneknapp !!
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/23098#discussion_r235830558
--- Diff: bin/load-spark-env.cmd ---
@@ -21,37 +21,42 @@ rem This script loads spark-env.cmd if it exists, and
ensures it is only loaded
rem
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23119
**[Test build #99203 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99203/testReport)**
for PR 23119 at commit
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23119
Retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23119
The previous successful one was `sbt`.
The current on-going one fails with another flacky test,
[SPARK-25903](https://issues.apache.org/jira/browse/SPARK-25903).
```
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/23118#discussion_r235830226
--- Diff: build/mvn ---
@@ -116,7 +116,8 @@ install_zinc() {
# the build/ folder
install_scala() {
# determine the Scala version used in
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99196/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23119
**[Test build #99196 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99196/testReport)**
for PR 23119 at commit
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22979
@HyukjinKwon Could it be related to recent changes in python tests?
---
-
To unsubscribe, e-mail:
Github user MaxGekk commented on the issue:
https://github.com/apache/spark/pull/22979
No chance to pass tests in the PR ;-)
```
test_aggregator (pyspark.sql.tests.test_group.GroupTests) ... #
# A fatal error has been detected by the Java Runtime Environment:
#
#
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23116
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23119
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23119
**[Test build #99202 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99202/testReport)**
for PR 23119 at commit
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23118
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user dongjoon-hyun commented on the issue:
https://github.com/apache/spark/pull/23116
Thank you, @jerryshao . Merged to master.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
1 - 100 of 387 matches
Mail list logo
