Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23152
thanks, merging to master/2.4!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23209
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5675/
---
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23204#discussion_r238323331
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala
---
@@ -483,8 +470,6 @@ private[execution] final class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23204#discussion_r238322699
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala
---
@@ -483,8 +470,6 @@ private[execution] final class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23136
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5674/
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23136
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99619/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23209
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23209
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99605/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23136
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23136
**[Test build #99619 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99619/testReport)**
for PR 23136 at commit
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238321460
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala
---
@@ -145,9 +145,14 @@ case class
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23209
**[Test build #99605 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99605/testReport)**
for PR 23209 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23150
**[Test build #4449 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4449/testReport)**
for PR 23150 at commit
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238318256
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala
---
@@ -145,9 +145,14 @@ case class
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22957
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23208
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23207
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22957
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23207
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22683
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23201
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22683
**[Test build #99611 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99611/testReport)**
for PR 22683 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23208
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23204
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20433
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21599
Build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23207
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22683
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99611/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99610/
Test FAILed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23201
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22468
**[Test build #99610 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99610/testReport)**
for PR 22468 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23207
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23094
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22204
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23094
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23150
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23202
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23202
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/20433
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23204
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23208
**[Test build #99620 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99620/testReport)**
for PR 23208 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23207
**[Test build #99616 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99616/testReport)**
for PR 23207 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22468
**[Test build #99610 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99610/testReport)**
for PR 22468 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23207
**[Test build #99617 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99617/testReport)**
for PR 23207 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23136
**[Test build #99619 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99619/testReport)**
for PR 23136 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21599
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23150
**[Test build #99608 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99608/testReport)**
for PR 23150 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23209
**[Test build #99605 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99605/testReport)**
for PR 23209 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23094
**[Test build #99612 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99612/testReport)**
for PR 23094 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22204
Merged build finished. Test FAILed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/23150
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22468
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22683
**[Test build #99611 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99611/testReport)**
for PR 22683 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/20433
**[Test build #99614 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99614/testReport)**
for PR 20433 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22957
**[Test build #99618 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99618/testReport)**
for PR 22957 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23202
**[Test build #99607 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99607/testReport)**
for PR 23202 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23195
**[Test build #99609 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99609/testReport)**
for PR 23195 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22204
**[Test build #99606 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99606/testReport)**
for PR 22204 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23201
**[Test build #99613 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99613/testReport)**
for PR 23201 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/23204
**[Test build #99615 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99615/testReport)**
for PR 23204 at commit
GitHub user skonto opened a pull request:
https://github.com/apache/spark/pull/23209
[SPARK-26256][K8s] Fix labels for pod deletion
## What changes were proposed in this pull request?
Adds proper labels when deleting executor pods.
## How was this patch tested?
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23208
cc @rdblue @rxin @jose-torres @gatorsmile @HyukjinKwon @gengliangwang
---
-
To unsubscribe, e-mail:
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23208#discussion_r238313454
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
---
@@ -17,52 +17,49 @@
package
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23208#discussion_r238313221
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ---
@@ -241,32 +241,28 @@ final class DataFrameWriter[T] private[sql](ds:
GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/23208
[SPARK-25530][SQL] data source v2 API refactor (batch write)
## What changes were proposed in this pull request?
Adjust the batch write API to match the read API refactor after
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/23207
@SparkQA test this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/23207
test this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/23207
@SparkQA retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands,
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/23207
@AmplabJenkins retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/23207
@AmplabJenkins test this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/23207
@AmplabJenkins
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mgaido91 commented on the issue:
https://github.com/apache/spark/pull/23057
@mccheah this is waiting for reviews by committers
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/23207
retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user skonto commented on the issue:
https://github.com/apache/spark/pull/23136
@liyinan926 updated the PR,here is the
[jira](https://issues.apache.org/jira/browse/SPARK-26256) for the bug wil
create a PR for it shortly.
---
Github user xuanyuanking commented on the issue:
https://github.com/apache/spark/pull/23207
test this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238292468
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/PlannerSuite.scala ---
@@ -780,6 +780,23 @@ class PlannerSuite extends SharedSQLContext {
Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22957#discussion_r238291824
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
---
@@ -223,14 +223,35 @@ abstract class Expression
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/18784
@skonto do you want to proceed with this?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/22952
> HDFS does not support it yet, though on the way, see
https://issues.apache.org/jira/browse/HADOOP-10019
That's an old patch; I don't know of any active dev there.
---
Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/22952
Hadoop FS glob filtering is pathologically bad on object stores.
I have tried in the past to do an ~O(1) impl for S3
[HADOOP-13371](https://issues.apache.org/jira/browse/HADOOP-13371).
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/22952
@HeartSaVioR
Related the glob part @zsxwing pointed out an important problem. Glob
pattern is much more than checking `*` and `?`, see the link up. For simplicity
take this test:
```
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/23205
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/22683
Add to this PR. The change goes logically together.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/23205
Merging as a follow up to https://github.com/apache/spark/pull/21688
---
-
To unsubscribe, e-mail:
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/23204
Is this observable in general hash join query, except for TPC-DS Q19?
---
-
To unsubscribe, e-mail:
Github user juliuszsompolski commented on the issue:
https://github.com/apache/spark/pull/23152
LGTM, thanks for the change!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
GitHub user xuanyuanking opened a pull request:
https://github.com/apache/spark/pull/23207
[SPARK-26193][SQL] Implement shuffle write metrics in SQL
## What changes were proposed in this pull request?
1. Implement `SQLShuffleWriteMetricsReporter` on the SQL side as the
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/23204#discussion_r238270550
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala
---
@@ -483,8 +470,6 @@ private[execution] final class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23204#discussion_r238264122
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLMetrics.scala
---
@@ -57,12 +57,6 @@ class SQLMetric(val metricType: String,
Github user gaborgsomogyi commented on the issue:
https://github.com/apache/spark/pull/22952
@HeartSaVioR
I've taken a deeper look at the overlap thing and found the following.
* Added an additional test which produced odd result:
```
...
val sourcePath
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22198
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99603/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/22198
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/22198
**[Test build #99603 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99603/testReport)**
for PR 22198 at commit
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23204
I'm fine to revert it if it caused a significant performance regression, we
should revisit it later, with different ideas, like updating the metrics for
each batch instead of each record.
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23204#discussion_r23825
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/TungstenAggregationIterator.scala
---
@@ -374,22 +374,6 @@ class
Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/23204#discussion_r238257371
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala
---
@@ -63,7 +63,7 @@ case class HashAggregateExec(
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/23194#discussion_r238256775
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/command/DDLSuite.scala
---
@@ -439,31 +440,22 @@ abstract class DDLSuite extends QueryTest
Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/23204
ok to test
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
401 - 500 of 626 matches
Mail list logo
