tanelk opened a new pull request #30018:
URL: https://github.com/apache/spark/pull/30018
### What changes were proposed in this pull request?
Added optimizer rule `RemoveRedundantAggregates`. It removes redundant
aggregates from a query plan. A redundant aggregate is an
SparkQA commented on pull request #29792:
URL: https://github.com/apache/spark/pull/29792#issuecomment-707326402
**[Test build #129704 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129704/testReport)**
for PR 29792 at commit
AmplabJenkins commented on pull request #30012:
URL: https://github.com/apache/spark/pull/30012#issuecomment-707324055
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins removed a comment on pull request #30012:
URL: https://github.com/apache/spark/pull/30012#issuecomment-707324055
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA commented on pull request #30012:
URL: https://github.com/apache/spark/pull/30012#issuecomment-707324042
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34308/
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707323256
**[Test build #129703 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129703/testReport)**
for PR 29995 at commit
AmplabJenkins removed a comment on pull request #30012:
URL: https://github.com/apache/spark/pull/30012#issuecomment-707319635
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #30012:
URL: https://github.com/apache/spark/pull/30012#issuecomment-707319635
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #30012:
URL: https://github.com/apache/spark/pull/30012#issuecomment-707319619
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34307/
SparkQA commented on pull request #30012:
URL: https://github.com/apache/spark/pull/30012#issuecomment-707316427
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34308/
SparkQA commented on pull request #30012:
URL: https://github.com/apache/spark/pull/30012#issuecomment-707311997
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34307/
SparkQA removed a comment on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707221578
**[Test build #129698 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129698/testReport)**
for PR 30017 at commit
AmplabJenkins commented on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707309278
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins removed a comment on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707309278
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA commented on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707308803
**[Test build #129698 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129698/testReport)**
for PR 30017 at commit
AmplabJenkins removed a comment on pull request #29855:
URL: https://github.com/apache/spark/pull/29855#issuecomment-707307030
This is an automated message from the Apache Git Service.
To respond to the message, please log on
sunchao commented on a change in pull request #29959:
URL: https://github.com/apache/spark/pull/29959#discussion_r503493043
##
File path: core/src/main/scala/org/apache/spark/util/HadoopFSUtils.scala
##
@@ -207,18 +166,14 @@ private[spark] object HadoopFSUtils extends Logging
AmplabJenkins commented on pull request #29855:
URL: https://github.com/apache/spark/pull/29855#issuecomment-707307030
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA removed a comment on pull request #29855:
URL: https://github.com/apache/spark/pull/29855#issuecomment-707225366
**[Test build #129699 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129699/testReport)**
for PR 29855 at commit
SparkQA commented on pull request #29855:
URL: https://github.com/apache/spark/pull/29855#issuecomment-707305860
**[Test build #129699 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129699/testReport)**
for PR 29855 at commit
SparkQA commented on pull request #30012:
URL: https://github.com/apache/spark/pull/30012#issuecomment-707298231
**[Test build #129702 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129702/testReport)**
for PR 30012 at commit
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707297786
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707297786
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707228914
**[Test build #129700 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129700/testReport)**
for PR 29995 at commit
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707297095
**[Test build #129700 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129700/testReport)**
for PR 29995 at commit
AmplabJenkins removed a comment on pull request #29092:
URL: https://github.com/apache/spark/pull/29092#issuecomment-707293693
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #29092:
URL: https://github.com/apache/spark/pull/29092#issuecomment-707293693
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA removed a comment on pull request #29092:
URL: https://github.com/apache/spark/pull/29092#issuecomment-707159456
**[Test build #129695 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129695/testReport)**
for PR 29092 at commit
SparkQA commented on pull request #29092:
URL: https://github.com/apache/spark/pull/29092#issuecomment-707292765
**[Test build #129695 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129695/testReport)**
for PR 29092 at commit
SparkQA commented on pull request #30012:
URL: https://github.com/apache/spark/pull/30012#issuecomment-707291707
**[Test build #129701 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129701/testReport)**
for PR 30012 at commit
planga82 commented on a change in pull request #30014:
URL: https://github.com/apache/spark/pull/30014#discussion_r503475781
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/SparkSqlParserSuite.scala
##
@@ -160,6 +160,15 @@ class SparkSqlParserSuite
AmplabJenkins removed a comment on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707268129
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
AmplabJenkins removed a comment on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707268117
Merged build finished. Test FAILed.
This is an automated message from the Apache Git Service.
To
SparkQA removed a comment on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707176191
**[Test build #129696 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129696/testReport)**
for PR 29983 at commit
AmplabJenkins commented on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707268117
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707267634
**[Test build #129696 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129696/testReport)**
for PR 29983 at commit
AmplabJenkins removed a comment on pull request #29933:
URL: https://github.com/apache/spark/pull/29933#issuecomment-707258732
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
AmplabJenkins removed a comment on pull request #29933:
URL: https://github.com/apache/spark/pull/29933#issuecomment-707258719
Merged build finished. Test FAILed.
This is an automated message from the Apache Git Service.
To
AmplabJenkins commented on pull request #29933:
URL: https://github.com/apache/spark/pull/29933#issuecomment-707258719
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #29933:
URL: https://github.com/apache/spark/pull/29933#issuecomment-707258023
**[Test build #129694 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129694/testReport)**
for PR 29933 at commit
SparkQA removed a comment on pull request #29933:
URL: https://github.com/apache/spark/pull/29933#issuecomment-707132759
**[Test build #129694 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129694/testReport)**
for PR 29933 at commit
AmplabJenkins removed a comment on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707256000
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707255972
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34305/
AmplabJenkins commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707256000
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins removed a comment on pull request #29855:
URL: https://github.com/apache/spark/pull/29855#issuecomment-707252274
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA commented on pull request #29855:
URL: https://github.com/apache/spark/pull/29855#issuecomment-707252239
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34306/
AmplabJenkins commented on pull request #29855:
URL: https://github.com/apache/spark/pull/29855#issuecomment-707252274
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dbtsai commented on pull request #29843:
URL: https://github.com/apache/spark/pull/29843#issuecomment-707247213
@HeartSaVioR I believe @dongjoon-hyun also agreed if Hadoop 3 client is
fully compatible with Hadoop 2 clusters, we should plan to remove Hadoop 2.7
client in Spark to simplify
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707246872
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34305/
AmplabJenkins removed a comment on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707245897
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA commented on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707245875
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34303/
AmplabJenkins commented on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707245897
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
cloud-fan commented on pull request #28026:
URL: https://github.com/apache/spark/pull/28026#issuecomment-707245498
This branch has gone stale very far, so it's hard for me to make changes
against this branch. If you insist on continuing this PR, I can wait for you
and review it later.
SparkQA commented on pull request #29855:
URL: https://github.com/apache/spark/pull/29855#issuecomment-707244739
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34306/
rdblue commented on pull request #28026:
URL: https://github.com/apache/spark/pull/28026#issuecomment-707240152
@cloud-fan, feel free to open a PR against my branch. That would be helpful.
If you are instead suggesting a separate PR for Spark authored by you, then
I would rather
AmplabJenkins removed a comment on pull request #29933:
URL: https://github.com/apache/spark/pull/29933#issuecomment-707233031
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
viirya commented on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707239790
cc @cloud-fan
This is an automated message from the Apache Git Service.
To respond to the message, please log
SparkQA removed a comment on pull request #29933:
URL: https://github.com/apache/spark/pull/29933#issuecomment-707096695
**[Test build #129692 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129692/testReport)**
for PR 29933 at commit
AmplabJenkins removed a comment on pull request #29933:
URL: https://github.com/apache/spark/pull/29933#issuecomment-707233019
Merged build finished. Test FAILed.
This is an automated message from the Apache Git Service.
To
SparkQA removed a comment on pull request #29797:
URL: https://github.com/apache/spark/pull/29797#issuecomment-707089778
**[Test build #129691 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129691/testReport)**
for PR 29797 at commit
AmplabJenkins removed a comment on pull request #29797:
URL: https://github.com/apache/spark/pull/29797#issuecomment-707235054
This is an automated message from the Apache Git Service.
To respond to the message, please log on
cloud-fan commented on a change in pull request #30014:
URL: https://github.com/apache/spark/pull/30014#discussion_r503423737
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/SparkSqlParserSuite.scala
##
@@ -160,6 +160,15 @@ class SparkSqlParserSuite
AmplabJenkins removed a comment on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707237445
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA commented on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707237406
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34304/
AmplabJenkins commented on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707237445
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins commented on pull request #29797:
URL: https://github.com/apache/spark/pull/29797#issuecomment-707235054
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dongjoon-hyun commented on a change in pull request #30012:
URL: https://github.com/apache/spark/pull/30012#discussion_r503420430
##
File path: .github/workflows/build_and_test.yml
##
@@ -42,9 +42,11 @@ jobs:
mllib-local, mllib,
yarn, mesos,
SparkQA commented on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707234745
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34303/
SparkQA commented on pull request #29797:
URL: https://github.com/apache/spark/pull/29797#issuecomment-707233900
**[Test build #129691 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129691/testReport)**
for PR 29797 at commit
dongjoon-hyun commented on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707233230
cc @anuragmantri
This is an automated message from the Apache Git Service.
To respond to the message,
AmplabJenkins commented on pull request #29933:
URL: https://github.com/apache/spark/pull/29933#issuecomment-707233019
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
cloud-fan commented on a change in pull request #29983:
URL: https://github.com/apache/spark/pull/29983#discussion_r503418293
##
File path:
sql/core/src/test/resources/sql-tests/results/udf/postgreSQL/udf-aggregates_part1.sql.out
##
@@ -141,17 +141,17 @@ struct
+struct
dongjoon-hyun commented on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707232227
Thank you, @viirya .
This is an automated message from the Apache Git Service.
To respond to the message,
SparkQA commented on pull request #29933:
URL: https://github.com/apache/spark/pull/29933#issuecomment-707232491
**[Test build #129692 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129692/testReport)**
for PR 29933 at commit
cloud-fan commented on a change in pull request #29983:
URL: https://github.com/apache/spark/pull/29983#discussion_r503417842
##
File path:
sql/core/src/test/resources/sql-tests/results/udf/postgreSQL/udf-aggregates_part1.sql.out
##
@@ -141,17 +141,17 @@ struct
+struct
SparkQA commented on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707231289
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34304/
gemelen commented on a change in pull request #30007:
URL: https://github.com/apache/spark/pull/30007#discussion_r503416343
##
File path:
common/kvstore/src/main/java/org/apache/spark/util/kvstore/InMemoryStore.java
##
@@ -164,7 +164,7 @@ public void clear() {
}
/**
-
tanelk commented on a change in pull request #29092:
URL: https://github.com/apache/spark/pull/29092#discussion_r503416308
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
##
@@ -122,6 +122,7 @@ abstract class
SparkQA commented on pull request #29995:
URL: https://github.com/apache/spark/pull/29995#issuecomment-707228914
**[Test build #129700 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129700/testReport)**
for PR 29995 at commit
cloud-fan commented on pull request #28026:
URL: https://github.com/apache/spark/pull/28026#issuecomment-707228025
Hi @rdblue , I've already ported the parser change of this PR to our
internal fork and tested it with real queries, do you mind if I create a PR for
the parser change first?
SparkQA commented on pull request #29855:
URL: https://github.com/apache/spark/pull/29855#issuecomment-707225366
**[Test build #129699 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129699/testReport)**
for PR 29855 at commit
gemelen commented on a change in pull request #29995:
URL: https://github.com/apache/spark/pull/29995#discussion_r503410206
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/connector/InMemoryTable.scala
##
@@ -114,18 +115,21 @@ class InMemoryTable(
dongjoon-hyun commented on a change in pull request #30012:
URL: https://github.com/apache/spark/pull/30012#discussion_r503408898
##
File path: .github/workflows/build_and_test.yml
##
@@ -42,9 +42,11 @@ jobs:
mllib-local, mllib,
yarn, mesos,
dongjoon-hyun commented on a change in pull request #30012:
URL: https://github.com/apache/spark/pull/30012#discussion_r503408347
##
File path: .github/workflows/build_and_test.yml
##
@@ -42,9 +42,11 @@ jobs:
mllib-local, mllib,
yarn, mesos,
Victsm commented on a change in pull request #29855:
URL: https://github.com/apache/spark/pull/29855#discussion_r503408196
##
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalBlockHandler.java
##
@@ -373,6 +427,54 @@ public
Victsm commented on a change in pull request #29855:
URL: https://github.com/apache/spark/pull/29855#discussion_r503407485
##
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalBlockHandler.java
##
@@ -373,6 +427,54 @@ public
dongjoon-hyun commented on a change in pull request #30012:
URL: https://github.com/apache/spark/pull/30012#discussion_r503407286
##
File path: .github/workflows/build_and_test.yml
##
@@ -42,9 +42,11 @@ jobs:
mllib-local, mllib,
yarn, mesos,
viirya commented on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707221573
cc @dongjoon-hyun
This is an automated message from the Apache Git Service.
To respond to the message, please
SparkQA commented on pull request #30017:
URL: https://github.com/apache/spark/pull/30017#issuecomment-707221578
**[Test build #129698 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129698/testReport)**
for PR 30017 at commit
viirya opened a new pull request #30017:
URL: https://github.com/apache/spark/pull/30017
### What changes were proposed in this pull request?
In Spark 2.3.0 and previous versions, Hive CTAS command will convert to use
data source to write data into the table when the
SparkQA commented on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707213682
**[Test build #129697 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129697/testReport)**
for PR 29983 at commit
AmplabJenkins removed a comment on pull request #30011:
URL: https://github.com/apache/spark/pull/30011#issuecomment-707209998
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #30011:
URL: https://github.com/apache/spark/pull/30011#issuecomment-707209998
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA removed a comment on pull request #30011:
URL: https://github.com/apache/spark/pull/30011#issuecomment-707068685
**[Test build #129687 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129687/testReport)**
for PR 30011 at commit
SparkQA commented on pull request #30011:
URL: https://github.com/apache/spark/pull/30011#issuecomment-707208810
**[Test build #129687 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129687/testReport)**
for PR 30011 at commit
AmplabJenkins removed a comment on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707208138
This is an automated message from the Apache Git Service.
To respond to the message, please log on
viirya commented on pull request #29975:
URL: https://github.com/apache/spark/pull/29975#issuecomment-707208744
Thanks all!
This is an automated message from the Apache Git Service.
To respond to the message, please log on
SparkQA commented on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707208116
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34302/
AmplabJenkins commented on pull request #29983:
URL: https://github.com/apache/spark/pull/29983#issuecomment-707208138
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA removed a comment on pull request #29800:
URL: https://github.com/apache/spark/pull/29800#issuecomment-707058481
**[Test build #129686 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/129686/testReport)**
for PR 29800 at commit
301 - 400 of 836 matches
Mail list logo
