SparkQA commented on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721198104
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35169/
AmplabJenkins removed a comment on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721196324
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins commented on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721196324
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721196293
Kubernetes integration test status success
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35168/
imback82 commented on a change in pull request #30229:
URL: https://github.com/apache/spark/pull/30229#discussion_r516749544
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzePartitionCommand.scala
##
@@ -75,6 +75,9 @@ case class
jkleckner commented on a change in pull request #29496:
URL: https://github.com/apache/spark/pull/29496#discussion_r516741734
##
File path:
resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/LoggingPodStatusWatcher.scala
##
@@ -177,4 +188,27
HyukjinKwon edited a comment on pull request #30232:
URL: https://github.com/apache/spark/pull/30232#issuecomment-720965482
BTW, @dongjoon-hyun, seems like Ubuntu 20.04 changed its default Python
version to Python 3 (?). I think that's why the error message shows a bit weird
like
SparkQA commented on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721182558
Kubernetes integration test starting
URL:
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35168/
srowen commented on a change in pull request #30203:
URL: https://github.com/apache/spark/pull/30203#discussion_r516730762
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala
##
@@ -218,13 +218,22 @@ object ExtractPythonUDFs
cloud-fan commented on a change in pull request #30229:
URL: https://github.com/apache/spark/pull/30229#discussion_r516728706
##
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala
##
@@ -175,7 +175,10 @@ abstract class SQLViewSuite extends
cloud-fan commented on a change in pull request #30229:
URL: https://github.com/apache/spark/pull/30229#discussion_r516727702
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
##
@@ -280,6 +280,9 @@ class
cloud-fan commented on a change in pull request #30229:
URL: https://github.com/apache/spark/pull/30229#discussion_r516726862
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/AnalyzePartitionCommand.scala
##
@@ -75,6 +75,9 @@ case class
srowen commented on pull request #30185:
URL: https://github.com/apache/spark/pull/30185#issuecomment-721169846
This still doesn't look right. I'd squash your branch changes and rebase on
master.
This is an automated
srowen commented on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721165959
Hoo boy that is a lot of code to touch. It sounds like eventually it has to
happen for Scala 3 and avoids some deprecation noise. I'm not against it though
as usual there is a
cloud-fan commented on a change in pull request #30229:
URL: https://github.com/apache/spark/pull/30229#discussion_r516725745
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala
##
@@ -419,17 +410,16 @@ class
srowen closed pull request #30182:
URL: https://github.com/apache/spark/pull/30182
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
srowen commented on pull request #30182:
URL: https://github.com/apache/spark/pull/30182#issuecomment-721162096
Merged to master/3.0
This is an automated message from the Apache Git Service.
To respond to the message, please
cloud-fan commented on a change in pull request #30193:
URL: https://github.com/apache/spark/pull/30193#discussion_r516722182
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2Exec.scala
##
@@ -479,15 +432,16 @@ object
cloud-fan commented on a change in pull request #30203:
URL: https://github.com/apache/spark/pull/30203#discussion_r516720258
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala
##
@@ -218,13 +218,22 @@ object ExtractPythonUDFs
SparkQA commented on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721156123
**[Test build #130568 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130568/testReport)**
for PR 30234 at commit
AmplabJenkins removed a comment on pull request #30151:
URL: https://github.com/apache/spark/pull/30151#issuecomment-721154544
This is an automated message from the Apache Git Service.
To respond to the message, please log on
akiyamaneko edited a comment on pull request #30182:
URL: https://github.com/apache/spark/pull/30182#issuecomment-721154044
Jenkins test this please
This is an automated message from the Apache Git Service.
To
AmplabJenkins commented on pull request #30151:
URL: https://github.com/apache/spark/pull/30151#issuecomment-721154544
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA removed a comment on pull request #30151:
URL: https://github.com/apache/spark/pull/30151#issuecomment-721027459
**[Test build #130563 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130563/testReport)**
for PR 30151 at commit
akiyamaneko commented on pull request #30182:
URL: https://github.com/apache/spark/pull/30182#issuecomment-721154044
retest this please
This is an automated message from the Apache Git Service.
To respond to the message,
SparkQA commented on pull request #30151:
URL: https://github.com/apache/spark/pull/30151#issuecomment-721153265
**[Test build #130563 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130563/testReport)**
for PR 30151 at commit
HyukjinKwon closed pull request #30227:
URL: https://github.com/apache/spark/pull/30227
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
AmplabJenkins commented on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721144076
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins removed a comment on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721144076
This is an automated message from the Apache Git Service.
To respond to the message, please log on
LuciferYang opened a new pull request #30234:
URL: https://github.com/apache/spark/pull/30234
### What changes were proposed in this pull request?
The purpose of this pr is to partial resolve SPARK-33285.
`Auto-application` is dropped in Scala 3 and deprecated in 2.13
SparkQA commented on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721142854
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
LuciferYang commented on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721140762
cc @srowen this is the first part to fix this compilation warnings, if it's
too big, I can split multiple small prs.
SparkQA removed a comment on pull request #30234:
URL: https://github.com/apache/spark/pull/30234#issuecomment-721142854
**[Test build #130567 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130567/testReport)**
for PR 30234 at commit
zero323 commented on pull request #30227:
URL: https://github.com/apache/spark/pull/30227#issuecomment-721137978
Thanks @HyukjinKwon!
This is an automated message from the Apache Git Service.
To respond to the message,
SparkQA removed a comment on pull request #30038:
URL: https://github.com/apache/spark/pull/30038#issuecomment-721069806
**[Test build #130566 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130566/testReport)**
for PR 30038 at commit
AmplabJenkins removed a comment on pull request #30151:
URL: https://github.com/apache/spark/pull/30151#issuecomment-721055509
This is an automated message from the Apache Git Service.
To respond to the message, please log on
ScrapCodes commented on pull request #27735:
URL: https://github.com/apache/spark/pull/27735#issuecomment-721070338
Hi @holdenk and @dongjoon-hyun,
Hope you are doing good !
Do you have a view around this one way or the other?
Thanks !
gengliangwang commented on pull request #30221:
URL: https://github.com/apache/spark/pull/30221#issuecomment-720397753
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
tgravescs commented on a change in pull request #30186:
URL: https://github.com/apache/spark/pull/30186#discussion_r516080247
##
File path:
core/src/main/resources/org/apache/spark/ui/static/executorspage-template.html
##
@@ -86,6 +86,22 @@ Executors
dongjoon-hyun closed pull request #30226:
URL: https://github.com/apache/spark/pull/30226
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
HyukjinKwon commented on a change in pull request #30203:
URL: https://github.com/apache/spark/pull/30203#discussion_r516423241
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/python/ExtractPythonUDFs.scala
##
@@ -218,13 +218,22 @@ object
AmplabJenkins removed a comment on pull request #30227:
URL: https://github.com/apache/spark/pull/30227#issuecomment-720494098
This is an automated message from the Apache Git Service.
To respond to the message, please log on
ScrapCodes commented on pull request #30038:
URL: https://github.com/apache/spark/pull/30038#issuecomment-720278520
Hi @huaxingao and @cloud-fan , This is ready for review !
This is an automated message from the Apache Git
AmplabJenkins removed a comment on pull request #30223:
URL: https://github.com/apache/spark/pull/30223#issuecomment-720382265
This is an automated message from the Apache Git Service.
To respond to the message, please log on
HeartSaVioR commented on pull request #26935:
URL: https://github.com/apache/spark/pull/26935#issuecomment-720970588
@viirya Kindly reminder. @gaborgsomogyi and @xuanyuanking gave +1 - would it
be good to persuade your call?
dongjoon-hyun closed pull request #29533:
URL: https://github.com/apache/spark/pull/29533
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to
AmplabJenkins commented on pull request #30230:
URL: https://github.com/apache/spark/pull/30230#issuecomment-720890323
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins commented on pull request #30216:
URL: https://github.com/apache/spark/pull/30216#issuecomment-720328716
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
ulysses-you opened a new pull request #28647:
URL: https://github.com/apache/spark/pull/28647
### What changes were proposed in this pull request?
At CreateTableLikeCommand, we use the new tblproperties with merge source
tblproperties.
### Why are the changes needed?
cloud-fan commented on a change in pull request #30202:
URL: https://github.com/apache/spark/pull/30202#discussion_r515753866
##
File path: docs/sql-migration-guide.md
##
@@ -52,7 +52,7 @@ license: |
- In Spark 3.1, the `schema_of_json` and `schema_of_csv` functions
AmplabJenkins removed a comment on pull request #30224:
URL: https://github.com/apache/spark/pull/30224#issuecomment-720385125
This is an automated message from the Apache Git Service.
To respond to the message, please log on
dongjoon-hyun commented on pull request #30231:
URL: https://github.com/apache/spark/pull/30231#issuecomment-720885674
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA commented on pull request #30164:
URL: https://github.com/apache/spark/pull/30164#issuecomment-720725184
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
cloud-fan commented on a change in pull request #30144:
URL: https://github.com/apache/spark/pull/30144#discussion_r515765822
##
File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
##
@@ -3691,6 +3691,32 @@ class SQLQuerySuite extends QueryTest with
sunchao removed a comment on pull request #30176:
URL: https://github.com/apache/spark/pull/30176#issuecomment-720671002
@HyukjinKwon @Obbay2 is there a JIRA for this?
This is an automated message from the Apache Git
HeartSaVioR commented on pull request #28841:
URL: https://github.com/apache/spark/pull/28841#issuecomment-720900095
Sigh. Now it has conflicts so I can't go with giving +1 and merging.
@cchighman Could you please fix the conflicts? It would be totally OK if
you'd like to let me
AmplabJenkins commented on pull request #26319:
URL: https://github.com/apache/spark/pull/26319#issuecomment-720287298
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA removed a comment on pull request #26319:
URL: https://github.com/apache/spark/pull/26319#issuecomment-720206210
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
xkrogen commented on pull request #29906:
URL: https://github.com/apache/spark/pull/29906#issuecomment-720575364
Huge thanks for pushing this through @tgravescs ! It was no small effort!
This is an automated message from the
SparkQA removed a comment on pull request #28647:
URL: https://github.com/apache/spark/pull/28647#issuecomment-720570503
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
SparkQA removed a comment on pull request #30164:
URL: https://github.com/apache/spark/pull/30164#issuecomment-720725184
**[Test build #130533 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130533/testReport)**
for PR 30164 at commit
AmplabJenkins removed a comment on pull request #29935:
URL: https://github.com/apache/spark/pull/29935#issuecomment-702872843
This is an automated message from the Apache Git Service.
To respond to the message, please log on
HeartSaVioR commented on a change in pull request #30151:
URL: https://github.com/apache/spark/pull/30151#discussion_r515757520
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/streaming/ui/StreamingQueryStatisticsPage.scala
##
@@ -126,6 +126,123 @@ private[ui]
AmplabJenkins removed a comment on pull request #30009:
URL: https://github.com/apache/spark/pull/30009#issuecomment-720876636
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins removed a comment on pull request #28841:
URL: https://github.com/apache/spark/pull/28841#issuecomment-720292747
This is an automated message from the Apache Git Service.
To respond to the message, please log on
otterc commented on pull request #30062:
URL: https://github.com/apache/spark/pull/30062#issuecomment-720702382
With the latest commit, all the review comments have been addressed.
@Ngone51 @tgravescs @attilapiros @mridulm @Victsm @jiangxb1987 Please take
another look at the changes.
AmplabJenkins removed a comment on pull request #30230:
URL: https://github.com/apache/spark/pull/30230#issuecomment-720890323
This is an automated message from the Apache Git Service.
To respond to the message, please log on
Ngone51 commented on a change in pull request #30062:
URL: https://github.com/apache/spark/pull/30062#discussion_r516467454
##
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java
##
@@ -0,0 +1,966 @@
+/*
+ * Licensed
SparkQA removed a comment on pull request #29082:
URL: https://github.com/apache/spark/pull/29082#issuecomment-720852047
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
HyukjinKwon commented on pull request #30177:
URL: https://github.com/apache/spark/pull/30177#issuecomment-720243217
Let me monitor a bit more before reverting it. I just saw it once.
This is an automated message from the
Victsm commented on a change in pull request #30062:
URL: https://github.com/apache/spark/pull/30062#discussion_r516167838
##
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol/PushBlockStream.java
##
@@ -23,24 +23,31 @@
import
HyukjinKwon commented on pull request #30227:
URL: https://github.com/apache/spark/pull/30227#issuecomment-720813038
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub
AmplabJenkins commented on pull request #30232:
URL: https://github.com/apache/spark/pull/30232#issuecomment-720895587
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
bozhang2820 opened a new pull request #30224:
URL: https://github.com/apache/spark/pull/30224
### What changes were proposed in this pull request?
This change is to support user provided nullable Avro schema for data with
non-nullable catalyst schema in Avro writing.
Without
SparkQA commented on pull request #30221:
URL: https://github.com/apache/spark/pull/30221#issuecomment-720879566
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
otterc commented on a change in pull request #30062:
URL: https://github.com/apache/spark/pull/30062#discussion_r516175694
##
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol/PushBlockStream.java
##
@@ -23,24 +23,31 @@
import
zero323 commented on a change in pull request #30216:
URL: https://github.com/apache/spark/pull/30216#discussion_r515789891
##
File path: R/pkg/DESCRIPTION
##
@@ -42,6 +42,7 @@ Collate:
'context.R'
'deserialize.R'
'functions.R'
+'functions_avro.R'
Review
zero323 opened a new pull request #30227:
URL: https://github.com/apache/spark/pull/30227
### What changes were proposed in this pull request?
This PR adds support for passing `Column`s as input to PySpark sorting
functions.
### Why are the changes needed?
zhengruifeng commented on pull request #30009:
URL: https://github.com/apache/spark/pull/30009#issuecomment-720851384
retest this please
This is an automated message from the Apache Git Service.
To respond to the message,
AmplabJenkins commented on pull request #30038:
URL: https://github.com/apache/spark/pull/30038#issuecomment-720294931
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
AmplabJenkins removed a comment on pull request #26319:
URL: https://github.com/apache/spark/pull/26319#issuecomment-720287298
This is an automated message from the Apache Git Service.
To respond to the message, please log on
dongjoon-hyun commented on pull request #30233:
URL: https://github.com/apache/spark/pull/30233#issuecomment-720928756
Thank you, @HyukjinKwon . Merged to master for Apache Spark 3.1.
This is an automated message from the
attilapiros commented on a change in pull request #30062:
URL: https://github.com/apache/spark/pull/30062#discussion_r516643629
##
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java
##
@@ -0,0 +1,966 @@
+/*
+ *
HeartSaVioR commented on a change in pull request #30221:
URL: https://github.com/apache/spark/pull/30221#discussion_r515822838
##
File path:
external/avro/src/main/scala/org/apache/spark/sql/avro/AvroUtils.scala
##
@@ -171,9 +171,15 @@ private[sql] object AvroUtils extends
AmplabJenkins commented on pull request #28841:
URL: https://github.com/apache/spark/pull/28841#issuecomment-720292747
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
dongjoon-hyun opened a new pull request #30232:
URL: https://github.com/apache/spark/pull/30232
### What changes were proposed in this pull request?
This PR aims to upgrade `Github Action` runner image from `Ubuntu 18.04
(LTS)` to `Ubuntu 20.04 (LTS)`.
### Why are the changes
HeartSaVioR edited a comment on pull request #30210:
URL: https://github.com/apache/spark/pull/30210#issuecomment-720876379
No I don't have real case for knowing and taking the risk. Probably I could
create some query which could evade the issue, but I agree that's more likely
in theory
SparkQA commented on pull request #30151:
URL: https://github.com/apache/spark/pull/30151#issuecomment-721027459
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
AmplabJenkins commented on pull request #30225:
URL: https://github.com/apache/spark/pull/30225#issuecomment-720385116
This is an automated message from the Apache Git Service.
To respond to the message, please log on to
HeartSaVioR commented on pull request #30210:
URL: https://github.com/apache/spark/pull/30210#issuecomment-720876379
No I don't have real case for knowing and taking the risk. Probably I could
create some query which could evade the issue, but I agree that's more likely
in theory and not
cloud-fan closed pull request #30200:
URL: https://github.com/apache/spark/pull/30200
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
tgravescs commented on pull request #30204:
URL: https://github.com/apache/spark/pull/30204#issuecomment-720557444
seems like random test failures happening. I tried the recent
YarnClusterSuite locally and it works fine.
AngersZh commented on a change in pull request #30144:
URL: https://github.com/apache/spark/pull/30144#discussion_r515926187
##
File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
##
@@ -3691,6 +3691,32 @@ class SQLQuerySuite extends QueryTest with
kiszk commented on pull request #26319:
URL: https://github.com/apache/spark/pull/26319#issuecomment-720634523
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
SparkQA removed a comment on pull request #30144:
URL: https://github.com/apache/spark/pull/30144#issuecomment-720433951
**[Test build #130526 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130526/testReport)**
for PR 30144 at commit
SparkQA commented on pull request #30230:
URL: https://github.com/apache/spark/pull/30230#issuecomment-720875095
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
AmplabJenkins removed a comment on pull request #30038:
URL: https://github.com/apache/spark/pull/30038#issuecomment-720294931
This is an automated message from the Apache Git Service.
To respond to the message, please log on
AmplabJenkins removed a comment on pull request #30186:
URL: https://github.com/apache/spark/pull/30186#issuecomment-720874715
This is an automated message from the Apache Git Service.
To respond to the message, please log on
HyukjinKwon commented on a change in pull request #30216:
URL: https://github.com/apache/spark/pull/30216#discussion_r515852857
##
File path: R/pkg/R/functions_avro.R
##
@@ -0,0 +1,117 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor
SparkQA commented on pull request #30228:
URL: https://github.com/apache/spark/pull/30228#issuecomment-720515948
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and
401 - 500 of 680 matches
Mail list logo