[GitHub] [spark] dongjoon-hyun commented on pull request #32811: [SPARK-35671][SHUFFLE][CORE] Add support in the ESS to serve merged shuffle block meta and data to executors
dongjoon-hyun commented on pull request #32811: URL: https://github.com/apache/spark/pull/32811#issuecomment-864360682 I have no other comments, @mridulm . :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
AmplabJenkins removed a comment on pull request #32970: URL: https://github.com/apache/spark/pull/32970#issuecomment-864358889 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140016/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
SparkQA removed a comment on pull request #32970: URL: https://github.com/apache/spark/pull/32970#issuecomment-864348284 **[Test build #140016 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140016/testReport)** for PR 32970 at commit [`7f755c6`](https://github.com/apache/spark/commit/7f755c64d9a1fd0dcaf93c07de5c7c1b90847dca). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
AmplabJenkins commented on pull request #32970: URL: https://github.com/apache/spark/pull/32970#issuecomment-864358889 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140016/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
SparkQA commented on pull request #32970: URL: https://github.com/apache/spark/pull/32970#issuecomment-864358686 **[Test build #140016 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140016/testReport)** for PR 32970 at commit [`7f755c6`](https://github.com/apache/spark/commit/7f755c64d9a1fd0dcaf93c07de5c7c1b90847dca). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
AmplabJenkins removed a comment on pull request #32970: URL: https://github.com/apache/spark/pull/32970#issuecomment-864357504 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44542/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32867: [SPARK-35721][PYTHON] Path level discover for python unittests
AmplabJenkins removed a comment on pull request #32867: URL: https://github.com/apache/spark/pull/32867#issuecomment-864357505 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140014/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
AmplabJenkins removed a comment on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864357506 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44541/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32867: [SPARK-35721][PYTHON] Path level discover for python unittests
AmplabJenkins commented on pull request #32867: URL: https://github.com/apache/spark/pull/32867#issuecomment-864357505 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140014/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
AmplabJenkins commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864357506 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44541/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
AmplabJenkins commented on pull request #32970: URL: https://github.com/apache/spark/pull/32970#issuecomment-864357504 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44542/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] xuanyuanking commented on pull request #32702: [SPARK-35565][SS] Add config for ignoring metadata directory of FileStreamSink
xuanyuanking commented on pull request #32702: URL: https://github.com/apache/spark/pull/32702#issuecomment-864357220 Thanks @viirya ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32867: [SPARK-35721][PYTHON] Path level discover for python unittests
SparkQA removed a comment on pull request #32867: URL: https://github.com/apache/spark/pull/32867#issuecomment-864343365 **[Test build #140014 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140014/testReport)** for PR 32867 at commit [`74e3dd1`](https://github.com/apache/spark/commit/74e3dd124cb6a0f75d4a75639e38b3804519fa1e). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32867: [SPARK-35721][PYTHON] Path level discover for python unittests
SparkQA commented on pull request #32867: URL: https://github.com/apache/spark/pull/32867#issuecomment-864357104 **[Test build #140014 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140014/testReport)** for PR 32867 at commit [`74e3dd1`](https://github.com/apache/spark/commit/74e3dd124cb6a0f75d4a75639e38b3804519fa1e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
SparkQA commented on pull request #32970: URL: https://github.com/apache/spark/pull/32970#issuecomment-864356941 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44542/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
SparkQA commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864354340 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44541/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
SparkQA commented on pull request #32970: URL: https://github.com/apache/spark/pull/32970#issuecomment-864353739 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44542/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
AmplabJenkins removed a comment on pull request #32969: URL: https://github.com/apache/spark/pull/32969#issuecomment-864352442 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44537/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32881: [SPARK-33298][CORE] Decouple file naming from FileCommitProtocol
AmplabJenkins removed a comment on pull request #32881: URL: https://github.com/apache/spark/pull/32881#issuecomment-864352444 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44538/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31677: [SPARK-34565][SQL] Collapse Window nodes with Project between them
AmplabJenkins removed a comment on pull request #31677: URL: https://github.com/apache/spark/pull/31677#issuecomment-864352441 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44539/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
AmplabJenkins removed a comment on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864352443 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140015/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31677: [SPARK-34565][SQL] Collapse Window nodes with Project between them
AmplabJenkins commented on pull request #31677: URL: https://github.com/apache/spark/pull/31677#issuecomment-864352441 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44539/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
AmplabJenkins commented on pull request #32969: URL: https://github.com/apache/spark/pull/32969#issuecomment-864352442 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44537/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32881: [SPARK-33298][CORE] Decouple file naming from FileCommitProtocol
AmplabJenkins commented on pull request #32881: URL: https://github.com/apache/spark/pull/32881#issuecomment-864352444 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44538/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
AmplabJenkins commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864352443 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140015/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31677: [SPARK-34565][SQL] Collapse Window nodes with Project between them
SparkQA commented on pull request #31677: URL: https://github.com/apache/spark/pull/31677#issuecomment-864352104 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44539/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
SparkQA commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864351612 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44541/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32881: [SPARK-33298][CORE] Decouple file naming from FileCommitProtocol
SparkQA commented on pull request #32881: URL: https://github.com/apache/spark/pull/32881#issuecomment-864351607 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44538/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
SparkQA commented on pull request #32969: URL: https://github.com/apache/spark/pull/32969#issuecomment-864351056 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44537/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
SparkQA removed a comment on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864347389 **[Test build #140015 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140015/testReport)** for PR 32964 at commit [`03e4342`](https://github.com/apache/spark/commit/03e43422d3ed208ea671b5bfc958d19a13b54137). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
SparkQA commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864349985 **[Test build #140015 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140015/testReport)** for PR 32964 at commit [`03e4342`](https://github.com/apache/spark/commit/03e43422d3ed208ea671b5bfc958d19a13b54137). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu commented on pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
AngersZh commented on pull request #32970: URL: https://github.com/apache/spark/pull/32970#issuecomment-864349819 @MaxGekk Since Hive's type not support startField/endField, should we support such as in script transform, input as YearMonthIntervalType(YEAR, MONTH) and value is 13 then output defined as YearMonthIntervalType(YEAR, YEAR) and result is 12? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mridulm commented on pull request #32811: [SPARK-35671][SHUFFLE][CORE] Add support in the ESS to serve merged shuffle block meta and data to executors
mridulm commented on pull request #32811: URL: https://github.com/apache/spark/pull/32811#issuecomment-864349119 Are there any other comments @Ngone51, @dongjoon-hyun ? I was planning to merge it in next couple of days if there are none. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] mridulm commented on a change in pull request #32811: [SPARK-35671][SHUFFLE][CORE] Add support in the ESS to serve merged shuffle block meta and data to executors
mridulm commented on a change in pull request #32811:
URL: https://github.com/apache/spark/pull/32811#discussion_r654741215
##
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolver.java
##
@@ -361,8 +361,8 @@ public int removeBlocks(String appId, String execId,
String[] blockIds) {
return numRemovedBlocks;
}
- public Map getLocalDirs(String appId, String[] execIds) {
-return Arrays.stream(execIds)
+ public Map getLocalDirs(String appId, Set execIds)
{
Review comment:
+CC @dongjoon-hyun any thoughts on the above ? Thx
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31677: [SPARK-34565][SQL] Collapse Window nodes with Project between them
SparkQA commented on pull request #31677: URL: https://github.com/apache/spark/pull/31677#issuecomment-864348544 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44539/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32881: [SPARK-33298][CORE] Decouple file naming from FileCommitProtocol
SparkQA commented on pull request #32881: URL: https://github.com/apache/spark/pull/32881#issuecomment-864348427 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44538/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
SparkQA commented on pull request #32970: URL: https://github.com/apache/spark/pull/32970#issuecomment-864348284 **[Test build #140016 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140016/testReport)** for PR 32970 at commit [`7f755c6`](https://github.com/apache/spark/commit/7f755c64d9a1fd0dcaf93c07de5c7c1b90847dca). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AngersZhuuuu opened a new pull request #32970: [SPARK-35772][SQL] Check all year-month interval types in HiveInspectors tests
AngersZh opened a new pull request #32970: URL: https://github.com/apache/spark/pull/32970 ### What changes were proposed in this pull request? Check all year-month interval types in HiveInspectors tests ### Why are the changes needed? Check all year-month interval types in HiveInspectors tests ### Does this PR introduce _any_ user-facing change? NO ### How was this patch tested? Added UT -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
SparkQA commented on pull request #32969: URL: https://github.com/apache/spark/pull/32969#issuecomment-864347830 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44537/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32867: [SPARK-35721][PYTHON] Path level discover for python unittests
AmplabJenkins removed a comment on pull request #32867: URL: https://github.com/apache/spark/pull/32867#issuecomment-864347588 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44540/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32867: [SPARK-35721][PYTHON] Path level discover for python unittests
SparkQA commented on pull request #32867: URL: https://github.com/apache/spark/pull/32867#issuecomment-864347581 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44540/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32867: [SPARK-35721][PYTHON] Path level discover for python unittests
AmplabJenkins commented on pull request #32867: URL: https://github.com/apache/spark/pull/32867#issuecomment-864347588 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44540/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
SparkQA commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864347389 **[Test build #140015 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140015/testReport)** for PR 32964 at commit [`03e4342`](https://github.com/apache/spark/commit/03e43422d3ed208ea671b5bfc958d19a13b54137). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast
AmplabJenkins removed a comment on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-864347258 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44535/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
AmplabJenkins removed a comment on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864347256 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44533/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32968: [SPARK-35470][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.base
AmplabJenkins removed a comment on pull request #32968: URL: https://github.com/apache/spark/pull/32968#issuecomment-864347255 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44536/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32967: [SPARK-35593][K8S][TESTS][FOLLOWUP] Increase timeout in KubernetesLocalDiskShuffleDataIOSuite
AmplabJenkins removed a comment on pull request #32967: URL: https://github.com/apache/spark/pull/32967#issuecomment-864347254 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44532/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast
AmplabJenkins commented on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-864347258 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44535/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
AmplabJenkins removed a comment on pull request #32969: URL: https://github.com/apache/spark/pull/32969#issuecomment-864347257 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140011/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
AmplabJenkins commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864347256 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44533/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32967: [SPARK-35593][K8S][TESTS][FOLLOWUP] Increase timeout in KubernetesLocalDiskShuffleDataIOSuite
AmplabJenkins commented on pull request #32967: URL: https://github.com/apache/spark/pull/32967#issuecomment-864347254 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44532/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
AmplabJenkins commented on pull request #32969: URL: https://github.com/apache/spark/pull/32969#issuecomment-864347257 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140011/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32968: [SPARK-35470][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.base
AmplabJenkins commented on pull request #32968: URL: https://github.com/apache/spark/pull/32968#issuecomment-864347255 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44536/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32968: [SPARK-35470][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.base
SparkQA commented on pull request #32968: URL: https://github.com/apache/spark/pull/32968#issuecomment-864345896 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44536/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
SparkQA commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864345280 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44533/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32967: [SPARK-35593][K8S][TESTS][FOLLOWUP] Increase timeout in KubernetesLocalDiskShuffleDataIOSuite
SparkQA commented on pull request #32967: URL: https://github.com/apache/spark/pull/32967#issuecomment-864345237 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44532/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
SparkQA removed a comment on pull request #32969: URL: https://github.com/apache/spark/pull/32969#issuecomment-864341696 **[Test build #140011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140011/testReport)** for PR 32969 at commit [`298a867`](https://github.com/apache/spark/commit/298a867750027c3331c096b3b55968ee70fb9c84). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
SparkQA commented on pull request #32969: URL: https://github.com/apache/spark/pull/32969#issuecomment-864345026 **[Test build #140011 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140011/testReport)** for PR 32969 at commit [`298a867`](https://github.com/apache/spark/commit/298a867750027c3331c096b3b55968ee70fb9c84). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast
SparkQA commented on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-864344186 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44535/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] pingsutw commented on a change in pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
pingsutw commented on a change in pull request #32964: URL: https://github.com/apache/spark/pull/32964#discussion_r654736129 ## File path: python/pyspark/pandas/frame.py ## @@ -4815,6 +4815,13 @@ def to_spark_io( index_col: Optional[Union[str, List[str]]] = None, **options ) -> None: +"""An alias for :func:`spark.to_spark_io`. +See :meth:`pyspark.pandas.DataFrame.spark.to_spark_io`. + +.. deprecated:: 3.2.0 +Use :func:`spark.to_spark_io` instead. Review comment: @HyukjinKwon Thanks for the review. Updated it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32867: [SPARK-35721][PYTHON] Path level discover for python unittests
SparkQA commented on pull request #32867: URL: https://github.com/apache/spark/pull/32867#issuecomment-864343365 **[Test build #140014 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140014/testReport)** for PR 32867 at commit [`74e3dd1`](https://github.com/apache/spark/commit/74e3dd124cb6a0f75d4a75639e38b3804519fa1e). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32968: [SPARK-35470][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.base
SparkQA commented on pull request #32968: URL: https://github.com/apache/spark/pull/32968#issuecomment-864342362 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44536/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31677: [SPARK-34565][SQL] Collapse Window nodes with Project between them
SparkQA commented on pull request #31677: URL: https://github.com/apache/spark/pull/31677#issuecomment-864341819 **[Test build #140013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140013/testReport)** for PR 31677 at commit [`7097ec3`](https://github.com/apache/spark/commit/7097ec3c724356a0d4c226b1222845b6df738e39). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
SparkQA commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864341764 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44533/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32881: [SPARK-33298][CORE] Decouple file naming from FileCommitProtocol
SparkQA commented on pull request #32881: URL: https://github.com/apache/spark/pull/32881#issuecomment-864341736 **[Test build #140012 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140012/testReport)** for PR 32881 at commit [`0f3df0f`](https://github.com/apache/spark/commit/0f3df0f45dc12768c7b9843e84e28a50b279443e). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
SparkQA commented on pull request #32969: URL: https://github.com/apache/spark/pull/32969#issuecomment-864341696 **[Test build #140011 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140011/testReport)** for PR 32969 at commit [`298a867`](https://github.com/apache/spark/commit/298a867750027c3331c096b3b55968ee70fb9c84). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32968: [SPARK-35470][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.base
AmplabJenkins removed a comment on pull request #32968: URL: https://github.com/apache/spark/pull/32968#issuecomment-864341555 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140010/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps
AmplabJenkins removed a comment on pull request #32859: URL: https://github.com/apache/spark/pull/32859#issuecomment-864341556 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140005/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32957: [SPARK-35472][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.generic
AmplabJenkins removed a comment on pull request #32957: URL: https://github.com/apache/spark/pull/32957#issuecomment-864341557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
AmplabJenkins removed a comment on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864341558 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140007/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32968: [SPARK-35470][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.base
AmplabJenkins commented on pull request #32968: URL: https://github.com/apache/spark/pull/32968#issuecomment-864341555 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140010/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
AmplabJenkins commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864341558 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140007/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32957: [SPARK-35472][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.generic
AmplabJenkins commented on pull request #32957: URL: https://github.com/apache/spark/pull/32957#issuecomment-864341557 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps
AmplabJenkins commented on pull request #32859: URL: https://github.com/apache/spark/pull/32859#issuecomment-864341556 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140005/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32967: [SPARK-35593][K8S][TESTS][FOLLOWUP] Increase timeout in KubernetesLocalDiskShuffleDataIOSuite
SparkQA commented on pull request #32967: URL: https://github.com/apache/spark/pull/32967#issuecomment-864341393 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44532/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] bersprockets commented on a change in pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
bersprockets commented on a change in pull request #32969:
URL: https://github.com/apache/spark/pull/32969#discussion_r654733861
##
File path:
external/avro/src/test/scala/org/apache/spark/sql/execution/benchmark/AvroWriteBenchmark.scala
##
@@ -31,7 +36,34 @@ package org.apache.spark.sql.execution.benchmark
* }}}
*/
object AvroWriteBenchmark extends DataSourceWriteBenchmark {
+ private def wideColumnsBenchmark: Unit = {
+import spark.implicits._
+
+withTempPath { dir =>
+ withTempTable("t1") {
+val width = 1000
+val values = 50
+val files = 20
+val selectExpr = (1 to width).map(i => s"value as c$i")
+// repartition to ensure we will write multiple files
+val df = spark.range(values)
+ .map(_ => Random.nextInt).selectExpr(selectExpr:
_*).repartition(files)
+ .persist(StorageLevel.DISK_ONLY)
+// cache the data to ensure we are not benchmarking range or
repartition
+df.filter("(c1*c2) = 12").collect
+df.createOrReplaceTempView("t1")
+val benchmark = new Benchmark(s"Write wide rows into $files files",
values)
+benchmark.addCase("Write wide rows") { _ =>
+ spark.sql("SELECT * FROM t1").
+
write.format("avro").save(s"${dir.getCanonicalPath}/${Random.nextLong.abs}")
+}
+benchmark.run()
Review comment:
This is not quite working. Results for this benchmark get printed in
stdout, but they don't show up in AvroWriteBenchmark-results.txt
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] bersprockets opened a new pull request #32969: [WIP][SPARK-35817][SQL] Restore performance of queries against wide Avro tables
bersprockets opened a new pull request #32969: URL: https://github.com/apache/spark/pull/32969 ### What changes were proposed in this pull request? When creating a record writer in an AvroDeserializer, or creating a struct converter in an AvroSerializer, look up Avro fields using a map rather than scanning the entire list of Avro fields. ### Why are the changes needed? A query against an Avro table can be quite slow when all are true: * There are many columns in the Avro file * The query contains a wide projection * There are many splits in the input * Some of the splits are read serially (e.g., less executors than there are tasks) A write to an Avro table can be quite slow when all are true: * There are many columns in the new rows * The operation is creating many files For example, a single-threaded query against a 6000 column Avro data set with 50K rows and 20 files takes less than a minute with Spark 3.0.1 but over 7 minutes with Spark 3.2.0-SNAPSHOT. This PR restores the faster time. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? * Ran existing unit tests * Added new unit tests * Added new benchmarks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast
SparkQA commented on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-864340793 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44535/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32957: [SPARK-35472][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.generic
SparkQA commented on pull request #32957: URL: https://github.com/apache/spark/pull/32957#issuecomment-864340585 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44534/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32968: [SPARK-35470][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.base
SparkQA removed a comment on pull request #32968: URL: https://github.com/apache/spark/pull/32968#issuecomment-864336384 **[Test build #140010 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140010/testReport)** for PR 32968 at commit [`fae2c02`](https://github.com/apache/spark/commit/fae2c029c4330075a8c38d24fb61c078f3882806). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32968: [SPARK-35470][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.base
SparkQA commented on pull request #32968: URL: https://github.com/apache/spark/pull/32968#issuecomment-864339945 **[Test build #140010 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140010/testReport)** for PR 32968 at commit [`fae2c02`](https://github.com/apache/spark/commit/fae2c029c4330075a8c38d24fb61c078f3882806). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `class SparkIndexOpsMethods(Generic[T_IndexOps], metaclass=ABCMeta):` * `class SparkSeriesMethods(SparkIndexOpsMethods[\"ps.Series\"]):` * `class SparkIndexMethods(SparkIndexOpsMethods[\"ps.Index\"]):` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ueshin closed pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps
ueshin closed pull request #32859: URL: https://github.com/apache/spark/pull/32859 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] ueshin commented on pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps
ueshin commented on pull request #32859: URL: https://github.com/apache/spark/pull/32859#issuecomment-864339663 Thanks! merging to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32957: [SPARK-35472][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.generic
SparkQA removed a comment on pull request #32957: URL: https://github.com/apache/spark/pull/32957#issuecomment-864335533 **[Test build #140008 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140008/testReport)** for PR 32957 at commit [`4d8a751`](https://github.com/apache/spark/commit/4d8a751ba145d7f2f59cb6521790a9d0c4240269). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32957: [SPARK-35472][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.generic
SparkQA commented on pull request #32957: URL: https://github.com/apache/spark/pull/32957#issuecomment-864339043 **[Test build #140008 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140008/testReport)** for PR 32957 at commit [`4d8a751`](https://github.com/apache/spark/commit/4d8a751ba145d7f2f59cb6521790a9d0c4240269). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] wangyum commented on pull request #31677: [SPARK-34565][SQL] Collapse Window nodes with Project between them
wangyum commented on pull request #31677: URL: https://github.com/apache/spark/pull/31677#issuecomment-864338820 retest this please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
SparkQA removed a comment on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864335530 **[Test build #140007 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140007/testReport)** for PR 32964 at commit [`6d3d9ef`](https://github.com/apache/spark/commit/6d3d9efd36b1b7c7df2dbc1c9b1a01e5c3ab1219). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
SparkQA commented on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864338725 **[Test build #140007 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140007/testReport)** for PR 32964 at commit [`6d3d9ef`](https://github.com/apache/spark/commit/6d3d9efd36b1b7c7df2dbc1c9b1a01e5c3ab1219). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] aokolnychyi commented on a change in pull request #32921: [WIP][SPARK-35779][SQL] Dynamic filtering for Data Source V2
aokolnychyi commented on a change in pull request #32921: URL: https://github.com/apache/spark/pull/32921#discussion_r654731571 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -96,6 +96,7 @@ case class AdaptiveSparkPlanExec( @transient private val queryStageOptimizerRules: Seq[Rule[SparkPlan]] = Seq( PlanAdaptiveDynamicPruningFilters(this), ReuseAdaptiveSubquery(context.subqueryCache), +PrepareScans, Review comment: It would be nice to move `PlanAdaptiveDynamicPruningFilters` and `PrepareScans` to prep rules in AQE just like I added `PrepareScans` to prep rules for non-AQE path. It is a bit tricky, though. `PlanAdaptiveDynamicPruningFilters` references the root node while prep rules are run immediately upon construction. Let me give it a try. ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -96,6 +96,7 @@ case class AdaptiveSparkPlanExec( @transient private val queryStageOptimizerRules: Seq[Rule[SparkPlan]] = Seq( PlanAdaptiveDynamicPruningFilters(this), ReuseAdaptiveSubquery(context.subqueryCache), +PrepareScans, Review comment: It would be nice to move `PlanAdaptiveDynamicPruningFilters` and `PrepareScans` to prep rules in AQE just like I added `PrepareScans` to prep rules for non-AQE path. It is a bit tricky, though. `PlanAdaptiveDynamicPruningFilters` references the root node while prep rules are run immediately upon construction. Let me give it a try. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps
SparkQA removed a comment on pull request #32859: URL: https://github.com/apache/spark/pull/32859#issuecomment-864314848 **[Test build #140005 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140005/testReport)** for PR 32859 at commit [`22530f4`](https://github.com/apache/spark/commit/22530f43ef9b4c44e141ae3349d6301d0ab4214e). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps
SparkQA commented on pull request #32859: URL: https://github.com/apache/spark/pull/32859#issuecomment-864337312 **[Test build #140005 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140005/testReport)** for PR 32859 at commit [`22530f4`](https://github.com/apache/spark/commit/22530f43ef9b4c44e141ae3349d6301d0ab4214e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #32967: [SPARK-35593][K8S][TESTS][FOLLOWUP] Increase timeout in KubernetesLocalDiskShuffleDataIOSuite
SparkQA removed a comment on pull request #32967: URL: https://github.com/apache/spark/pull/32967#issuecomment-864335520 **[Test build #140006 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140006/testReport)** for PR 32967 at commit [`05342b4`](https://github.com/apache/spark/commit/05342b4de541686dc5bdbddd882a603a7bded0b1). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32967: [SPARK-35593][K8S][TESTS][FOLLOWUP] Increase timeout in KubernetesLocalDiskShuffleDataIOSuite
AmplabJenkins removed a comment on pull request #32967: URL: https://github.com/apache/spark/pull/32967#issuecomment-864337216 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140006/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #32967: [SPARK-35593][K8S][TESTS][FOLLOWUP] Increase timeout in KubernetesLocalDiskShuffleDataIOSuite
AmplabJenkins commented on pull request #32967: URL: https://github.com/apache/spark/pull/32967#issuecomment-864337216 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140006/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #32967: [SPARK-35593][K8S][TESTS][FOLLOWUP] Increase timeout in KubernetesLocalDiskShuffleDataIOSuite
SparkQA commented on pull request #32967: URL: https://github.com/apache/spark/pull/32967#issuecomment-864337183 **[Test build #140006 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140006/testReport)** for PR 32967 at commit [`05342b4`](https://github.com/apache/spark/commit/05342b4de541686dc5bdbddd882a603a7bded0b1). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32957: [SPARK-35472][PYTHON] Fix disallow_untyped_defs mypy checks for pyspark.pandas.generic
AmplabJenkins removed a comment on pull request #32957: URL: https://github.com/apache/spark/pull/32957#issuecomment-864303594 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44528/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32859: [SPARK-35708][PYTHON][TEST] Add BaseTest for DataTypeOps
AmplabJenkins removed a comment on pull request #32859: URL: https://github.com/apache/spark/pull/32859#issuecomment-864335413 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44531/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32867: [SPARK-35721][PYTHON] Path level discover for python unittests
AmplabJenkins removed a comment on pull request #32867: URL: https://github.com/apache/spark/pull/32867#issuecomment-864335412 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140004/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io
AmplabJenkins removed a comment on pull request #32964: URL: https://github.com/apache/spark/pull/32964#issuecomment-864074058 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast
AmplabJenkins removed a comment on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-864153654 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44521/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
